nova-libvirt-compare.py should handle VERIFY_RESIZE better

[Simon Leinen]

I just migrated a few SHUTOFF instances to other hypervisors. They have been moved to other hypervisors and are in VERIFY_RESIZE state now. nova-libvirt-compare.py (from osops-tools-contrib) gives a bit too much output about those:

$ ./nova-libvirt-compare.py -b
Possible inconsistency: Instance f783ca49-4609-4f70-aec3-00c7f04479bf (Nova status VERIFY_RESIZE)
  On hypervisor zhdk0067.zhdk.cloud.switch.ch, it has state shut off
Possible inconsistency: Instance a7b25635-96d6-4336-a32d-167f082a7198 (Nova status VERIFY_RESIZE)
  On hypervisor zhdk0065.zhdk.cloud.switch.ch, it has state shut off
Possible inconsistency: Instance 32a01399-7423-46a3-8d6b-c402947f72eb (Nova status VERIFY_RESIZE)
  On hypervisor zhdk0065.zhdk.cloud.switch.ch, it has state shut off
Possible inconsistency: Instance e0208935-a6ac-4e3b-bbc8-52e7f893ed68 (Nova status VERIFY_RESIZE)
  On hypervisor zhdk0065.zhdk.cloud.switch.ch, it has state shut off
Possible inconsistency: Instance cfa840b7-e0ae-4ad6-bd64-747e605f5022 (Nova status VERIFY_RESIZE)
  On hypervisor zhdk0065.zhdk.cloud.switch.ch, it has state shut off
Possible inconsistency: Instance d0ad605e-0f13-4c15-b44d-ebef4d6c264c (Nova status VERIFY_RESIZE)
  On hypervisor zhdk0065.zhdk.cloud.switch.ch, it has state shut off
Possible inconsistency: Instance 062944e5-68d9-44f5-b047-75a3dc4699ef (Nova status VERIFY_RESIZE)
  On hypervisor zhdk0065.zhdk.cloud.switch.ch, it has state shut off
Possible inconsistency: Instance ecff603b-def7-4c2c-86eb-323526b3ad62 (Nova status VERIFY_RESIZE)
  On hypervisor zhdk0067.zhdk.cloud.switch.ch, it has state shut off
Possible inconsistency: Instance e62feb7f-8cc5-4c03-bd8b-c88b2a6fef88 (Nova status VERIFY_RESIZE)
  On hypervisor zhdk0069.zhdk.cloud.switch.ch, it has state shut off
Possible inconsistency: Instance 3633420d-61d1-4d32-8912-3a94f79b5f74 (Nova status VERIFY_RESIZE)
  On hypervisor zhdk0065.zhdk.cloud.switch.ch, it has state shut off
Instance cfa840b7-e0ae-4ad6-bd64-747e605f5022 (Nova status VERIFY_RESIZE):
  Found on hypervisor zhdk0036.zhdk.cloud.switch.ch (state: shut off)
  Should be running on zhdk0065.zhdk.cloud.switch.ch, and it is (state: shut off).
Instance ecff603b-def7-4c2c-86eb-323526b3ad62 (Nova status VERIFY_RESIZE):
  Found on hypervisor zhdk0036.zhdk.cloud.switch.ch (state: shut off)
  Should be running on zhdk0067.zhdk.cloud.switch.ch, and it is (state: shut off).
Instance e62feb7f-8cc5-4c03-bd8b-c88b2a6fef88 (Nova status VERIFY_RESIZE):
  Found on hypervisor zhdk0041.zhdk.cloud.switch.ch (state: shut off)
  Should be running on zhdk0069.zhdk.cloud.switch.ch, and it is (state: shut off).
Instance 3633420d-61d1-4d32-8912-3a94f79b5f74 (Nova status VERIFY_RESIZE):
  Found on hypervisor zhdk0035.zhdk.cloud.switch.ch (state: shut off)
  Should be running on zhdk0065.zhdk.cloud.switch.ch, and it is (state: shut off).
Instance e0208935-a6ac-4e3b-bbc8-52e7f893ed68 (Nova status VERIFY_RESIZE):
  Found on hypervisor zhdk0035.zhdk.cloud.switch.ch (state: shut off)
  Should be running on zhdk0065.zhdk.cloud.switch.ch, and it is (state: shut off).
Instance f783ca49-4609-4f70-aec3-00c7f04479bf (Nova status VERIFY_RESIZE):
  Found on hypervisor zhdk0035.zhdk.cloud.switch.ch (state: shut off)
  Should be running on zhdk0067.zhdk.cloud.switch.ch, and it is (state: shut off).
Instance a7b25635-96d6-4336-a32d-167f082a7198 (Nova status VERIFY_RESIZE):
  Found on hypervisor zhdk0037.zhdk.cloud.switch.ch (state: shut off)
  Should be running on zhdk0065.zhdk.cloud.switch.ch, and it is (state: shut off).
Instance d0ad605e-0f13-4c15-b44d-ebef4d6c264c (Nova status VERIFY_RESIZE):
  Found on hypervisor zhdk0037.zhdk.cloud.switch.ch (state: shut off)
  Should be running on zhdk0065.zhdk.cloud.switch.ch, and it is (state: shut off).
Instance 062944e5-68d9-44f5-b047-75a3dc4699ef (Nova status VERIFY_RESIZE):
  Found on hypervisor zhdk0037.zhdk.cloud.switch.ch (state: shut off)
  Should be running on zhdk0065.zhdk.cloud.switch.ch, and it is (state: shut off).
Instance 32a01399-7423-46a3-8d6b-c402947f72eb (Nova status VERIFY_RESIZE):
  Found on hypervisor zhdk0037.zhdk.cloud.switch.ch (state: shut off)
  Should be running on zhdk0065.zhdk.cloud.switch.ch, and it is (state: shut off).

Is not bad that the script warns about instances in VERIFY_RESIZE, because if instances remain in that state for extended periods of time, that may point to a problem. But the warnings could be made less verbose. In particular, the fact that the instances are known to two hypervisors is completely normal.

In addition, we might only warn about VERIFY_RESIZE instances when desired (via a to-be-defined command flag) and/or when the state hasn't been modified more than a given amount of time ago.