OpenStack Compute (nova)

instance.availability_zone reports wrong az after live migration

Bug #1771860 reported by Matt Riedemann
30
This bug affects 5 people
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Fix Released
Medium
Matt Riedemann
Pike
Fix Committed
Medium
Matt Riedemann
Queens
Fix Committed
Medium
Matt Riedemann
Rocky
Fix Committed
Medium
Matt Riedemann

Bug Description

This is a follow up to bug 1768876 and was discovered via functional testing for that bug here:

https://review.openstack.org/#/c/567701/

The scenario is:

1. create two compute hosts in separate AZs (zone1 and zone2)
2. create an instance without specifying an AZ so it can move freely between hosts in different zones
3. live migrate the instance - since there is only one other host, the instance has to move to the other zone
4. assert the instance.availability_zone is updated to reflect the other zone

#4 fails because the live migration conductor task doesn't update the instance.az once a destination host is picked by the scheduler. Note that the other move operations like unshelve, evacuate and cold migrate do update the instance.az once a host is picked.

Matt Riedemann (mriedem)
Changed in nova:
assignee: nobody → Matt Riedemann (mriedem)
OpenStack Infra (hudson-openstack)
Changed in nova:
status: Triaged → In Progress
OpenStack Infra (hudson-openstack)
Changed in nova:
assignee: Matt Riedemann (mriedem) → Zhenyu Zheng (zhengzhenyu)
Matt Riedemann (mriedem)
Changed in nova:
assignee: Zhenyu Zheng (zhengzhenyu) → Matt Riedemann (mriedem)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (stable/rocky)

Fix proposed to branch: stable/rocky
Review: https://review.openstack.org/643173

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (master)

Reviewed: https://review.openstack.org/567701
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=0ed68c76fa8a84d1d5f0ab945e34c8e16341d627
Submitter: Zuul
Branch: master

commit 0ed68c76fa8a84d1d5f0ab945e34c8e16341d627
Author: Matt Riedemann <email address hidden>
Date: Thu May 10 19:27:36 2018 -0400

    Update instance.availability_zone during live migration

    While triaging bug 1768876 there was some concern
    that change I8d426f2635232ffc4b510548a905794ca88d7f99
    in Pike had regressed some behavior where a user that
    does not explicitly request a specific AZ during server
    create is then later restricted to only move operations
    within that same AZ.

    This test shows that is not a regression because the
    AvailabilityZoneFilter looks at RequestSpec.availability_zone
    rather than instance.availabililty_zone, so the instance
    is free to be moved across zones.

    As a result of the test, however, it was noticed that
    the instance.availability_zone isn't updated during live
    migration once the destination host is selected. The other
    move operations like unshelve, evacuate and cold migrate
    all update the instance.availabiltiy_zone, so this copies
    the same logic.

    Change-Id: I9f73c237923fdcbf4096edc5aedd2c968d4b893e
    Closes-Bug: #1771860
    Related-Bug: #1768876

Changed in nova:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (stable/rocky)

Reviewed: https://review.openstack.org/643173
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=133763d3582c2e85e4e5962b542294135d1a7f4c
Submitter: Zuul
Branch: stable/rocky

commit 133763d3582c2e85e4e5962b542294135d1a7f4c
Author: Matt Riedemann <email address hidden>
Date: Thu May 10 19:27:36 2018 -0400

    Update instance.availability_zone during live migration

    While triaging bug 1768876 there was some concern
    that change I8d426f2635232ffc4b510548a905794ca88d7f99
    in Pike had regressed some behavior where a user that
    does not explicitly request a specific AZ during server
    create is then later restricted to only move operations
    within that same AZ.

    This test shows that is not a regression because the
    AvailabilityZoneFilter looks at RequestSpec.availability_zone
    rather than instance.availabililty_zone, so the instance
    is free to be moved across zones.

    As a result of the test, however, it was noticed that
    the instance.availability_zone isn't updated during live
    migration once the destination host is selected. The other
    move operations like unshelve, evacuate and cold migrate
    all update the instance.availabiltiy_zone, so this copies
    the same logic.

    Conflicts:
          nova/tests/unit/conductor/tasks/test_live_migrate.py

    NOTE(mriedem): The conflict is due to not having change
    I8e47cac8bab50a086b98f37c2f9f659b10009cf1 in Rocky.
    Also note that the func_fixtures import in the functional
    test was changed since it was added in Stein with change
    Idaed39629095f86d24a54334c699a26c218c6593.

    Change-Id: I9f73c237923fdcbf4096edc5aedd2c968d4b893e
    Closes-Bug: #1771860
    Related-Bug: #1768876
    (cherry picked from commit 0ed68c76fa8a84d1d5f0ab945e34c8e16341d627)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/nova 19.0.0.0rc1

This issue was fixed in the openstack/nova 19.0.0.0rc1 release candidate.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/nova 18.2.0

This issue was fixed in the openstack/nova 18.2.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (stable/queens)

Fix proposed to branch: stable/queens
Review: https://review.openstack.org/647623

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (stable/pike)

Fix proposed to branch: stable/pike
Review: https://review.openstack.org/647630

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (stable/queens)

Reviewed: https://review.openstack.org/647623
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=ee07c1c67d4ee6b146783c40826c4a28bd7367c3
Submitter: Zuul
Branch: stable/queens

commit ee07c1c67d4ee6b146783c40826c4a28bd7367c3
Author: Matt Riedemann <email address hidden>
Date: Thu May 10 19:27:36 2018 -0400

    Update instance.availability_zone during live migration

    While triaging bug 1768876 there was some concern
    that change I8d426f2635232ffc4b510548a905794ca88d7f99
    in Pike had regressed some behavior where a user that
    does not explicitly request a specific AZ during server
    create is then later restricted to only move operations
    within that same AZ.

    This test shows that is not a regression because the
    AvailabilityZoneFilter looks at RequestSpec.availability_zone
    rather than instance.availabililty_zone, so the instance
    is free to be moved across zones.

    As a result of the test, however, it was noticed that
    the instance.availability_zone isn't updated during live
    migration once the destination host is selected. The other
    move operations like unshelve, evacuate and cold migrate
    all update the instance.availabiltiy_zone, so this copies
    the same logic.

    Change-Id: I9f73c237923fdcbf4096edc5aedd2c968d4b893e
    Closes-Bug: #1771860
    Related-Bug: #1768876
    (cherry picked from commit 0ed68c76fa8a84d1d5f0ab945e34c8e16341d627)
    (cherry picked from commit 133763d3582c2e85e4e5962b542294135d1a7f4c)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (stable/pike)

Reviewed: https://review.opendev.org/647630
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=d6832b0e070cebbe48a71a46d88b9412375989c1
Submitter: Zuul
Branch: stable/pike

commit d6832b0e070cebbe48a71a46d88b9412375989c1
Author: Matt Riedemann <email address hidden>
Date: Thu May 10 19:27:36 2018 -0400

    Update instance.availability_zone during live migration

    While triaging bug 1768876 there was some concern
    that change I8d426f2635232ffc4b510548a905794ca88d7f99
    in Pike had regressed some behavior where a user that
    does not explicitly request a specific AZ during server
    create is then later restricted to only move operations
    within that same AZ.

    This test shows that is not a regression because the
    AvailabilityZoneFilter looks at RequestSpec.availability_zone
    rather than instance.availabililty_zone, so the instance
    is free to be moved across zones.

    As a result of the test, however, it was noticed that
    the instance.availability_zone isn't updated during live
    migration once the destination host is selected. The other
    move operations like unshelve, evacuate and cold migrate
    all update the instance.availabiltiy_zone, so this copies
    the same logic.

    Conflicts:
          nova/conductor/tasks/live_migrate.py
          nova/tests/unit/conductor/tasks/test_live_migrate.py

    NOTE(mriedem): The conflicts were due to changes
    Idad5cdbb2c5647c469e4ad5e9393564255df0f7f and
    I9068a5a5b47cef565802a6d58f37777464644100 in Queens.
    The _wait_for_migration_status interface also changed
    with I752617066bb2167b49239ab9d17b0c89754a3e12 in Queens.

    Change-Id: I9f73c237923fdcbf4096edc5aedd2c968d4b893e
    Closes-Bug: #1771860
    Related-Bug: #1768876
    (cherry picked from commit 0ed68c76fa8a84d1d5f0ab945e34c8e16341d627)
    (cherry picked from commit 133763d3582c2e85e4e5962b542294135d1a7f4c)
    (cherry picked from commit ee07c1c67d4ee6b146783c40826c4a28bd7367c3)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/nova 16.1.8

This issue was fixed in the openstack/nova 16.1.8 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/nova 17.0.11

This issue was fixed in the openstack/nova 17.0.11 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.