Transforming the RequestSpec object into legacy dicts doesn't support the requested_destination field

Bug #1702454 reported by Sylvain Bauza on 2017-07-05
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
High
Matt Riedemann
Newton
High
Tony Breeds
Ocata
High
Matt Riedemann
Pike
Medium
Matt Riedemann

Bug Description

We added a new field in the RequestSpec object called 'requested_destination' and we began using it for evacuations by https://review.openstack.org/#/c/315572/ (Newton)

That object was tranformed into legacy dictionaries (called "filter_properties" and "request_spec") before being rehydrated for the rebuild_instance() method in the conductor service. That said, when transforming, we were forgetting about the 'requested_destination' field in the object so that when we were calling the scheduler, we were never using that field.

That bug was fixed implicitly by https://review.openstack.org/#/c/469037/ which is now merged in master, but the issue is still there in stable branches, and if you need to use the legacy methods, you'll not have it.

As a consequence, the feature to pass a destination for evacuation is not working in Newton and Ocata. Fortunately, given we didn't transformed the object into dicts before calling the scheduler for live-migrations, it does work for that action.

A proper resolution would be to make sure that we pass the requested_destination field into 'filter_properties' so that when transforming again into an object, we set again the field.

Fix proposed to branch: master
Review: https://review.openstack.org/481116

Changed in nova:
status: Confirmed → In Progress
tags: added: pike-rc-potential
Matt Riedemann (mriedem) wrote :

Why is this tagged as pike-rc-potential, it's a latent issue in stable branches, correct?

tags: added: evacuate
removed: pike-rc-potential
Matt Riedemann (mriedem) on 2017-08-01
tags: removed: rebuild
Changed in nova:
assignee: Sylvain Bauza (sylvain-bauza) → Matt Riedemann (mriedem)
Matt Riedemann (mriedem) on 2017-10-29
Changed in nova:
assignee: Matt Riedemann (mriedem) → Sylvain Bauza (sylvain-bauza)
Changed in nova:
assignee: Sylvain Bauza (sylvain-bauza) → Matt Riedemann (mriedem)

Reviewed: https://review.openstack.org/490997
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=6e332cb7f1f1b18d117685a0a0f5bb6a8e0c8e93
Submitter: Zuul
Branch: master

commit 6e332cb7f1f1b18d117685a0a0f5bb6a8e0c8e93
Author: Sylvain Bauza <email address hidden>
Date: Wed Aug 2 19:28:13 2017 +0200

    Functional regression test for evacuate with a target

    In I6ddcaaca37fc5387c2d2e9f51c67ea9e85acb5c5 we forgot to update the
    legacy filter properties dictionary so the requested target wasn't
    passed to the scheduler when evacuating.
    Adding a functional test for verifying the behaviour.

    NOTE(sbauza): The issue has been incendentally fixed in Pike by
    I434af8e4ad991ac114dd67d66797a562d16bafe2 so the regression test just
    verifies that the expected behaviour works.
    The Newton and Ocata backports will be slightly different from that one as we
    need to verify that host3 will be preferred eventually over host2.

    Related-Bug: #1702454

    Change-Id: Id9adb10d2ef821c8b61d8f1d5dc9dd66ec7aaac8

Reviewed: https://review.openstack.org/481116
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=7317afd3201c4f9f21d3f37dab08423dae236b17
Submitter: Zuul
Branch: master

commit 7317afd3201c4f9f21d3f37dab08423dae236b17
Author: Sylvain Bauza <email address hidden>
Date: Thu Jul 6 17:17:31 2017 +0200

    Pass requested_destination in filter_properties

    When we added the requested_destination field for the RequestSpec object
    in Newton, we forgot to pass it to the legacy dictionary when wanting to
    use scheduler methods not yet supporting the NovaObject.
    As a consequence, when we were transforming the RequestSpec object into a
    tuple of (request_spec, filter_props) dicts and then rehydrating a new
    RequestSpec object using those dicts, the newly created object was not
    keeping that requested_destination field from the original.

    Change-Id: Iba0b88172e9a3bfd4f216dd364d70f7e01c60ee2
    Closes-Bug: #1702454

Changed in nova:
status: In Progress → Fix Released

Reviewed: https://review.openstack.org/491000
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=0b672420bceaf5a6e264a9362eeb6e29badd09d8
Submitter: Zuul
Branch: stable/ocata

commit 0b672420bceaf5a6e264a9362eeb6e29badd09d8
Author: Sylvain Bauza <email address hidden>
Date: Wed Aug 2 19:28:13 2017 +0200

    Functional regression test for evacuate with a target

    In I6ddcaaca37fc5387c2d2e9f51c67ea9e85acb5c5 we forgot to update the
    legacy filter properties dictionary so the requested target wasn't
    passed to the scheduler when evacuating.
    Adding a functional test for verifying the behaviour.

    NOTE(sbauza): The issue has been incendentally fixed in Pike by
    I434af8e4ad991ac114dd67d66797a562d16bafe2 so the regression test just
    verifies that the expected behaviour works.
    That specific backport is consequently different from the master change
    as the final target is different from the one we expected.

    Related-Bug: #1702454

    Change-Id: Id9adb10d2ef821c8b61d8f1d5dc9dd66ec7aaac8
    (cherry picked from commit e0e2e065a495b4fa9ebdec987c935e3c83118c46)
    (cherry picked from commit b575a2645137d0e8f1f622ee45fca89605d2fdb9)

tags: added: in-stable-ocata
tags: added: in-stable-pike

Reviewed: https://review.openstack.org/516111
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=de27ede60dd66f7c4b76fe39c63c0305424a7903
Submitter: Zuul
Branch: stable/pike

commit de27ede60dd66f7c4b76fe39c63c0305424a7903
Author: Sylvain Bauza <email address hidden>
Date: Wed Aug 2 19:28:13 2017 +0200

    Functional regression test for evacuate with a target

    In I6ddcaaca37fc5387c2d2e9f51c67ea9e85acb5c5 we forgot to update the
    legacy filter properties dictionary so the requested target wasn't
    passed to the scheduler when evacuating.
    Adding a functional test for verifying the behaviour.

    NOTE(sbauza): The issue has been incendentally fixed in Pike by
    I434af8e4ad991ac114dd67d66797a562d16bafe2 so the regression test just
    verifies that the expected behaviour works.
    The Newton and Ocata backports will be slightly different from that one as we
    need to verify that host3 will be preferred eventually over host2.

    Related-Bug: #1702454

    Change-Id: Id9adb10d2ef821c8b61d8f1d5dc9dd66ec7aaac8
    (cherry picked from commit e0e2e065a495b4fa9ebdec987c935e3c83118c46)

Reviewed: https://review.openstack.org/516112
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=0c0023d0759f54f5bd99d9ed29c2a2ccb3965cba
Submitter: Zuul
Branch: stable/pike

commit 0c0023d0759f54f5bd99d9ed29c2a2ccb3965cba
Author: Sylvain Bauza <email address hidden>
Date: Thu Jul 6 17:17:31 2017 +0200

    Pass requested_destination in filter_properties

    When we added the requested_destination field for the RequestSpec object
    in Newton, we forgot to pass it to the legacy dictionary when wanting to
    use scheduler methods not yet supporting the NovaObject.
    As a consequence, when we were transforming the RequestSpec object into a
    tuple of (request_spec, filter_props) dicts and then rehydrating a new
    RequestSpec object using those dicts, the newly created object was not
    keeping that requested_destination field from the original.

    Change-Id: Iba0b88172e9a3bfd4f216dd364d70f7e01c60ee2
    Closes-Bug: #1702454
    (cherry picked from commit 69bef428bd555bb31f43db6ca9c21db8aeb9007e)

Reviewed: https://review.openstack.org/491001
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=fe8acf003e8e4d26f02ec38034c4715dfb9888ba
Submitter: Zuul
Branch: stable/ocata

commit fe8acf003e8e4d26f02ec38034c4715dfb9888ba
Author: Sylvain Bauza <email address hidden>
Date: Thu Jul 6 17:17:31 2017 +0200

    Pass requested_destination in filter_properties

    When we added the requested_destination field for the RequestSpec object
    in Newton, we forgot to pass it to the legacy dictionary when wanting to
    use scheduler methods not yet supporting the NovaObject.
    As a consequence, when we were transforming the RequestSpec object into a
    tuple of (request_spec, filter_props) dicts and then rehydrating a new
    RequestSpec object using those dicts, the newly created object was not
    keeping that requested_destination field from the original.

    Conflicts:
          nova/tests/unit/objects/test_request_spec.py

    NOTE(mriedem): Conflict is due to d288132dca7cc76dfc6679eda17bb8fcc62577de
    not being in Ocata, but it's not needed either.

    Change-Id: Iba0b88172e9a3bfd4f216dd364d70f7e01c60ee2
    Closes-Bug: #1702454
    (cherry picked from commit 69bef428bd555bb31f43db6ca9c21db8aeb9007e)
    (cherry picked from commit bceb176fb00104fb0e39d6f1f3183f616c2b90f7)

Reviewed: https://review.openstack.org/516118
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=32094105285daa7c420189f73025dd24437843bb
Submitter: Zuul
Branch: stable/newton

commit 32094105285daa7c420189f73025dd24437843bb
Author: Sylvain Bauza <email address hidden>
Date: Wed Aug 2 19:28:13 2017 +0200

    Functional regression test for evacuate with a target

    In I6ddcaaca37fc5387c2d2e9f51c67ea9e85acb5c5 we forgot to update the
    legacy filter properties dictionary so the requested target wasn't
    passed to the scheduler when evacuating.
    Adding a functional test for verifying the behaviour.

    NOTE(sbauza): The issue has been incendentally fixed in Pike by
    I434af8e4ad991ac114dd67d66797a562d16bafe2 so the regression test just
    verifies that the expected behaviour works.
    That specific backport is consequently different from the master change
    as the final target is different from the one we expected.

    Related-Bug: #1702454

    NOTE(mriedem): There are a few changes needed in newton to make
    the test work, like there is no PlacementFixture, and some of the
    configs changed.

    Change-Id: Id9adb10d2ef821c8b61d8f1d5dc9dd66ec7aaac8
    (cherry picked from commit e0e2e065a495b4fa9ebdec987c935e3c83118c46)
    (cherry picked from commit b575a2645137d0e8f1f622ee45fca89605d2fdb9)
    (cherry picked from commit 2f9ce628a527c5cffb6cde5b9ef6490c0d4af60c)

tags: added: in-stable-newton

This issue was fixed in the openstack/nova 15.0.8 release.

This issue was fixed in the openstack/nova 16.0.3 release.

Reviewed: https://review.openstack.org/516119
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=6ce8bcab7cd73e27adfd26fceee7ec7c397bfa9b
Submitter: Zuul
Branch: stable/newton

commit 6ce8bcab7cd73e27adfd26fceee7ec7c397bfa9b
Author: Sylvain Bauza <email address hidden>
Date: Thu Jul 6 17:17:31 2017 +0200

    Pass requested_destination in filter_properties

    When we added the requested_destination field for the RequestSpec object
    in Newton, we forgot to pass it to the legacy dictionary when wanting to
    use scheduler methods not yet supporting the NovaObject.
    As a consequence, when we were transforming the RequestSpec object into a
    tuple of (request_spec, filter_props) dicts and then rehydrating a new
    RequestSpec object using those dicts, the newly created object was not
    keeping that requested_destination field from the original.

    Conflicts:
          nova/objects/request_spec.py

    NOTE(mriedem): The conflict is due to not having change
    b26ef568077f1bd9af8076c362bad06da3ec7e43 in Newton.

    Conflicts:
          nova/tests/unit/objects/test_request_spec.py

    NOTE(tonyb): Due to having Icb418f2be575bb2ba82756fdeb67b24a28950746

    Depends-On: I344d8fdded9b7d5385fcb41b699f1352acb4cda7

    Change-Id: Iba0b88172e9a3bfd4f216dd364d70f7e01c60ee2
    Closes-Bug: #1702454
    (cherry picked from commit 69bef428bd555bb31f43db6ca9c21db8aeb9007e)
    (cherry picked from commit bceb176fb00104fb0e39d6f1f3183f616c2b90f7)
    (cherry picked from commit 841121a95d76b10e6738156b55ee8ebc648054b4)

This issue was fixed in the openstack/nova 14.0.10 release.

This issue was fixed in the openstack/nova 17.0.0.0b2 development milestone.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers