volume-backed instance rebuild with no image change is still going through scheduler

[Matt Riedemann]

Due to this recent change:

https://github.com/openstack/nova/commit/984dd8ad6add4523d93c7ce5a666a32233e02e34

And the fact that volume-backed instances don't store the instance.image_ref value to indicate they are volume-backed, this condition will always evaluate to True:

https://github.com/openstack/nova/blob/984dd8ad6add4523d93c7ce5a666a32233e02e34/nova/compute/api.py#L2959

And we'll unnecessarily call through the scheduler again to validate the instance on the original host during the rebuild even if the image isn't changing.

For a volume-backed instance, we have to get the original image_id from the volume's volume_image_metadata field:

https://review.openstack.org/#/c/520686/

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/521200

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/521391

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/521200
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=a4eebd5de7ef4b63082536718ba7b993a66d47e7
Submitter: Zuul
Branch: master

commit a4eebd5de7ef4b63082536718ba7b993a66d47e7
Author: Matt Riedemann <email address hidden>
Date: Fri Nov 17 16:53:39 2017 -0500

    Add regression test for rebuilding a volume-backed server

    Commit 984dd8ad6add4523d93c7ce5a666a32233e02e34 makes rebuild
    check to see if the user is rebuilding an instance with a new
    image and if so, to run the scheduler filters again since the
    new image might not work with the current host for the instance,
    and we rebuild to the same host that the instance is already
    running on.

    The problem is the instance.image_ref attribute is not set for
    a volume-backed (boot-from-volume) instance, so the conditional
    in the rebuild() method is always True, which means we always run
    through the scheduler for volume-backed instances during rebuild,
    even if the image in the root disk isn't changing.

    This adds a functional regression test to recreate the bug.

    Change-Id: If79c554b46c44a7f70f8367426e7da362d7234c8
    Related-Bug: #1732947

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/521391
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=54407afef3b446e34c44ba5392ccbf121e266de2
Submitter: Zuul
Branch: master

commit 54407afef3b446e34c44ba5392ccbf121e266de2
Author: Matt Riedemann <email address hidden>
Date: Sun Nov 19 17:25:56 2017 -0500

    Get original image_id from volume for volume-backed instance rebuild

    A volume-backed instance will not have the instance.image_ref
    attribute set, to indicate to the API user that it is a volume-backed
    instance.

    Commit 984dd8ad6add4523d93c7ce5a666a32233e02e34 missed this subtle
    difference with how instance.image_ref is used, which means a rebuild
    of any volume-backed instance will now run through the scheduler, even
    if the image_href passed to rebuild is the same image ID as for the
    root volume.

    This fixes that case in rebuild by getting the image metadata off the
    root volume for a volume-backed instance and compares that to the
    image_href passed to rebuild.

    Change-Id: I48cda813b9effa37f6c3e0cd2e8a22bb78c79d72
    Closes-Bug: #1732947

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/pike
Review: https://review.openstack.org/525632

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/pike
Review: https://review.openstack.org/525634

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/ocata
Review: https://review.openstack.org/525664

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/ocata
Review: https://review.openstack.org/525666

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/newton
Review: https://review.openstack.org/526542

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/526544

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/nova 17.0.0.0b2 development milestone.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/525632
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=c7991653a4e7cbb86656e6ba7579a86236801567
Submitter: Zuul
Branch: stable/pike

commit c7991653a4e7cbb86656e6ba7579a86236801567
Author: Matt Riedemann <email address hidden>
Date: Fri Nov 17 16:53:39 2017 -0500

    Add regression test for rebuilding a volume-backed server

    Commit 984dd8ad6add4523d93c7ce5a666a32233e02e34 makes rebuild
    check to see if the user is rebuilding an instance with a new
    image and if so, to run the scheduler filters again since the
    new image might not work with the current host for the instance,
    and we rebuild to the same host that the instance is already
    running on.

    The problem is the instance.image_ref attribute is not set for
    a volume-backed (boot-from-volume) instance, so the conditional
    in the rebuild() method is always True, which means we always run
    through the scheduler for volume-backed instances during rebuild,
    even if the image in the root disk isn't changing.

    This adds a functional regression test to recreate the bug.

    Change-Id: If79c554b46c44a7f70f8367426e7da362d7234c8
    Related-Bug: #1732947
    (cherry picked from commit a4eebd5de7ef4b63082536718ba7b993a66d47e7)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/525634
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=4e36c4bd0eab2be619991ae00454906e171720b6
Submitter: Zuul
Branch: stable/pike

commit 4e36c4bd0eab2be619991ae00454906e171720b6
Author: Matt Riedemann <email address hidden>
Date: Sun Nov 19 17:25:56 2017 -0500

    Get original image_id from volume for volume-backed instance rebuild

    A volume-backed instance will not have the instance.image_ref
    attribute set, to indicate to the API user that it is a volume-backed
    instance.

    Commit 984dd8ad6add4523d93c7ce5a666a32233e02e34 missed this subtle
    difference with how instance.image_ref is used, which means a rebuild
    of any volume-backed instance will now run through the scheduler, even
    if the image_href passed to rebuild is the same image ID as for the
    root volume.

    This fixes that case in rebuild by getting the image metadata off the
    root volume for a volume-backed instance and compares that to the
    image_href passed to rebuild.

    Change-Id: I48cda813b9effa37f6c3e0cd2e8a22bb78c79d72
    Closes-Bug: #1732947
    (cherry picked from commit 54407afef3b446e34c44ba5392ccbf121e266de2)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/525664
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=fe5e3e7e61d879f7a28eb177e4bdbc7ead3ae4ac
Submitter: Zuul
Branch: stable/ocata

commit fe5e3e7e61d879f7a28eb177e4bdbc7ead3ae4ac
Author: Matt Riedemann <email address hidden>
Date: Fri Nov 17 16:53:39 2017 -0500

    Add regression test for rebuilding a volume-backed server

    Commit 984dd8ad6add4523d93c7ce5a666a32233e02e34 makes rebuild
    check to see if the user is rebuilding an instance with a new
    image and if so, to run the scheduler filters again since the
    new image might not work with the current host for the instance,
    and we rebuild to the same host that the instance is already
    running on.

    The problem is the instance.image_ref attribute is not set for
    a volume-backed (boot-from-volume) instance, so the conditional
    in the rebuild() method is always True, which means we always run
    through the scheduler for volume-backed instances during rebuild,
    even if the image in the root disk isn't changing.

    This adds a functional regression test to recreate the bug.

    Conflicts:
          nova/tests/fixtures.py

    NOTE(mriedem): The CinderFixture conflict is due to not having
    1928ec5606491de95abba8791b1d7b888690dd3c in Ocata. However, this
    change needs the part of that patch to the CinderFixture to work,
    so it's added here.

    NOTE(mriedem): The functional test needed to change slightly to
    use the PlacementFixture and disable the DiskFilter.

    Change-Id: If79c554b46c44a7f70f8367426e7da362d7234c8
    Related-Bug: #1732947
    (cherry picked from commit a4eebd5de7ef4b63082536718ba7b993a66d47e7)
    (cherry picked from commit c7991653a4e7cbb86656e6ba7579a86236801567)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/525666
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=10bc4fdbc568670dfd8bec109e8b854ba8ad7949
Submitter: Zuul
Branch: stable/ocata

commit 10bc4fdbc568670dfd8bec109e8b854ba8ad7949
Author: Matt Riedemann <email address hidden>
Date: Sun Nov 19 17:25:56 2017 -0500

    Get original image_id from volume for volume-backed instance rebuild

    A volume-backed instance will not have the instance.image_ref
    attribute set, to indicate to the API user that it is a volume-backed
    instance.

    Commit 984dd8ad6add4523d93c7ce5a666a32233e02e34 missed this subtle
    difference with how instance.image_ref is used, which means a rebuild
    of any volume-backed instance will now run through the scheduler, even
    if the image_href passed to rebuild is the same image ID as for the
    root volume.

    This fixes that case in rebuild by getting the image metadata off the
    root volume for a volume-backed instance and compares that to the
    image_href passed to rebuild.

    Change-Id: I48cda813b9effa37f6c3e0cd2e8a22bb78c79d72
    Closes-Bug: #1732947
    (cherry picked from commit 54407afef3b446e34c44ba5392ccbf121e266de2)
    (cherry picked from commit 4e36c4bd0eab2be619991ae00454906e171720b6)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/526542
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=5ec5886fe74514c65908ff3adc2ca46610b595ad
Submitter: Zuul
Branch: stable/newton

commit 5ec5886fe74514c65908ff3adc2ca46610b595ad
Author: Matt Riedemann <email address hidden>
Date: Fri Nov 17 16:53:39 2017 -0500

    Add regression test for rebuilding a volume-backed server

    Commit 984dd8ad6add4523d93c7ce5a666a32233e02e34 makes rebuild
    check to see if the user is rebuilding an instance with a new
    image and if so, to run the scheduler filters again since the
    new image might not work with the current host for the instance,
    and we rebuild to the same host that the instance is already
    running on.

    The problem is the instance.image_ref attribute is not set for
    a volume-backed (boot-from-volume) instance, so the conditional
    in the rebuild() method is always True, which means we always run
    through the scheduler for volume-backed instances during rebuild,
    even if the image in the root disk isn't changing.

    This adds a functional regression test to recreate the bug.

    Conflicts:
          nova/tests/fixtures.py

    NOTE(mriedem): The fixture conflict is due to not having the
    CinderFixture in Newton. That was added in Ocata via commit
    47fb8b757994688a995ec3bdfa5dccc72f870f34. It's added here in
    it's entirety since that is simpler than trying to basically
    re-write it from scratch in a backport.

    NOTE(mriedem): The functional test is slightly modified to
    not rely on Placement and because of old config option names.

    Change-Id: If79c554b46c44a7f70f8367426e7da362d7234c8
    Related-Bug: #1732947
    (cherry picked from commit a4eebd5de7ef4b63082536718ba7b993a66d47e7)
    (cherry picked from commit c7991653a4e7cbb86656e6ba7579a86236801567)
    (cherry picked from commit fe5e3e7e61d879f7a28eb177e4bdbc7ead3ae4ac)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/526544
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=24953a95e5989f44acaf488c253230e476c2fa59
Submitter: Zuul
Branch: stable/newton

commit 24953a95e5989f44acaf488c253230e476c2fa59
Author: Matt Riedemann <email address hidden>
Date: Sun Nov 19 17:25:56 2017 -0500

    Get original image_id from volume for volume-backed instance rebuild

    A volume-backed instance will not have the instance.image_ref
    attribute set, to indicate to the API user that it is a volume-backed
    instance.

    Commit 984dd8ad6add4523d93c7ce5a666a32233e02e34 missed this subtle
    difference with how instance.image_ref is used, which means a rebuild
    of any volume-backed instance will now run through the scheduler, even
    if the image_href passed to rebuild is the same image ID as for the
    root volume.

    This fixes that case in rebuild by getting the image metadata off the
    root volume for a volume-backed instance and compares that to the
    image_href passed to rebuild.

    Change-Id: I48cda813b9effa37f6c3e0cd2e8a22bb78c79d72
    Closes-Bug: #1732947
    (cherry picked from commit 54407afef3b446e34c44ba5392ccbf121e266de2)
    (cherry picked from commit 4e36c4bd0eab2be619991ae00454906e171720b6)
    (cherry picked from commit 10bc4fdbc568670dfd8bec109e8b854ba8ad7949)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/nova 16.0.4 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/nova 15.1.0 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/nova 14.1.0 release.