openstack server list --deleted --limit -1 hangs

[Keith Berger]

OpenStack Rocky:

When running `openstack server list --deleted --limit -1 hangs` it will hang and not return.

A debug output is found in this pastebin

http://paste.openstack.org/show/785497/

and a direct curl is shown here

http://paste.openstack.org/show/785488/

It seems to be related to the marker as discussed here.

http://eavesdrop.openstack.org/irclogs/%23openstack-nova/%23openstack-nova.2019-10-22.log.html#t2019-10-22T20:35:11

after the initial collection of 18 servers, for some reason it than tried to grab with

https://192.168.23.35:8774/v2.1/428982d4248a419a933668b6a4dd14a0/servers/dtail?deleted=True&marker=8aab9854-af8e-4e98-840a-192f15ae01f9

where the marker is the 18th deleted server in the list. it then loops on this until you breakout.

[Matt Riedemann (mriedem)]

I've got the fix started here: https://review.opendev.org/#/c/690429/

[Matt Riedemann (mriedem)]

The regression was introduced with https://review.opendev.org/#/q/I1aa3ca6cc70cef65d24dec1e7db9491c9b73f7ab which started in Queens and was backported to Newton. We can only fix upstream going back to Ocata though since the newton branch is end of life upstream.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/690428
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=45c2752f2ce08b314012eff044b01aa7d626b43d
Submitter: Zuul
Branch: master

commit 45c2752f2ce08b314012eff044b01aa7d626b43d
Author: Matt Riedemann <email address hidden>
Date: Tue Oct 22 17:53:20 2019 -0400

    Add functional regression test for bug 1849409

    Change I1aa3ca6cc70cef65d24dec1e7db9491c9b73f7ab in Queens,
    which was backported through to Newton, introduced a regression
    when listing deleted servers with a marker because it assumes
    that if BuildRequestList.get_by_filters does not raise
    MarkerNotFound that the marker was found among the build requests
    and does not account for that get_by_filters method short-circuiting
    if filtering servers with deleted/cleaned/limit=0. The API code
    then nulls out the marker which means you'll continue to get the
    marker instance back in the results even though you shouldn't,
    and that can cause an infinite loop in some client-side tooling like
    nova's CLI:

      nova list --deleted --limit -1

    This adds a functional recreate test for the regression which will
    be updated when the bug is fixed.

    Change-Id: I324193129acb6ac739133c7e76920762a8987a84
    Related-Bug: #1849409

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/train
Review: https://review.opendev.org/690710

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/train
Review: https://review.opendev.org/690711

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/stein
Review: https://review.opendev.org/690715

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/690716

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/rocky
Review: https://review.opendev.org/690720

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/rocky
Review: https://review.opendev.org/690721

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/queens
Review: https://review.opendev.org/690724

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/queens
Review: https://review.opendev.org/690725

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/pike
Review: https://review.opendev.org/690734

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/pike
Review: https://review.opendev.org/690735

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/ocata
Review: https://review.opendev.org/690746

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/ocata
Review: https://review.opendev.org/690747

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/690429
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=df03499843aa7fd6089bd4d07b9d0eb5a8c14b47
Submitter: Zuul
Branch: master

commit df03499843aa7fd6089bd4d07b9d0eb5a8c14b47
Author: Matt Riedemann <email address hidden>
Date: Tue Oct 22 17:59:32 2019 -0400

    Fix listing deleted servers with a marker

    Change I1aa3ca6cc70cef65d24dec1e7db9491c9b73f7ab in Queens,
    which was backported through to Newton, introduced a regression
    when listing deleted servers with a marker because it assumes
    that if BuildRequestList.get_by_filters does not raise
    MarkerNotFound that the marker was found among the build requests
    and does not account for that get_by_filters method short-circuiting
    if filtering servers with deleted/cleaned/limit=0. The API code
    then nulls out the marker which means you'll continue to get the
    marker instance back in the results even though you shouldn't,
    and that can cause an infinite loop in some client-side tooling like
    nova's CLI:

      nova list --deleted --limit -1

    This fixes the bug by raising MarkerNotFound from
    BuildRequestList.get_by_filters if we have a marker but are
    short-circuiting and returning early from the method based on
    limit or filters.

    Change-Id: Ic2b19c2aa06b3059ab0344b6ac56ffd62b3f755d
    Closes-Bug: #1849409

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/690710
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=727d942b2830fb6970d99507f2b5eb1a28df01b2
Submitter: Zuul
Branch: stable/train

commit 727d942b2830fb6970d99507f2b5eb1a28df01b2
Author: Matt Riedemann <email address hidden>
Date: Tue Oct 22 17:53:20 2019 -0400

    Add functional regression test for bug 1849409

    Change I1aa3ca6cc70cef65d24dec1e7db9491c9b73f7ab in Queens,
    which was backported through to Newton, introduced a regression
    when listing deleted servers with a marker because it assumes
    that if BuildRequestList.get_by_filters does not raise
    MarkerNotFound that the marker was found among the build requests
    and does not account for that get_by_filters method short-circuiting
    if filtering servers with deleted/cleaned/limit=0. The API code
    then nulls out the marker which means you'll continue to get the
    marker instance back in the results even though you shouldn't,
    and that can cause an infinite loop in some client-side tooling like
    nova's CLI:

      nova list --deleted --limit -1

    This adds a functional recreate test for the regression which will
    be updated when the bug is fixed.

    Change-Id: I324193129acb6ac739133c7e76920762a8987a84
    Related-Bug: #1849409
    (cherry picked from commit 45c2752f2ce08b314012eff044b01aa7d626b43d)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/690711
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=03a2508362ecf50463d0659142312a30d0fb91f3
Submitter: Zuul
Branch: stable/train

commit 03a2508362ecf50463d0659142312a30d0fb91f3
Author: Matt Riedemann <email address hidden>
Date: Tue Oct 22 17:59:32 2019 -0400

    Fix listing deleted servers with a marker

    Change I1aa3ca6cc70cef65d24dec1e7db9491c9b73f7ab in Queens,
    which was backported through to Newton, introduced a regression
    when listing deleted servers with a marker because it assumes
    that if BuildRequestList.get_by_filters does not raise
    MarkerNotFound that the marker was found among the build requests
    and does not account for that get_by_filters method short-circuiting
    if filtering servers with deleted/cleaned/limit=0. The API code
    then nulls out the marker which means you'll continue to get the
    marker instance back in the results even though you shouldn't,
    and that can cause an infinite loop in some client-side tooling like
    nova's CLI:

      nova list --deleted --limit -1

    This fixes the bug by raising MarkerNotFound from
    BuildRequestList.get_by_filters if we have a marker but are
    short-circuiting and returning early from the method based on
    limit or filters.

    Change-Id: Ic2b19c2aa06b3059ab0344b6ac56ffd62b3f755d
    Closes-Bug: #1849409
    (cherry picked from commit df03499843aa7fd6089bd4d07b9d0eb5a8c14b47)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/690715
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=47caaccd4a03660d7df144f2eadd821d36baeaa8
Submitter: Zuul
Branch: stable/stein

commit 47caaccd4a03660d7df144f2eadd821d36baeaa8
Author: Matt Riedemann <email address hidden>
Date: Tue Oct 22 17:53:20 2019 -0400

    Add functional regression test for bug 1849409

    Change I1aa3ca6cc70cef65d24dec1e7db9491c9b73f7ab in Queens,
    which was backported through to Newton, introduced a regression
    when listing deleted servers with a marker because it assumes
    that if BuildRequestList.get_by_filters does not raise
    MarkerNotFound that the marker was found among the build requests
    and does not account for that get_by_filters method short-circuiting
    if filtering servers with deleted/cleaned/limit=0. The API code
    then nulls out the marker which means you'll continue to get the
    marker instance back in the results even though you shouldn't,
    and that can cause an infinite loop in some client-side tooling like
    nova's CLI:

      nova list --deleted --limit -1

    This adds a functional recreate test for the regression which will
    be updated when the bug is fixed.

    Change-Id: I324193129acb6ac739133c7e76920762a8987a84
    Related-Bug: #1849409
    (cherry picked from commit 45c2752f2ce08b314012eff044b01aa7d626b43d)
    (cherry picked from commit 727d942b2830fb6970d99507f2b5eb1a28df01b2)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/690716
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=6038455e1dafeaf8ccfdd4be9e993beea3c9fb45
Submitter: Zuul
Branch: stable/stein

commit 6038455e1dafeaf8ccfdd4be9e993beea3c9fb45
Author: Matt Riedemann <email address hidden>
Date: Tue Oct 22 17:59:32 2019 -0400

    Fix listing deleted servers with a marker

    Change I1aa3ca6cc70cef65d24dec1e7db9491c9b73f7ab in Queens,
    which was backported through to Newton, introduced a regression
    when listing deleted servers with a marker because it assumes
    that if BuildRequestList.get_by_filters does not raise
    MarkerNotFound that the marker was found among the build requests
    and does not account for that get_by_filters method short-circuiting
    if filtering servers with deleted/cleaned/limit=0. The API code
    then nulls out the marker which means you'll continue to get the
    marker instance back in the results even though you shouldn't,
    and that can cause an infinite loop in some client-side tooling like
    nova's CLI:

      nova list --deleted --limit -1

    This fixes the bug by raising MarkerNotFound from
    BuildRequestList.get_by_filters if we have a marker but are
    short-circuiting and returning early from the method based on
    limit or filters.

    Change-Id: Ic2b19c2aa06b3059ab0344b6ac56ffd62b3f755d
    Closes-Bug: #1849409
    (cherry picked from commit df03499843aa7fd6089bd4d07b9d0eb5a8c14b47)
    (cherry picked from commit 03a2508362ecf50463d0659142312a30d0fb91f3)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/nova 20.0.1 release.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/690720
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=08337cccb060d0b3cad388004c1f802d5d053813
Submitter: Zuul
Branch: stable/rocky

commit 08337cccb060d0b3cad388004c1f802d5d053813
Author: Matt Riedemann <email address hidden>
Date: Tue Oct 22 17:53:20 2019 -0400

    Add functional regression test for bug 1849409

    Change I1aa3ca6cc70cef65d24dec1e7db9491c9b73f7ab in Queens,
    which was backported through to Newton, introduced a regression
    when listing deleted servers with a marker because it assumes
    that if BuildRequestList.get_by_filters does not raise
    MarkerNotFound that the marker was found among the build requests
    and does not account for that get_by_filters method short-circuiting
    if filtering servers with deleted/cleaned/limit=0. The API code
    then nulls out the marker which means you'll continue to get the
    marker instance back in the results even though you shouldn't,
    and that can cause an infinite loop in some client-side tooling like
    nova's CLI:

      nova list --deleted --limit -1

    This adds a functional recreate test for the regression which will
    be updated when the bug is fixed.

    NOTE(mriedem): In this backport we get the PlacementFixture from
    the usual nova_fixtures import since change
    Idaed39629095f86d24a54334c699a26c218c6593 is not in Rocky. Also,
    test_concurrent_request_server_group_members_over_quota leaks deleted
    servers from cell0 for some reason in Rocky which can make this test
    intermittently fail. This is not a problem after Rocky for some reason.
    To avoid intermittent failures due to that leak, the test is modified
    to filter on the name of the server created in the test only.

    Change-Id: I324193129acb6ac739133c7e76920762a8987a84
    Related-Bug: #1849409
    (cherry picked from commit 45c2752f2ce08b314012eff044b01aa7d626b43d)
    (cherry picked from commit 727d942b2830fb6970d99507f2b5eb1a28df01b2)
    (cherry picked from commit 47caaccd4a03660d7df144f2eadd821d36baeaa8)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/690721
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=c84846dbf93588e5420b3565d163c0393443e6d6
Submitter: Zuul
Branch: stable/rocky

commit c84846dbf93588e5420b3565d163c0393443e6d6
Author: Matt Riedemann <email address hidden>
Date: Tue Oct 22 17:59:32 2019 -0400

    Fix listing deleted servers with a marker

    Change I1aa3ca6cc70cef65d24dec1e7db9491c9b73f7ab in Queens,
    which was backported through to Newton, introduced a regression
    when listing deleted servers with a marker because it assumes
    that if BuildRequestList.get_by_filters does not raise
    MarkerNotFound that the marker was found among the build requests
    and does not account for that get_by_filters method short-circuiting
    if filtering servers with deleted/cleaned/limit=0. The API code
    then nulls out the marker which means you'll continue to get the
    marker instance back in the results even though you shouldn't,
    and that can cause an infinite loop in some client-side tooling like
    nova's CLI:

      nova list --deleted --limit -1

    This fixes the bug by raising MarkerNotFound from
    BuildRequestList.get_by_filters if we have a marker but are
    short-circuiting and returning early from the method based on
    limit or filters.

    Conflicts:

      nova/tests/functional/regressions/test_bug_1849409.py

    NOTE(mriedem): The conflict is due to the Rocky-specific change
    to the functional test in I324193129acb6ac739133c7e76920762a8987a84
    due to test_concurrent_request_server_group_members_over_quota leaking
    deleted servers for some reason.

    Change-Id: Ic2b19c2aa06b3059ab0344b6ac56ffd62b3f755d
    Closes-Bug: #1849409
    (cherry picked from commit df03499843aa7fd6089bd4d07b9d0eb5a8c14b47)
    (cherry picked from commit 03a2508362ecf50463d0659142312a30d0fb91f3)
    (cherry picked from commit 6038455e1dafeaf8ccfdd4be9e993beea3c9fb45)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/690724
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=f03f5075e3751ccd03a60999eab1d7f4bf7c4f02
Submitter: Zuul
Branch: stable/queens

commit f03f5075e3751ccd03a60999eab1d7f4bf7c4f02
Author: Matt Riedemann <email address hidden>
Date: Tue Oct 22 17:53:20 2019 -0400

    Add functional regression test for bug 1849409

    Change I1aa3ca6cc70cef65d24dec1e7db9491c9b73f7ab in Queens,
    which was backported through to Newton, introduced a regression
    when listing deleted servers with a marker because it assumes
    that if BuildRequestList.get_by_filters does not raise
    MarkerNotFound that the marker was found among the build requests
    and does not account for that get_by_filters method short-circuiting
    if filtering servers with deleted/cleaned/limit=0. The API code
    then nulls out the marker which means you'll continue to get the
    marker instance back in the results even though you shouldn't,
    and that can cause an infinite loop in some client-side tooling like
    nova's CLI:

      nova list --deleted --limit -1

    This adds a functional recreate test for the regression which will
    be updated when the bug is fixed.

    NOTE(mriedem): In this backport the test is modified to create the
    server without networking to avoid a NetworkAmbiguous error
    which is probably due to some missing NeutronFixture sugar added
    in Rocky.

    Change-Id: I324193129acb6ac739133c7e76920762a8987a84
    Related-Bug: #1849409
    (cherry picked from commit 45c2752f2ce08b314012eff044b01aa7d626b43d)
    (cherry picked from commit 727d942b2830fb6970d99507f2b5eb1a28df01b2)
    (cherry picked from commit 47caaccd4a03660d7df144f2eadd821d36baeaa8)
    (cherry picked from commit 08337cccb060d0b3cad388004c1f802d5d053813)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/690725
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=8c18293f93a0ef957010198ae3ef2f336364b97f
Submitter: Zuul
Branch: stable/queens

commit 8c18293f93a0ef957010198ae3ef2f336364b97f
Author: Matt Riedemann <email address hidden>
Date: Tue Oct 22 17:59:32 2019 -0400

    Fix listing deleted servers with a marker

    Change I1aa3ca6cc70cef65d24dec1e7db9491c9b73f7ab in Queens,
    which was backported through to Newton, introduced a regression
    when listing deleted servers with a marker because it assumes
    that if BuildRequestList.get_by_filters does not raise
    MarkerNotFound that the marker was found among the build requests
    and does not account for that get_by_filters method short-circuiting
    if filtering servers with deleted/cleaned/limit=0. The API code
    then nulls out the marker which means you'll continue to get the
    marker instance back in the results even though you shouldn't,
    and that can cause an infinite loop in some client-side tooling like
    nova's CLI:

      nova list --deleted --limit -1

    This fixes the bug by raising MarkerNotFound from
    BuildRequestList.get_by_filters if we have a marker but are
    short-circuiting and returning early from the method based on
    limit or filters.

    Change-Id: Ic2b19c2aa06b3059ab0344b6ac56ffd62b3f755d
    Closes-Bug: #1849409
    (cherry picked from commit df03499843aa7fd6089bd4d07b9d0eb5a8c14b47)
    (cherry picked from commit 03a2508362ecf50463d0659142312a30d0fb91f3)
    (cherry picked from commit 6038455e1dafeaf8ccfdd4be9e993beea3c9fb45)
    (cherry picked from commit c84846dbf93588e5420b3565d163c0393443e6d6)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/690734
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=e2ba87cb7b1cfd4226cfd8028436ecf87ad8a2be
Submitter: Zuul
Branch: stable/pike

commit e2ba87cb7b1cfd4226cfd8028436ecf87ad8a2be
Author: Matt Riedemann <email address hidden>
Date: Tue Oct 22 17:53:20 2019 -0400

    Add functional regression test for bug 1849409

    Change I1aa3ca6cc70cef65d24dec1e7db9491c9b73f7ab in Queens,
    which was backported through to Newton, introduced a regression
    when listing deleted servers with a marker because it assumes
    that if BuildRequestList.get_by_filters does not raise
    MarkerNotFound that the marker was found among the build requests
    and does not account for that get_by_filters method short-circuiting
    if filtering servers with deleted/cleaned/limit=0. The API code
    then nulls out the marker which means you'll continue to get the
    marker instance back in the results even though you shouldn't,
    and that can cause an infinite loop in some client-side tooling like
    nova's CLI:

      nova list --deleted --limit -1

    This adds a functional recreate test for the regression which will
    be updated when the bug is fixed.

    Change-Id: I324193129acb6ac739133c7e76920762a8987a84
    Related-Bug: #1849409
    (cherry picked from commit 45c2752f2ce08b314012eff044b01aa7d626b43d)
    (cherry picked from commit 727d942b2830fb6970d99507f2b5eb1a28df01b2)
    (cherry picked from commit 47caaccd4a03660d7df144f2eadd821d36baeaa8)
    (cherry picked from commit 08337cccb060d0b3cad388004c1f802d5d053813)
    (cherry picked from commit f03f5075e3751ccd03a60999eab1d7f4bf7c4f02)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/690735
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=b277ec154f2a175c4cab609b89fd8d3c7de4d0e9
Submitter: Zuul
Branch: stable/pike

commit b277ec154f2a175c4cab609b89fd8d3c7de4d0e9
Author: Matt Riedemann <email address hidden>
Date: Tue Oct 22 17:59:32 2019 -0400

    Fix listing deleted servers with a marker

    Change I1aa3ca6cc70cef65d24dec1e7db9491c9b73f7ab in Queens,
    which was backported through to Newton, introduced a regression
    when listing deleted servers with a marker because it assumes
    that if BuildRequestList.get_by_filters does not raise
    MarkerNotFound that the marker was found among the build requests
    and does not account for that get_by_filters method short-circuiting
    if filtering servers with deleted/cleaned/limit=0. The API code
    then nulls out the marker which means you'll continue to get the
    marker instance back in the results even though you shouldn't,
    and that can cause an infinite loop in some client-side tooling like
    nova's CLI:

      nova list --deleted --limit -1

    This fixes the bug by raising MarkerNotFound from
    BuildRequestList.get_by_filters if we have a marker but are
    short-circuiting and returning early from the method based on
    limit or filters.

    Change-Id: Ic2b19c2aa06b3059ab0344b6ac56ffd62b3f755d
    Closes-Bug: #1849409
    (cherry picked from commit df03499843aa7fd6089bd4d07b9d0eb5a8c14b47)
    (cherry picked from commit 03a2508362ecf50463d0659142312a30d0fb91f3)
    (cherry picked from commit 6038455e1dafeaf8ccfdd4be9e993beea3c9fb45)
    (cherry picked from commit c84846dbf93588e5420b3565d163c0393443e6d6)
    (cherry picked from commit 8c18293f93a0ef957010198ae3ef2f336364b97f)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/nova 19.1.0 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/nova 18.3.0 release.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/690746
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=b6c7456face630086a0eb7fd1c8335d42ab0456a
Submitter: Zuul
Branch: stable/ocata

commit b6c7456face630086a0eb7fd1c8335d42ab0456a
Author: Matt Riedemann <email address hidden>
Date: Tue Oct 22 17:53:20 2019 -0400

    Add functional regression test for bug 1849409

    Change I1aa3ca6cc70cef65d24dec1e7db9491c9b73f7ab in Queens,
    which was backported through to Newton, introduced a regression
    when listing deleted servers with a marker because it assumes
    that if BuildRequestList.get_by_filters does not raise
    MarkerNotFound that the marker was found among the build requests
    and does not account for that get_by_filters method short-circuiting
    if filtering servers with deleted/cleaned/limit=0. The API code
    then nulls out the marker which means you'll continue to get the
    marker instance back in the results even though you shouldn't,
    and that can cause an infinite loop in some client-side tooling like
    nova's CLI:

      nova list --deleted --limit -1

    This adds a functional recreate test for the regression which will
    be updated when the bug is fixed.

    NOTE(mriedem): In this backport the test is modified to disable the
    DiskFilter since we're using Placement for filtering on DISK_GB.
    Also, _wait_until_deleted is moved to InstanceHelperMixin since
    If7b02bcd8d77e94c7fb42b721792c1391bc0e3b7 is not in Ocata.

    Change-Id: I324193129acb6ac739133c7e76920762a8987a84
    Related-Bug: #1849409
    (cherry picked from commit 45c2752f2ce08b314012eff044b01aa7d626b43d)
    (cherry picked from commit 727d942b2830fb6970d99507f2b5eb1a28df01b2)
    (cherry picked from commit 47caaccd4a03660d7df144f2eadd821d36baeaa8)
    (cherry picked from commit 08337cccb060d0b3cad388004c1f802d5d053813)
    (cherry picked from commit f03f5075e3751ccd03a60999eab1d7f4bf7c4f02)
    (cherry picked from commit e2ba87cb7b1cfd4226cfd8028436ecf87ad8a2be)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/690747
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=b91592595141d7766c00255ffff04a25ca81c834
Submitter: Zuul
Branch: stable/ocata

commit b91592595141d7766c00255ffff04a25ca81c834
Author: Matt Riedemann <email address hidden>
Date: Tue Oct 22 17:59:32 2019 -0400

    Fix listing deleted servers with a marker

    Change I1aa3ca6cc70cef65d24dec1e7db9491c9b73f7ab in Queens,
    which was backported through to Newton, introduced a regression
    when listing deleted servers with a marker because it assumes
    that if BuildRequestList.get_by_filters does not raise
    MarkerNotFound that the marker was found among the build requests
    and does not account for that get_by_filters method short-circuiting
    if filtering servers with deleted/cleaned/limit=0. The API code
    then nulls out the marker which means you'll continue to get the
    marker instance back in the results even though you shouldn't,
    and that can cause an infinite loop in some client-side tooling like
    nova's CLI:

      nova list --deleted --limit -1

    This fixes the bug by raising MarkerNotFound from
    BuildRequestList.get_by_filters if we have a marker but are
    short-circuiting and returning early from the method based on
    limit or filters.

    Change-Id: Ic2b19c2aa06b3059ab0344b6ac56ffd62b3f755d
    Closes-Bug: #1849409
    (cherry picked from commit df03499843aa7fd6089bd4d07b9d0eb5a8c14b47)
    (cherry picked from commit 03a2508362ecf50463d0659142312a30d0fb91f3)
    (cherry picked from commit 6038455e1dafeaf8ccfdd4be9e993beea3c9fb45)
    (cherry picked from commit c84846dbf93588e5420b3565d163c0393443e6d6)
    (cherry picked from commit 8c18293f93a0ef957010198ae3ef2f336364b97f)
    (cherry picked from commit b277ec154f2a175c4cab609b89fd8d3c7de4d0e9)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/nova ocata-eol release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/nova pike-eol release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/nova queens-eol release.