Pagination of instances works incorrect

[Andriy Kurilin]

Listing of instances returns a limited number of resources. To list all instances, "marker" parameter can be used to list all the pages, i.e to obtain all the servers.

    # novaclient implements the same logic for `limit=-1`.
    results = []
    marker = None
    while True:
         servers = list_server(marker)
         if not servers:
             break

         results.extend(servers)
         marker = results[-1].id

For some reasons, it stopped working. Listing instances starting from X instance returns a list of resources with X instance in it.

Caught by the scenario when 2 VMs are created at almost the same time (<1sec between requests) in the same tenant by the same user, but with different names.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/510140

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/510203

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/510203
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=00bbc2ffd356ed17b11d432875c172141cd50b24
Submitter: Jenkins
Branch: master

commit 00bbc2ffd356ed17b11d432875c172141cd50b24
Author: Dan Smith <email address hidden>
Date: Fri Oct 6 10:33:33 2017 -0700

    Fix instance_get_by_sort_filters() for multiple sort keys

    This method would not actually work for any query where multiple sort
    keys were provided. Since it effectively ANDed all of the sort_key > val
    conditions in the query, any multi-key sort would exclude a lot of
    results.

    This fix actually replicates much of the logic from the base
    paginate_query() utility method, which properly handles multiple
    keys by creating key1>val1 OR (key1=val2 AND key2>=val2) WHERE
    clauses necessary for proper ordering.

    Change-Id: I3dac96759f7c7f11a0e0e9d86731dd4d22462d33
    Partial-Bug: #1721791

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/510140
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=add69c05072b4ee7efd4e75debf2172ed2269c86
Submitter: Jenkins
Branch: master

commit add69c05072b4ee7efd4e75debf2172ed2269c86
Author: Dan Smith <email address hidden>
Date: Fri Oct 6 07:40:42 2017 -0700

    Always put 'uuid' into sort_keys for stable instance lists

    If we're listing by sort keys that yield many ambiguous results, we
    may exacerbate issues in client pagination because we're not even
    bound by insertion order given that we have multiple databases being
    queried in parallel. So, even if the client didn't ask for it, throw
    'uuid' into the end of sort_keys to provide us a stable ordering. This
    was done for the default case by always including 'id' in the default
    set of sort_keys, although a user could still break if they request
    their own keys.

    Note this also removes the recently-added explicit sort in the
    test_bug_1689692 case, since we're enforcing a strict ordering with
    this patch. Also, mriedem is awesome.

    Change-Id: Ida446acb1286a8b215451a5d8d7d23882643ef13
    Closes-Bug: #1721791

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/nova 17.0.0.0b1 development milestone.