Get servers REST reply does not have marker when default limit is reached

[Steven Kaufer]

Both the /servers and /servers/details APIs support pagination. When the request includes the "limit" parameters, then a "next" link is included in the reply if the number of servers that match the query is greater than or equal to the limit.

The problem occurs when the caller does not include the limit parameter but the total number of servers is greater than or equal to the default "CONF.osapi_max_limit". When this occurs, the number of servers in the reply is "osapi_max" but there is no "next" link. Therefore, the caller cannot determine if there are any more servers and has no marker value such that they can retrieve the rest of the servers.

The fix for this is to include the "next" link when the total number of servers is greater than or equal to the default limit, even if the "limit" parameter is not supplied.

The documentation also says that the "next" link is required: http://docs.openstack.org/api/openstack-compute/2/content/Paginated_Collections-d1e664.html

The fix appears to be in the _get_collection_links function in nova/api/openstack/common.py. The logic needs to be updated so that the "next"
link is included if the total number of items returned equals the minimum of either the "limit" paramater or the "CONF.osapi_max_limit" value.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/78800

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/78800
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=968c01afe7ff43346f5817e2a3adf97fd03106b1
Submitter: Jenkins
Branch: master

commit 968c01afe7ff43346f5817e2a3adf97fd03106b1
Author: Steven Kaufer <email address hidden>
Date: Thu Mar 6 22:35:48 2014 +0000

    Include next link when default limit is reached

    The /servers and /servers/detail APIs support pagination and a
    "next" link should be included when more data is available, see
    http://docs.openstack.org/api/openstack-compute/2/content/
    Paginated_Collections-d1e664.html. In the documentation it states
    that 'collections are required to contain atom "next" links'.

    When the default "osapi_max" limit is reached then the "next" link
    is not included in the API reply. In this case, the caller cannot
    determine if there are any more servers and has no marker value
    such that they can retrieve the rest of the servers.

    For example, assume that there are 1050 VMs and the osapi_max value
    is the default of 1000. The /servers API will return the first 1000
    VMs but it will not include the "next" link; the caller has no way
    to determine if there are more then 1000 VMs and no way to retrieve
    the other 50 VMs.

    The fix for this is to include the "next" link when the number of
    servers being returned is the maximum limit, even if the "limit"
    parameter is not supplied. Note: the caller could workaround this
    by specifying the exact "osapi_max" limit value (ie, limit=1000)
    on the API call but this requires that the caller knows the
    configured limit.

    Change-Id: I9b3da852b0ab11dc980524e2317ee876a3ec28b8
    Closes-bug: 1288466

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/icehouse
Review: https://review.openstack.org/114487

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/114487
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=e5e6bc77e8fff614d5dff3a4a6fc4f2ddc18d215
Submitter: Jenkins
Branch: stable/icehouse

commit e5e6bc77e8fff614d5dff3a4a6fc4f2ddc18d215
Author: Steven Kaufer <email address hidden>
Date: Thu Mar 6 22:35:48 2014 +0000

    Include next link when default limit is reached

    The /servers and /servers/detail APIs support pagination and a
    "next" link should be included when more data is available, see
    http://docs.openstack.org/api/openstack-compute/2/content/
    Paginated_Collections-d1e664.html. In the documentation it states
    that 'collections are required to contain atom "next" links'.

    When the default "osapi_max" limit is reached then the "next" link
    is not included in the API reply. In this case, the caller cannot
    determine if there are any more servers and has no marker value
    such that they can retrieve the rest of the servers.

    For example, assume that there are 1050 VMs and the osapi_max value
    is the default of 1000. The /servers API will return the first 1000
    VMs but it will not include the "next" link; the caller has no way
    to determine if there are more then 1000 VMs and no way to retrieve
    the other 50 VMs.

    The fix for this is to include the "next" link when the number of
    servers being returned is the maximum limit, even if the "limit"
    parameter is not supplied. Note: the caller could workaround this
    by specifying the exact "osapi_max" limit value (ie, limit=1000)
    on the API call but this requires that the caller knows the
    configured limit.

    Change-Id: I9b3da852b0ab11dc980524e2317ee876a3ec28b8
    Closes-bug: 1288466
    (cherry picked from commit 968c01afe7ff43346f5817e2a3adf97fd03106b1)