Create servergroup failed with unexpected error

[Ethan Lynn]

Create a serverGroup with policies set to [] will cause nova api return unexpected error.

Aug 25 15:16:16 controller-qf47h5jn62 nova-osapi[752]: 2020-08-25 15:16:16.307 9 DEBUG nova.api.openstack.wsgi [req-d2edaa0f-bc9f-42ac-ba85-1273d25f3799 451d8d24f6428a3cef7409695abb6cbd3a1c4d2ef2b459e1fcdf75a848be8294 9425beb60c8746518fd77513191623ea - default default] Action: 'create', calling method: <function Controller.__getattribute__.<locals>.version_select at 0x7f0600e5d680>, body: {"server_group": {"name": "tpp-test-archetype-app-sandbox-vlyo", "policies": []}} _process_stack /usr/lib/python3.7/site-packages/nova/api/openstack/wsgi.py:520
Aug 25 15:16:16 controller-qf47h5jn62 nova-osapi[752]: 2020-08-25 15:16:16.336 9 DEBUG nova.quota [req-d2edaa0f-bc9f-42ac-ba85-1273d25f3799 451d8d24f6428a3cef7409695abb6cbd3a1c4d2ef2b459e1fcdf75a848be8294 9425beb60c8746518fd77513191623ea - default default] Getting quotas for project 9425beb60c8746518fd77513191623ea. Resources: {'server_groups'} _get_quotas /usr/lib/python3.7/site-packages/nova/quota.py:390
Aug 25 15:16:16 controller-qf47h5jn62 nova-osapi[752]: 2020-08-25 15:16:16.340 9 DEBUG nova.quota [req-d2edaa0f-bc9f-42ac-ba85-1273d25f3799 451d8d24f6428a3cef7409695abb6cbd3a1c4d2ef2b459e1fcdf75a848be8294 9425beb60c8746518fd77513191623ea - default default] Getting quotas for user 451d8d24f6428a3cef7409695abb6cbd3a1c4d2ef2b459e1fcdf75a848be8294 and project 9425beb60c8746518fd77513191623ea. Resources: {'server_groups'} _get_quotas /usr/lib/python3.7/site-packages/nova/quota.py:382
Aug 25 15:16:16 controller-qf47h5jn62 nova-osapi[752]: 2020-08-25 15:16:16.351 9 ERROR nova.api.openstack.wsgi [req-d2edaa0f-bc9f-42ac-ba85-1273d25f3799 451d8d24f6428a3cef7409695abb6cbd3a1c4d2ef2b459e1fcdf75a848be8294 9425beb60c8746518fd77513191623ea - default default] Unexpected exception in API method: IndexError: list index out of range
Aug 25 15:16:16 controller-qf47h5jn62 nova-osapi[752]: 2020-08-25 15:16:16.351 9 ERROR nova.api.openstack.wsgi Traceback (most recent call last):
Aug 25 15:16:16 controller-qf47h5jn62 nova-osapi[752]: 2020-08-25 15:16:16.351 9 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.7/site-packages/nova/api/openstack/wsgi.py", line 671, in wrapped
Aug 25 15:16:16 controller-qf47h5jn62 nova-osapi[752]: 2020-08-25 15:16:16.351 9 ERROR nova.api.openstack.wsgi return f(*args, **kwargs)
Aug 25 15:16:16 controller-qf47h5jn62 nova-osapi[752]: 2020-08-25 15:16:16.351 9 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.7/site-packages/nova/api/validation/__init__.py", line 110, in wrapper
Aug 25 15:16:16 controller-qf47h5jn62 nova-osapi[752]: 2020-08-25 15:16:16.351 9 ERROR nova.api.openstack.wsgi return func(*args, **kwargs)
Aug 25 15:16:16 controller-qf47h5jn62 nova-osapi[752]: 2020-08-25 15:16:16.351 9 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.7/site-packages/nova/api/validation/__init__.py", line 110, in wrapper
Aug 25 15:16:16 controller-qf47h5jn62 nova-osapi[752]: 2020-08-25 15:16:16.351 9 ERROR nova.api.openstack.wsgi return func(*args, **kwargs)
Aug 25 15:16:16 controller-qf47h5jn62 nova-osapi[752]: 2020-08-25 15:16:16.351 9 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.7/site-packages/nova/api/validation/__init__.py", line 110, in wrapper
Aug 25 15:16:16 controller-qf47h5jn62 nova-osapi[752]: 2020-08-25 15:16:16.351 9 ERROR nova.api.openstack.wsgi return func(*args, **kwargs)
Aug 25 15:16:16 controller-qf47h5jn62 nova-osapi[752]: 2020-08-25 15:16:16.351 9 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.7/site-packages/nova/api/openstack/compute/server_groups.py", line 201, in create
Aug 25 15:16:16 controller-qf47h5jn62 nova-osapi[752]: 2020-08-25 15:16:16.351 9 ERROR nova.api.openstack.wsgi sg = objects.InstanceGroup(context, policy=policies[0])
Aug 25 15:16:16 controller-qf47h5jn62 nova-osapi[752]: 2020-08-25 15:16:16.351 9 ERROR nova.api.openstack.wsgi IndexError: list index out of range
Aug 25 15:16:16 controller-qf47h5jn62 nova-osapi[752]: 2020-08-25 15:16:16.351 9 ERROR nova.api.openstack.wsgi
Aug 25 15:16:16 controller-qf47h5jn62 nova-osapi[752]: 2020-08-25 15:16:16.354 9 INFO nova.api.openstack.wsgi [req-d2edaa0f-bc9f-42ac-ba85-1273d25f3799 451d8d24f6428a3cef7409695abb6cbd3a1c4d2ef2b459e1fcdf75a848be8294 9425beb60c8746518fd77513191623ea - default default] HTTP exception thrown: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
Aug 25 15:16:16 controller-qf47h5jn62 nova-osapi[752]: <class 'IndexError'>
Aug 25 15:16:16 controller-qf47h5jn62 nova-osapi[752]: 2020-08-25 15:16:16.355 9 DEBUG nova.api.openstack.wsgi [req-d2edaa0f-bc9f-42ac-ba85-1273d25f3799 451d8d24f6428a3cef7409695abb6cbd3a1c4d2ef2b459e1fcdf75a848be8294 9425beb60c8746518fd77513191623ea - default default] Returning 500 to user: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.

[Sylvain Bauza (sylvain-bauza)]

Which release are you running with and do you specify a specific API microversion ?

Our latest release (Ussuri) includes API validation on the policies field to be a list of either 'affinity' or 'anti-affinity' enums on v2.1 and more on later microversions.

https://github.com/openstack/nova/blob/stable/ussuri/nova/api/openstack/compute/schemas/server_groups.py#L28-L39

[Balazs Gibizer (balazs-gibizer)]

I can reproduce the problem on the latest master

stack@aio:/opt/stack/nova$ curl -g -i -X POST http://192.168.121.129/compute/v2.1/os-server-groups -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: $TOKEN" -H "X-OpenStack-Nova-API-Version: 2.1" -d '{"server_group": {"name": "my-grp", "policies": []}}'
HTTP/1.1 500 Internal Server Error
Date: Tue, 15 Sep 2020 10:34:08 GMT
Server: Apache/2.4.29 (Ubuntu)
OpenStack-API-Version: compute 2.1
X-OpenStack-Nova-API-Version: 2.1
Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version
Content-Type: application/json; charset=UTF-8
Content-Length: 184
x-openstack-request-id: req-6651c4cd-bae4-4011-8975-cb00460d36d3
x-compute-request-id: req-6651c4cd-bae4-4011-8975-cb00460d36d3
Connection: close

{"computeFault": {"code": 500, "message": "Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.\n<class 'IndexError'>"}}stack@aio:/opt/stack/nova$

[Balazs Gibizer (balazs-gibizer)]

The above reproduction holds for microversion 2.1 - 2.63. In microversion 2.64 the API has changed, the policies field was renamed to policy and the type changed from a list to a single string. So the problem does not appear with microversion 2.64 or above

[Ghanshyam Mann (ghanshyammann)]

yeah in 2.64 microversion request has been changed from 'policies' as array to single items as string
- https://review.opendev.org/#/c/567534/36/nova/api/openstack/compute/rest_api_version_history.rst

empty 'policies' was anyways wrong request which has been fixed in 2.64 by converting the array type to string field. What we can do is to improve the error code from 500 to 400 but do we need to improve the error code for older microversion? or should ask to use new microversion which already fixed the issue.

[Brin Zhang (zhangbailin)]

Agree improve the error code from 500 to 400, and IMO we also need to improve the error code for older microversion, it's friendly for users.

[Brin Zhang (zhangbailin)]

This is happened in microversion 2.1~2.63, there is no much benefits to improve error code for older microversion.

A good way is to change the docs.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/752133

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/752210

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/752211

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/752210
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=2c66962c7a40d8ef4fab54324e06edcdec1bd716
Submitter: Zuul
Branch: master

commit 2c66962c7a40d8ef4fab54324e06edcdec1bd716
Author: Stephen Finucane <email address hidden>
Date: Wed Sep 16 10:52:17 2020 +0100

    tests: Add regression test for bug 1894966

    You must specify the 'policies' field. Currently, not doing so will
    result in a HTTP 500 error code. This should be a 4xx error. Add a test
    to demonstrate the bug before we provide a fix.

    Change-Id: I72e85855f621d3a51cd58d14247abd302dcd958b
    Signed-off-by: Stephen Finucane <email address hidden>
    Related-Bug: #1894966

[OpenStack Infra (hudson-openstack)]

Change abandoned by Brin Zhang (<email address hidden>) on branch: master
Review: https://review.opendev.org/752133

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/752371

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/752211
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=32c43fc8017ee89d4e6cdf79086d87735a00f0c0
Submitter: Zuul
Branch: master

commit 32c43fc8017ee89d4e6cdf79086d87735a00f0c0
Author: Stephen Finucane <email address hidden>
Date: Wed Sep 16 10:59:49 2020 +0100

    api: Set min, maxItems for server_group.policies field

    As noted inline, the 'policies' field may be a list but it expects one
    of two items.

    Change-Id: I34c68df1e6330dab1524aa0abec733610211a407
    Signed-off-by: Stephen Finucane <email address hidden>
    Closes-Bug: #1894966

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/752702

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/train
Review: https://review.opendev.org/752706

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/train
Review: https://review.opendev.org/752707

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/752371
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=94d24e3e8d04488abdebd4969daf98b780125297
Submitter: Zuul
Branch: stable/ussuri

commit 94d24e3e8d04488abdebd4969daf98b780125297
Author: Stephen Finucane <email address hidden>
Date: Wed Sep 16 10:52:17 2020 +0100

    tests: Add regression test for bug 1894966

    You must specify the 'policies' field. Currently, not doing so will
    result in a HTTP 500 error code. This should be a 4xx error. Add a test
    to demonstrate the bug before we provide a fix.

    Change-Id: I72e85855f621d3a51cd58d14247abd302dcd958b
    Signed-off-by: Stephen Finucane <email address hidden>
    Related-Bug: #1894966
    (cherry picked from commit 2c66962c7a40d8ef4fab54324e06edcdec1bd716)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/752702
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=781210bd598c3e0ee9bd6a7db5d25688b5fc0131
Submitter: Zuul
Branch: stable/ussuri

commit 781210bd598c3e0ee9bd6a7db5d25688b5fc0131
Author: Stephen Finucane <email address hidden>
Date: Wed Sep 16 10:59:49 2020 +0100

    api: Set min, maxItems for server_group.policies field

    As noted inline, the 'policies' field may be a list but it expects one
    of two items.

    Change-Id: I34c68df1e6330dab1524aa0abec733610211a407
    Signed-off-by: Stephen Finucane <email address hidden>
    Closes-Bug: #1894966
    (cherry picked from commit 32c43fc8017ee89d4e6cdf79086d87735a00f0c0)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/752706
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=cf6db29168c4ee9a34f3b3ebdd1deae31f95f203
Submitter: Zuul
Branch: stable/train

commit cf6db29168c4ee9a34f3b3ebdd1deae31f95f203
Author: Stephen Finucane <email address hidden>
Date: Wed Sep 16 10:52:17 2020 +0100

    tests: Add regression test for bug 1894966

    You must specify the 'policies' field. Currently, not doing so will
    result in a HTTP 500 error code. This should be a 4xx error. Add a test
    to demonstrate the bug before we provide a fix.

    Changes:
      nova/tests/functional/regressions/test_bug_1894966.py

    NOTE(stephenfin): Need to update 'super' call to Python 2-compatible
    variant.

    Change-Id: I72e85855f621d3a51cd58d14247abd302dcd958b
    Signed-off-by: Stephen Finucane <email address hidden>
    Related-Bug: #1894966
    (cherry picked from commit 2c66962c7a40d8ef4fab54324e06edcdec1bd716)
    (cherry picked from commit 94d24e3e8d04488abdebd4969daf98b780125297)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/752707
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=1634d3f59ace0206131992e31ee2d4b64123d7e8
Submitter: Zuul
Branch: stable/train

commit 1634d3f59ace0206131992e31ee2d4b64123d7e8
Author: Stephen Finucane <email address hidden>
Date: Wed Sep 16 10:59:49 2020 +0100

    api: Set min, maxItems for server_group.policies field

    As noted inline, the 'policies' field may be a list but it expects one
    of two items.

    Change-Id: I34c68df1e6330dab1524aa0abec733610211a407
    Signed-off-by: Stephen Finucane <email address hidden>
    Closes-Bug: #1894966
    (cherry picked from commit 32c43fc8017ee89d4e6cdf79086d87735a00f0c0)
    (cherry picked from commit 781210bd598c3e0ee9bd6a7db5d25688b5fc0131)

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Elod Illes <email address hidden>" on branch: stable/stein
Review: https://review.opendev.org/c/openstack/nova/+/772720
Reason: This branch transitioned to End of Life for this project, open patches needs to be closed to be able to delete the branch.

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Elod Illes <email address hidden>" on branch: stable/stein
Review: https://review.opendev.org/c/openstack/nova/+/772721
Reason: This branch transitioned to End of Life for this project, open patches needs to be closed to be able to delete the branch.