security group create errors without description

Bug #1434172 reported by Dean Troyer
14
This bug affects 3 people
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Won't Fix
Low
Sean Dague
python-novaclient
Won't Fix
Low
Alan
python-openstackclient
Fix Released
Low
Dean Troyer

Bug Description

security group create returns an error without --description supplied. This appears to be the server rejecting the request so we should set a default value rather than sending None.

  $ openstack security group create qaz
  ERROR: openstack Security group description is not a string or unicode (HTTP 400) (Request-ID: req-dee03de3-893a-4d58-bc3d-de87d09c3fb8)

Sent body:

  {"security_group": {"name": "qaz2", "description": null}}

Tags: api
Dean Troyer (dtroyer)
Changed in python-openstackclient:
status: New → Confirmed
importance: Undecided → Low
assignee: nobody → Dean Troyer (dtroyer)
milestone: none → m9
Revision history for this message
Dean Troyer (dtroyer) wrote :

The description also can not be empty ("") so the default could be the security group name repeated?

description: updated
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to python-openstackclient (master)

Fix proposed to branch: master
Review: https://review.openstack.org/165956

Changed in python-openstackclient:
status: Confirmed → In Progress
Revision history for this message
Steve Martinelli (stevemar) wrote :

@Dean, whats the API say? I would think null is a valid description value

Revision history for this message
Steve Martinelli (stevemar) wrote :

AFAICT - the API says a request with no description in the body should be allowed:
http://developer.openstack.org/api-ref-compute-v2-ext.html

In the controller, the description value is not checked but goes to validate_property...
https://github.com/openstack/nova/blob/3e5062a8ddf94351389f7e201959591b26f338a4/nova/api/openstack/compute/contrib/security_groups.py#L190-L207

At validate_property though, it errors out when any value is not a string
https://github.com/openstack/nova/blob/d33184b6f890256b332e75bd46a58823cac663ff/nova/compute/api.py#L3855-L3880

To me, this seems like an issue with nova's implementation not matching it's API. In this case the API seems correct as description should always be optional.

We can and should proceed with the OSC fix as a band-aid solution, but should mark a TODO to change it when the nova side is resolved.

Revision history for this message
Steve Martinelli (stevemar) wrote :

Potentially, we could change novaclient to check if description is not null, and not include the key in the body. Not sure if this will result in a different error.

Revision history for this message
Sean Dague (sdague) wrote :

description is optional, optional does not mean it can be null, it means it shouldn't be in the payload at all.

Changed in nova:
status: New → Invalid
Revision history for this message
Steve Martinelli (stevemar) wrote :
Revision history for this message
Steve Martinelli (stevemar) wrote :
Download full text (4.0 KiB)

@sdague, I made a change to novaclient to ensure the body does not include the description value.

You can see in the log below that the body is just: {"security_group": {"name": "tempo"}}

But, the result remains the same (400 with description is not a string or unicode):
2015-03-25 00:22:48.445 DEBUG nova.api.openstack.wsgi [req-0c501906-c662-4afd-a3fc-3b6e5e22caf9 admin admin] Action: 'create', calling method: <bound method SecurityGroupController.create of <nova.api.openstack.compute.contrib.security_groups.SecurityGroupController object at 0x7f0768586cd0>>, body: {"security_group": {"name": "tempo"}} from (pid=14913) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:780
2015-03-25 00:22:48.447 INFO nova.api.openstack.wsgi [req-0c501906-c662-4afd-a3fc-3b6e5e22caf9 admin admin] HTTP exception thrown: Security group description is not a string or unicode
2015-03-25 00:22:48.447 DEBUG nova.api.openstack.wsgi [req-0c501906-c662-4afd-a3fc-3b6e5e22caf9 admin admin] Returning 400 to user: Security group description is not a string or unicode from (pid=14913) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:1166
2015-03-25 00:22:48.450 INFO nova.osapi_compute.wsgi.server [req-0c501906-c662-4afd-a3fc-3b6e5e22caf9 admin admin] 10.0.2.15 "POST /v2/36c7f6452c394b44ad4ae1f2bfe07800/os-security-groups HTTP/1.1" status: 400 len: 317 time: 0.1127591

This is the change that I made to novaclient

steve:python-novaclient$ git diff
diff --git a/novaclient/v2/security_groups.py b/novaclient/v2/security_groups.py
index 40d1e7f..0cd4960 100644
--- a/novaclient/v2/security_groups.py
+++ b/novaclient/v2/security_groups.py
@@ -45,7 +45,9 @@ class SecurityGroupManager(base.ManagerWithFind):
         :param description: description of the security group
         :rtype: the security group object
         """
- body = {"security_group": {"name": name, 'description': description}}
+ body = {"security_group": {"name": name}}
+ if description:
+ body['security_group']['description'] = description
         return self._create('/os-security-groups', body, 'security_group')

Double checked that it still works for the case with a description, and it does.

2015-03-25 00:25:28.870 DEBUG nova.api.openstack.wsgi [req-e2be9899-c50a-40fc-9eaa-258db57b5cf3 admin admin] Action: 'create', calling method: <bound method SecurityGroupController.create of <nova.api.openstack.compute.contrib.security_groups.SecurityGroupController object at 0x7f0768586cd0>>, body: {"security_group": {"name": "tempo", "description": "tempo_desc"}} from (pid=14913) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:780
2015-03-25 00:25:28.871 DEBUG oslo_db.api [req-e2be9899-c50a-40fc-9eaa-258db57b5cf3 admin admin] Loading backend 'sqlalchemy' from 'nova.db.sqlalchemy.api' from (pid=14913) _load_backend /usr/local/lib/python2.7/dist-packages/oslo_db/api.py:214
2015-03-25 00:25:28.872 WARNING oslo_config.cfg [req-e2be9899-c50a-40fc-9eaa-258db57b5cf3 admin admin] Option "sql_connection" from group "DEFAULT" is deprecated. Use option "connection" from group "database".
2015-03-25 00:25:28.892 DEBUG oslo_db.sqlalchemy.session [req-e2be9899-c50a-40fc-9eaa-258d...

Read more...

Changed in nova:
status: Invalid → New
Revision history for this message
Steve Martinelli (stevemar) wrote :

Switched it back to new, let me know if you disagree.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to python-novaclient (master)

Fix proposed to branch: master
Review: https://review.openstack.org/167487

Changed in python-novaclient:
assignee: nobody → Steve Martinelli (stevemar)
status: New → In Progress
Revision history for this message
Davanum Srinivas (DIMS) (dims-v) wrote :

Steve, Dean,

Is there anything to do in Nova itself?

thanks,
dims

Changed in nova:
status: New → Incomplete
Revision history for this message
Steve Martinelli (stevemar) wrote :

@Dims, yes there is.

This request to create a new security group fails: body = {"security_group": {"name": name}}.

This is actually shown in one of nova's tests: https://github.com/openstack/nova/blob/master/nova/tests/unit/api/openstack/compute/contrib/test_security_groups.py#L171-L179

    def test_create_security_group_with_no_description(self):
        sg = security_group_template()
        del sg['description']

        req = fakes.HTTPRequest.blank('/v2/fake/os-security-groups')
        self.assertRaises(webob.exc.HTTPBadRequest, self.controller.create,
                          req, {'security_group': sg})

So not including an optional attribute results in a bad request?

Revision history for this message
Steve Martinelli (stevemar) wrote :

I realize this is a low priority bug, and it's RC time. But I honestly can't show how this is any less valid.

Revision history for this message
melanie witt (melwitt) wrote :

According to the documentation for the security groups api, the description parameter is supposed to be optional:

http://developer.openstack.org/api-ref-compute-v2-ext.html#ext-os-security-groups

Revision history for this message
Sean Dague (sdague) wrote :

Right, optional means you are allowed to not provide the parameter. Not that you can provide it as null. There is a python-novaclient bug here.

Changed in nova:
status: Incomplete → Won't Fix
Revision history for this message
melanie witt (melwitt) wrote :

Hi Sean, in comment #8 Steve said he still got a 400 from nova after omitting the description parameter from the request body.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on python-novaclient (master)

Change abandoned by Steve Martinelli (<email address hidden>) on branch: master
Review: https://review.openstack.org/167487
Reason: meh

Dean Troyer (dtroyer)
Changed in python-openstackclient:
milestone: m9 → m10
Revision history for this message
melanie witt (melwitt) wrote :

There is also a bug in nova. Both the documentation and the data model show "description" as an optional field, but the implementation erroneously requires it as it calls a validate_property function on "description" whether it was provided or not:

http://git.openstack.org/cgit/openstack/nova/tree/nova/api/openstack/compute/contrib/security_groups.py#n201
http://git.openstack.org/cgit/openstack/nova/tree/nova/compute/api.py#n3838

Changed in nova:
importance: Undecided → Low
status: Won't Fix → Confirmed
melanie witt (melwitt)
tags: added: api
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to python-openstackclient (master)

Reviewed: https://review.openstack.org/165956
Committed: https://git.openstack.org/cgit/openstack/python-openstackclient/commit/?id=11c39530f5f97a14d534c8d5b7160a1e74f6cdf8
Submitter: Jenkins
Branch: master

commit 11c39530f5f97a14d534c8d5b7160a1e74f6cdf8
Author: Dean Troyer <email address hidden>
Date: Thu Mar 19 11:46:05 2015 -0500

    Fix security group create description bug

    --description is optional in our CLI but the server requires it to be
    non-empty. Set a default value of the given name.

    Closes-Bug: #1434172
    Change-Id: I81507a77ad8d815000ff411784ae71e229c77f78

Changed in python-openstackclient:
status: In Progress → Fix Committed
Revision history for this message
melanie witt (melwitt) wrote :

We can work around the nova bug in novaclient by passing a default value.

Changed in python-novaclient:
assignee: Steve Martinelli (stevemar) → nobody
importance: Undecided → Low
status: In Progress → Confirmed
Revision history for this message
Steve Martinelli (stevemar) wrote :

@melwitt we worked around this in OSC by sending a description that was the same as the name (if no name was passed in).

Dean Troyer (dtroyer)
Changed in python-openstackclient:
status: Fix Committed → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/183909

Changed in nova:
assignee: nobody → Adrien Vergé (adrien-verge)
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on nova (master)

Change abandoned by Adrien Vergé (<email address hidden>) on branch: master
Review: https://review.openstack.org/183909

hgangwx (hgangwx)
Changed in python-novaclient:
assignee: nobody → ibm-cloud-qa (ibm-cloud-qa)
hgangwx (hgangwx)
Changed in python-novaclient:
assignee: ibm-cloud-qa (ibm-cloud-qa) → nobody
Alan (kaihongd)
Changed in python-novaclient:
assignee: nobody → Alan (kaihongd)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to python-novaclient (master)

Fix proposed to branch: master
Review: https://review.openstack.org/262655

Changed in python-novaclient:
status: Confirmed → In Progress
Changed in nova:
assignee: Adrien Vergé (adrien-verge) → Sean Dague (sdague)
Revision history for this message
Sean Dague (sdague) wrote :

We've updated the Nova docs that this is no longer optional.

Changed in nova:
status: In Progress → Won't Fix
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on nova (master)

Change abandoned by Sean Dague (<email address hidden>) on branch: master
Review: https://review.openstack.org/279552

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on python-novaclient (master)

Change abandoned by Kevin L. Mitchell (<email address hidden>) on branch: master
Review: https://review.openstack.org/262655
Reason: Not updated since December.

Revision history for this message
Takashi Natsume (natsume-takashi) wrote :

In python-novaclient, the "nova secgroup-create" command has been removed since https://review.openstack.org/#/c/437145/.

So it is no longer necessary to fix it in python-novaclient.

Changed in python-novaclient:
status: In Progress → Won't Fix
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.