Bulk creation for security group returns 500 error.

[Kengo Hobo]

API request
============
vagrant@ubuntu:~$ curl -i -X POST -H "X-Auth-Token: $TOKEN" http://192.168.122.139:9696/v2.0/security-groups -d '{"security_groups":[{"security_group":{"name":"hobo1"}}]}'
HTTP/1.1 500 Internal Server Error
Content-Type: application/json
Content-Length: 150
X-Openstack-Request-Id: req-48d5282e-f0b6-48b8-887c-7aa0c953ee88
Date: Wed, 20 Jul 2016 03:54:06 GMT

{"NeutronError": {"message": "Request Failed: internal server error while processing your request.", "type": "HTTPInternalServerError", "detail": ""}}

trace in neutron server
=======================
2016-07-20 12:54:06.234 5351 ERROR neutron.api.v2.resource [req-48d5282e-f0b6-48b8-887c-7aa0c953ee88 e01bc3eadeb045edb02fc6b2af4b5d49 867929bfedca4a719e17a7f3293845de -
 - -] create failed: No details.
2016-07-20 12:54:06.234 5351 ERROR neutron.api.v2.resource Traceback (most recent call last):
2016-07-20 12:54:06.234 5351 ERROR neutron.api.v2.resource File "/opt/stack/neutron/neutron/api/v2/resource.py", line 79, in resource
2016-07-20 12:54:06.234 5351 ERROR neutron.api.v2.resource result = method(request=request, **args)
2016-07-20 12:54:06.234 5351 ERROR neutron.api.v2.resource File "/opt/stack/neutron/neutron/api/v2/base.py", line 401, in create
2016-07-20 12:54:06.234 5351 ERROR neutron.api.v2.resource return self._create(request, body, **kwargs)
2016-07-20 12:54:06.234 5351 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/oslo_db/api.py", line 148, in wrapper
2016-07-20 12:54:06.234 5351 ERROR neutron.api.v2.resource ectxt.value = e.inner_exc
2016-07-20 12:54:06.234 5351 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2016-07-20 12:54:06.234 5351 ERROR neutron.api.v2.resource self.force_reraise()
2016-07-20 12:54:06.234 5351 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2016-07-20 12:54:06.234 5351 ERROR neutron.api.v2.resource six.reraise(self.type_, self.value, self.tb)
2016-07-20 12:54:06.234 5351 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/oslo_db/api.py", line 138, in wrapper
2016-07-20 12:54:06.234 5351 ERROR neutron.api.v2.resource return f(*args, **kwargs)
2016-07-20 12:54:06.234 5351 ERROR neutron.api.v2.resource File "/opt/stack/neutron/neutron/api/v2/base.py", line 500, in _create
2016-07-20 12:54:06.234 5351 ERROR neutron.api.v2.resource objs = do_create(body, bulk=True)
2016-07-20 12:54:06.234 5351 ERROR neutron.api.v2.resource File "/opt/stack/neutron/neutron/api/v2/base.py", line 496, in do_create
2016-07-20 12:54:06.234 5351 ERROR neutron.api.v2.resource request.context, reservation.reservation_id)
2016-07-20 12:54:06.234 5351 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2016-07-20 12:54:06.234 5351 ERROR neutron.api.v2.resource self.force_reraise()
2016-07-20 12:54:06.234 5351 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2016-07-20 12:54:06.234 5351 ERROR neutron.api.v2.resource six.reraise(self.type_, self.value, self.tb)
2016-07-20 12:54:06.234 5351 ERROR neutron.api.v2.resource File "/opt/stack/neutron/neutron/api/v2/base.py", line 489, in do_create
2016-07-20 12:54:06.234 5351 ERROR neutron.api.v2.resource return obj_creator(request.context, **kwargs)
2016-07-20 12:54:06.234 5351 ERROR neutron.api.v2.resource TypeError: create_security_group_bulk() got an unexpected keyword argument 'security_groups'
2016-07-20 12:54:06.234 5351 ERROR neutron.api.v2.resource
2016-07-20 12:54:06.241 5351 INFO neutron.wsgi [req-48d5282e-f0b6-48b8-887c-7aa0c953ee88 e01bc3eadeb045edb02fc6b2af4b5d49 867929bfedca4a719e17a7f3293845de - - -] 192.168.122.139 - - [20/Jul/2016 12:54:06] "POST /v2.0/security-groups HTTP/1.1" 500 344 0.203171

[Miguel Lavalle (minsel)]

We should be able to create security groups in bulk. The extension starts the API controller with bulk operations enabled: https://github.com/openstack/neutron/blob/master/neutron/extensions/securitygroup.py#L320

The docs (http://developer.openstack.org/api-ref-networking-v2-ext.html) don't show explicitly the request body for a bulk create operation. This is an example for networks (which I confirmed works correctly):

{
    "networks": [
        {
            "name": "sample_network3",
            "admin_state_up": true
        },
        {
            "name": "sample_network4",
            "admin_state_up": true
        }
    ]
}

Following this example, in my opinion the body for a security groups bulk create should be:

{
    "security_groups": [
        {
            "name": "my-security-group-1",
            "description": "My security group number 1"
        },
        {
            "name": "my-security-group-2",
            "description": "My security group number 2"
        }
    ]
}

It also returns a 500 and the Traceback is http://paste.openstack.org/show/538906/

[Reedip (reedip-banerjee-deactivatedaccount)]

Python-neutronclient and openstackclient also do not support bulk creation of security groups.

[Reedip (reedip-banerjee-deactivatedaccount)]

Hi Kengo Hobo, I would like to take this issue up.
Please let me know if you already have a patch ready for the server side. if not, then I will take it up.

[Kengo Hobo (hobo-kengo)]

Hi Reedip, thanks for notification.
I haven't implemented patch for this bug. Please take it.

[Reedip (reedip-banerjee-deactivatedaccount)]

Hi Kengo Hobo,
I have taken the patch

[Akihiro Motoki (amotoki)]

The bug is valid in the server side.

In neutronclient, the python binding layer already supports bulk creation (though the method name is still create_security_group()).

From the CLI point of view, it is not easy to use bulk creation CLI command. I don't think it is useful and I believe this is the reason OSC has no support for bulk create operations. Neutron CLI has no plan to support bulk creation now. I think this bug does not make sense in OSC too.

[Reedip (reedip-banerjee-deactivatedaccount)]

Akihiro San,
As per our earlier discussion, as NeutronClient may be deprecated in Octa, therefore there is no point of participating in NeutronClient.
For OpenstackClient as well, as you said there is no reason.
And for SDK, I will create a BP to track Bulk Create/Delete support

[OpenStack Infra (hudson-openstack)]

Change abandoned by Armando Migliaccio (<email address hidden>) on branch: master
Review: https://review.openstack.org/365990
Reason: This review is > 4 weeks without comment, and failed Jenkins the last time it was checked. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and leaving a 'recheck' comment to get fresh test results.

[Reedip (reedip-banerjee-deactivatedaccount)]

Similar bug logged in https://bugs.launchpad.net/neutron/+bug/1639539

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/365990
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=1866bd4b45f0eb6266ff5a608729f24e4090cfa4
Submitter: Jenkins
Branch: master

commit 1866bd4b45f0eb6266ff5a608729f24e4090cfa4
Author: Reedip <email address hidden>
Date: Tue Sep 6 14:12:00 2016 +0530

    Bulk creation of SecurityGroups

    Currently, Bulk Creation of security groups
    is enabled, but due to a mismatch in the function
    signature, it is not possible to create security
    groups in bulk.
    This patch allows the same.

    Change-Id: Id1c7a520aa1bb58984b199c86aca2267b98e6b3e
    Closes-Bug: #1604662

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 10.0.0.0b3 development milestone.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/437794

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/437794
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=e4f27f459eb723ae7de9d6b2504f4675f7ca7b33
Submitter: Jenkins
Branch: stable/newton

commit e4f27f459eb723ae7de9d6b2504f4675f7ca7b33
Author: Reedip <email address hidden>
Date: Tue Sep 6 14:12:00 2016 +0530

    Bulk creation of SecurityGroups

    Currently, Bulk Creation of security groups
    is enabled, but due to a mismatch in the function
    signature, it is not possible to create security
    groups in bulk.
    This patch allows the same.

    Change-Id: Id1c7a520aa1bb58984b199c86aca2267b98e6b3e
    Closes-Bug: #1604662
    (cherry picked from commit 1866bd4b45f0eb6266ff5a608729f24e4090cfa4)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 9.3.0 release.