NSXt CI is broken due to exception when creating security group

Bug #1668975 reported by Gary Kotton on 2017-03-01
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
vmware-nsx
Undecided
Unassigned

Bug Description

2017-02-27 16:50:59.984 15828 WARNING vmware_nsxlib.v3.client [req-76df86e6-19d3-45c2-9e0b-470d6065d91f - admin] The HTTP request returned error code 500, whereas 201/200 response codes were expected. Response body {u'module_name': u'common-services', u'error_message': u'General error has occurred.', u'error_code': u'100', u'details': u'java.lang.NullPointerException'}
2017-02-27 16:50:59.984 15828 ERROR vmware_nsx.plugins.nsx_v3.plugin [req-76df86e6-19d3-45c2-9e0b-470d6065d91f - admin] Failed to create backend firewall rules for security-group default (1e2544c1-b347-4e65-9c03-66e1462c39a1), rolling back changes.
2017-02-27 16:50:59.984 15828 ERROR vmware_nsx.plugins.nsx_v3.plugin Traceback (most recent call last):
2017-02-27 16:50:59.984 15828 ERROR vmware_nsx.plugins.nsx_v3.plugin File "/opt/stack/vmware-nsx/vmware_nsx/plugins/nsx_v3/plugin.py", line 3384, in create_security_group
2017-02-27 16:50:59.984 15828 ERROR vmware_nsx.plugins.nsx_v3.plugin logging, action, sg_rules)
2017-02-27 16:50:59.984 15828 ERROR vmware_nsx.plugins.nsx_v3.plugin File "/opt/stack/vmware-nsx/vmware_nsx/plugins/nsx_v3/plugin.py", line 3316, in _create_firewall_rules
2017-02-27 16:50:59.984 15828 ERROR vmware_nsx.plugins.nsx_v3.plugin ruleid_2_remote_nsgroup_map)
2017-02-27 16:50:59.984 15828 ERROR vmware_nsx.plugins.nsx_v3.plugin File "/usr/local/lib/python2.7/dist-packages/vmware_nsxlib/v3/security.py", line 455, in create_rules
2017-02-27 16:50:59.984 15828 ERROR vmware_nsx.plugins.nsx_v3.plugin return self.add_rules(firewall_rules, section_id)
2017-02-27 16:50:59.984 15828 ERROR vmware_nsx.plugins.nsx_v3.plugin File "/usr/local/lib/python2.7/dist-packages/vmware_nsxlib/v3/security.py", line 396, in add_rules
2017-02-27 16:50:59.984 15828 ERROR vmware_nsx.plugins.nsx_v3.plugin return self.client.create(resource + params, {'rules': rules})
2017-02-27 16:50:59.984 15828 ERROR vmware_nsx.plugins.nsx_v3.plugin File "/usr/local/lib/python2.7/dist-packages/vmware_nsxlib/v3/client.py", line 86, in create
2017-02-27 16:50:59.984 15828 ERROR vmware_nsx.plugins.nsx_v3.plugin return self.url_post(resource, body, headers=headers)
2017-02-27 16:50:59.984 15828 ERROR vmware_nsx.plugins.nsx_v3.plugin File "/usr/local/lib/python2.7/dist-packages/vmware_nsxlib/v3/client.py", line 111, in url_post
2017-02-27 16:50:59.984 15828 ERROR vmware_nsx.plugins.nsx_v3.plugin return self._rest_call(url, method='POST', body=body, headers=headers)
2017-02-27 16:50:59.984 15828 ERROR vmware_nsx.plugins.nsx_v3.plugin File "/usr/local/lib/python2.7/dist-packages/vmware_nsxlib/v3/client.py", line 207, in _rest_call
2017-02-27 16:50:59.984 15828 ERROR vmware_nsx.plugins.nsx_v3.plugin result = super(JSONRESTClient, self)._rest_call(*args, **kwargs)
2017-02-27 16:50:59.984 15828 ERROR vmware_nsx.plugins.nsx_v3.plugin File "/usr/local/lib/python2.7/dist-packages/vmware_nsxlib/v3/client.py", line 182, in _rest_call
2017-02-27 16:50:59.984 15828 ERROR vmware_nsx.plugins.nsx_v3.plugin _("%(verb)s %(url)s") % {'verb': method, 'url': request_url})
2017-02-27 16:50:59.984 15828 ERROR vmware_nsx.plugins.nsx_v3.plugin File "/usr/local/lib/python2.7/dist-packages/vmware_nsxlib/v3/client.py", line 140, in _validate_result
2017-02-27 16:50:59.984 15828 ERROR vmware_nsx.plugins.nsx_v3.plugin error_code=error_code)
2017-02-27 16:50:59.984 15828 ERROR vmware_nsx.plugins.nsx_v3.plugin File "/usr/local/lib/python2.7/dist-packages/vmware_nsxlib/v3/client.py", line 250, in _raise_error
2017-02-27 16:50:59.984 15828 ERROR vmware_nsx.plugins.nsx_v3.plugin error_code=error_code)
2017-02-27 16:50:59.984 15828 ERROR vmware_nsx.plugins.nsx_v3.plugin ManagerError: Unexpected error from backend manager (['10.192.68.26']) for POST api/v1/firewall/sections/e149598f-f3e4-4fdb-94e3-1af30437ad14/rules?action=create_multiple&operation=insert_bottom : General error has occurred.
2017-02-27 16:50:59.984 15828 ERROR vmware_nsx.plugins.nsx_v3.plugin
2017-02-27 16:51:00.010 15828 DEBUG neutron.callbacks.manager [req-76df86e6-19d3-45c2-9e0b-470d6065d91f - admin] Notify callbacks [] for security_group, before_delete _notify_loop /opt/stack/neutron/neutron/callbacks/manager.py:142
2017-02-27 16:51:00.011 15828 DEBUG neutron.callbacks.manager [req-76df86e6-19d3-45c2-9e0b-470d6065d91f - admin] Notify callbacks [] for security_group, precommit_delete _notify_loop /opt/stack/neutron/neutron/callbacks/manager.py:142
2017-02-27 16:51:00.025 15828 DEBUG neutron.callbacks.manager [req-76df86e6-19d3-45c2-9e0b-470d6065d91f - admin] Notify callbacks [] for security_group, after_delete _notify_loop /opt/stack/neutron/neutron/callbacks/manager.py:142
2017-02-27 16:51:00.025 15828 DEBUG vmware_nsxlib.v3.client [req-76df86e6-19d3-45c2-9e0b-470d6065d91f - admin] REST call: DELETE api/v1/ns-groups/25021437-74c9-45b4-ad54-833c708c542b?force=true
Headers: {'Content-Type': 'application/json', 'Accept': 'application/json'}
Body: None _rest_call /usr/local/lib/python2.7/dist-packages/vmware_nsxlib/v3/client.py:168
2017-02-27 16:51:00.025 15828 DEBUG vmware_nsxlib.v3.cluster [req-76df86e6-19d3-45c2-9e0b-470d6065d91f - admin] API cluster proxy DELETE /api/v1/ns-groups/25021437-74c9-45b4-ad54-833c708c542b?force=true to https://10.192.68.26/api/v1/ns-groups/25021437-74c9-45b4-ad54-833c708c542b?force=true _proxy /usr/local/lib/python2.7/dist-packages/vmware_nsxlib/v3/cluster.py:465
/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/connectionpool.py:843: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning)
2017-02-27 16:51:00.102 15828 DEBUG vmware_nsxlib.v3.client [req-76df86e6-19d3-45c2-9e0b-470d6065d91f - admin] REST call: DELETE api/v1/ns-groups/25021437-74c9-45b4-ad54-833c708c542b?force=true
Response: _rest_call /usr/local/lib/python2.7/dist-packages/vmware_nsxlib/v3/client.py:178
2017-02-27 16:51:00.103 15828 DEBUG vmware_nsxlib.v3.client [req-76df86e6-19d3-45c2-9e0b-470d6065d91f - admin] REST call: DELETE api/v1/firewall/sections/e149598f-f3e4-4fdb-94e3-1af30437ad14?cascade=true
Headers: {'Content-Type': 'application/json', 'Accept': 'application/json'}
Body: None _rest_call /usr/local/lib/python2.7/dist-packages/vmware_nsxlib/v3/client.py:168
2017-02-27 16:51:00.103 15828 DEBUG vmware_nsxlib.v3.cluster [req-76df86e6-19d3-45c2-9e0b-470d6065d91f - admin] API cluster proxy DELETE /api/v1/firewall/sections/e149598f-f3e4-4fdb-94e3-1af30437ad14?cascade=true to https://10.192.68.26/api/v1/firewall/sections/e149598f-f3e4-4fdb-94e3-1af30437ad14?cascade=true _proxy /usr/local/lib/python2.7/dist-packages/vmware_nsxlib/v3/cluster.py:465
/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/connectionpool.py:843: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning)
2017-02-27 16:51:00.132 15828 DEBUG vmware_nsxlib.v3.client [req-76df86e6-19d3-45c2-9e0b-470d6065d91f - admin] REST call: DELETE api/v1/firewall/sections/e149598f-f3e4-4fdb-94e3-1af30437ad14?cascade=true
Response: _rest_call /usr/local/lib/python2.7/dist-packages/vmware_nsxlib/v3/client.py:178
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource [req-76df86e6-19d3-45c2-9e0b-470d6065d91f - admin] create failed: No details.
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource Traceback (most recent call last):
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource File "/opt/stack/neutron/neutron/api/v2/resource.py", line 79, in resource
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource result = method(request=request, **args)
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource File "/opt/stack/neutron/neutron/api/v2/base.py", line 436, in create
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource return self._create(request, body, **kwargs)
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource File "/opt/stack/neutron/neutron/db/api.py", line 95, in wrapped
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource setattr(e, '_RETRY_EXCEEDED', True)
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource self.force_reraise()
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource six.reraise(self.type_, self.value, self.tb)
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource File "/opt/stack/neutron/neutron/db/api.py", line 91, in wrapped
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource return f(*args, **kwargs)
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/oslo_db/api.py", line 151, in wrapper
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource ectxt.value = e.inner_exc
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource self.force_reraise()
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource six.reraise(self.type_, self.value, self.tb)
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/oslo_db/api.py", line 139, in wrapper
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource return f(*args, **kwargs)
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource File "/opt/stack/neutron/neutron/db/api.py", line 131, in wrapped
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource traceback.format_exc())
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource self.force_reraise()
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource six.reraise(self.type_, self.value, self.tb)
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource File "/opt/stack/neutron/neutron/db/api.py", line 126, in wrapped
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource return f(*dup_args, **dup_kwargs)
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource File "/opt/stack/neutron/neutron/api/v2/base.py", line 549, in _create
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource obj = do_create(body)
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource File "/opt/stack/neutron/neutron/api/v2/base.py", line 531, in do_create
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource request.context, reservation.reservation_id)
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource self.force_reraise()
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource six.reraise(self.type_, self.value, self.tb)
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource File "/opt/stack/neutron/neutron/api/v2/base.py", line 524, in do_create
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource return obj_creator(request.context, **kwargs)
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource File "/opt/stack/vmware-nsx/vmware_nsx/plugins/nsx_v3/plugin.py", line 799, in create_network
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource self._ensure_default_security_group(context, tenant_id)
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource File "/opt/stack/vmware-nsx/vmware_nsx/plugins/nsx_v3/plugin.py", line 3250, in _ensure_default_security_group
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource context, tenant_id)
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource File "/opt/stack/neutron/neutron/db/securitygroups_db.py", line 703, in _ensure_default_security_group
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource default_sg=True)['id']
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource File "/opt/stack/vmware-nsx/vmware_nsx/plugins/nsx_v3/plugin.py", line 3397, in create_security_group
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource self.nsxlib.firewall_section.delete(firewall_section['id'])
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource self.force_reraise()
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource six.reraise(self.type_, self.value, self.tb)
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource File "/opt/stack/vmware-nsx/vmware_nsx/plugins/nsx_v3/plugin.py", line 3384, in create_security_group
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource logging, action, sg_rules)
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource File "/opt/stack/vmware-nsx/vmware_nsx/plugins/nsx_v3/plugin.py", line 3316, in _create_firewall_rules
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource ruleid_2_remote_nsgroup_map)
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/vmware_nsxlib/v3/security.py", line 455, in create_rules
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource return self.add_rules(firewall_rules, section_id)
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/vmware_nsxlib/v3/security.py", line 396, in add_rules
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource return self.client.create(resource + params, {'rules': rules})
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/vmware_nsxlib/v3/client.py", line 86, in create
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource return self.url_post(resource, body, headers=headers)
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/vmware_nsxlib/v3/client.py", line 111, in url_post
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource return self._rest_call(url, method='POST', body=body, headers=headers)
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/vmware_nsxlib/v3/client.py", line 207, in _rest_call
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource result = super(JSONRESTClient, self)._rest_call(*args, **kwargs)
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/vmware_nsxlib/v3/client.py", line 182, in _rest_call
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource _("%(verb)s %(url)s") % {'verb': method, 'url': request_url})
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/vmware_nsxlib/v3/client.py", line 140, in _validate_result
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource error_code=error_code)
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource File "/usr/local/lib/python2.7/dist-packages/vmware_nsxlib/v3/client.py", line 250, in _raise_error
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource error_code=error_code)
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource ManagerError: Unexpected error from backend manager (['10.192.68.26']) for POST api/v1/firewall/sections/e149598f-f3e4-4fdb-94e3-1af30437ad14/rules?action=create_multiple&operation=insert_bottom : General error has occurred.
2017-02-27 16:51:00.144 15828 ERROR neutron.api.v2.resource

This issue was fixed in the openstack/requirements 1.1.0 release.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers