vmware nsx: db flush error during router operations

Bug #1266482 reported by Armando Migliaccio
16
This bug affects 2 people
Affects Status Importance Assigned to Milestone
neutron
Fix Released
Medium
Salvatore Orlando

Bug Description

This stacktrace has been observed on master:

http://paste.openstack.org/show/60437/

This ultimately causes Tempest failures like the one below:

time: 2014-01-06 06:53:23.851259Z
tags: worker-0
test: tearDownClass (tempest.api.compute.security_groups.test_security_group_rules.SecurityGroupRulesTestXML)
time: 2014-01-06 06:53:23.851259Z
failure: tearDownClass (tempest.api.compute.security_groups.test_security_group_rules.SecurityGroupRulesTestXML) [ multipart
Content-Type: text/x-traceback;charset="utf8",language="python"
traceback
42D
Traceback (most recent call last):
  File "tempest/api/compute/base.py", line 112, in tearDownClass
    cls.clear_isolated_creds()
  File "tempest/test.py", line 289, in clear_isolated_creds
    cls.isolated_creds.clear_isolated_creds()
  File "tempest/common/isolated_creds.py", line 457, in clear_isolated_creds
    self._clear_isolated_net_resources()
  File "tempest/common/isolated_creds.py", line 436, in _clear_isolated_net_resources
    router['id'], subnet['id'])
  File "tempest/services/network/json/network_client.py", line 289, in remove_router_interface_with_subnet_id
    resp, body = self.put(uri, update_body, self.headers)
  File "tempest/common/rest_client.py", line 314, in put
    return self.request('PUT', url, headers, body)
  File "tempest/common/rest_client.py", line 436, in request
    resp, resp_body)
  File "tempest/common/rest_client.py", line 530, in _error_checker
    raise exceptions.ServerFault(message)
ServerFault: Got server fault
Details: {"NeutronError": "Request Failed: internal server error while processing your request."}

Tags: nicira
Changed in neutron:
assignee: nobody → Armando Migliaccio (armando-migliaccio)
Revision history for this message
Armando Migliaccio (armando-migliaccio) wrote :

This might help address the issue:

https://review.openstack.org/#/c/58017/

Revision history for this message
Salvatore Orlando (salvatore-orlando) wrote :

There is a chance the issue might be originating from the fact that handle_router_metadata_access perform multiple transactions with the same db session, and these transaction might end up operating on the same objects.

I've tried to cleanup the session between the two transaction and so far I've not seen the bug anymore.

Check: https://review.openstack.org/#/c/66070/

Changed in neutron:
assignee: Armando Migliaccio (armando-migliaccio) → Salvatore Orlando (salvatore-orlando)
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (master)

Reviewed: https://review.openstack.org/66070
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=f6d384d7a2df5ba603bf7862853e9cb9fc70fcb4
Submitter: Jenkins
Branch: master

commit f6d384d7a2df5ba603bf7862853e9cb9fc70fcb4
Author: Salvatore Orlando <email address hidden>
Date: Wed Jan 15 12:45:22 2014 -0800

    Expunge session contents between plugin requests

    In the NVP plugin, metadata processing performs several plugin operations
    with the same context (and db session). The first operation might leave
    persisted objects in the session instance which then conflict with objects
    created in the second operation.

    Closes-Bug: #1266482

    Change-Id: Ic9e2b952013610647aa222f1a8b6e33504b54b37

Changed in neutron:
status: In Progress → Fix Committed
Changed in neutron:
importance: Undecided → Medium
milestone: none → icehouse-3
Thierry Carrez (ttx)
Changed in neutron:
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in neutron:
milestone: icehouse-3 → 2014.1
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.