DBNotAllowed raised if trying to create network with VlanManager from nova-manage network create

Bug #1304724 reported by Matt Riedemann
22
This bug affects 3 people
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Fix Released
High
Dan Smith

Bug Description

Steps to reproduce:
- Setup a devstack from scratch using nova-network
- delete the default network
  # nova-manage network delete 10.0.0.0/24
- change nova.conf to use VlanManager:
network_manager = nova.network.manager.VlanManager
- restart nova-network
- create a new network with a vlan id:
nova-manage network create --label=network --fixed_range_v4 10.0.1.0/24 --vlan 42
- boot a vm on the cirros image:
nova --debug boot --flavor 1 --image 0b969819-2d85-4f7f-af76-125c5bb5789f test

Expected behavior: The new VM goes to Active state

Actual behavior: The new VM goes to Error state, also nova-network log has this exception:
a7-abaf-78db50a4b62c] network allocations from (pid=13676) allocate_for_instance /opt/stack/nova/nova/network/manager.py:494
2014-04-07 15:32:02.137 ERROR nova.network [req-87a65a9e-9196-4203-9de2-f6911d2aef4b admin demo] No db access allowed in nova-network: File "/usr/local/lib/python2.7/dist-packages/eventlet/greenthread.py", line 194, in main
    result = function(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 128, in <lambda>
    yield lambda: self._dispatch_and_reply(incoming)
  File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 133, in _dispatch_and_reply
    incoming.message))
  File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 176, in _dispatch
    return self._do_dispatch(endpoint, method, ctxt, args)
  File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 122, in _do_dispatch
    result = getattr(endpoint, method)(ctxt, **new_args)
  File "/opt/stack/nova/nova/network/floating_ips.py", line 119, in allocate_for_instance
    **kwargs)
  File "/opt/stack/nova/nova/network/manager.py", line 497, in allocate_for_instance
    requested_networks=requested_networks)
  File "/opt/stack/nova/nova/network/manager.py", line 1837, in _get_networks_for_instance
    networks = self.db.project_get_networks(context, project_id)
  File "/opt/stack/nova/nova/db/api.py", line 1370, in project_get_networks
    return IMPL.project_get_networks(context, project_id, associate)
  File "/opt/stack/nova/nova/cmd/network.py", line 47, in __call__
    stacktrace = "".join(traceback.format_stack())

I think the exception was introduced by this patch that disables direct database access from nova-network: https://review.openstack.org/#/c/79716/
However, VlanManager still relies on database access for the given scenario, and there are 3 other places in manager.py that rely on direct db access:
devuser@ubuntu:/opt/stack/nova$ grep self.db nova/network/manager.py -n
1389: vifs = self.db.virtual_interface_get_by_instance(context,
1446: vif = self.db.virtual_interface_get_by_address(context,
1837: networks = self.db.project_get_networks(context, project_id)
1914: not self.db.network_in_use_on_host(context, network['id'],
Therefore, I cannot currently use conductor with nova-network VlanManager, which is a regression from Havana.

===

devstack defaults the network_manager to FlatDHCPManager so we don't test VlanManager in the gate.

Tags: network
Revision history for this message
Matt Riedemann (mriedem) wrote :
Changed in nova:
status: New → In Progress
importance: Undecided → High
assignee: nobody → Dan Smith (danms)
milestone: none → icehouse-rc2
tags: added: icehouse-rc-potential
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (milestone-proposed)

Fix proposed to branch: milestone-proposed
Review: https://review.openstack.org/86194

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (master)

Reviewed: https://review.openstack.org/86190
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=275a165cf0e74112d7ec9addacb1f84d703977c8
Submitter: Jenkins
Branch: master

commit 275a165cf0e74112d7ec9addacb1f84d703977c8
Author: Dan Smith <email address hidden>
Date: Tue Apr 8 16:55:43 2014 -0700

    Fix straggling uses of direct-to-database queries in nova-network

    There were a few remaining calls directly to the database module
    left in nova-network. These would fail because of the wedge we
    put in place specifically to catch this case. However, we don't
    test VlanManager in the gate, which means we didn't catch them
    until now.

    This required adding two more methods, one each to Network and
    NetworkList, but they are both extremely simple.

    Change-Id: Iecc382074f060da1bd6f740d7bde0b20a0be2fcf
    Closes-bug: #1304724

Changed in nova:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (milestone-proposed)

Reviewed: https://review.openstack.org/86194
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=51992642c09cf09cef7ea0efe8a9ed6fbdfdf02d
Submitter: Jenkins
Branch: milestone-proposed

commit 51992642c09cf09cef7ea0efe8a9ed6fbdfdf02d
Author: Dan Smith <email address hidden>
Date: Tue Apr 8 16:55:43 2014 -0700

    Fix straggling uses of direct-to-database queries in nova-network

    There were a few remaining calls directly to the database module
    left in nova-network. These would fail because of the wedge we
    put in place specifically to catch this case. However, we don't
    test VlanManager in the gate, which means we didn't catch them
    until now.

    This required adding two more methods, one each to Network and
    NetworkList, but they are both extremely simple.

    Change-Id: Iecc382074f060da1bd6f740d7bde0b20a0be2fcf
    Closes-bug: #1304724
    (cherry picked from commit 275a165cf0e74112d7ec9addacb1f84d703977c8)

Changed in nova:
status: Fix Committed → Fix Released
Matt Riedemann (mriedem)
tags: removed: icehouse-rc-potential
Thierry Carrez (ttx)
Changed in nova:
milestone: icehouse-rc2 → 2014.1
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.