nova-network crashes if dhcp_domain = ""

Bug #1161027 reported by Sean Dague
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Fix Released
High
Sean Dague

Bug Description

if /etc/nova/nova.conf includes the line:

dhcp_domain = ""

Then nova-network crashes, because --dhcp-domain= is not a valid argument to pass to dnsmasq. The following is seen in the logs under this circumstance:

2013-03-27 11:49:26 DEBUG [nova.utils] Running cmd (subprocess): sudo nova-rootwrap /etc/nova/rootwrap.conf env CONFIG_FILE=["/etc/nova/nova.conf"] NETWORK_ID=1 dnsmasq --strict-order --bind-interfaces --conf-file= --domain= --pid-file=/opt/stack/grizzly/data/nova/networks/nova-br100.pid --listen-address=10.4.128.1 --except-interface=lo --dhcp-range=set:private,10.4.128.2,static,120s --dhcp-lease-max=4096 --dhcp-hostsfile=/opt/stack/grizzly/data/nova/networks/nova-br100.conf --dhcp-script=/opt/stack/grizzly/nova/bin/nova-dhcpbridge --leasefile-ro
2013-03-27 11:49:27 DEBUG [nova.utils] Result was 1
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 346, in fire_timers
    timer()
  File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/timer.py", line 56, in __call__
    cb(*args, **kw)
  File "/usr/local/lib/python2.7/dist-packages/eventlet/greenthread.py", line 194, in main
    result = function(*args, **kwargs)
  File "/opt/stack/grizzly/nova/nova/service.py", line 147, in run_server
    server.start()
  File "/opt/stack/grizzly/nova/nova/service.py", line 429, in start
    self.manager.init_host()
  File "/opt/stack/grizzly/nova/nova/network/manager.py", line 1604, in init_host
    super(FlatDHCPManager, self).init_host()
  File "/opt/stack/grizzly/nova/nova/network/manager.py", line 346, in init_host
    self._setup_network_on_host(ctxt, network)
  File "/opt/stack/grizzly/nova/nova/network/manager.py", line 1619, in _setup_network_on_host
    self.driver.update_dhcp(elevated, dev, network)
  File "/opt/stack/grizzly/nova/nova/network/linux_net.py", line 938, in update_dhcp
    _execute(*cmd, run_as_root=True)
  File "/opt/stack/grizzly/nova/nova/network/linux_net.py", line 1141, in _execute
    return utils.execute(*cmd, **kwargs)
  File "/opt/stack/grizzly/nova/nova/utils.py", line 239, in execute
    cmd=' '.join(cmd))
ProcessExecutionError: Unexpected error while running command.
Command: sudo nova-rootwrap /etc/nova/rootwrap.conf env CONFIG_FILE=["/etc/nova/nova.conf"] NETWORK_ID=1 dnsmasq --strict-order --bind-interfaces --conf-file= --domain= --pid-file=/opt/stack/grizzly/data/nova/networks/nova-br100.pid --listen-address=10.4.128.1 --except-interface=lo --dhcp-range=set:private,10.4.128.2,static,120s --dhcp-lease-max=4096 --dhcp-hostsfile=/opt/stack/grizzly/data/nova/networks/nova-br100.conf --dhcp-script=/opt/stack/grizzly/nova/bin/nova-dhcpbridge --leasefile-ro
Exit code: 1
Stdout: ''
Stderr: '\ndnsmasq: bad command line options: try --help\n'
2013-03-27 11:49:27 CRITICAL [nova] Unexpected error while running command.
Command: sudo nova-rootwrap /etc/nova/rootwrap.conf env CONFIG_FILE=["/etc/nova/nova.conf"] NETWORK_ID=1 dnsmasq --strict-order --bind-interfaces --conf-file= --domain= --pid-file=/opt/stack/grizzly/data/nova/networks/nova-br100.pid --listen-address=10.4.128.1 --except-interface=lo --dhcp-range=set:private,10.4.128.2,static,120s --dhcp-lease-max=4096 --dhcp-hostsfile=/opt/stack/grizzly/data/nova/networks/nova-br100.conf --dhcp-script=/opt/stack/grizzly/nova/bin/nova-dhcpbridge --leasefile-ro
Exit code: 1
Stdout: ''
Stderr: '\ndnsmasq: bad command line options: try --help\n'
Traceback (most recent call last):
  File "/usr/local/bin/nova-network", line 7, in <module>
    execfile(__file__)
  File "/opt/stack/grizzly/nova/bin/nova-network", line 54, in <module>
    service.wait()
  File "/opt/stack/grizzly/nova/nova/service.py", line 689, in wait
    _launcher.wait()
  File "/opt/stack/grizzly/nova/nova/service.py", line 209, in wait
    super(ServiceLauncher, self).wait()
  File "/opt/stack/grizzly/nova/nova/service.py", line 179, in wait
    service.wait()
  File "/usr/local/lib/python2.7/dist-packages/eventlet/greenthread.py", line 168, in wait
    return self._exit_event.wait()
  File "/usr/local/lib/python2.7/dist-packages/eventlet/event.py", line 116, in wait
    return hubs.get_hub().switch()
  File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 187, in switch
    return self.greenlet.switch()
  File "/usr/local/lib/python2.7/dist-packages/eventlet/greenthread.py", line 194, in main
    result = function(*args, **kwargs)
  File "/opt/stack/grizzly/nova/nova/service.py", line 147, in run_server
    server.start()
  File "/opt/stack/grizzly/nova/nova/service.py", line 429, in start
    self.manager.init_host()
  File "/opt/stack/grizzly/nova/nova/network/manager.py", line 1604, in init_host
    super(FlatDHCPManager, self).init_host()
  File "/opt/stack/grizzly/nova/nova/network/manager.py", line 346, in init_host
    self._setup_network_on_host(ctxt, network)
  File "/opt/stack/grizzly/nova/nova/network/manager.py", line 1619, in _setup_network_on_host
    self.driver.update_dhcp(elevated, dev, network)
  File "/opt/stack/grizzly/nova/nova/network/linux_net.py", line 938, in update_dhcp
    restart_dhcp(context, dev, network_ref)
  File "/opt/stack/grizzly/nova/nova/openstack/common/lockutils.py", line 242, in inner
    retval = f(*args, **kwargs)
  File "/opt/stack/grizzly/nova/nova/network/linux_net.py", line 1048, in restart_dhcp
    _execute(*cmd, run_as_root=True)
  File "/opt/stack/grizzly/nova/nova/network/linux_net.py", line 1141, in _execute
    return utils.execute(*cmd, **kwargs)
  File "/opt/stack/grizzly/nova/nova/utils.py", line 239, in execute
    cmd=' '.join(cmd))
ProcessExecutionError: Unexpected error while running command.
Command: sudo nova-rootwrap /etc/nova/rootwrap.conf env CONFIG_FILE=["/etc/nova/nova.conf"] NETWORK_ID=1 dnsmasq --strict-order --bind-interfaces --conf-file= --domain= --pid-file=/opt/stack/grizzly/data/nova/networks/nova-br100.pid --listen-address=10.4.128.1 --except-interface=lo --dhcp-range=set:private,10.4.128.2,static,120s --dhcp-lease-max=4096 --dhcp-hostsfile=/opt/stack/grizzly/data/nova/networks/nova-br100.conf --dhcp-script=/opt/stack/grizzly/nova/bin/nova-dhcpbridge --leasefile-ro
Exit code: 1
Stdout: ''
Stderr: '\ndnsmasq: bad command line options: try --help\n'

Sean Dague (sdague)
Changed in nova:
importance: Undecided → High
Sean Dague (sdague)
Changed in nova:
assignee: nobody → Sean Dague (sdague)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/25543

Changed in nova:
status: New → In Progress
Changed in nova:
milestone: none → grizzly-rc2
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (master)

Reviewed: https://review.openstack.org/25543
Committed: http://github.com/openstack/nova/commit/8d68ed3583df6ced4b4c8b6d6bb3569e03992cfe
Submitter: Jenkins
Branch: master

commit 8d68ed3583df6ced4b4c8b6d6bb3569e03992cfe
Author: Sean Dague <email address hidden>
Date: Wed Mar 27 13:29:15 2013 -0400

    always quote dhcp-domain, otherwise dnsmasq can fail to start

    the dnsmasq arg --dhcp-domain can't be passed to dnsmasq as empty
    (i.e. --dhcp-domain= ) unlike many other arguments. If you do pass
    it as empty, dnsmasq won't start, throwing a ProcessException which
    then crashes the rest of nova-network.

    Fix this by quoting dhcp-domain so that empty domains are allowed.

    Fixed bug #1161027

    This was found by grenade upgrade testing

    Change-Id: Id24814339d8c8a709f5041f0aa98b10ba1e0e595

Changed in nova:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (milestone-proposed)

Fix proposed to branch: milestone-proposed
Review: https://review.openstack.org/25617

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (milestone-proposed)

Reviewed: https://review.openstack.org/25617
Committed: http://github.com/openstack/nova/commit/2a47b329da09e858c6dfe3c5860e2ee088a8a307
Submitter: Jenkins
Branch: milestone-proposed

commit 2a47b329da09e858c6dfe3c5860e2ee088a8a307
Author: Sean Dague <email address hidden>
Date: Wed Mar 27 13:29:15 2013 -0400

    always quote dhcp-domain, otherwise dnsmasq can fail to start

    the dnsmasq arg --dhcp-domain can't be passed to dnsmasq as empty
    (i.e. --dhcp-domain= ) unlike many other arguments. If you do pass
    it as empty, dnsmasq won't start, throwing a ProcessException which
    then crashes the rest of nova-network.

    Fix this by quoting dhcp-domain so that empty domains are allowed.

    Fixed bug #1161027

    This was found by grenade upgrade testing

    Change-Id: Id24814339d8c8a709f5041f0aa98b10ba1e0e595
    (cherry picked from commit 8d68ed3583df6ced4b4c8b6d6bb3569e03992cfe)

Changed in nova:
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in nova:
milestone: grizzly-rc2 → 2013.1
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.