Must start quantum-dhcp-agent by hand first time after devstack install

[Syd Logan]

Steps to reproduce:

- Fresh install of Ubuntu 12.04 server (dual homed, eth0 is setup via DHCP, eth1 is configured with 192.168.4.0 net and has static IP 192.168.4.1)
- Download latest devstack
- Create the following localrc in the devstac directory:

HOST_IP=192.168.4.1
FLAT_INTERFACE=eth1
FIXED_RANGE=10.4.128.0/20
FIXED_NETWORK_SIZE=4096
FLOATING_RANGE=192.168.4.128/25
MULTI_HOST=True
Q_INTERFACE=eth1
LOGFILE=/opt/stack/logs/stack.sh.log
ADMIN_PASSWORD=password
MYSQL_PASSWORD=password
RABBIT_PASSWORD=password
SERVICE_PASSWORD=password
SERVICE_TOKEN=xyzpdqlazydog
LIBVIRT_FIREWALL_DRIVER=nova.virt.firewall.NoopFirewallDriver
ENABLED_SERVICES=g-api,g-reg,key,n-api,n-cpu,n-sch,n-vnc,mysql,rabbit,openstackx,q-svc,quantum,q-agt,n-cpu,q-dhcp
Q_PLUGIN=openvswitch

- sudo ./stack.sh

Expected results:

dnsmasq and quantum-dhcp-agent are running

Observed results:

Neither is running. But they can be started by hand, e.g.,

$ sudo python /opt/stack/quantum/quantum/plugins/openvswitch/agent/ovs_quantum_agent.py --config-file /etc/quantum/quantum.conf --config-file /etc/quantum/plugins/openvswitch/ovs_quantum_plugin.ini

[Syd Logan (slogan-r)]

Also, if I reboot the host, and re-run devstack, the services start without intervention, so this looks to be a start state sort of bug with respect to installation.

[dan wendlandt (danwent)]

what does the window labeled q-dhcp say? Is there an error there?

[Gary Kotton (garyk)]

Hi,
In the stackrc do you have q-dhcp defined?
Thanks
Gary

On 08/30/2012 04:06 AM, Syd Logan wrote:
> Also, if I reboot the host, and re-run devstack, the services start
> without intervention, so this looks to be a start state sort of bug with
> respect to installation.
>
> ** Summary changed:
>
> - Ubuntu 12.04: must start quantum-dhcp-agent by hand
> + Ubuntu 12.04: must start quantum-dhcp-agent by hand first time after devstack install
>
> ** Summary changed:
>
> - Ubuntu 12.04: must start quantum-dhcp-agent by hand first time after devstack install
> + Must start quantum-dhcp-agent by hand first time after devstack install
>

[Aaron Rosen (arosen)]

He posted that he has this in localrc:

ENABLED_SERVICES=g-api,g-reg,key,n-api,n-cpu,n-sch,n-vnc,mysql,rabbit,openstackx,q-svc,quantum,q-agt,n-cpu,q-dhcp

In either case I this bug should be against devstack and not quantum.

[dan wendlandt (danwent)]

yeah, i'm looking for any output (e.g., tracebacks) that may be in the window where devstack initially tried to start quantum-dhcp-agent.

[Gary Kotton (garyk)]

The dhcp screen output would be a great start.

[Syd Logan (slogan-r)]

Could I get a quick how to on capturing screen output from quantum-dhcp-agent?

[Aaron Rosen (arosen)]

screen -x stack and then control a + n over to q-dhcp.

I was able to reproduce this here's the output.

nicira@ubuntu:/opt/stack/quantum$ sudo python /opt/stack/quantum/bin/quantum-dhcp-agent --config-file /etc/quantum/quantum.conf --config-file=/etc/quantum/dhcp_agent.ini
2012-08-31 11:02:17 DEBUG [amqplib] Start from server, version: 8.0, properties: {u'information': 'Licensed under the MPL. See http://www.rabbitmq.com/', u'product': 'RabbitMQ', u'copyright': 'Copyright (C) 2007-2011 VMware, Inc.', u'capabilities': {}, u'platform': 'Erlang/OTP', u'version': '2.7.1'}, mechanisms: ['PLAIN', 'AMQPLAIN'], locales: ['en_US']
2012-08-31 11:02:17 DEBUG [amqplib] Open OK! known_hosts []
2012-08-31 11:02:17 DEBUG [amqplib] using channel_id: 1
2012-08-31 11:02:17 DEBUG [amqplib] Channel open
2012-08-31 11:02:17 INFO [quantum.openstack.common.rpc.common] Connected to AMQP server on localhost:5672
2012-08-31 11:02:17 DEBUG [quantum.openstack.common.rpc.amqp] Making asynchronous call on q-plugin ...
2012-08-31 11:02:17 DEBUG [quantum.openstack.common.rpc.amqp] MSG_ID is 13af1461000d4341beb1cc88496ce660
2012-08-31 11:02:17 DEBUG [quantum.openstack.common.rpc.amqp] Pool creating new connection
2012-08-31 11:02:17 DEBUG [amqplib] Start from server, version: 8.0, properties: {u'information': 'Licensed under the MPL. See http://www.rabbitmq.com/', u'product': 'RabbitMQ', u'copyright': 'Copyright (C) 2007-2011 VMware, Inc.', u'capabilities': {}, u'platform': 'Erlang/OTP', u'version': '2.7.1'}, mechanisms: ['PLAIN', 'AMQPLAIN'], locales: ['en_US']
2012-08-31 11:02:17 DEBUG [amqplib] Open OK! known_hosts []
2012-08-31 11:02:17 DEBUG [amqplib] using channel_id: 1
2012-08-31 11:02:17 DEBUG [amqplib] Channel open
2012-08-31 11:02:17 INFO [quantum.openstack.common.rpc.common] Connected to AMQP server on localhost:5672
2012-08-31 11:02:17 DEBUG [amqplib] Closed channel #1
2012-08-31 11:02:17 DEBUG [amqplib] using channel_id: 1
2012-08-31 11:02:17 DEBUG [amqplib] Channel open
Traceback (most recent call last):
  File "/opt/stack/quantum/bin/quantum-dhcp-agent", line 20, in <module>
    main()
  File "/opt/stack/quantum/quantum/agent/dhcp_agent.py", line 495, in main
    mgr.run()
  File "/opt/stack/quantum/quantum/agent/dhcp_agent.py", line 75, in run
    self.lease_relay.start()
  File "/opt/stack/quantum/quantum/agent/dhcp_agent.py", line 480, in start
    family=socket.AF_UNIX)
  File "/usr/local/lib/python2.7/dist-packages/eventlet/convenience.py", line 38, in listen
    sock.bind(addr)
  File "/usr/lib/python2.7/socket.py", line 224, in meth
    return getattr(self._sock,name)(*args)
socket.error: [Errno 2] No such file or directory

sudo python /opt/stack/quantum/bin/quantum-dhcp-agent --config-file /etc/quantum/quantum.conf --config-file=/etc/quantum/dhcp_agent.ini

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/12284

[Aaron Rosen (arosen)]

This occurs if no networks exist since. If networks exist it creates the directory /opt/stack/dhcp/... for the dnsmasq processes to store their data. Then it sets up the lease_relay which tries to write to /opt/stack/dhcp which wasn't created.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/12284
Committed: http://github.com/openstack/quantum/commit/3d089c43c460f89f564a4c0c69d468b27a2a6074
Submitter: Jenkins
Branch: master

commit 3d089c43c460f89f564a4c0c69d468b27a2a6074
Author: Aaron Rosen <email address hidden>
Date: Fri Aug 31 15:34:49 2012 -0700

    Quantum dhcp crashes if no networks exist

    Fixes bug 1043583

    Change-Id: Ibdfeb98c094b80ea63e79b23f14811e00525d96b