| 2014-06-04 11:08:19 |
Derek Higgins |
description |
Failure started between 1 and 2 AM UTC
Exception while spawning instances
2014-06-04 08:24:23.468 2985 ERROR neutron.agent.dhcp_agent [req-8211911f-5281-460c-8ccf-6407b00f7296 None] Failed reporting state!
2014-06-04 08:24:23.468 2985 TRACE neutron.agent.dhcp_agent Traceback (most recent call last):
2014-06-04 08:24:23.468 2985 TRACE neutron.agent.dhcp_agent File "/opt/stack/venvs/openstack/lib/python2.7/site-packages/neutron/agent/dhcp_agent.py", line 565, in _report_state
2014-06-04 08:24:23.468 2985 TRACE neutron.agent.dhcp_agent self.state_rpc.report_state(ctx, self.agent_state, self.use_call)
2014-06-04 08:24:23.468 2985 TRACE neutron.agent.dhcp_agent File "/opt/stack/venvs/openstack/lib/python2.7/site-packages/neutron/agent/rpc.py", line 72, in report_state
2014-06-04 08:24:23.468 2985 TRACE neutron.agent.dhcp_agent return self.call(context, msg, topic=self.topic)
2014-06-04 08:24:23.468 2985 TRACE neutron.agent.dhcp_agent File "/opt/stack/venvs/openstack/lib/python2.7/site-packages/neutron/openstack/common/rpc/proxy.py", line 129, in call
2014-06-04 08:24:23.468 2985 TRACE neutron.agent.dhcp_agent exc.info, real_topic, msg.get('method'))
2014-06-04 08:24:23.468 2985 TRACE neutron.agent.dhcp_agent Timeout: Timeout while waiting on RPC response - topic: "q-plugin", RPC method: "report_state" info: "<unknown>"
2014-06-04 08:24:23.468 2985 TRACE neutron.agent.dhcp_agent
2014-06-04 08:24:23.475 2985 WARNING neutron.openstack.common.loopingcall [-] task run outlasted interval by 30.586129 sec
neutron : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/bin/neutron-rootwrap /etc/neutron/rootwrap.conf ip netns exec qdhcp-2484b872-21cf-4daf-980a-9e4e8003f363 ip link set tap76ef60c2-73 up
2014-06-04 08:24:23.660 2985 ERROR neutron.agent.linux.utils [req-eccd15d4-39b3-4ea7-af59-2a71077850f2 None]
Command: ['sudo', '/usr/bin/neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qdhcp-2484b872-21cf-4daf-980a-9e4e8003f363', 'ip', 'link', 'set', 'tap76ef60c2-73', 'up']
Exit code: 1
Stdout: ''
Stderr: 'Cannot open network namespace "qdhcp-2484b872-21cf-4daf-980a-9e4e8003f363": No such file or directory\n' |
Failure started between 1 and 2 AM UTC
Running nova in debug mode shows the problem
Jun 04 09:15:55 localhost nova-scheduler[9605]: 2014-06-04 09:15:55.259 9605 DEBUG nova.filters [req-c37d26da-66de-4658-ba6f-a06a775f1a28 None] Filter ComputeFilter returned 1 host(s) get_filtered_objects /opt/stack/venvs/nova/lib/python2.7/site-packages/nova/filters.py:88
Jun 04 09:15:55 localhost nova-scheduler[9605]: 2014-06-04 09:15:55.259 9605 DEBUG nova.scheduler.filters.compute_capabilities_filter [req-c37d26da-66de-4658-ba6f-a06a775f1a28 None] (seed, 8f3d2259-ef0b-44fc-a0c4-4d5cc2ef1443) ram:3072 disk:40960 io_ops:0 instances:0 fails instance_type extra_specs requirements host_passes /opt/stack/venvs/nova/lib/python2.7/site-packages/nova/scheduler/filters/compute_capabilities_filter.py:72
Jun 04 09:15:55 localhost nova-scheduler[9605]: 2014-06-04 09:15:55.260 9605 INFO nova.filters [req-c37d26da-66de-4658-ba6f-a06a775f1a28 None] Filter ComputeCapabilitiesFilter returned 0 hosts |
|
