tripleo

Upgrade of overcloud failed with "Connection to neutron failed: Maximum attempts reached"

Bug #1315355 reported by Cian O'Driscoll
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
Expired
Medium
Unassigned

Bug Description

nova list
+--------------------------------------+-------------------------------------+--------+------------+-------------+---------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+-------------------------------------+--------+------------+-------------+---------------------+
| f3348009-f5c7-43b8-9083-deb792dc0596 | overcloud-NovaCompute0-jffpsf77j3de | ACTIVE | - | Running | ctlplane=192.0.2.37 |
| 736338c8-b1ac-40e5-946d-2afdabf2160e | overcloud-NovaCompute1-glax3c2dkbt4 | ACTIVE | - | Running | ctlplane=192.0.2.38 |
| 2e265fe0-29bb-42b9-872d-d2a4308b72c2 | overcloud-controller0-opwusydrzngd | ERROR | - | Running | ctlplane=192.0.2.39 |
+--------------------------------------+-------------------------------------+--------+------------+-------------+---------------------+

heat resource-list overcloud

......
| NovaCompute0 | OS::Nova::Server | UPDATE_IN_PROGRESS | 2014-05-02T11:33:35Z |
| controller0 | OS::Nova::Server | UPDATE_FAILED | 2014-05-02T11:33:37Z |
| NovaCompute1 | OS::Nova::Server | UPDATE_IN_PROGRESS | 2014-05-02T11:33:38Z |
+--------------------------+--------------------------------+--------------------+----------------------+

2014-05-02 11:33:43.257 4913 ERROR nova.compute.manager [req-a53c4ad1-94ac-4844-bcaf-ff67bc0cf47e None] [instance: 2e265fe0-29bb-42b9-872d-d2a4308b72c2] Setting instance vm_state to ERROR
2014-05-02 11:33:43.257 4913 TRACE nova.compute.manager [instance: 2e265fe0-29bb-42b9-872d-d2a4308b72c2] Traceback (most recent call last):
2014-05-02 11:33:43.257 4913 TRACE nova.compute.manager [instance: 2e265fe0-29bb-42b9-872d-d2a4308b72c2] File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/compute/manager.py", line 5575, in _error_out_instance_on_exception
2014-05-02 11:33:43.257 4913 TRACE nova.compute.manager [instance: 2e265fe0-29bb-42b9-872d-d2a4308b72c2] yield
2014-05-02 11:33:43.257 4913 TRACE nova.compute.manager [instance: 2e265fe0-29bb-42b9-872d-d2a4308b72c2] File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/compute/manager.py", line 2516, in rebuild_instance
2014-05-02 11:33:43.257 4913 TRACE nova.compute.manager [instance: 2e265fe0-29bb-42b9-872d-d2a4308b72c2] network_info = self._get_instance_nw_info(context, instance)
2014-05-02 11:33:43.257 4913 TRACE nova.compute.manager [instance: 2e265fe0-29bb-42b9-872d-d2a4308b72c2] File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/compute/manager.py", line 1153, in _get_instance_nw_info
2014-05-02 11:33:43.257 4913 TRACE nova.compute.manager [instance: 2e265fe0-29bb-42b9-872d-d2a4308b72c2] instance)
2014-05-02 11:33:43.257 4913 TRACE nova.compute.manager [instance: 2e265fe0-29bb-42b9-872d-d2a4308b72c2] File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 466, in get_instance_nw_info
2014-05-02 11:33:43.257 4913 TRACE nova.compute.manager [instance: 2e265fe0-29bb-42b9-872d-d2a4308b72c2] port_ids)
2014-05-02 11:33:43.257 4913 TRACE nova.compute.manager [instance: 2e265fe0-29bb-42b9-872d-d2a4308b72c2] File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 477, in _get_instance_nw_info
2014-05-02 11:33:43.257 4913 TRACE nova.compute.manager [instance: 2e265fe0-29bb-42b9-872d-d2a4308b72c2] port_ids)
2014-05-02 11:33:43.257 4913 TRACE nova.compute.manager [instance: 2e265fe0-29bb-42b9-872d-d2a4308b72c2] File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 1104, in _build_network_info_model
2014-05-02 11:33:43.257 4913 TRACE nova.compute.manager [instance: 2e265fe0-29bb-42b9-872d-d2a4308b72c2] data = client.list_ports(**search_opts)
2014-05-02 11:33:43.257 4913 TRACE nova.compute.manager [instance: 2e265fe0-29bb-42b9-872d-d2a4308b72c2] File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 111, in with_params
2014-05-02 11:33:43.257 4913 TRACE nova.compute.manager [instance: 2e265fe0-29bb-42b9-872d-d2a4308b72c2] ret = self.function(instance, *args, **kwargs)
2014-05-02 11:33:43.257 4913 TRACE nova.compute.manager [instance: 2e265fe0-29bb-42b9-872d-d2a4308b72c2] File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 306, in list_ports
2014-05-02 11:33:43.257 4913 TRACE nova.compute.manager [instance: 2e265fe0-29bb-42b9-872d-d2a4308b72c2] **_params)
2014-05-02 11:33:43.257 4913 TRACE nova.compute.manager [instance: 2e265fe0-29bb-42b9-872d-d2a4308b72c2] File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 1250, in list
2014-05-02 11:33:43.257 4913 TRACE nova.compute.manager [instance: 2e265fe0-29bb-42b9-872d-d2a4308b72c2] for r in self._pagination(collection, path, **params):
2014-05-02 11:33:43.257 4913 TRACE nova.compute.manager [instance: 2e265fe0-29bb-42b9-872d-d2a4308b72c2] File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 1263, in _pagination
2014-05-02 11:33:43.257 4913 TRACE nova.compute.manager [instance: 2e265fe0-29bb-42b9-872d-d2a4308b72c2] res = self.get(path, params=params)
2014-05-02 11:33:43.257 4913 TRACE nova.compute.manager [instance: 2e265fe0-29bb-42b9-872d-d2a4308b72c2] File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 1236, in get
2014-05-02 11:33:43.257 4913 TRACE nova.compute.manager [instance: 2e265fe0-29bb-42b9-872d-d2a4308b72c2] headers=headers, params=params)
2014-05-02 11:33:43.257 4913 TRACE nova.compute.manager [instance: 2e265fe0-29bb-42b9-872d-d2a4308b72c2] File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 1228, in retry_request
2014-05-02 11:33:43.257 4913 TRACE nova.compute.manager [instance: 2e265fe0-29bb-42b9-872d-d2a4308b72c2] raise exceptions.ConnectionFailed(reason=_("Maximum attempts reached"))
2014-05-02 11:33:43.257 4913 TRACE nova.compute.manager [instance: 2e265fe0-29bb-42b9-872d-d2a4308b72c2] ConnectionFailed: Connection to neutron failed: Maximum attempts reached

Revision history for this message
Ladislav Smola (lsmola) wrote :

What do you mean by 'Upgrade of overcloud', what action is that. heat stack-update ?

Revision history for this message
Steve Kowalik (stevenk) wrote :

I think there might be more clues in the neutron log, but I'm not sure. How reproducible is this?

Changed in tripleo:
status: New → Incomplete
importance: Undecided → Medium
Revision history for this message
Launchpad Janitor (janitor) wrote :

[Expired for tripleo because there has been no activity for 60 days.]

Changed in tripleo:
status: Incomplete → Expired
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.