Failed to attach vif after container is rebooted on dashboard.

Bug #1516510 reported by Xiao Li Xu
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
nova-docker
New
Undecided
Unassigned

Bug Description

Setup info:
1 controller node + 2 network nodes + 6 compute nodes

Reproduce steps:
1. Create 10 tenants, 6 containers in each tenant.
2. Associate floatingip to 60 containers. All containers can be accessed via floatingip.
3. On dashboard, restart all these 60 containers tenant by tenant.
4. After containers are up, try to access all containers through floatingip. 4 of them can't be accessed.
Login these 4 containers on compute nodes, found that there's no network interface attached to these 4 containes. Take one container of example:
root@quasarsdnve1:~# docker exec -it 3ed3deaa8697 bash
root@instance-00001d14:/# ifconfig
lo Link encap:Local Loopback
          inet addr:127.0.0.1 Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING MTU:65536 Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)

root@instance-00001d14:/#
 <<<<<<<<< only loopback interface, no network interface.

Check nova log, it says " Failed to attach vif":
2015-11-13 01:25:56.087 37642 INFO nova.compute.manager [req-9c5bc8d2-50e7-4227-baea-816dc1c8ef09 ee03468688eb4339ac41b4c185f243ee 86c4f04df63b44e484d3927355101f59 - - -] [instance: cdb312d4-e269-4de2-a41c-23aa996227d3] Rebooting instance
2015-11-13 01:25:59.076 37642 ERROR novadocker.virt.docker.vifs [req-9c5bc8d2-50e7-4227-baea-816dc1c8ef09 ee03468688eb4339ac41b4c185f243ee 86c4f04df63b44e484d3927355101f59 - - -] Failed to attach vif
2015-11-13 01:25:59.076 37642 TRACE novadocker.virt.docker.vifs Traceback (most recent call last):
2015-11-13 01:25:59.076 37642 TRACE novadocker.virt.docker.vifs File "/usr/local/lib/python2.7/dist-packages/novadocker/virt/docker/vifs.py", line 418, in attach
2015-11-13 01:25:59.076 37642 TRACE novadocker.virt.docker.vifs container_id, run_as_root=True)
2015-11-13 01:25:59.076 37642 TRACE novadocker.virt.docker.vifs File "/usr/lib/python2.7/dist-packages/nova/utils.py", line 207, in execute
2015-11-13 01:25:59.076 37642 TRACE novadocker.virt.docker.vifs return processutils.execute(*cmd, **kwargs)
2015-11-13 01:25:59.076 37642 TRACE novadocker.virt.docker.vifs File "/usr/local/lib/python2.7/dist-packages/oslo_concurrency/processutils.py", line 295, in execute
2015-11-13 01:25:59.076 37642 TRACE novadocker.virt.docker.vifs cmd=sanitized_cmd)
2015-11-13 01:25:59.076 37642 TRACE novadocker.virt.docker.vifs ProcessExecutionError: Unexpected error while running command.
2015-11-13 01:25:59.076 37642 TRACE novadocker.virt.docker.vifs Command: sudo nova-rootwrap /etc/nova/rootwrap.conf ip link set ns242e07cf-c4 netns 3ed3deaa8697a017c983a6c7af71d5f355027740efb50d6545d59c5e353336b0
2015-11-13 01:25:59.076 37642 TRACE novadocker.virt.docker.vifs Exit code: 1
2015-11-13 01:25:59.076 37642 TRACE novadocker.virt.docker.vifs Stdout: u''
2015-11-13 01:25:59.076 37642 TRACE novadocker.virt.docker.vifs Stderr: u'Cannot find device "ns242e07cf-c4"\n'
2015-11-13 01:25:59.076 37642 TRACE novadocker.virt.docker.vifs

Tags: nova-docker
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.