nova booted with neutron fails to obtain dhcp lease

Bug #1444219 reported by Steven Dake
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
kolla
Fix Released
Critical
Steven Dake

Bug Description

Console log:

http://ur1.ca/k6d3r -> http://paste.fedoraproject.org/211045/90601101

Script I ran to configure:

http://ur1.ca/k6d3z -> http://paste.fedoraproject.org/211046/06019914

Note I had to apply two patches:
https://review.openstack.org/#/c/173590/
https://review.openstack.org/#/c/173627/

Neutron works pretty well, can create ports etc was able to associate a floating ip with the machine, its just that its network interface doesn't come up, so the float doesn't work.

Steven Dake (sdake)
Changed in kolla:
status: New → Triaged
importance: Undecided → Critical
assignee: nobody → Daneyon Hansen (danehans)
Revision history for this message
Daneyon Hansen (danehans) wrote :

Steve,

Thanks for the information. I believe we are experiencing the same issue. As you can see in your console log, the instance is not getting an IP from the dhcp agent and therefore can not reach metadata, work with a floating ip, etc.. Here is what you can do to work around the issue:

1. Stop and remove your neutron server and agents containers.
2. Drop the neutron DB in your mariadbapp container.
3. Start the neutron-server container. Look at the container logs and you should see that the DB gets recreated.
4. Make sure neutron-server is running by issuing a neutron ext-list and you should see a list of extensions loaded.
5. Start the agents container. Look at the logs and you should see all the agents connect to amqp.
6. The agents container is functioning when you can do a neutron agent-list and see a :-) next to each agent.

Now go ahead and run your neutron setup script. When the VM is active, view the console log and you should see that it obtains an IP from DHCP and successfully pulls metadata.

Everything is great until you stop and restart your agents container. When the container is restarted, the agents can no longer access the network namespaces that were created. We need to figure out why the namespaces are no longer available to the container after the restart.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to kolla (master)

Reviewed: https://review.openstack.org/175754
Committed: https://git.openstack.org/cgit/stackforge/kolla/commit/?id=17fc2d9b475da15a571fe6ed1aa51fadc3b076cc
Submitter: Jenkins
Branch: master

commit 17fc2d9b475da15a571fe6ed1aa51fadc3b076cc
Author: Daneyon Hansen <email address hidden>
Date: Tue Apr 21 07:37:32 2015 +0000

    Removes existing network namespaces for neutron-agents container

    Previously, network namespaces would not work correctly when
    restarting or stop/rm/start the neutron-agents container. This
    is because network namespaces created within the container are
    associated to the container's pid: /pid/$CONTAINER_PID/ns/net.
    With this patch, when a container starts or restarts, the l3/dhcp
    agent start scripts will remove any existing qrouter/qdhcp network
    namespaces before starting the agents. The agents will recreate the
    necessary network namespaces since they are stored in the db.

    Closes-Bug: 1444219

    Change-Id: Ia86729766fe8c2fc145b3a02d519746b149a73bb

Changed in kolla:
status: Triaged → Fix Committed
Steven Dake (sdake)
no longer affects: kolla/milestone-4
Changed in kolla:
milestone: none → liberty-1
Sam Yaple (s8m)
Changed in kolla:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.