overcloud metadata requests hit seed

Bug #1241042 reported by Derek Higgins
12
This bug affects 2 people
Affects Status Importance Assigned to Milestone
tripleo
Fix Released
Critical
Roman Podoliaka

Bug Description

On a default devtest run (tested on Fedora) requests from the overcloud to http://169.254.169.254 are hitting the seed vm and not the undercloud and they should be.

Derek Higgins (derekh)
Changed in tripleo:
importance: Undecided → Critical
Revision history for this message
Derek Higgins (derekh) wrote :

I've tried the attached patch

Note I had to put in a sleep before the call to "neutron agent-list" in order to wait for the DHCP agent to become available, then when
neutron dhcp-agent-network-add 679d2a45-c48b-4a5d-9e5d-652114b48f0d ctlplane

ran it failed with the error
409-{u'NeutronError': {u'message': u'The network 9257ab5f-a3dd-47d9-abc9-c1970d485eb6 has been already hosted by the DHCP Agent 679d2a45-c48b-4a5d-9e5d-652114b48f0d.', u'type': u'NetworkHostedByDHCPAgent', u'detail': u''}}

Finally when the overcloud booted up it couldn't connect to a PXE server.

Changed in tripleo:
status: New → Triaged
Revision history for this message
Robert Collins (lifeless) wrote :

That has file injection set to true, but it should be false :) The neutron sleep thing probably needs to be a separate patch.

Revision history for this message
Roman Podoliaka (rpodolyaka) wrote :

I'm hitting this with Ubuntu 13.04 too. Going to look at tcpdump logs.

Revision history for this message
Roman Podoliaka (rpodolyaka) wrote :

Ooops. I was wrong. Works for me with Ubuntu 13.04 when use_file_injection is set to False.

The tricky thing is that:
      {{#use_file_injection}}use_file_injection={{use_file_injection}}{{/use_file_injection}}
evaluates to nothing in nova config file when use_file_injection is set to False.

At the same time, in Nova use_file_injection defaults to True, and this is not what we need here to make host_routes work.

Should we change the default value in Nova or there is some way to make {{#}} conditionals work with false values?

Revision history for this message
Derek Higgins (derekh) wrote :

Ahh so basically nova.baremetal.use_file_injection is getting morphed into a Boolean instead of a string, I'll test to see what happens if wrapped in quotes.

Revision history for this message
Derek Higgins (derekh) wrote :

Attaching second attempt, this didn't work either both overcloud nodes went to Error state, retrying and then will try to find out why.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tripleo-incubator (master)

Fix proposed to branch: master
Review: https://review.openstack.org/53325

Changed in tripleo:
assignee: nobody → Roman Podolyaka (rpodolyaka)
status: Triaged → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Fix proposed to branch: master
Review: https://review.openstack.org/53331

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-incubator (master)

Reviewed: https://review.openstack.org/53331
Committed: http://github.com/openstack/tripleo-incubator/commit/af22d19c6b534231e29b2a54996c2ba107bc0b7f
Submitter: Jenkins
Branch: master

commit af22d19c6b534231e29b2a54996c2ba107bc0b7f
Author: Roman Podolyaka <email address hidden>
Date: Wed Oct 23 14:58:06 2013 +0300

    Ensure the network is scheduled to an dhcp-agent

    For some reason DHCP-agent is not present in neutron
    agent-list output immediately after start, so we have
    to wait a bit.

    By the time we are going to schedule the neutron network
    to the dhcp agent it might have already been scheduled,
    so we should be able to handle this case too.

    Partial-Bug: #1241042
    Related-Bug: #1231366

    Change-Id: Ib40183a21e6ca99fa46e5b94782535cad980eec2

Changed in tripleo:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Reviewed: https://review.openstack.org/53325
Committed: http://github.com/openstack/tripleo-incubator/commit/347bd58498fd8cd6806e7546bfdd8894f9c51f34
Submitter: Jenkins
Branch: master

commit 347bd58498fd8cd6806e7546bfdd8894f9c51f34
Author: Roman Podolyaka <email address hidden>
Date: Wed Oct 23 14:43:01 2013 +0300

    Use neutron-dhcp-agent dhcp driver by default

    We are now relying on neutron-dhcp-agent features
    like host routes, so we must use neutron-dhcp-agent
    in undercloud instead of bm-dnsmasq in order to
    deploy overcloud properly.

    Co-Authored-By: Derek Higgins <email address hidden>

    Closes-Bug: #1241042

    Change-Id: I71b5efa72eaddb3ec07eda25336f421393635c1a

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.