Cloud-init randomly fail to get nova metadata

Bug #1766824 reported by Damien Ciabrini
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
novajoin
Fix Released
Undecided
Damien Ciabrini

Bug Description

From time to time it seems that we experience contention on novajoin, which makes cloud-init fail to retrieve metadata when doing:

curl -s http://169.254.169.254/openstack/2016-10-06/vendor_data2.json

To workaround the chances of contention we currently have a random sleep before that call so lower the odds that many nodes call novajoin at the same time.

This random sleep reduces the failure rate at deploy time but we can still experience it fairly often.

Changed in novajoin:
assignee: nobody → Damien Ciabrini (dciabrin)
description: updated
Changed in novajoin:
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to novajoin (master)

Reviewed: https://review.openstack.org/564137
Committed: https://git.openstack.org/cgit/openstack/novajoin/commit/?id=ed5de4f1383bda69464fccd08b512088f72bd8d7
Submitter: Zuul
Branch: master

commit ed5de4f1383bda69464fccd08b512088f72bd8d7
Author: Damien Ciabrini <email address hidden>
Date: Wed Apr 25 10:53:02 2018 +0200

    Add a retry loop when fetching vendor_data from novajoin

    In [1] we added a random sleep in the script called by cloud-init
    to fetch vendor_data, to lower chances that many nodes connect to
    novajoin concurrently. Still, curl failures still happen from
    time to time.

    Add a retry loop around the call to curl to further increase the
    chances of success, as this seems to give good results in practice
    when deploying overclouds with tripleo.

    [1] 188f09ef22a233323464fcb0a8c6942d6981032a

    Closes-Bug: #1766824

    Change-Id: I728c2cb0c8a7433b68dd7de2de242e922974d713

Changed in novajoin:
status: In Progress → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.