OpenStack datasource should not retry user-data on 404

Bug #1702160 reported by Chad Smith on 2017-07-03
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Chad Smith

Bug Description

When receiving a 404 from user_data attempts, cloud-init should not attempt to retry the errant user_data file as it is wasted effort. If the metadata service is already up, then any user-data should already be present, so a 404 at this time means that no user data exists so retries will not result in finding user-data at a later time.

Excerpt of logs indicating metadata is up but user-data does not exist:

2017-07-03 18:26:50,408 -[DEBUG]: Seeing if we can get any data from <class 'cloudinit.sources.DataSourceOpenStack.DataSourceOpenStack'>
2017-07-03 18:26:50,558 -[DEBUG]: [0/1] open '' with {'timeout': 10.0, 'method': 'GET', 'allow_redirects': True, 'headers': {'User-Agent': 'Cloud-Init/0.7.9'}, 'url': ''} configuration
2017-07-03 18:26:51,434 -[DEBUG]: Read from (200, 83b) after 1 attempts
2017-07-03 18:26:51,445 -[DEBUG]: [0/6] open '' with {'timeout': 10.0, 'method': 'GET', 'allow_redirects': True, 'headers': {'User-Agent': 'Cloud-Init/0.7.9'}, 'url': ''} configuration
2017-07-03 18:26:52,514 -[DEBUG]: Please wait 1 seconds while we wait to try again
...2017-07-03 18:26:53,957 -[DEBUG]: Please wait 1 seconds while we wait to try again
2017-07-03 18:26:59,989 -[DEBUG]: Failed reading optional path due to: 404 Client Error: Not Found for url:

Validated as well on cmdline minutes later:
--2017-07-03 18:51:33--
Connecting to connected.
HTTP request sent, awaiting response... 404 Not Found
2017-07-03 18:51:34 ERROR 404: Not Found.

Chad Smith (chad.smith) on 2017-07-03
description: updated
Chad Smith (chad.smith) on 2017-07-03
summary: - OpenStack datasource should not retry user-data on 404s
+ OpenStack datasource should not retry user-data on 404
Chad Smith (chad.smith) on 2017-07-03
Changed in cloud-init:
importance: Undecided → Medium
assignee: nobody → Chad Smith (chad.smith)
status: New → In Progress
Kurt Garloff (kgarloff) wrote :

sources/helpers/ has the exception_cb logic reversed.

Attached patch fixes it.

Note: I special cased 'meta_data.json' as this is required and we might want to have retries for this file even though we have a response code >= 400. This is not because I believe that this is the best possible approach, but because there might be environments out there that relied on the unintended retrials and I want to avoid this patch possibly causing a regression anywhere.

Patch is against 0.7.9 from Xenial (where I tested this), let me know if you need this rediffed against some current git tree.

Kurt Garloff (kgarloff) wrote :

Two more notes:
1. This saves up to 18s boot time (when no user_data, network_data.json and vendor_data.json is provided)
2. The logic in url_helper.readurl() was different before, so one might argue that it should be reverted back there and the helper/ logic be left as is ... Would need to check and to ensure we don't break anything. I did not go through the commit logs to understand what the author intended with this reversion and whether it was intentional.

Scott Moser (smoser) wrote :

Thanks for the patch.
In order to accept it you will need to sign the Canonical Contributors Agreement, see for more information.

Also, a git merge proposal is ideal rather than a patch on the bug (see also in HACKING)


To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers