openstack: looping over simplestream urls stops too soon
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
juju-core |
Fix Released
|
High
|
Ian Booth |
Bug Description
From what I understood, there is a list for simplestream URL candidates that the code loops over:
- start off with the juju tools url
- append from product-streams if such a service was found in keystone catalog
- append the default cloud-images.
If you start clean, with nothing in swift, this loop will stop early because swift returns a 401 for the first candidate when there is nothing there, and the code only moves on to the next one in the list if the error is environs.
for _, baseURL := range baseURLs {
indexRef, err := getIndexWithFor
if err != nil {
if environs.
}
return nil, err
}
I think it should also consider the 401 HTTP error case and move on to the next URL in the list in that case. Or maybe publicStorage() from openstack/
// If there is a public bucket URL defined, create a public storage using that,
// otherwise create the public bucket using the user's credentials on the authenticated client.
publicBucketURL := e.publicBucketURL()
if publicBucketURL == "" {
// this is possibly just a hack - if the ACL is swift.Private,
// the machine won't be able to get the tools (401 error)
swift: swift.New(
Here is a sample run:
- swift is made sure to be empty:
$ swift list
juju-dist
$ swift delete juju-dist
streams/v1
streams/
streams/
Ok, let's bootstrap:
$ juju bootstrap -v --upload-tools
2013/05/28 17:57:19 INFO environs/openstack: opening environment "canonistack1"
2013/05/28 17:57:22 INFO environs/tools: built 1.11.1.
2013/05/28 17:57:22 INFO environs/tools: uploading 1.11.1.
2013/05/28 17:57:47 INFO environs/tools: uploading 1.11.1.
2013/05/28 17:58:09 INFO environs/openstack: bootstrapping environment "canonistack1"
2013/05/28 17:58:19 INFO environs: reading tools with major version 1
2013/05/28 17:58:19 INFO environs: filtering tools by series: precise
2013/05/28 17:58:19 INFO environs: filtering tools by version: 1.11.1.1
2013/05/28 17:58:21 ERROR command failed: cannot start bootstrap instance: cannot read index data, cannot access URL "https:/
error: cannot start bootstrap instance: cannot read index data, cannot access URL "https:/
Unauthorized. See how it only tried the first candidate, the URL derived from the tools.
Bootstrap failed, and I only have the tools now (that's my control bucket):
$ swift list
juju-51e360xxxxx
Let's try again, creating "juju-dist" this time:
$ swift post juju-dist
$ swift list
juju-51e360xxxx
juju-dist
$ juju bootstrap -v --upload-tools
2013/05/28 18:08:46 INFO environs/openstack: opening environment "canonistack1"
2013/05/28 18:08:49 INFO environs/tools: built 1.11.1.
2013/05/28 18:08:49 INFO environs/tools: uploading 1.11.1.
2013/05/28 18:09:13 INFO environs/tools: uploading 1.11.1.
2013/05/28 18:09:35 INFO environs/openstack: bootstrapping environment "canonistack1"
2013/05/28 18:09:45 INFO environs: reading tools with major version 1
2013/05/28 18:09:46 INFO environs: filtering tools by series: precise
2013/05/28 18:09:46 INFO environs: filtering tools by version: 1.11.1.1
2013/05/28 18:09:48 ERROR command failed: cannot start bootstrap instance: cannot read index data, cannot access URL "https:/
error: cannot start bootstrap instance: cannot read index data, cannot access URL "https:/
Again unauthorized. Let's fix the container ACLs:
$ swift post juju-dist -r '.r:*'
$ juju bootstrap -v --upload-tools
2013/05/28 18:11:12 INFO environs/openstack: opening environment "canonistack1"
2013/05/28 18:11:15 INFO environs/tools: built 1.11.1.
2013/05/28 18:11:15 INFO environs/tools: uploading 1.11.1.
2013/05/28 18:11:39 INFO environs/tools: uploading 1.11.1.
2013/05/28 18:12:02 INFO environs/openstack: bootstrapping environment "canonistack1"
2013/05/28 18:12:12 INFO environs: reading tools with major version 1
2013/05/28 18:12:12 INFO environs: filtering tools by series: precise
2013/05/28 18:12:12 INFO environs: filtering tools by version: 1.11.1.1
2013/05/28 18:12:26 INFO environs/openstack: started instance "f6d260c8-
2013/05/28 18:12:28 INFO command finished
Now it worked:
$ swift list
juju-51e360f7f9
juju-dist
The juju-dist container is still empty:
$ swift list juju-dist
$
Meaning that the code finally saw the NotFoundError that made it try the next URL in the list, which comes from the "product-stream" keystone service:
$ keystone catalog
(...)
Service: product-streams
+------
| Property | Value |
+------
| adminURL | https:/
| id | db3cxxxxxxxxxxx
| internalURL | https:/
| publicURL | https:/
| region | xxxx1 |
+------
description: | updated |
Changed in juju-core: | |
status: | New → In Progress |
assignee: | nobody → Ian Booth (wallyworld) |
importance: | Undecided → High |
Changed in juju-core: | |
status: | In Progress → Fix Committed |
Changed in juju-core: | |
milestone: | none → 1.11.1 |
Changed in juju-core: | |
status: | Fix Committed → Fix Released |
tags: | added: landscape |
I think there's some confusion around the public bucket behaviour. If a public bucket url is defined, the code assumes the bucket has been created and set up by the cloud administrator and so the code merely creates a storage client to access the already created bucket. That's what this part of the comment is referring to:
// If there is a public bucket URL defined, create a public storage using that,
If there's no public bucket url defined, the code creates a so called "public" bucket on the user's account and calls the bucket whatever the public bucket name is configured to be (default "juju-dist"). That's what this part of the comment is referring to:
// otherwise create the public bucket using the user's credentials on the
So AFAICT, this part of the code is behaving as expected.