openstack provider should have config option to ignore invalid certs

[Kapil Thangavelu]

its very common with private openstack clouds to have invalid certs for the https api endpoints. Juju-core should support a config option on the env to allow for using these clouds.

$ juju status --debug -v
2013-07-17 11:50:09 INFO juju provider.go:117 environs/openstack: opening environment "openstack"
2013-07-17 11:50:09 ERROR juju supercommand.go:234 command failed: failed to GET object provider-state from container juju-2bbd5234abc3ad8e84445e487c6b0f76
caused by: failed executing the request https://mdyja.maas.example.com:8080/v1/AUTH_3f7edbf6d457496a8d16acb954cd727e/juju-2bbd5234abc3ad8e84445e487c6b0f76/provider-state
caused by: Get https://mdyja.maas.example.com:8080/v1/AUTH_3f7edbf6d457496a8d16acb954cd727e/juju-2bbd5234abc3ad8e84445e487c6b0f76/provider-state: x509: certificate is valid for CN, not mdyja.maas.example.com
error: failed to GET object provider-state from container juju-2bbd5234abc3ad8e84445e487c6b0f76
caused by: failed executing the request https://mdyja.maas.example.com:8080/v1/AUTH_3f7edbf6d457496a8d16acb954cd727e/juju-2bbd5234abc3ad8e84445e487c6b0f76/provider-state
caused by: Get https://mdyja.maas.example.com:8080/v1/AUTH_3f7edbf6d457496a8d16acb954cd727e/juju-2bbd5234abc3ad8e84445e487c6b0f76/provider-state: x509: certificate is valid for CN, not mdyja.maas.example.com

[Kapil Thangavelu (hazmat)]

<rogpeppe> you can configure the http stack to always ignore invalid certs, or ignore them for just a single request
<rogpeppe> for the former, you can set http.DefaultClient.Transport to &http.Transport{TLSClientConfig: tls.Config{InsecureSkipVerify: true, (maybe more here)}}
<rogpeppe> for the latter, you can create a new http.Client and call Do (or Get or whatever) on that

[John A Meinel (jameinel)]

Are these valid self-signed certs, or just completely invalid certificates?

[Kapil Thangavelu (hazmat)]

Self-signed by our openstack charms, but getting those ca certs in place for all the users of the cloud is a non trivial for usage. i think this is a higher priority as its going to effect our openstack deployments, which typically set the following pyjuju flag on the openstack env to combat this.
    ssl-hostname-verification: false

[Jonathan Davies (jpds)]

This bug is preventing me from using juju-core for our deployments.

[Dave Cheney (dave-cheney)]

Raising priority at the request of jpds who hit it during a customer deployment.

[Mark Ramm (mark-ramm)]

If this is a customer facing issue that works in pyjuju but not in core, we should resolve it as quickly as possible. Otherwise we are forcing users to choose between old unsupported juju, and does not yet work for me juju, which is definitely not a good thing.

[Martin Packman (gz)]

So, what we probably want here is to resurrect the ssl-hostname-verification environments.yaml config key, but now defaulted to "true" everywhere. Perhaps we only want to support setting it to false for Openstack even. When it's manually set to false, we should use InsecureSkipVerify for all http communication with cloud api endpoints:

<http://stackoverflow.com/questions/12122159/golang-how-to-do-a-https-request-with-bad-certificate>

The tricky part, is it's not just the state server that will be talking to object-store, as all machines will want to get tools and such like from there. So, the config will need to get propogated, and read, before tools are downloaded or anything else is fetched from swift containers.

[John A Meinel (jameinel)]

so to be clear, the specific hard part is that while we can easily configure and pass around "ssl-hostname-verification: false" to Goose code, we will *also* likely need to be passing that all the way down to the "wget https://.../juju-1.13.2-precise-amd64.tar.gz" call.

[John A Meinel (jameinel)]

The related branch allows insecure requests, but I haven't quite gotten everything wired up. The goose.Client code seems wired up reasonably well, but identity.Authorizer objects actually have their own goose/http.Client connections.

[John A Meinel (jameinel)]

I think the necessary changes to goose have already landed.

[John A Meinel (jameinel)]

Initial steps have landed, remaining steps split into bug #1234577 and bug #1234576