manual provider client cache prevents reuse of env by name

[Kapil Thangavelu]

so i had failed manual bootstrap due to bug https://bugs.launchpad.net/juju-core/+bug/1238934 .. I went to try it with a more recent release (raring instead of precise). Changing the bootstrap-host ip address and re-bootstrapping will bomb out with the previously configure bootstrap host's ip address and a check against that.

2013-10-11 20:00:59 INFO juju.environs.boostrap bootstrap.go:68 environs: picked newest version: 1.17.0.1
2013-10-11 20:00:59 INFO juju.environs.manual detection.go:27 Checking if root@162.243.42.177 is already provisioned
2013-10-11 20:00:59 INFO juju.environs.manual detection.go:41 root@162.243.42.177 is already provisioned ["juju-db.conf"]
2013-10-11 20:00:59 ERROR juju supercommand.go:282 machine is already provisioned

And you can't destroy-environment on a manual env to reset the client side state/cache.

[Kapil Thangavelu (hazmat)]

deleting the previous instance still results in an attempt to reset/clear the old instance.. but if the instance is gone it just hangs for 3m before failing.

2013-10-11 20:09:58 DEBUG juju.environs open.go:75 ConfigForName found bootstrap config map[string]interface {}{"admin-secret":"b6d86b997f8a80f37bc96fda8941e0fc", "api-port":17070, "default-series":"precise", "firewall-mode":"instance", "ssl-hostname-verification":true, "tools-url":"", "bootstrap-host":"162.243.42.177", "image-metadata-url":"", "storage-auth-key":"fa32204f792c74a7a29ab59ac3ce3226", "type":"null", "bootstrap-user":"root", "development":false, "logging-config":"<root>=DEBUG", "name":"null", "state-port":37017, "authorized-keys":"ssh-dss A ender@linux\n", "ca-cert":"-----BEGIN CERTIFICATE-----\nMIICVzCCAcKgAwIBAgIBADALBgkqhkiG9w0BAQUwQjENMAsGA1UEChMEanVqdTEx\nMC8GA1UEAwwoanVqdS1nZW5lcmF0ZWQgQ0EgZm9yIGVudmlyb25tZW50ICJudWxs\nIjAeFw0xMzEwMTExODU3NDFaFw0yMzEwMTExOTAyNDFaMEIxDTALBgNVBAoTpUDqFfegzQAUjdk0J1P0a/jG3NWdMQbdxUnflFhY\nGPWfyO3ym9csHnluXtn/o9Y6N75O5pCtbNwygHVfAXc4J5J2VkWJGQSGNDI+bbqC\nMyAiFhlb6akJHwDGGnHjOpcDO3s+we8q8FaL\n-----END CERTIFICATE-----\n", "ca-private-key":"-----BEGIN RSA PRIVATE KEY-----qY0+iguC0cbXmTQH4wLHC\nP2KIvQOSit6S2POEylPpOCynfEeKWGxRjZI2twr35oOBIhEx0axZMSVTvycdAkBR\n6GObG4Kud+bQCxAcg5QpiRGSv/Z2fpAA/NurbuegBVXlK7jx9PwzQN9immxJPVJI\nuG6DU7geib/6wwfAe4uFAkAudIW8Xo3miJzjp2jBVZEkj6lbiF3w2vSnkuUhIKul\n4K4TM7ZKD0jnb7WbWSrYy6NppYQI9eJXY19s/9M45ibm\n-----END RSA PRIVATE KEY-----\n"}
2013-10-11 20:09:58 DEBUG juju.environs.configstore disk.go:77 Making /opt/juju/environments
2013-10-11 20:09:58 INFO juju.environs open.go:156 environment info already exists; using New not Prepare
2013-10-11 20:12:05 ERROR juju supercommand.go:282 failed to enable bootstrap storage: failed to create storage dir: exit status 255 (ssh: connect to host 162.243.42.177 port 22: Connection timed out)

[Kapil Thangavelu (hazmat)]

clearing out JUJU_HOME/environments/null.jenv does clear the client cache and allow the procedure to proceed. There should be a sanity check though within juju that the two addresses match before using the cache details.

[Andrew Wilkins (axwalk)]

This is a bigger problem than just the null/manual provider. I logged lp:1239550 yesterday, after helping bigjools diagnose a problem with the MAAS provider. If you change the oauth token after bootstrapping, it's ignored.

Of course, the problem is exacerbated due to the lack of destroy-environment...

[Andrew Wilkins (axwalk)]

Really the only sane way of *fixing* this is to implement destroy-environment. For a first step, I'm going to work on resolving lp:1239550. My intention is to make bootstrap fail if environments.yaml has been changed, with a message about how to fix it (manually, since it was modified manually).

[Andrew Wilkins (axwalk)]

I have an MP awaiting review for lp:1239550 that will warn about changes to environments.yaml after bootstrapping/preparing an environment.

I'll look at this now, though I think it's unlikely to get implemented for 1.17.0, assuming that's happening within a couple of weeks.

[Andrew Wilkins (axwalk)]

I have the beginnings of something working: a new "worker" in jujud that waits for SIGABRT, and uninstalls the machine agent. I've updated the uninstall code to also remove the data-dir, and I'll have to do something about stopping/removing mongo.

Apart from that, I need to figure out what to do about removing manually provisioned machines, along with their services and units.