juju controller caches credentials from bootstrap

[Andrew Woodward]

juju 2.2.2

I installed a controller via openstack, using a userpass credential about two weeks ago. Said credential was changed (as they do) and I noticed "something" locked my account with in a few minutes, So I figured that there was some nonsense that the controller might be doing on my behalf, so I updated my credentials on my juju client node, then issued `juju update-credential <cloud> <credential>` it was like OK, so I unlocked my account. Then I come to find that a few min later my account was locked again. This processes repeated through out the day, until I popped on IRC to determine if there was a way to rule out juju. The feeback was to add a new unit so I ran `juju deploy juju-gui` came back and the machine was still in a pending state, oh and my account is locked again.

Now I'm completely fed up with the controller and figure It's going to easier to delete it and start over, yes that's right its going to be less painful to start over. x.x so I `juju destroy-controller --destroy-all-models <controller> --debug` Nope, now its stuck deleting waiting for the unit in pending state, and to take the cake, my account is locking every few seconds. A few minutes later I just terminate the controllers instance via the cloud's api. YAY! My credential hasn't locked since.

Problems
* juju update-credential doesn't appear to do anything it's described to do. It didn't ask me about the updates, so I just went straight to the file, and updated it and figured it did something useful.
* Can't validate the credential via the cli.
* There is 0% documentation about controllers caching credentials, or how to update them
* Controller cant be destroyed because a unit that was never created, is waited on infinitely.
* There isn't clear feedback about the failed credential. (Sure I didn't go pull the debug log, but I did look at juju status multiple times, and it didn't indicate the problem)

[Anastasia (anastasia-macmood)]

Credential management does need some attention.

All credential commands, bar 'update-credential', only deal with client side cloud credentials. Consequently, "go to the file and updating it" would only change what your client know, not what your controller knows.

'update-credential' command allows you to update a credential cached on the controller. Note it is an update not a replace, so only the values of the credential may be changed not its name, for example. And, yes, we will soon validate whether newly supplied credential will work on the affected models. It's not happening yet.

We are working on making this area better from usability perspective. It will be more intuitive once we are done. Docs will be updated too.

[Andrew Woodward (xarses)]

To clarify, I initially ran 'update-credential' and expected for it to ask me about what to update, so I went to the files and updated them my self, then ran 'update-credential' again, this didn't resolve the account-locking so I'm going to go out on a limb and say that didn't do anything to whatever credential the controller had cached.

[Anastasia (anastasia-macmood)]

I am marking this report as a duplicate since we are now tracking progress on credential UX improvements through bug # 1735402.

Essentially, we have not cleanly disclosed that we have credentials stored in 2 different places.

There are credentials that Juju stores locally on a client machine and only uses for 'bootstrap' and 'add-model' when requested. Local credential storage is managed via these commands:

* 'add-credential' (creates a new credential locally on this client);
* 'add-credential --replace' (updates existing credential stored locally on this client);
* 'set-default-credential' (marks a locally stored credential as default for a specified cloud. This means that a user can run 'bootstrap' with just specifying a cloud and this credential will be picked up.);
* 'remove-credential' (deletes given credential from a local store on this client).

Then there are credentials that are stored on the controller once you bootstrap. These credentials are per user/per cloud; are used by models but are not currently visible. The only thing you, as a user, can do with these credentials is to update their content via 'update-credential' command. However, this command has limitations:

1. it's controller based. You must know what your credential is called and there is no means to find out what it is called on the controller. (There is work being done to fix that for 2.4, see the bug above);

2. it does not check that the new contents work for your model (as per 1);

3. you can only change the content of the credential that a model uses, you cannot replace model credential with another credential.

While we are working on changing the UX to be more friendly, the process on how to update model credential is neatly described in comment # 12 of this bug.