AWS credentials should not be in Zookeeper

[Clint Byrum]

The AWS credentials can cost a user quite a bit of money if they are compromised, and can be used to overwrite charms with trojaned charms, leading to full compromise (rather than just juju DoS).

The AWS credentials should be passed directly to agents that need them. Right now, that is just the single provisioning agent. This way even if somebody roots one box in the juju cluster, they can only alter zookeeper, but can't start new machines pointed at their own trojaned charms.

[Scott Moser (smoser)]

On Tue, 20 Dec 2011, Clint Byrum wrote:

> Public bug reported:
>
> The AWS credentials can cost a user quite a bit of money if they are
> compromised, and can be used to overwrite charms with trojaned charms,
> leading to full compromise (rather than just juju DoS).
>
> The AWS credentials should be passed directly to agents that need them.
> Right now, that is just the single provisioning agent. This way even if
> somebody roots one box in the juju cluster, they can only alter
> zookeeper, but can't start new machines pointed at their own trojaned
> charms.

It doesn't fix this issue but Amazon Identity and Access Management has
loads of things you can do to massively improve the situation.

Some ideas:
 * on creation of bootstrap node, you create a set of credentials that can
   only be used from the source IP of that node.
 * those credentials can only do the things they're expected to do
   (whatever that list is)
 * If there is a need for credentials on other systems, temporary
   credentials (with timeouts) can be created on the fly, with access to
   only do what they need to do.

At very least, there should be JuJu documentation telling the user how to
create a set of IAM credentials that are restricted to the broadest
use-case of JuJu. The docs should indicate to the user that they should
not put "master" credentials into juju.

Yes, thats probably at least 1 different bug.

[Gustavo Niemeyer (niemeyer)]

The real issue is that zookeeper right now is unprotected, but there are changes we want to do for solving that, and there are tickets open about these (and some branches laying over somewhere, actually).

[Clint Byrum (clint-fewbar)]

Excerpts from Gustavo Niemeyer's message of Wed Dec 21 02:27:52 UTC 2011:
> The real issue is that zookeeper right now is unprotected, but there are
> changes we want to do for solving that, and there are tickets open about
> these (and some branches laying over somewhere, actually).
>

With the ACL's that have been proposed, this definitely lowers in
priority. Even so, I question the need to have these credentials in ZK
directly when they carry so much power. I love Scott's idea, and think
we should probably just morph this into a doc bug once the ACL's are
in place.

[Gustavo Niemeyer (niemeyer)]

> With the ACL's that have been proposed, this definitely lowers in
> priority. Even so, I question the need to have these credentials in ZK
> directly when they carry so much power.

The coordination service is the heart of the system. It's what distributed the credentials to the actual machines for them to get access to everything. If we don't trust that system, we should fix it so we can trust it, rather than avoiding its use.

[Clint Byrum (clint-fewbar)]

Just to revive this debate, we should trust ZK, no doubt. However, we don't *have* to trust it with everything.

I think its worth a long term discussion around having ZK just represent the intended *structure* and to coordinate agents' reactions to changes. I'd like to consider letting agents keep hold of data and pass it directly to eachother, so that there is not one central service which contains every critical piece of data needed to compromise the whole environment.