1.21 alpha dumps a stack trace when ec2 credentials are missing

[Jeff Lane ]

Created a fresh ec2 instance of 14.04 and installed juju from the devel PPA to get 1.21-alpha1 to test out as I was asked to set up a demo next week using juju 1.21.

So I installed juju:
ubuntu@ip-10-0-0-212:~$ dpkg -l |grep juju
ii juju 1.21-alpha1-0ubuntu1~14.04.1~juju1 all next generation service orchestration system
ii juju-core 1.21-alpha1-0ubuntu1~14.04.1~juju1 amd64 Juju is devops distilled - client

Then edited environments. yaml to add my AWS api key and add the devel tools stream pointer that is mentioned on the devel ppa page:
amazon:
        type: ec2

        # region specifies the EC2 region. It defaults to us-east-1.
        #
        # region: us-east-1

        # access-key holds the EC2 access key. It defaults to the
        # environment variable AWS_ACCESS_KEY_ID.
        #
        # access-key:

        # secret-key holds the EC2 secret key. It defaults to the
        # environment variable AWS_SECRET_ACCESS_KEY.
        #
        secret-key: MY_SECRET_KEY_REDACTED

        # image-stream chooses a simplestreams stream to select OS images
        # from, for example daily or released images (or any other stream
        # available on simplestreams).
        #
        # image-stream: "released"

        # Whether or not to refresh the list of available updates for an
        # OS. The default option of true is recommended for use in
        # production systems, but disabling this can speed up local
        # deployments for development or testing.
        #
        # enable-os-refresh-update: true

        # Whether or not to perform OS upgrades when machines are
        # provisioned. The default option of true is recommended for use
        # in production systems, but disabling this can speed up local
        # deployments for development or testing.
        #
        # enable-os-upgrade: true
        tools-metadata-url: https://streams.canonical.com/juju/devel/tools

Then I attempted a bootstrap and the following is what happened:
ubuntu@ip-10-0-0-212:~$ juju bootstrap --debug
2014-10-14 21:28:11 INFO juju.cmd supercommand.go:37 running juju [1.21-alpha1-trusty-amd64 gc]
2014-10-14 21:28:11 INFO juju.provider.ec2 ec2.go:216 opening environment "amazon"
2014-10-14 21:28:11 INFO juju.cmd cmd.go:113 Bootstrap failed, destroying environment
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x40 pc=0x4e0957]

goroutine 1 [running]:
runtime.panic(0xd57c40, 0x1b6b9e8)
 /usr/lib/go/src/pkg/runtime/panic.c:266 +0xb6
github.com/juju/juju/environs.Destroy(0x0, 0x0, 0x7f30c8b48a60, 0xc2101af730, 0x1, ...)
 /build/buildd/juju-core-1.21-alpha1/src/github.com/juju/juju/environs/open.go:291 +0x27
main.destroyPreparedEnviron(0xc21001f870, 0x0, 0x0, 0x7f30c8b48a60, 0xc2101af730, ...)
 /build/buildd/juju-core-1.21-alpha1/src/github.com/juju/juju/cmd/juju/common.go:27 +0x117
main.func·020()
 /build/buildd/juju-core-1.21-alpha1/src/github.com/juju/juju/cmd/juju/common.go:65 +0x6a
main.func·018()
 /build/buildd/juju-core-1.21-alpha1/src/github.com/juju/juju/cmd/juju/bootstrap.go:188 +0x87
main.(*BootstrapCommand).Run(0xc21006c600, 0xc21001f870, 0x7f30c8b3cd10, 0xc2101d5e60)
 /build/buildd/juju-core-1.21-alpha1/src/github.com/juju/juju/cmd/juju/bootstrap.go:260 +0x2fd
github.com/juju/juju/cmd/envcmd.(*environCommandWrapper).Run(0xc2101b5060, 0xc21001f870, 0xb9c860, 0xc2101af600)
 /build/buildd/juju-core-1.21-alpha1/src/github.com/juju/juju/cmd/envcmd/environmentcommand.go:1 +0x3d
main.(*envCmdWrapper).Run(0xc2101b5080, 0xc21001f870, 0x0, 0x0)
 /build/buildd/juju-core-1.21-alpha1/src/github.com/juju/juju/cmd/juju/addmachine.go:1 +0x3d
github.com/juju/cmd.(*SuperCommand).Run(0xc210195630, 0xc21001f870, 0xc21001f870, 0x0)
 /build/buildd/juju-core-1.21-alpha1/src/github.com/juju/cmd/supercommand.go:321 +0x3ca
github.com/juju/cmd.Main(0x7f30c8b48610, 0xc210195630, 0xc21001f870, 0xc21000a010, 0x2, ...)
 /build/buildd/juju-core-1.21-alpha1/src/github.com/juju/cmd/cmd.go:247 +0x283
main.Main(0xc21000a000, 0x3, 0x3)
 /build/buildd/juju-core-1.21-alpha1/src/github.com/juju/juju/cmd/juju/main.go:76 +0xada
main.main()
 /build/buildd/juju-core-1.21-alpha1/src/github.com/juju/juju/cmd/juju/main.go:172 +0x44

goroutine 3 [syscall]:
os/signal.loop()
 /usr/lib/go/src/pkg/os/signal/signal_unix.go:21 +0x1e
created by os/signal.init·1
 /usr/lib/go/src/pkg/os/signal/signal_unix.go:27 +0x31

goroutine 8 [finalizer wait]:
runtime.park(0x43fe00, 0x1b70b28, 0x1b6d768)
 /usr/lib/go/src/pkg/runtime/proc.c:1342 +0x66
runfinq()
 /usr/lib/go/src/pkg/runtime/mgc0.c:2279 +0x84
runtime.goexit()
 /usr/lib/go/src/pkg/runtime/proc.c:1394

The instance is a standard m1.tiny aws instance running 14.04.1:
ubuntu@ip-10-0-0-212:~$ cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=14.04
DISTRIB_CODENAME=trusty
DISTRIB_DESCRIPTION="Ubuntu 14.04.1 LTS"

[Jeff Lane  (bladernr)]

Ok, found the issue. I backed down to 1.18 on trusty. According to the documentation on setting up aws:
"All you need to do to get this configuration to work is to either set the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY via environment variables, or uncomment and add the values to the configuration file."

in environments.yaml above note that I was ONLY setting secret_key.

I tried with 1.18 setting AWS_SECRET_ACCESS_KEY and AWS_ACCESS_KEY_ID env vars and when trying to bootstrap, I got this error:
ubuntu@ip-10-0-0-212:~$ export AWS_ACCESS_KEY_ID=<MY ACCESS KEY ID>
ubuntu@ip-10-0-0-212:~$ export AWS_SECRET_ACCESS_KEY=<MY ACCESS KEY>
ubuntu@ip-10-0-0-212:~$ juju bootstrap
ERROR environment has no access-key or secret-key

I retried using only secret-key in environments.yaml and got the same error. Finally, I filled in BOTH access-key and secret-key in envionrments and was able to bootstrap 1.18.

So with that, I removed 1.18 and re-installed 1.21 and using both access-key and secret-key in the environments.yaml file, I was able to successfully start a bootstrap with 1.21.

So first issue, the docs and boilerplate environments.yaml need to be fixed to say you need BOTH secret-key and access-key to bootstrap ec2. (it would also help to add info on using IAM as that's Amazon's recommended way of using api access now).

Second issue, however, is the stack dump that occurs on 1.21 when the bootstrap fails.

Third issue, on 1.21 there was no error telling me WHY bootstrap failed. With 1.18 the message pointed me in the right directly, if not necessarily completely accurate. But on 1.21 there is no such message before the stacktrace is dumped on teardown.

[Ian Booth (wallyworld)]

The stacktrace issue has been fixed in latest source code.
The behaviour now looks like this:

$ juju bootstrap
Bootstrap failed, cleaning up the environment.
ERROR there was an issue examining the environment: environment has no access-key or secret-key

Both access key id AND secret-key are required. These can be set via either entries in the yaml file or via env variables.