win client bootstrap fails because it uses private ip

[Curtis Hovey]

While testing the windows client (vetting the CI scripts) we can see that the windows client cannot complete a bootstrap. It rejects the public IP address in favour of the private IP address. Manual runs of the 1.17.1 client verify this behaviour.

juju --show-log bootstrap -e test-win-client --constraints mem=2G
2014-01-14 19:53:37 INFO juju.provider.ec2 ec2.go:196 opening environment "test-win-client"
2014-01-14 19:53:37 INFO juju.environs.bootstrap bootstrap.go:46 bootstrapping environment "test-win-client"
2014-01-14 19:53:42 INFO juju.environs.tools tools.go:85 reading tools with major.minor version 1.17
2014-01-14 19:53:42 INFO juju.environs.tools tools.go:96 filtering tools by series: precise
2014-01-14 19:53:43 INFO juju.environs.bootstrap bootstrap.go:58 picked newest version: 1.17.0
Launching instance
2014-01-14 19:53:45 INFO juju.provider.ec2 ec2.go:438 started instance "i-b3ca129d"
 - i-b3ca129d
Waiting for address
Attempting to connect to ec2-50-17-93-169.compute-1.amazonaws.com:22
Attempting to connect to domU-12-31-39-0C-74-B8.compute-1.internal:22
Attempting to connect to 50.17.93.169:22
Attempting to connect to 10.215.119.70:22

[Curtis Hovey (sinzui)]

This is the debug log of the failed bootstrap

[Curtis Hovey (sinzui)]

This issue is still present in r2212.

[Andrew Wilkins (axwalk)]

Is this still occurring on trunk? r2217 introduced an alternative embedded SSH client intended for use on Windows. It'd be good to know whether *that* works, even if the OpenSSH client method doesn't (though I wouldn't consider this Fixed until both are working).

[Curtis Hovey (sinzui)]

Win-client still fails with r2235. openssh is not in $PATH. should something else like putty.exe be in the $PATH? We still see the same issue, public addresses are quickly rejected, and then the test is killed after 5 minutes because juju cannot connect to the private address.
    http://162.213.35.54:8080/job/win-client-deploy/18/console

[John A Meinel (jameinel)]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 2014-01-22 1:10, Curtis Hovey wrote:
> Win-client still fails with r2235. openssh is not in $PATH. should
> something else like putty.exe be in the $PATH? We still see the
> same issue, public addresses are quickly rejected, and then the
> test is killed after 5 minutes because juju cannot connect to the
> private address.
> http://162.213.35.54:8080/job/win-client-deploy/18/console
>

I'm pretty sure putty.exe needs very different arguments to ssh.exe,
and I'm equally pretty sure that we don't have any logic internally
that contain the string "putty". So if we want to try putting like
OpenSSH for testing (I've used it in the past, because I already have
it), that might be something useful for people getting started until
we actually have other support.

John
=:->

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.13 (Cygwin)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEARECAAYFAlLfjbAACgkQJdeBCYSNAAP4awCeJXP3m3k4rgckfONdjjqxc62m
Uk8AoIPuJO/62dzfVh6M3KQ7+XdG448f
=6X9U
-----END PGP SIGNATURE-----

[Andrew Wilkins (axwalk)]

I would like to clarify: when bootstrap says "Attempting to connect to X" for a bunch of different addresses in a row, it's not trying sequentially. Bootstrap tries *all* the addresses for the full 10 minutes, in a loop.

I'm pretty sure I've reproduced this, so I'll dig in.

[Andrew Wilkins (axwalk)]

If OpenSSH is available in %PATH%, but has no default keys, then bootstrap times out. This is because the auto-generated client keys are not added as optional identities to the ssh invocation. I'm preparing a CL to fix that.

If OpenSSH is *not* available in in %PATH%, then juju will attempt with an embedded SSH client which does use the auto-generated client keys. If this fails, I suspect it's because you've got an existing .jenv file that either doesn't include authorised-keys or has an entry that doesn't include the auto-generated public key. I have reverified that without an existing .jenv file, I'm able to bootstrap without OpenSSH available.

[Curtis Hovey (sinzui)]

openssh is not in the path. not is putty. The user does have ssh keys setup and uses them with juju 1.16.5 to deploy services on aws.

[Andrew Wilkins (axwalk)]

Okay, so that leaves paragraph two of comment #7. Does the .jenv file exist before you try to bootstrap? Do you have some non-default authorized-keys setup in environments.yaml?

[Curtis Hovey (sinzui)]

Good news. The juju 1.17.2 created from r2274 can bootstrap. This was within powershell. We modified CI to use the new --force when destroy-environment is called. I did delete all the jenv files. The configs do not use a custom ssh key because juju 1.16.x only works with .ssh/id_rsa in the Windows.

The tests do not pass, still reporting permission denied try to connect. I modified the tests to always remove OpenSSH from the path in case it is not found. I will tinker with the tests again to see if I can ensure a sane env.

[Curtis Hovey (sinzui)]

Tests pass! The problem was that the combination of openssh/cygwin/sh.exe/switch.exe always finds ssh.exe even though OpenShh/bin is never in the path. I renamed ssh.exe to ossh.exe. This let juju fallback to the native ssh code.

[Curtis Hovey (sinzui)]

I am not marking the bug as fixed committed since I see a branch linked, but I am convinced juju tip allows me to bootstrap just as I can using juju 1.16.5.

[Andrew Wilkins (axwalk)]

Woo! Thanks for confirming. The problems you had when ssh.exe *was* being picked up may be due to the .ssh dir being inside the Cygwin home. Even so, with the linked branch it should be possible to bootstrap when ssh.exe is found or not.