manual provider uses reverse-lookup result instead of "bootstrap-host"

[Zachery Hostens]

When using the manual provider, juju uses the result of doing a reverse lookup of bootstrap-host, rather than bootstrap-host itself. If it is an IP, juju should use it verbatim, or be at the mercy of stale DNS entries.

Original description:

after getting proper ppa adds to a 12.04 server (talked to a couple people on IRC) i was able to get machine-0 up and running.

after getting it up and running how ever `juju status` just sits there, and does nothing.

Recreate:

server: apt-get install -y ubuntu-cloud-keyring
server: apt-get install python-software-properties
server: add-apt-repository cloud-archive:tools

workstation: juju bootstrap
workstation: juju status

[Tim Penhey (thumper)]

2013-10-31 21:37:15 DEBUG juju.environs.config config.go:598 coercion failed attributes: map[string]interface {}{"bootstrap-host":"162.219.7.68"}, checker: schema.fieldMapC{fields:schema.Fields{"bootstrap-host":schema.stringC{}, "bootstrap-user":schema.stringC{}, "storage-listen-ip":schema.stringC{}, "storage-port":schema.intC{}, "storage-auth-key":schema.stringC{}}, defaults:schema.Defaults{"bootstrap-user":"", "storage-listen-ip":"", "storage-port":8040}, strict:false}, storage-auth-key: expected string, got nothing
2013-10-31 21:37:15 ERROR juju.worker environ.go:50 loaded invalid environment configuration: storage-auth-key: expected string, got nothing

The storage-auth-key is listed as a secret, so is being stripped out of the environ config. Technically the caller is a management node, so it shouldn't be.

[Andrew Wilkins (axwalk)]

@thumper: It's normal to get an error about storage-auth-key; as it's secret, it is only provided on the first API connection.

@zacheryph: Would you please run this command, and attach the output? "juju status --debug". The command is meant to be more or less instantaneous, so just run it for a few seconds before killing.

Please note that this will contain private keys, but they are specific to the environment. Once this issue is resolved, I'd suggest regenerating the environment (you'll probably have to anyway). Alternatively, you could wipe out the private key parts of the log.

Also, can you please verify that the API server port is not firewalled: "telnet <bootstrap-host> 17070". If it hangs too, there's your problem.

[Zachery Hostens (zacheryph)]

here is the log. keys redacted but im sure ill end up re-doing the env from scratch when all is cured. i AM able to telnet to 17070 as well as 37017 (mongo.) i did this just cause i seen it in the log. I let the command run for a good few minutes and nothing else came out besides this.

2013-11-01 13:35:29 DEBUG juju.environs open.go:75 ConfigForName found bootstrap config map[string]interface {}{"authorized-keys":"ssh-rsa [redact]\n", "ca-private-key":"-----BEGIN RSA PRIVATE KEY-----\n [redact] \n-----END RSA PRIVATE KEY-----\n", "name":"null", "ssl-hostname-verification":true, "storage-auth-key":"60f[redact]195c", "admin-secret":"84d[redact]5f9", "ca-cert":"-----BEGIN CERTIFICATE-----\n [redact] \n-----END CERTIFICATE-----\n", "image-metadata-url":"", "api-port":17070, "bootstrap-host":"162.219.7.68", "default-series":"precise", "type":"null", "development":false, "firewall-mode":"instance", "logging-config":"<root>=DEBUG", "state-port":37017, "tools-url":""}
2013-11-01 13:35:30 DEBUG juju state.go:160 waiting for DNS name(s) of state server instances [manual:]
2013-11-01 13:35:30 INFO juju.state open.go:68 opening state; mongo addresses: ["c1210-1595.cloudatcost.com.:37017"]; entity ""

[Zachery Hostens (zacheryph)]

This came up after the command ran for 10 minutes:

2013-11-01 13:45:30 ERROR juju supercommand.go:282 Unable to connect to environment "null".
Please check your credentials or use 'juju bootstrap' to create a new environment.

Error details:
no reachable servers

[Andrew Wilkins (axwalk)]

Sorry, I missed your update.

So, it looks like maybe the DNS was stale? I've just done a reverse-lookup on 162.219.7.68, and it comes back with topher.zro.net, not the "c1210-1595.cloudatcost.com" that you were apparently getting.

[Zachery Hostens (zacheryph)]

Yeah I noticed that after posting here. I was thinking that might be the issue, though it confused me why it would use reverse dns at all to connect, when i put the IP address in the config yaml. trying now i get issues regarding sftp being an unknown protocol (which i have seen a ticket for.) maybe this was the only issue for that? the first time i was getting issues like this i did some sort of install of juju on the server manually as well. that might have had something to do with it. this ticket might be a none issue now :-/

some how before i installed juju on the server and bootstrap "went with it" i guess. though now even with it installed it still tries pulling down 1.16.2 (cloud-tools only shows 1.16.0)

zephyr:~:% host topher.zro.net
  topher.zro.net has address 162.219.7.68
zephyr:~:% host 162.219.7.68
  68.7.219.162.in-addr.arpa domain name pointer topher.zro.net.

new output:

zephyr:~:% juju bootstrap
WARNING no tools available, attempting to retrieve from https://juju-dist.s3.amazonaws.com/
ERROR Get sftp://162.219.7.68//var/lib/juju/storage/tools/releases/juju-1.16.2-precise-amd64.tgz: unsupported protocol scheme "sftp"

Is there anyway manually around this? (say manually install the tgz myself then let bootstrap complete?)

[Zachery Hostens (zacheryph)]

OK! maybe the DNS was the only issue here. All I did was:

rm -rf /var/lib/juju and added ppa:juju/stable and installed juju-1.16.3 locally and everything is working now.

bootstrap succeeded and:

zephyr:~:% juju status
environment: "null"
machines:
  "0":
    agent-state: started
    agent-version: 1.16.3
    dns-name: topher.zro.net.
    instance-id: 'manual:'
    series: precise
    hardware: arch=amd64 cpu-cores=1 mem=490M
services: {}

[Andrew Wilkins (axwalk)]

Glad to hear you got it working, thanks for reporting back.

The reverse-lookup bit, I think, is mired in history. But, I agree - it would be better if it just used the IP you entered. I'll leave this ticket open so we can investigate changing that.