debug-log doesn't work with lxc provider

[Andreas Hasenack]

andreas@nsn7:~$ juju debug-log -v
2013-07-18 13:38:27 INFO juju.environs.local environprovider.go:32 opening environment "local"
2013-07-18 13:38:27 INFO juju open.go:69 state: opening state; mongo addresses: ["10.0.3.1:37017"]; entity ""
2013-07-18 13:38:27 INFO juju open.go:107 state: connection established
2013-07-18 13:38:27 INFO juju ssh.go:79 looking up address for machine 0...
Warning: Permanently added '10.0.3.1' (ECDSA) to the list of known hosts.
Permission denied (publickey).
2013-07-18 13:38:27 ERROR juju supercommand.go:235 command failed: exit status 255
error: exit status 255
andreas@nsn7:~$

It probably shouldn't even try to ssh, and instead grab the log file(s) directly.

[Matthew Williams (mattyw)]

This looks similar to a bug I'm getting trying to "juju ssh" into a unit deployed using the lxc provider. Is it likely to be the same root cause?

[John A Meinel (jameinel)]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 2013-09-05 16:44, Matthew Williams wrote:
> This looks similar to a bug I'm getting trying to "juju ssh" into a
> unit deployed using the lxc provider. Is it likely to be the same
> root cause?
>

I would expect if 'juju ssh' doesn't work with the local provider,
then definitely "juju debug-log" won't work, since it goes via "juju ssh".

John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.13 (Cygwin)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEARECAAYFAlIsNSwACgkQJdeBCYSNAAN12gCgjNQtEwzTsyeUvjh4sN/Qb3Y1
gwMAn12BHGmQdkY4YSqe/ks/Gd8uR1yg
=LECa
-----END PGP SIGNATURE-----

[Curtis Hovey (sinzui)]

This looks very similar to the errors I see testing on lxc and canonistack. My quick fix was to remove all the 10.0.3.* from my known hosts. This isn't a true juju problem, it is an ssh issue that is obfuscated by juju. We are accepting the keys to all the machines we create, but do not remove them when the machine is destroyed. Juju could suggest that known_hosts may have a key from a previous machine registered machine and that can be addressed by adding
    ssh-keygen -R 10.0.3.1

[Adam Stokes (adam-stokes)]

Even removing the offending ip in known_hosts the problem persists:

~ » ssh-keygen -R 10.0.3.1 adam@beatnuts
/home/adam/.ssh/known_hosts updated.
Original contents retained as /home/adam/.ssh/known_hosts.old
------------------------------------------------------------
~ » sudo juju debug-log adam@beatnuts
[sudo] password for adam:
ssh: connect to host 10.0.3.1 port 22: Connection refused
ERROR exit status 255
------------------------------------------------------------
~ » sudo juju status adam@beatnuts
environment: local
machines:
  "0":
    agent-state: started
    agent-version: 1.16.0.1
    dns-name: 10.0.3.1
    instance-id: localhost
    series: saucy
  "1":
    agent-state: started
    agent-version: 1.16.0.1
    instance-id: adam-local-machine-1
    series: precise
services:
  joomla:
    charm: cs:precise/joomla-1
    exposed: false
    units:
      joomla/0:
        agent-state: pending
        agent-version: 1.16.0.1
        machine: "1"
        public-address: 10.0.3.152

 Running juju-core 1.16.0-0ubuntu1 on saucy amd64

[Marco Ceppi (marcoceppi)]

As a work around, you will find all the log files that debug-log scans in ~/.juju/local/log where "local" is the name of your bootstrapped local environment. To duplicate debug-log in some sense, you could `tail -f ~/.juju/local/logs/unit-*.log` which will follow all the unit logs for your deployment.

[Frank Mueller (themue)]

Latest information: debug-log will get an own API command instead of using ssh and tail. The CLI command will then use that API command.

The ssh addressing issue with the local provider is an own issue and already in the backlog.

[Adam Stokes (adam-stokes)]

Should note also if you run debug-log against a local provider using kvm it will also fail with a publickey denied ssh error. Just fyi as it looks like it'll be interfacing with the api going forward.

[Matt Rae (mattrae)]

i was trying juju debug-log using the local provider with the latest juju fixing lp:1202682. so far i'm only seeing what looks like the machine agent/api logs, and not unit agent logs.. if i check under /var/log/syslog i can see the unit logs being logged there.

For example, generally i'd see output from 'apt-get' in the debug-log output. Now juju debug-log --replay | grep apt-get shows nothing. Tailing /var/log/syslog shows the unit log as normal. Is this expected?