API server inaccessible for roughly 5 seconds after bootstrap

[Adam Stokes]

when bootstrapping and attempting to access the api service directly after juju responds with:

Traceback (most recent call last):
  File "/tmp/jujudumpstatus.py", line 10, in <module>
    j.login()
  File "/home/adam/.pyenv/versions/3.4.0/lib/python3.4/site-packages/macumba-0.1-py3.4.egg/macumba/__init__.py", line 100, in login
  File "/home/adam/.pyenv/versions/3.4.0/lib/python3.4/site-packages/ws4py/client/__init__.py", line 209, in connect
    self.sock.connect(self.bind_addr)
  File "/home/adam/.pyenv/versions/3.4.0/lib/python3.4/ssl.py", line 803, in connect
    self._real_connect(addr, False)
  File "/home/adam/.pyenv/versions/3.4.0/lib/python3.4/ssl.py", line 790, in _real_connect
    socket.connect(self, addr)
ConnectionRefusedError: [Errno 111] Connection refused

This is a part of a python library we use to access juju api but the error is obviously in response to the api server being unavailable.

To reproduce you can use our python library and the following code:

$ pip install macumba

Script:

#!/bin/bash

cat <<-EOF > ~/.juju/environments.yaml
default: local

environments:
  local:
    type: local
    container: kvm
    lxc-clone: true
    admin-secret: pass
EOF

juju bootstrap

cat <<-EOF > /tmp/jujudumpstatus.py
#!/usr/bin/env python3
import macumba
from pprint import pprint

JUJU_URL = 'wss://localhost:17070/'
JUJU_PASS = 'pass'

if __name__ == "__main__":
    j = macumba.JujuClient(url=JUJU_URL, password=JUJU_PASS)
    j.login()
    ret = j.status()
    pprint(ret)
EOF

chmod +x /tmp/jujudumpstatus.py
/tmp/jujudumpstatus.py

After waiting roughly 5-10 seconds the api server will become available. This did not use to happen in juju version 1.18.

Currently seen on Trusty, juju-core 1.20.x

Thanks
Adam

[Adam Stokes (adam-stokes)]

Attaching machine-0.log which contains output from failed connection and shortly after where the api server started accepting requests

[Menno Finlay-Smits (menno.smits)]

Thanks for the log.

They show that about 4 seconds is spent ensuring mongo is configured correctly and started up. The next 1.20 point release will be much faster in this respect if mongo is already up and running. Hopefully that will help.

AFAICT everything else looks about right. There may be other opportunities for reducing the API server startup time but the mongo initialisation is the lowest hang fruit.

[Ian Booth (wallyworld)]

As per our conversation, I'm closing this bug since the startup time has been reduced assuming mongo is already configured correctly. Please reopen if you feel there's still an issue.