No module named docker - compute node

[alpha23]

stable/rocky, centos/source containers
Host docker version on COMPUTE_NODE: Server Version: 19.03.12

I receive the following error error when reconfiguring kuryr. I did not receive this error with nova, neutron, etc.

failed: [<COMPUTE_NODE>] (item={'key': 'kuryr', 'value': {'container_name': 'kuryr', 'group': 'compute', 'enabled': True, 'image': 'kolla/centos-source-kuryr-libnetwork:rocky', 'privileged': True, 'cap_add': ['NET_ADMIN'], 'volumes': ['/etc/kolla//kuryr/:/var/lib/kolla/config_files/:ro', '/etc/localtime:/etc/localtime:ro', '/lib/modules:/lib/modules:ro', '/run:/run:shared', '/usr/lib/docker:/usr/lib/docker', '', '', 'kolla_logs:/var/log/kolla/'], 'dimensions': {}}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "kuryr", "value": {"cap_add": ["NET_ADMIN"], "container_name": "kuryr", "dimensions": {}, "enabled": true, "group": "compute", "image": "kolla/centos-source-kuryr-libnetwork:rocky", "privileged": true, "volumes": ["/etc/kolla//kuryr/:/var/lib/kolla/config_files/:ro", "/etc/localtime:/etc/localtime:ro", "/lib/modules:/lib/modules:ro", "/run:/run:shared", "/usr/lib/docker:/usr/lib/docker", "", "", "kolla_logs:/var/log/kolla/"]}}, "module_stderr": "Shared connection to <COMPUTE_NODE> closed.\r\n", "module_stdout": "\r\nTraceback (most recent call last):\r\n File \"<HOME_PATH>/.ansible/tmp/ansible-tmp-1601140556.2102048-25749545366289/AnsiballZ_kolla_docker.py\", line 114, in <module>\r\n _ansiballz_main()\r\n File \"<HOME_PATH>/.ansible/tmp/ansible-tmp-1601140556.2102048-25749545366289/AnsiballZ_kolla_docker.py\", line 106, in _ansiballz_main\r\n invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\r\n File \"<HOME_PATH>/.ansible/tmp/ansible-tmp-1601140556.2102048-25749545366289/AnsiballZ_kolla_docker.py\", line 49, in invoke_module\r\n imp.load_module('__main__', mod, module, MOD_DESC)\r\n File \"/tmp/ansible_kolla_docker_payload_CUNHhP/__main__.py\", line 226, in <module>\r\nImportError: No module named docker\r\n", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1}

[alpha23 (alpha23)]

I should mention that the host of the compute node is:

Distributor ID: Ubuntu
Description: Ubuntu 18.04.5 LTS
Release: 18.04
Codename: bionic

4.15.0-112-generic

Any thoughts on how to fix this issue?

[Radosław Piliszek (yoctozepto)]

The code does not differ. It is likely that your Python environment got garbled between runs. Ensure prechecks still pass for the buggy host.

[alpha23 (alpha23)]

Running prechecks does fail on the compute node, however, I'm still unsure on how to resolve this issue. ansible_playbook_interpreter=/usr/bin/python in inventory file for the compute node.

    "module_stderr": "OpenSSH_7.6p1 Ubuntu-4ubuntu0.3, OpenSSL 1.0.2n 7 Dec 2017\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 19: Applying options for *\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 25120\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 1\r\nShared connection to <COMPUTE_NODE> closed.\r\n",
    "module_stdout": "\r\nTraceback (most recent call last):\r\n File \"<HOME_DIR>/.ansible/tmp/ansible-tmp-1601678859.358308-246129810625221/AnsiballZ_kolla_container_facts.py\", line 114, in <module>\r\n _ansiballz_main()\r\n File \"<HOME_DIR>/.ansible/tmp/ansible-tmp-1601678859.358308-246129810625221/AnsiballZ_kolla_container_facts.py\", line 106, in _ansiballz_main\r\n invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\r\n File \"<HOME_DIR>/.ansible/tmp/ansible-tmp-1601678859.358308-246129810625221/AnsiballZ_kolla_container_facts.py\", line 49, in invoke_module\r\n imp.load_module('__main__', mod, module, MOD_DESC)\r\n File \"/tmp/ansible_kolla_container_facts_payload_6u9POR/__main__.py\", line 52, in <module>\r\nImportError: No module named docker\r\n",
    "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error",
    "rc": 1

[Radosław Piliszek (yoctozepto)]

Hmm, normally bootstrap-servers installs the docker package via the baremetal role.
It's very not recommended to rerun this step on a live system so it's best to just install the docker package oneself:

  sudo python -m pip install docker

[alpha23 (alpha23)]

I had already executed 'sudo python -m pip install docker' on the compute node. Re-executed and reconfigured just to verify.

Same issue. Please advise.

[Radosław Piliszek (yoctozepto)]

When you do:

    sudo python

and then try:

    import docker

does that work or errors out?

[alpha23 (alpha23)]

This is resolved. I'm pretty sure this was resolved by installing docker via pip2.7 and then ensuring that the docker module is installed.