lookup url function proxy issue with venv checksum

[Alexander J. Maidak]

I'm trying to setup an AIO behind a proxy.

I have these environment variables:

proxy_env_url: http://myproxy.local:3128
no_proxy_env: ".example.com,localhost,127.0.0.1,{{ internal_lb_vip_address }},{{ external_lb_vip_address }},{% for host in groups['all_containers'] %}{{ hostvars[host]['container_address'] }}{% if not loop.last %},{% endif %}{% endfor %}"
global_environment_variables:
  HTTP_PROXY: "{{ proxy_env_url }}"
  HTTPS_PROXY: "{{ proxy_env_url }}"
  NO_PROXY: "{{ no_proxy_env }}"
  http_proxy: "{{ proxy_env_url }}"
  https_proxy: "{{ proxy_env_url }}"
  no_proxy: "{{ no_proxy_env }}"

/etc/environment:
no_proxy=.example.com,localhost,127.0.0.1,172.29.236.100,10.141.213.33,172.29.238.127,172.29.236.101,172.29.236.233,172.29.239.9,172.29.236.186,172.29.238.11,172.29.238.244,172.29.238.61,172.29.236.246,172.29.236.164,172.29.237.156,172.29.237.34,172.29.236.119,172.29.237.101,172.29.239.40,172.29.236.151,172.29.237.241,172.29.236.75,172.29.237.162,172.29.239.177,172.29.238.2,172.29.236.201,172.29.237.50,172.29.239.60,172.29.236.197,172.29.236.131
https_proxy=http://myproxy.local:3128
NO_PROXY=.example.com,localhost,127.0.0.1,172.29.236.100,10.141.213.33,172.29.238.127,172.29.236.101,172.29.236.233,172.29.239.9,172.29.236.186,172.29.238.11,172.29.238.244,172.29.238.61,172.29.236.246,172.29.236.164,172.29.237.156,172.29.237.34,172.29.236.119,172.29.237.101,172.29.239.40,172.29.236.151,172.29.237.241,172.29.236.75,172.29.237.162,172.29.239.177,172.29.238.2,172.29.236.201,172.29.237.50,172.29.239.60,172.29.236.197,172.29.236.131
http_proxy=http://myproxy.local:3128
HTTPS_PROXY=http://myproxy.local:3128
HTTP_PROXY=http://myproxy.local:3128

This play in os_keystone/tasks/keystone_install.yml:
- name: Attempt venv download
  get_url:
    url: "{{ keystone_venv_download_url }}"
    dest: "/var/cache/{{ keystone_venv_download_url | basename }}"
    checksum: "sha1:{{ lookup('url', keystone_venv_download_url | replace('tgz', 'checksum')) }}"
  register: keystone_get_venv
  when: not keystone_developer_mode | bool

fails with:

TASK [os_keystone : Attempt venv download] *************************************
task path: /etc/ansible/roles/os_keystone/tasks/keystone_install.yml:57
url lookup connecting to http://172.29.236.100:8181/venvs/14.0.7/ubuntu/keystone-14.0.7-x86_64.checksum
fatal: [aio1_keystone_container-724c264a]: FAILED! => {"failed": true, "msg": "An unhandled exception occurred while running the lookup plugin 'url'. Error was a <class 'socket.timeout'>, original message: timed out"}

NO MORE HOSTS LEFT *************************************************************

If I access the keystone container this works fine:

root@aio1-keystone-container-724c264a:/tmp# wget http://172.29.236.100:8181/venvs/14.0.7/ubuntu/keystone-14.0.7-x86_64.checksum
--2017-02-24 14:19:51-- http://172.29.236.100:8181/venvs/14.0.7/ubuntu/keystone-14.0.7-x86_64.checksum
Connecting to 172.29.236.100:8181... connected.
HTTP request sent, awaiting response... 200 OK
Length: 41 [application/octet-stream]
Saving to: ‘keystone-14.0.7-x86_64.checksum.1’

keystone-14.0.7-x86_64.checksum.1 100%[================================================================================================================================>] 41 --.-KB/s in 0s

2017-02-24 14:19:51 (3.81 MB/s) - ‘keystone-14.0.7-x86_64.checksum.1’ saved [41/41]

So proxy env appears to be working fine. Perhaps there's an issue with the url lookup plugin? but I'm not sure how to trouble shoot this.

Not sure what the issue is. I found this bug: #1556975 (https://bugs.launchpad.net/openstack-ansible/+bug/1556975) looks similar, but the playbooks are quite different... Not sure if this is the same thing or not.

[Alexander J. Maidak (ajmaidak)]

I'm running the 14.0.8 branch which installs ansible 2.1.4...

[Jean-Philippe Evrard (jean-philippe-evrard)]

Could you curl http://172.29.236.100:8181/venvs/14.0.7/ubuntu/keystone-14.0.7-x86_64.checksum from the deploy node?

[Alexander J. Maidak (ajmaidak)]

Thanks for responding to my bug. I must admit that my issue was simple user error.

When setting up my proxy env I was lazy in the environment variable in the shell which ran ansible were:

NO_PROXY=localhost,127.0.0.1,.example.com
http_proxy=http://myproxy.local:3128
HTTPS_PROXY=http://myproxy.local:3128
https_proxy=http://myproxy.local:3128
no_proxy=.example.com,127.0.0.1,localhost
HTTP_PROXY=http://myproxy.local:3128

This lacked the 172.29.236.100 address in the no_proxy var which broke the lookup plugin.

After setting my proxy environment vars properly in the shell I was able to get the AIO deploy to work.

I did not test just adding the 172.29.236.100 address to my no_proxy, rather I adopted the complete no_proxy variable created by scripts/run-playbooks.sh... If values beyond 172.29.236.100 are required there's a little bit of a chicken and egg problem as scripts/run-playbooks.sh requires a detailed no_proxy variable but the best way to get one is via the run-playbooks.sh script.