neutron_agents_container GnuTLS recv error

[nexus]

9.0.6 repeatedly errors at this location.

TASK: [openstack_common | Get package from git] *******************************
ok: [infra1_neutron_server_container-1c8d00dc]
ok: [compute1]
failed: [infra1_neutron_agents_container-5cce3945] => {"attempts": 5, "cmd": "/usr/bin/git ls-remote origin -h refs/heads/fd1221124b8f53a61731e51b5f923af44ec5598f", "failed": true, "rc": 128}
stderr: fatal: unable to access 'https://git.openstack.org/openstack/neutron/': GnuTLS recv error (-9): A TLS packet with unexpected length was received.

msg: Task failed as maximum retries was encountered

This is what it's trying to pull from github.

root@infra1:~# ps auxf | grep git
root 3120 0.0 0.0 11740 932 pts/108 S+ 02:11 0:00 \_ grep --color=auto git
root 2999 0.0 0.0 17840 1228 ? S 02:07 0:00 | \_ /usr/bin/git ls-remote origin -t refs/tags/fd1221124b8f53a61731e51b5f923af44ec5598f
root 3000 0.1 0.1 166420 8796 ? S 02:07 0:00 | \_ git-remote-https origin https://git.openstack.org/openstack/neutron

The fallback git repo also appears to hang here. Any idea why this is erroring only for the neutron_agents container?

[nexus (nexus.nex)]

I thought it might help stepping through each playbook one at a time. 9.0.5 this time.

success - playbooks/setup/host-setup.yml
success - playbooks/infrastructure/haproxy-install.yml
success - playbooks/infrastructure/memcached-install.yml
success - playbooks/infrastructure/galera-install.yml
success - playbooks/infrastructure/rabbit-install.yml
success - playbooks/infrastructure/rsyslog-install.yml
success - playbooks/infrastructure/elasticsearch-install.yml
success - playbooks/infrastructure/logstash-install.yml
success - playbooks/infrastructure/kibana-install.yml
success - playbooks/infrastructure/es2unix-install.yml
success - playbooks/infrastructure/rsyslog-config.yml
success - playbooks/openstack/openstack-common.yml
success - playbooks/openstack/keystone.yml
success - playbooks/openstack/keystone-add-all-services.yml
success - playbooks/openstack/glance-all.yml
success - playbooks/openstack/heat-all.yml
success - playbooks/openstack/nova-all.yml
fail - playbooks/openstack/neutron-all.yml

 TASK: [openstack_common | Get package from git] *******************************
 failed: [infra1_neutron_agents_container-aaf5d1e0] => {"attempts": 5, "cmd": "/usr/bin/git ls-remote origin -h refs/heads/fd1221124b8f53a61731e51b5f923af44ec5598f", "failed": true, "rc": 128}
 stderr: fatal: unable to access 'https://git.openstack.org/openstack/neutron/': GnuTLS recv error (-9): A TLS packet with unexpected length was received.

 msg: Task failed as maximum retries was encountered
 ...ignoring

 TASK: [openstack_common | Get package from fallback git repo] *****************
 failed: [infra1_neutron_agents_container-aaf5d1e0] => {"attempts": 5, "cmd": "/usr/bin/git ls-remote origin -t refs/tags/fd1221124b8f53a61731e51b5f923af44ec5598f", "failed": true, "rc": 128}
 stderr: fatal: unable to access 'https://git.openstack.org/openstack/neutron/': GnuTLS recv error (-9): A TLS packet with unexpected length was received.

 msg: Task failed as maximum retries was encountered

 FATAL: all hosts have already failed -- aborting

Surely the other playbooks also use the same method to pull source from github. I don't understand why it's failing only on this container.

[Kevin Carter (kevin-carter)]

@nexus thank you for reporting this issue. we have a few questions regarding the deployment. We're curious about what the deployment looks like, if there were any overriden vars that pertain to the deployment that were effecting the packages that are being installed, e.g. a frozen repo. Also we'd like to know if you're behind some kind of a firewall/proxy that may be munging the bits?

This issue came up in our Tuesday meeting and we believe that this is an issue that can happen when using GNUTLS behind a proxy though none of us have seen this before. So any additional information that you can provide about this issue will be greatly appreciated.

Tuesday Bug Triage meeting notes:
https://etherpad.openstack.org/p/openstack_ansible_bug_triage.2015-03-03-16.00

General Meeting information:
https://wiki.openstack.org/wiki/Meetings/openstack-ansible

[nexus (nexus.nex)]

5 node POC. I re-did the entire deployment and did not come across this error again. It might have been due to some MTU issue/changes on our end. Thanks