Vcenter-as-compute: Fresh install 4.0 # 3069 provisioning fails install_contrail

[Sarath]

This seen on Sanity when install_contrail and when repeated second time this not seen.

2017-05-04 14:23:46:684615: [stack@10.84.24.64] sudo: python -c 'from platform import linux_distribution; print linux_distribution()'
No handlers could be found for logger "paramiko.transport"
2017-05-04 14:23:46:685958: 2017-05-04 14:23:46:684929: Traceback (most recent call last):
2017-05-04 14:23:46:686696: File "/usr/local/lib/python2.7/dist-packages/fabric/main.py", line 743, in main
2017-05-04 14:23:46:686765: *args, **kwargs
2017-05-04 14:23:46:688306: File "/usr/local/lib/python2.7/dist-packages/fabric/tasks.py", line 375, in execute
2017-05-04 14:23:46:688421: multiprocessing
2017-05-04 14:23:46:688790: File "/usr/local/lib/python2.7/dist-packages/fabric/tasks.py", line 271, in _execute
2017-05-04 14:23:46:688884: return task.run(*args, **kwargs)
2017-05-04 14:23:46:689085: File "/usr/local/lib/python2.7/dist-packages/fabric/tasks.py", line 171, in run
2017-05-04 14:23:46:689194: return self.wrapped(*args, **kwargs)
2017-05-04 14:23:46:689365: File "/usr/local/lib/python2.7/dist-packages/fabric/decorators.py", line 53, in inner_decorator
2017-05-04 14:23:46:689451: return func(*args, **kwargs)
2017-05-04 14:23:46:689761: File "/usr/local/lib/python2.7/dist-packages/fabric/tasks.py", line 168, in __call__
2017-05-04 14:23:46:689861: return self.run(*args, **kwargs)
2017-05-04 14:23:46:690039: File "/usr/local/lib/python2.7/dist-packages/fabric/tasks.py", line 171, in run
2017-05-04 14:23:46:690127: return self.wrapped(*args, **kwargs)
2017-05-04 14:23:46:690303: File "/root/fabric-utils/fabfile/tasks/install.py", line 953, in install_contrail
2017-05-04 14:23:46:690389: if detect_ostype() == 'ubuntu':
2017-05-04 14:23:46:691193: File "/root/fabric-utils/fabfile/utils/fabos.py", line 24, in detect_ostype
2017-05-04 14:23:46:691300: (dist, version, extra) = get_linux_distro()
2017-05-04 14:23:46:691617: File "/root/fabric-utils/fabfile/utils/fabos.py", line 20, in get_linux_distro
2017-05-04 14:23:46:691707: (dist, version, extra) = ast.literal_eval(sudo(linux_distro))
2017-05-04 14:23:46:691861: File "/usr/local/lib/python2.7/dist-packages/fabric/network.py", line 633, in host_prompting_wrapper
2017-05-04 14:23:46:691951: return func(*args, **kwargs)
2017-05-04 14:23:46:692445: File "/usr/local/lib/python2.7/dist-packages/fabric/operations.py", line 1095, in sudo
2017-05-04 14:23:46:692536: stderr=stderr, timeout=timeout, shell_escape=shell_escape,
2017-05-04 14:23:46:693205: File "/usr/local/lib/python2.7/dist-packages/fabric/operations.py", line 909, in _run_command
2017-05-04 14:23:46:693271: channel=default_channel(), command=wrapped_command, pty=pty,
2017-05-04 14:23:46:693519: File "/usr/local/lib/python2.7/dist-packages/fabric/state.py", line 387, in default_channel
2017-05-04 14:23:46:693579: chan = _open_session()
2017-05-04 14:23:46:693827: File "/usr/local/lib/python2.7/dist-packages/fabric/state.py", line 379, in _open_session
2017-05-04 14:23:46:693892: return connections[env.host_string].get_transport().open_session()
2017-05-04 14:23:46:694046: File "/usr/lib/python2.7/dist-packages/paramiko/transport.py", line 702, in open_session
2017-05-04 14:23:46:694104: timeout=timeout)
2017-05-04 14:23:46:694671: File "/usr/lib/python2.7/dist-packages/paramiko/transport.py", line 823, in open_channel
2017-05-04 14:23:46:694735: raise e
2017-05-04 14:23:46:694974: socket.error: [Errno 104] Connection reset by peer
2017-05-04 14:23:46:695235: Disconnecting from 10.87.36.20... done.

Detailed sanity logs can be found below
http://anamika.englab.juniper.net:8080/job/ubuntu-14-04_mitaka_vcenter_compute_HA_Sanity/97/console

[Sarath (nsarath)]

nsarath@ubuntu-build04:/auto/cores/1688403$ ls -l
total 110288
-rwxrwxrwx 1 nsarath test 17838080 May 4 14:49 Ctrl-A-log.tar
-rwxrwxrwx 1 nsarath test 4526080 May 4 14:49 Ctrl-A-Utils.tar
-rwxrwxrwx 1 nsarath test 17991680 May 4 14:49 Ctrl-B-log.tar
-rwxrwxrwx 1 nsarath test 18135040 May 4 14:49 Ctrl-C-log.tar
-rwxrwxrwx 1 nsarath test 18329600 May 4 14:50 Esxi-1-log.tar
-rwxrwxrwx 1 nsarath test 18319360 May 4 14:50 Esxi-2-log.tar
-rwxrwxrwx 1 nsarath test 17305600 May 4 14:49 Kvm-1-log.tar

[amudhar (amudha)]

Sarath,

Like you updated, if this is not seen with latest liberty and mitaka, please close appropriately.

Thanks,
Amudha

[Sarath (nsarath)]

Sanity setup hit this issue consistently and Amudha will triage using dev setup

[amudhar (amudha)]

Not reproducible manually. Waiting on sanity testbed to debug.

[amudhar (amudha)]

This is seen only during sanity run. The host_build used in sanity runs is stack@10.84.24.64 (Centos).

2017-05-16 18:47:52:801549: [stack@10.84.24.64] sudo: python -c 'from platform import linux_distribution; print linux_distribution()'
2017-05-16 18:47:52:801850: Disconnecting from 10.87.36.20... Done.

The above command run on the host_build does not display any output during the first run of install_contrail and hence it fails.

-Amudha

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/31756
Submitter: amudha ramachandran (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/31757
Submitter: amudha ramachandran (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/31756
Committed: http://github.com/Juniper/contrail-fabric-utils/commit/0d3aa96ce0f5493d2451cab90e450afbd052e4cc
Submitter: Zuul (<email address hidden>)
Branch: master

commit 0d3aa96ce0f5493d2451cab90e450afbd052e4cc
Author: Amudha <email address hidden>
Date: Wed May 17 12:24:48 2017 -0700

Fix install_contrail failure

Remove disable_sysv_auto_generation() and install_xenial_prerequisites(),
taken care of in packaging now

Change-Id: Ifda16bf548645fbffcaae0877289e751f6d4869c
Closes-Bug: #1688403

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/31757
Committed: http://github.com/Juniper/contrail-fabric-utils/commit/45fb42016edbea54c60bb0036c3b31bec3b1b92e
Submitter: Zuul (<email address hidden>)
Branch: R4.0

commit 45fb42016edbea54c60bb0036c3b31bec3b1b92e
Author: Amudha <email address hidden>
Date: Wed May 17 12:24:48 2017 -0700

Fix install_contrail failure

Remove disable_sysv_auto_generation() and install_xenial_prerequisites(),
taken care of in packaging now

Change-Id: Ifda16bf548645fbffcaae0877289e751f6d4869c
Closes-Bug: #1688403