Custom BVT job [1] was launched without any modification to fuel-qa code, all customizations were made via job parameters, in particular OPENSTACK_RELEASE = 'Ubuntu+UCA 16.04' to deploy Xenial based environment.
Deployment was successful, OSTF also passed (according to log [2]).
Overrall test FAILED becase fuel-qa doesn't support Xenial fully, it wrongy choose 'rpm' to get list of packages on Debian-based system.
* From the log [2] we can see fuel-qa tried to call get_node_packages and use 'rpm' for that:
---
2016-07-28 23:39:00,936 - DEBUG __init__.py:56 -- Calling: get_node_packages with args: (SSHClient(host=10.109.15.7, port=22, auth=SSHAuth(username=root, password=<*masked*>, key=<private for pub: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCtXgjw+TnNbZTzJpGvZq5Dwuq98ss7ChxSGZ33jvTQg0qiYbxJUO562b07CJxMvh25zFzRDvW70KXYNuZXvzew2cuuPOfV1Ht8GVhBowqJv+/mjWJfCaBywltdHjWdMZspo7vQExen2jzlCcvDzdZFQ5kHHH8Kswsz5RYEbeF+YHuy35hA/8FIp0M53Lolliav5d//g1fJ2L/KyFt08NeN448SoHHk6W5hXzN8hOxcaPLmfRMd5hxqogGnDwJp1OtQMRF2X0vNnE64in60sNfy6BPaFey78/oyVf4jLytzHO1B67FRrWGVNfINiaLtMdQO6Bt72HVcIy1mKbCJrOQN>, keys=[None])), 'ceph_rados_gw', u'compute_ceph-osd', {'ceph_rados_gw': {}}) {}
2016-07-28 23:39:00,936 - DEBUG ssh_client.py:740 -- Executing command: 'rpm -qa --qf "%{name} %{version}"'
2016-07-28 23:39:01,004 - DEBUG ssh_client.py:719 -- rpm -qa --qf "%{name} %{version}"
execution results: Exit code: 127
2016-07-28 23:39:01,004 - ERROR __init__.py:67 -- get_node_packages raised: IndexError('list index out of range',)
Traceback: Traceback (most recent call last):
File "/home/jenkins/workspace/10.0.custom.ubuntu.bvt_2/fuelweb_test/__init__.py", line 60, in wrapped
result = func(*args, **kwargs)
File "/home/jenkins/workspace/10.0.custom.ubuntu.bvt_2/fuelweb_test/helpers/utils.py", line 215, in get_node_packages
node_packages = remote.execute(cmd)['stdout'][0].split('\r')[:-1]
IndexError: list index out of range
2016-07-28 23:39:01,005 - ERROR __init__.py:67 -- store_packages_json raised: IndexError('list index out of range',)
Traceback: Traceback (most recent call last):
File "/home/jenkins/workspace/10.0.custom.ubuntu.bvt_2/fuelweb_test/__init__.py", line 60, in wrapped
result = func(*args, **kwargs)
File "/home/jenkins/workspace/10.0.custom.ubuntu.bvt_2/fuelweb_test/helpers/utils.py", line 234, in store_packages_json
packages = get_node_packages(remote, func_name, role, packages)
File "/home/jenkins/workspace/10.0.custom.ubuntu.bvt_2/fuelweb_test/__init__.py", line 60, in wrapped
result = func(*args, **kwargs)
File "/home/jenkins/workspace/10.0.custom.ubuntu.bvt_2/fuelweb_test/helpers/utils.py", line 215, in get_node_packages
node_packages = remote.execute(cmd)['stdout'][0].split('\r')[:-1]
IndexError: list index out of range
---
* From the master node we can see that OS on the node in question is Ubuntu:
---
[root@nailgun ~]# fuel node
id | status | name | cluster | ip | mac | roles | pending_roles | online | group_id
---+--------+---------------------------+---------+-------------+-------------------+-------------------+---------------+--------+---------
4 | ready | slave-02_controller | 1 | 10.109.15.5 | 64:4c:99:cb:76:d3 | controller | | 1 | 1
2 | ready | slave-06_compute_ceph-osd | 1 | 10.109.15.9 | 64:21:ec:4d:26:26 | ceph-osd, compute | | 1 | 1
1 | ready | slave-01_controller | 1 | 10.109.15.4 | 64:8c:36:6c:32:51 | controller | | 1 | 1
6 | ready | slave-03_controller | 1 | 10.109.15.6 | 64:6f:58:14:a4:2b | controller | | 1 | 1
5 | ready | slave-05_compute_ceph-osd | 1 | 10.109.15.8 | 64:b1:80:a4:15:59 | ceph-osd, compute | | 1 | 1
3 | ready | slave-04_compute_ceph-osd | 1 | 10.109.15.7 | 64:49:b3:00:4b:76 | ceph-osd, compute | | 1 | 1
[root@nailgun ~]# ssh node-3 cat /etc/lsb-release
Warning: Permanently added 'node-3' (ECDSA) to the list of known hosts.
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=16.04
DISTRIB_CODENAME=xenial
DISTRIB_DESCRIPTION="Ubuntu 16.04.1 LTS"
---
* get_node_packages code [3]:
---
def get_node_packages(remote, func_name, node_role,
packages_dict, release=settings.OPENSTACK_RELEASE):
if settings.OPENSTACK_RELEASE_UBUNTU in release:
cmd = "dpkg-query -W -f='${Package} ${Version}'\r"
else:
cmd = 'rpm -qa --qf "%{name} %{version}"\r'
---
[1] https://custom-ci.infra.mirantis.net/job/10.0.custom.ubuntu.bvt_2/186/
[2] https://custom-ci.infra.mirantis.net/job/10.0.custom.ubuntu.bvt_2/186/artifact/logs/sys_test.log
[3] https://github.com/openstack/fuel-qa/blob/c8642a7c1d49c9c69eb934171ca3d63c0825d430/fuelweb_test/helpers/utils.py#L209-L214
Fix proposed to branch: master
Review: https:/