OVB devmode does not inject repos into the overcloud image

[John Trowbridge]

When using libvirt based quickstart devmode, we have a task which updates the repos in the overcloud image.[1] We have similar functionality in our quickstart based OVB CI.[2]

However, if a developer tries to use devmode via OVB we have no such task. This makes any reproducer of CI on a package installed on the overcloud invalid, because the overcloud images will be exactly what is on the tripleo mirror server, rather than having the intended change.

I am marking this as critical, because OVB devmode is fairly crippled in usefulness until we fix this.

[1] https://github.com/openstack/tripleo-quickstart/blob/master/roles/libvirt/setup/undercloud/tasks/main.yml#L41-L45

[2] https://github.com/openstack-infra/tripleo-ci/blob/master/toci-quickstart/playbooks/ovb.yml#L53

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/472737

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/492453

[Ronelle Landy (rlandy)]

Rebased https://review.openstack.org/472737.
Will test this out and comment.

[Ronelle Landy (rlandy)]

In response to:

"However, if a developer tries to use devmode via OVB we have no such task. This makes any reproducer of CI on a package installed on the overcloud invalid, because the overcloud images will be exactly what is on the tripleo mirror server, rather than having the intended change."

Looking at:

https://github.com/openstack/tripleo-quickstart/blob/master/ci-scripts/full-deploy-ovb.sh#L76

If you add a change, you trigger this execution of quickstart where:
--extra-vars build_test_packages="true" \
--extra-vars ib_repo_image_inject="true" \

This results in the following steps being triggered:

20:13:09 TASK [modify-image : virt-customize args --> upload files] *********************
20:13:09 task path: /home/.../ansible/roles/modify-image/tasks/libguestfs.yml:24
20:13:09 Friday 15 September 2017 20:13:09 +0000 (0:00:00.264) 0:20:41.715 ******
20:13:09 ok: [undercloud] => (item={u'dest': u'/tmp/gating_repo.tar.gz', u'src': u'/tmp/gating_repo.tar.gz'}) => {"ansible_facts": {"_vc_args": " --upload /tmp/gating_repo.tar.gz:/tmp/gating_repo.tar.gz"}, "changed": false, "item": {"dest": "/tmp/gating_repo.tar.gz", "src": "/tmp/gating_repo.tar.gz"}}

20:13:10 TASK [modify-image : virt-customize args --> modify script] ********************
...
20:13:10 ok: [undercloud] => {"ansible_facts": {"_vc_args": " --upload /tmp/gating_repo.tar.gz:/tmp/gating_repo.tar.gz --run /home/stack/install-built-repo.sh"}, "changed": false}

20:13:11 TASK [modify-image : Run virt-customize on the provided image] *****************
20:13:11 task path: /home/rhos-ci/.../ansible/roles/modify-image/tasks/libguestfs.yml:40
20:13:11 Friday 15 September 2017 20:13:11 +0000 (0:00:00.987) 0:20:43.555 ******
20:39:41 changed: [undercloud] => {"changed": true, "cmd": "virt-customize --upload /tmp/gating_repo.tar.gz:/tmp/gating_repo.tar.gz --run /home/stack/install-built-repo.sh -a /home/stack/overcloud-full.qcow2 > /home/stack/_home_stack_install-built-repo.sh.log 2>&1", "delta": "0:26:29.622662", "end": "2017-09-15 20:39:41.561385", "rc": 0, "start": "2017-09-15 20:13:11.938723", "stderr": "", "stdout": "", "stdout_lines": [], "warnings": []}

[Ronelle Landy (rlandy)]

Looking at the repos on the overcloud, for example:

https://thirdparty.logs.rdoproject.org/jenkins-tq-gate-devmode-master-ovb-rdocloud-multiple-nics-231/overcloud-novacompute-0/etc/yum.repos.d/

See gating.repo.gz.

[Ronelle Landy (rlandy)]

Comparing rpms on the undercloud and compute overcloud node:

[stack@371-rhos-ci-247-undercloud ~]$ rpm -qa | grep 'openstack-tripleo'
openstack-tripleo-puppet-elements-7.0.0-0.20170901071200.165ffaf.el7.centos.noarch
openstack-tripleo-ui-7.3.1-0.20170908191546.9a22f02.el7.centos.noarch
openstack-tripleo-heat-templates-7.0.0-0.20170914171151.ab682ed.el7.centos.noarch
openstack-tripleo-common-7.5.1-0.20170915201020.9ee52dd.el7.centos.noarch
openstack-tripleo-image-elements-7.0.0-0.20170831032840.82fc1bb.el7.centos.noarch
openstack-tripleo-validations-7.3.1-0.20170904180256.cb5d380.el7.centos.noarch
openstack-tripleo-common-containers-7.5.1-0.20170915201020.9ee52dd.el7.centos.noarch

note triple-common-* is updated

Version from delorean repo:
[ ] openstack-tripleo-common-7.5.1-0.20170913110856.b46a35a.el7.centos.noarch.rpm 2017-09-13 11:09 192K
[ ] openstack-tripleo-common-7.5.1-0.20170913110856.b46a35a.el7.centos.src.rpm 2017-09-13 11:09 192K

On the overcloud compute:

[heat-admin@overcloud-novacompute-0 ~]$ ls /opt/gating_repo
openstack-tripleo-common-7.5.1-0.20170915201020.9ee52dd.el7.centos.noarch.rpm openstack-tripleo-common-containers-7.5.1-0.20170915201020.9ee52dd.el7.centos.noarch.rpm
openstack-tripleo-common-7.5.1-0.20170915201020.9ee52dd.el7.centos.src.rpm openstack-tripleo-common-devtools-7.5.1-0.20170915201020.9ee52dd.el7.centos.noarch.rpm
openstack-tripleo-common-container-base-7.5.1-0.20170915201020.9ee52dd.el7.centos.noarch.rpm repodata

[heat-admin@overcloud-novacompute-0 ~]$ rpm -qa | grep tripleo
puppet-tripleo-8.0.0-0.20170908063251.d5a2360.el7.centos.noarch

[heat-admin@overcloud-novacompute-0 ~]$ yum provides openstack-tripleo-common
Loaded plugins: fastestmirror, priorities
Determining fastest mirrors
 * base: mirror.hostduplex.com
 * extras: mirrors.oit.uci.edu
 * updates: centos-distro.cavecreek.net
openstack-tripleo-common-7.5.1-0.20170915201020.9ee52dd.el7.centos.noarch : Python library for code used by TripleO projects.
Repo : gating-repo

[Ronelle Landy (rlandy)]

Does the above meet the requirements for the change being on the overcloud?

[Ronelle Landy (rlandy)]

Chatted with John ...

<trown> rlandy: seems like it works from what you posted, not sure if that was always true or if something changed

Since this was logged a few months ago and we have had no reports of issues since, I am closing this out as fixed.

[OpenStack Infra (hudson-openstack)]

Change abandoned by John Trowbridge (<email address hidden>) on branch: master
Review: https://review.openstack.org/472737