Rocky promotion jobs failing on tripleo-modify-image-playbook

[Arx Cruz]

http://logs.rdoproject.org/openstack-periodic-24hr/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-rocky/d4afcf4/logs/undercloud/home/zuul/install-undercloud.log.txt.gz

INFO:tripleo_common.image.image_uploader:Running skopeo inspect docker://trunk.registry.rdoproject.org/tripleorocky/centos-binary-zaqar:5eb012b4f62615519707a926cc62528f5dc54a44_13fae5e9
INFO:tripleo_common.image.image_uploader:Running skopeo inspect docker://trunk.registry.rdoproject.org/tripleorocky/centos-binary-tempest:5eb012b4f62615519707a926cc62528f5dc54a44_13fae5e9
INFO:tripleo_common.image.image_uploader:Running skopeo inspect docker://trunk.registry.rdoproject.org/tripleorocky/centos-binary-rsyslog-base:5eb012b4f62615519707a926cc62528f5dc54a44_13fae5e9
INFO:tripleo_common.image.image_uploader:Using config files: ['/tmp/tmpKwCtqL']
INFO:tripleo_common.image.image_uploader:imagename: trunk.registry.rdoproject.org/tripleorocky/centos-binary-rsyslog-base:5eb012b4f62615519707a926cc62528f5dc54a44_13fae5e9
INFO:tripleo_common.image.image_uploader:Running skopeo inspect --tls-verify=false docker://192.168.24.1:8787/tripleorocky/centos-binary-rsyslog-base:5eb012b4f62615519707a926cc62528f5dc54a44_13fae5e9-updated-20190710050141
INFO:tripleo_common.image.image_uploader:Pulling trunk.registry.rdoproject.org/tripleorocky/centos-binary-rsyslog-base
INFO:tripleo_common.actions.ansible:Running ansible-playbook command: ['ansible-playbook', '-vvvvv', '/tmp/tripleo-modify-image-playbook-eZNCvb/playbook.yaml']
Traceback (most recent call last):
  File "/usr/bin/tripleo-container-image-prepare", line 100, in <module>
    env, roles_data, cleanup=args.cleanup)
  File "/usr/lib/python2.7/site-packages/tripleo_common/image/kolla_builder.py", line 193, in container_images_prepare_multi
    uploader.upload()
  File "/usr/lib/python2.7/site-packages/tripleo_common/image/image_uploader.py", line 132, in upload
    uploader.run_tasks()
  File "/usr/lib/python2.7/site-packages/tripleo_common/image/image_uploader.py", line 514, in run_tasks
    result = self.upload_image(*first)
  File "/usr/lib/python2.7/site-packages/tripleo_common/image/image_uploader.py", line 262, in upload_image
    target_image_source_tag, append_tag)
  File "/usr/lib/python2.7/site-packages/tripleo_common/image/image_uploader.py", line 212, in run_modify_playbook
    result = action.run(None)
  File "/usr/lib/python2.7/site-packages/tripleo_common/actions/ansible.py", line 589, in run
    log_errors=processutils.LogErrors.ALL)
  File "/usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py", line 424, in execute
    cmd=sanitized_cmd)
oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command.
Command: ansible-playbook -vvvvv /tmp/tripleo-modify-image-playbook-eZNCvb/playbook.yaml
Exit code: 2

[Sagi (Sergey) Shnaidman (sshnaidm)]

[tripleo-modify-image : Set original_user with Docker] ********************
task path: /usr/share/ansible/roles/tripleo-modify-image/tasks/get_original_user.yml:23
Wednesday 10 July 2019 05:13:19 +0000 (0:00:00.054) 0:00:02.284 ********
fatal: [localhost]: FAILED! => {

The task includes an option with an undefined variable. The error was: 'dict object' has no attribute 'images'

The error appears to have been in '/usr/share/ansible/roles/tripleo-modify-image/tasks/get_original_user.yml': line 23, column 7, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

      when: source_image_facts.images | length == 0
    - name: Set original_user with Docker
      ^ here

[Emilien Macchi (emilienm)]

It's a bug in Ansible, we can reproduce with this simple playbook:

https://paste.fedoraproject.org/paste/oChSmQdflHKwBJRcslbLzw

I'll file the issue and in the meantime we have a workaround : https://review.opendev.org/#/c/670065

[Emilien Macchi (emilienm)]

Turns out it's an expected behaviour in Ansible: https://github.com/ansible/ansible/issues/15710#issuecomment-216645922

[Alex Schultz (alex-schultz)]

https://review.opendev.org/#/c/670065/