Periodic jobs fail to in run virt-customize on RDO Cloud

[Alfredo Moralejo]

OVB jobs with upload in periodic pipelines are failing for ocata/pipe/master when doing virt-customize:

06:00:48 TASK [modify-image : Run virt-customize on the provided image] *****************
06:00:48 task path: /home/jenkins/workspace/periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset002-ocata-upload/.quickstart/usr/local/share/ansible/roles/modify-image/tasks/libguestfs.yml:40
06:01:08 fatal: [localhost]: FAILED! => {"changed": true, "cmd": "virt-customize --run .//repo_setup.sh -a overcloud-full.qcow2 > .//.__repo_setup.sh.log 2>&1", "delta": "0:00:25.713577", "end": "2018-01-17 06:01:08.968759", "failed": true, "msg": "non-zero return code", "rc": 1, "start": "2018-01-17 06:00:43.255182", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []}

Full logs in [1], [2] and [3]

Running command with debug and trace enabled in a local reproducer we find http://paste.openstack.org/show/646502/

Note error:

[ 2.573801] general protection fault: 0000 [#1] SMP
[ 2.574091] Modules linked in:
[ 2.574091] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 3.10.0-693.11.6.el7.x86_64 #1
[ 2.574091] Hardware name: Red Hat KVM, BIOS 1.10.2-3.el7_4.1 04/01/2014
[ 2.574091] task: ffff88001eac8000 ti: ffff88001ead0000 task.ti: ffff88001ead0000
[ 2.574091] RIP: 0010:[<ffffffff8120a3f0>] [<ffffffff8120a3f0>] flush_old_exec+0x3b0/0x930
[ 2.574091] RSP: 0000:ffff88001ead3d10 EFLAGS: 00010246
[ 2.574091] RAX: 0000000000000001 RBX: 0000000000000000 RCX: 0000000000000049
[ 2.574091] RDX: 0000000000000000 RSI: ffff88001d558000 RDI: ffffffff81a7c100
[ 2.574091] RBP: ffff88001ead3d68 R08: 000000000001bfc0 R09: ffffffff8120a1e2
[ 2.574091] R10: ffff88001ead3b30 R11: 0000000000000007 R12: ffff88001d558000
[ 2.574091] R13: ffffffff81a7c100 R14: ffff88001eac8000 R15: ffff88001d53e400
[ 2.574091] FS: 0000000000000000(0000) GS:ffff88001ee00000(0000) knlGS:0000000000000000
[ 2.574091] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 2.574091] CR2: 0000000000000000 CR3: 00000000019fa000 CR4: 00000000003606f0
[ 2.574091] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 2.574091] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[ 2.574091] Call Trace:
[ 2.574091] [<ffffffff8126199c>] load_elf_binary+0x33c/0xe00
[ 2.574091] [<ffffffff812d6743>] ? ima_get_action+0x23/0x30
[ 2.574091] [<ffffffff812d5d7e>] ? process_measurement+0x8e/0x250
[ 2.574091] [<ffffffff812d6239>] ? ima_bprm_check+0x49/0x50
[ 2.574091] [<ffffffff81261660>] ? load_elf_library+0x220/0x220
[ 2.574091] [<ffffffff81209b9d>] search_binary_handler+0xed/0x300
[ 2.574091] [<ffffffff8120b1d6>] do_execve_common.isra.24+0x5b6/0x6c0
[ 2.574091] [<ffffffff81694d50>] ? rest_init+0x80/0x80
[ 2.574091] [<ffffffff8120b2f8>] do_execve+0x18/0x20
[ 2.574091] [<ffffffff8100202b>] run_init_process+0x2b/0x30
[ 2.574091] [<ffffffff81694d8d>] kernel_init+0x3d/0xf0
[ 2.574091] [<ffffffff816b8798>] ret_from_fork+0x58/0x90
[ 2.574091] [<ffffffff81694d50>] ? rest_init+0x80/0x80
[ 2.574091] Code: 00 3e 0f ab 08 48 8b 05 db e5 90 00 48 c1 e8 34 a8 01 74 19 65 8b 05 28 2c e0 7e a8 02 74 0e 31 d2 b8 01 00 00 00 b9 49 00 00 00 <0f> 30 bf 00 00 00 80 49 03 7c 24 58 48 8b 05 0d 7c 7f 00 72 0e
[ 2.574091] RIP [<ffffffff8120a3f0>] flush_old_exec+0x3b0/0x930
[ 2.574091] RSP <ffff88001ead3d10>
[ 2.966949] ---[ end trace 9f4f0bee8dbd524c ]---

[1] https://review.rdoproject.org/jenkins/job/periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset002-ocata-upload/1/consoleFull
[2] https://review.rdoproject.org/jenkins/job/periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset002-pike-upload/1/consoleFull
[3] https://review.rdoproject.org/jenkins/job/periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset002-master-upload/1/consoleFull

[Alfredo Moralejo (amoralej)]

Duplicate of https://bugs.launchpad.net/tripleo/+bug/1743749