There are sometimes segfaults while running jobs on centos8:
https://431f4da4fd1116536511-20277ea6d4d3827fb7916ddfe4cdbf50.ssl.cf2.rackcdn.com/738469/2/check/tripleo-ci-centos-8-containers-undercloud-minion/f6d41b7/logs/undercloud/var/log/extra/journal.txt
Jun 30 11:55:27 centos-8-vexxhost-ca-ymq-1-0017476676 systemd-logind[751]: Removed session 22.
Jun 30 11:55:28 centos-8-vexxhost-ca-ymq-1-0017476676 kernel: ansible-playboo[35640]: segfault at 7ff652c63ba0 ip 00007ff65e9c5a68 sp 00007ff6544b3970 error 4 in libpython3.6m.so.1.0[7ff65e8b4000+2b2000]
Jun 30 11:55:28 centos-8-vexxhost-ca-ymq-1-0017476676 kernel: Code: c4 18 5b 5d c3 0f 1f 00 66 2e 0f 1f 84 00 00 00 00 00 53 48 89 fb 48 83 ec 10 48 8b 05 71 4f 43 00 48 85 c0 0f 84 a3 00 00 00 <48> 8b 48 18 48 8b 3d c5 4f 3a 00 48 c7 00 01 00 00 00 83 2d 57 4f
Jun 30 11:55:28 centos-8-vexxhost-ca-ymq-1-0017476676 systemd[1]: Created slice system-systemd\x2dcoredump.slice.
Jun 30 11:55:28 centos-8-vexxhost-ca-ymq-1-0017476676 systemd[1]: Started Process Core Dump (PID 35799/UID 0).
Jun 30 11:55:28 centos-8-vexxhost-ca-ymq-1-0017476676 systemd-coredump[35800]: Process 35468 (ansible-playboo) of user 1000 dumped core.
Stack trace of thread 35640:
#0 0x00007ff65e9c5a68 PyCFunction_NewEx (libpython3.6m.so.1.0)
#1 0x00007ff65e9cce2c method_get (libpython3.6m.so.1.0)
#2 0x00007ff65e9cd495 _PyObject_GenericGetAttrWithDict (libpython3.6m.so.1.0)
#3 0x00007ff65ea4470b _PyEval_EvalFrameDefault (libpython3.6m.so.1.0)
#4 0x00007ff65e9c8887 _PyEval_EvalCodeWithName (libpython3.6m.so.1.0)
#5 0x00007ff65e9f70d0 fast_function (libpython3.6m.so.1.0)
#6 0x00007ff65ea1beb6 call_function (libpython3.6m.so.1.0)
#7 0x00007ff65ea445b4 _PyEval_EvalFrameDefault (libpython3.6m.so.1.0)
#8 0x00007ff65e9c8887 _PyEval_EvalCodeWithName (libpython3.6m.so.1.0)
#9 0x00007ff65e9f70d0 fast_function (libpython3.6m.so.1.0)
#10 0x00007ff65ea1beb6 call_function (libpython3.6m.so.1.0)
#11 0x00007ff65ea445b4 _PyEval_EvalFrameDefault (libpython3.6m.so.1.0)
#12 0x00007ff65e9c8887 _PyEval_EvalCodeWithName (libpython3.6m.so.1.0)
#13 0x00007ff65e9f70d0 fast_function (libpython3.6m.so.1.0)
#14 0x00007ff65ea1beb6 call_function (libpython3.6m.so.1.0)
#15 0x00007ff65ea445b4 _PyEval_EvalFrameDefault (libpython3.6m.so.1.0)
#16 0x00007ff65e9c8887 _PyEval_EvalCodeWithName (libpython3.6m.so.1.0)
#17 0x00007ff65e9f70d0 fast_function (libpython3.6m.so.1.0)
#18 0x00007ff65ea1beb6 call_function (libpython3.6m.so.1.0)
#19 0x00007ff65ea445b4 _PyEval_EvalFrameDefault (libpython3.6m.so.1.0)
#20 0x00007ff65ea26727 PyEval_EvalCodeEx (libpython3.6m.so.1.0)
#21 0x00007ff65ea2769e function_call (libpython3.6m.so.1.0)
#22 0x00007ff65e9ca3a7 PyObject_Call (libpython3.6m.so.1.0)
#23 0x00007ff65ea4631a _PyEval_EvalFrameDefault (libpython3.6m.so.1.0)
#24 0x00007ff65e9f6f1a fast_function (libpython3.6m.so.1.0)
#25 0x00007ff65ea1beb6 call_function (libpython3.6m.so.1.0)
#26 0x00007ff65ea445b4 _PyEval_EvalFrameDefault (libpython3.6m.so.1.0)
#27 0x00007ff65e9f6f1a fast_function (libpython3.6m.so.1.0)
#28 0x00007ff65ea1beb6 call_function (libpython3.6m.so.1.0)
#29 0x00007ff65ea445b4 _PyEval_EvalFrameDefault (libpython3.6m.so.1.0)
#30 0x00007ff65e9c964a _PyFunction_FastCallDict (libpython3.6m.so.1.0)
#31 0x00007ff65e9ca00e _PyObject_FastCallDict (libpython3.6m.so.1.0)
#32 0x00007ff65e9d3211 _PyObject_Call_Prepend (libpython3.6m.so.1.0)
#33 0x00007ff65e9ca3a7 PyObject_Call (libpython3.6m.so.1.0)
#34 0x00007ff65eac7f12 t_bootstrap (libpython3.6m.so.1.0)
#35 0x00007ff65ea6b734 pythread_wrapper (libpython3.6m.so.1.0)
#36 0x00007ff65e69c2de start_thread (libpthread.so.0)
#37 0x00007ff65dc43e83 __clone (libc.so.6)
Stack trace of thread 35468:
#0 0x00007ff65e6a27da pthread_cond_timedwait@@GLIBC_2.3.2 (libpthread.so.0)
#1 0x00007ff65e9bdbd9 PyEval_RestoreThread (libpython3.6m.so.1.0)
#2 0x00007ff65ea97254 time_sleep (libpython3.6m.so.1.0)
#3 0x00007ff65ea1b981 _PyCFunction_FastCallDict (libpython3.6m.so.1.0)
#4 0x00007ff65ea1bdd0 call_function (libpython3.6m.so.1.0)
#5 0x00007ff65ea445b4 _PyEval_EvalFrameDefault (libpython3.6m.so.1.0)
#6 0x00007ff65e9f6f1a fast_function (libpython3.6m.so.1.0)
#7 0x00007ff65ea1beb6 call_function (libpython3.6m.so.1.0)
#8 0x00007ff65ea445b4 _PyEval_EvalFrameDefault (libpython3.6m.so.1.0)
#9 0x00007ff65e9c8887 _PyEval_EvalCodeWithName (libpython3.6m.so.1.0)
#10 0x00007ff65e9f70d0 fast_function (libpython3.6m.so.1.0)
#11 0x00007ff65ea1beb6 call_function (libpython3.6m.so.1.0)
#12 0x00007ff65ea445b4 _PyEval_EvalFrameDefault (libpython3.6m.so.1.0)
#13 0x00007ff65e9c8887 _PyEval_EvalCodeWithName (libpython3.6m.so.1.0)
#14 0x00007ff65e9f70d0 fast_function (libpython3.6m.so.1.0)
#15 0x00007ff65ea1beb6 call_function (libpython3.6m.so.1.0)
#16 0x00007ff65ea45152 _PyEval_EvalFrameDefault (libpython3.6m.so.1.0)
#17 0x00007ff65e9f6f1a fast_function (libpython3.6m.so.1.0)
#18 0x00007ff65ea1beb6 call_function (libpython3.6m.so.1.0)
#19 0x00007ff65ea445b4 _PyEval_EvalFrameDefault (libpython3.6m.so.1.0)
#20 0x00007ff65e9c8887 _PyEval_EvalCodeWithName (libpython3.6m.so.1.0)
#21 0x00007ff65e9f70d0 fast_function (libpython3.6m.so.1.0)
#22 0x00007ff65ea1beb6 call_function (libpython3.6m.so.1.0)
#23 0x00007ff65ea445b4 _PyEval_EvalFrameDefault (libpython3.6m.so.1.0)
#24 0x00007ff65ea26727 PyEval_EvalCodeEx (libpython3.6m.so.1.0)
#25 0x00007ff65ea2747b PyEval_EvalCode (libpython3.6m.so.1.0)
#26 0x00007ff65eac6882 run_mod (libpython3.6m.so.1.0)
#27 0x00007ff65e99e362 PyRun_FileExFlags (libpython3.6m.so.1.0)
#28 0x00007ff65e9a2d2a PyRun_SimpleFileExFlags (libpython3.6m.so.1.0)
#29 0x00007ff65e9a401d Py_Main.cold.2600 (libpython3.6m.so.1.0)
#30 0x0000558084992b96 n/a (/home/zuul/workspace/.quickstart/bin/python3)
Jun 30 11:55:41 centos-8-vexxhost-ca-ymq-1-0017476676 ansible-stat[35853]: Invoked with checksum_algorithm=sha1 get_checksum=True path=/home/zuul/workspace/logs/zuul_console.json follow=False get_mime=True get_attributes=True get_md5=None
Another example: https://5c06aefe35fb69c8caeb-e8808aa78747cb96cfa3bd19c511369e.ssl.cf2.rackcdn.com/737671/1/gate/tripleo-ci-centos-8-containers-multinode/7a83dc9/logs/undercloud/var/log/extra/journal.txt
un 29 14:34:17 centos-8-vexxhost-ca-ymq-1-0017449630 systemd-logind[759]: Session 61 logged out. Waiting for processes to exit.
Jun 29 14:34:17 centos-8-vexxhost-ca-ymq-1-0017449630 systemd-logind[759]: Removed session 61.
Jun 29 14:34:18 centos-8-vexxhost-ca-ymq-1-0017449630 kernel: ansible-playboo[35708]: segfault at 55e014884c18 ip 000055e014884c18 sp 00007ffce94ffee8 error 15
Jun 29 14:34:18 centos-8-vexxhost-ca-ymq-1-0017449630 kernel: Code: 00 00 00 00 00 00 00 00 00 00 b1 03 00 00 00 00 00 00 50 aa 6b c0 60 7f 00 00 90 b4 6b c0 60 7f 00 00 fa ff ff ff ff ff ff ff <cb> 01 00 00 00 00 00 00 a0 3c 8a c4 60 7f 00 00 00 00 00 00 00 00
Jun 29 14:34:18 centos-8-vexxhost-ca-ymq-1-0017449630 systemd[1]: Created slice system-systemd\x2dcoredump.slice.
Jun 29 14:34:18 centos-8-vexxhost-ca-ymq-1-0017449630 systemd[1]: Started Process Core Dump (PID 43445/UID 0).
Jun 29 14:34:18 centos-8-vexxhost-ca-ymq-1-0017449630 systemd-coredump[43446]: Process 35708 (ansible-playboo) of user 1000 dumped core.
Stack trace of thread 35708:
#0 0x000055e014884c18 n/a (n/a)
Jun 29 14:34:19 centos-8-vexxhost-ca-ymq-1-0017449630 sshd[30268]: pam_unix(sshd:session): session closed for user zuul
Jun 29 14:34:19 centos-8-vexxhost-ca-ymq-1-0017449630 systemd-logind[759]: Session 23 logged out. Waiting for processes to exit.
Jun 29 14:34:19 centos-8-vexxhost-ca-ymq-1-0017449630 systemd-logind[759]: Removed session 23.
