standalone f28 failing runtime error: invalid memory address or nil pointer dereference

[Quique Llorente]

At the fedora28 standalone job that uses centos-7 docker images we have the following error at noop review:
http://logs.openstack.org/45/560445/199/check/tripleo-ci-fedora-28-standalone/f515752/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz#_2018-11-20_08_46_18

3f20dba9b9f52d35e9b0143966208e986\\n\\r 0 B / 658 B \\r 658 B / 658 B 0s\\nCopying blob sha256:d80ce32896f73eeea378eb0ad02996695139ac78d486fb0f377535fcfc2afba3\\n\\r 0 B / 474 B \\r 474 B / 474 B 0s\\nCopying config sha256:2863e03ec7ada670a0826966f461263604a1719a3f8f56a7c69896b17bcb2543\\n\\r 0 B / 26.35 KB \\r 26.35 KB / 26.35 KB 0s\\nWriting manifest to image destination\\nStoring signatures\\npanic: runtime error: invalid memory address or nil pointer dereference\\n[signal SIGSEGV: segmentation violation code=0x1 addr=0x8 pc=0xc49a97]\

The hashes are the following:
centos-7:current-tripleo:3ed8ac0e93367a02ad53d9fa93467057724b6621_fd8eb74b
fedora-28:current-tripleo:9d86a14fa555e87e555aa43bc09160055d2821d4_9b470851
fedora-28:latest:5ff37f10280639737527226ca4bd97919d3725ea_4d4946bc

Could be related to this
https://github.com/containers/libpod/issues/1730

[Quique Llorente (quiquell)]

fedora28 is libpod is older than centos
http://logs.openstack.org/45/560445/199/check/tripleo-ci-fedora-28-standalone/f515752/logs/undercloud/var/log/extra/rpm-list.txt.gz
http://logs.openstack.org/09/618709/3/check/tripleo-ci-centos-7-standalone/c8b80dd/logs/undercloud/var/log/extra/rpm-list.txt.gz

[Quique Llorente (quiquell)]

Looks like we don't have newer version for podman from f28.

So from IRC:
13:24 <apevec> ykarel, podman is not getting updates in f28 I guess, we'd have to build Rawhide in copr :(
13:24 <quiquell> apevec: What's rawhide an copr ?
13:24 <apevec> quiquell, in centos-7 jobs, podman is coming from Virt SIG repo
13:24 <quiquell> apevec: well this is golang, we can ping directly to binaries if they are
13:24 <apevec> https://fedoraproject.org/wiki/Releases/Rawhide
13

[Alan Pevec (apevec)]

we'll need to start building podman for f28-stabilized in https://copr.fedorainfracloud.org/coprs/g/openstack-sig/fedora-overrides/

[Alan Pevec (apevec)]

podman-0.11.1.1-3.git594495d.fc28 Copr rebuild from CentOS VirtSIG CBS Koji SRPM worked: https://copr-be.cloud.fedoraproject.org/results/%40openstack-sig/fedora-overrides/fedora-28-x86_64/00827040-podman/
It is immediately available in the Copr repo (which is also bad, as new builds are not gated).

[yatin (yatinkarel)]

<<< Looks like we don't have newer version for podman from f28.

For CentOS:- it's latest because there we are using [centos7-virt-container] repo defined in delorean-deps:- https://trunk.rdoproject.org/centos7-master/delorean-deps.repo

For Fedora:- fedora stabilized repo is chasing fedora28 base and updates repo, so if the package is not in those repo it will not be in fedora stable-base repo:- https://trunk.rdoproject.org/fedora/stable-base/latest/

To get the builds not in fedora28 base repo to fedora stable-base repo:-
- can add it from koji build([1]), this doesn't work when the build in f29 or higher depends on package that's not available in f28 ex; python3.7, Also koji builds which don't have tags may be deleted.
- add it to copr[2](a member of openstack-sig can build there), can request on #rdo to get some build. Then with regular sync it will be available in fedora stable-base repo. In copr we can build for f28 from srpm or spec file.

For now podman-0.11.1.1-3.git594495d.fc28.x86_64.rpm is build in copr([3]) using srpm from CBS:- http://cbs.centos.org/kojifiles/packages/podman/0.11.1.1/3.git594495d.el7/src/podman-0.11.1.1-3.git594495d.el7.src.rpm

[1] https://review.rdoproject.org/r/#/c/17062/
[2] https://copr.fedorainfracloud.org/coprs/g/openstack-sig/fedora-overrides/packages/
[3] https://copr.fedorainfracloud.org/coprs/g/openstack-sig/fedora-overrides/build/827040/

[Alfredo Moralejo (amoralej)]

podman-0.11.1.1-3.git594495d and required dependencies have been updated to the fedora stable repo and should be fine now.