$ca_pkcs12 = '/var/lib/certmonger/local/creds' not found in centos-8 ussuri undercloud deployment

[wes hayutin]

CentOS-8 Ussuri undercloud

https://opendev.org/openstack/puppet-tripleo/src/branch/master/manifests/certmonger/ca/local.pp#L29

is failing..

    <13>Apr 8 19:26:38 puppet-user: Debug: /Stage[main]/Tripleo::Certmonger::Ca::Local/Exec[extract-and-trust-ca]/returns: Exec try 5/5
    <13>Apr 8 19:26:38 puppet-user: Debug: Exec[extract-and-trust-ca](provider=posix): Executing 'openssl pkcs12 -in /var/lib/certmonger/local/creds -out /etc/pki/ca-trust/source/anchors/cm-local-ca.pem -nokeys -nodes -passin pass:'' && update-ca-trust extract'
    <13>Apr 8 19:26:38 puppet-user: Debug: Executing: 'openssl pkcs12 -in /var/lib/certmonger/local/creds -out /etc/pki/ca-trust/source/anchors/cm-local-ca.pem -nokeys -nodes -passin pass:'' && update-ca-trust extract'
    <13>Apr 8 19:26:39 puppet-user: Debug: /Stage[main]/Tripleo::Certmonger::Ca::Local/Exec[extract-and-trust-ca]/returns: Sleeping for 1 seconds between tries
    <13>Apr 8 19:26:40 puppet-user: Notice: /Stage[main]/Tripleo::Certmonger::Ca::Local/Exec[extract-and-trust-ca]/returns: Can't open /var/lib/certmonger/local/creds for reading, No such file or directory
    <13>Apr 8 19:26:40 puppet-user: Notice: /Stage[main]/Tripleo::Certmonger::Ca::Local/Exec[extract-and-trust-ca]/returns: 140161319311168:error:02001002:system library:fopen:No such file or directory:crypto/bio/bss_file.c:72:fopen('/var/lib/certmonger/local/creds','rb')
    <13>Apr 8 19:26:40 puppet-user: Notice: /Stage[main]/Tripleo::Certmonger::Ca::Local/Exec[extract-and-trust-ca]/returns: 140161319311168:error:2006D080:BIO routines:BIO_new_file:no such file:crypto/bio/bss_file.c:79:
    <13>Apr 8 19:26:40 puppet-user: Error: /Stage[main]/Tripleo::Certmonger::Ca::Local/Exec[extract-and-trust-ca]: Failed to call refresh: 'openssl pkcs12 -in /var/lib/certmonger/local/creds -out /etc/pki/ca-trust/source/anchors/cm-local-ca.pem -nokeys -nodes -passin pass:'' && update-ca-trust extract' returned 1 instead of one of [0]
    <13>Apr 8 19:26:40 puppet-user: Error: /Stage[main]/Tripleo::Certmonger::Ca::Local/Exec[extract-and-trust-ca]: 'openssl pkcs12 -in /var/lib/certmonger/local/creds -out /etc/pki/ca-trust/source/anchors/cm-local-ca.pem -nokeys -nodes -passin pass:'' && update-ca-trust extract' returned 1 instead of one of [0]

[wes hayutin (weshayutin)]

[14:48:24] <ade_lee> so its something that is supposed to be created when certmonger starts up
[14:48:38] <ade_lee> certmonger creates it itself n startup

[Michele Baldessari (michele)]

Am hitting this one as well now in our env

[Michele Baldessari (michele)]

So something is removing the local folder:
[root@undercloud ~]# rpm -V certmonger
missing /var/lib/certmonger/local

Interestingly enough a bunch of certmonger stuff is missing in the image itself?
On the hypervisor I get:
virt-customize -v -a undercloud.qcow2 --run-command 'rpm -V certmonger'
missing /var/lib/certmonger/cas
missing /var/lib/certmonger/local
missing /var/lib/certmonger/requests
missing /var/run/certmonger

It's as if /var/lib/certmonger/{cas,requests} and /var/run/certmonger get recreated whereas /var/lib/certmonger/local does not.

In fact if I create /var/lib/certmonger/local, restart certmonger and then run the undercloud install it all works.

Now there are two possibilities:
A) Those folders were always missing and certmonger recreated all the things with no issue, but somehow it is unable to do so for some reason
B) Those folders were not always missing from the image, and something is removing them and certmonger cannot really copy with /var/lib/certmonger/local missing.

Now what I can observe is that certmonger does not create all these folders at startup?:
systemctl stop certmonger
rm -rf /var/lib/certmonger/*
systemctl start certmonger
ls -l /var/lib/certmonger
[root@undercloud certmonger]# ls -l
total 0
drwx------. 2 root root 124 Apr 9 07:21 cas
-rw-------. 1 root root 0 Apr 9 07:21 lock

Not sure if they get created at a certain time or when something gets triggered

[wes hayutin (weshayutin)]

caused by packaging fall out...

closing

[Marios Andreou (marios-b)]

Bug status has been set to 'Incomplete' and target milestone has been removed due to inactivity. If you disagree please re-set these values and reach out to us on freenode #tripleo

[Launchpad Janitor (janitor)]

[Expired for tripleo because there has been no activity for 60 days.]