FileNotFoundError: [Errno 2] No such file or directory: '/var/lib/dnf/rpmdb_lock.pid'

[chandan kumar]

https://zuul.opendev.org/t/openstack/builds?job_name=tripleo-ansible-centos-stream-molecule-tripleo_packages&project=openstack/tripleo-ansible is failing continuously on
848810: Use ubi9-init image as base | https://review.opendev.org/c/openstack/tripleo-ansible/+/848810
with following error:
```
TASK [tripleo_packages : Update all packages] **********************************
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: FileNotFoundError: [Errno 2] No such file or directory: '/var/lib/dnf/rpmdb_lock.pid'
fatal: [centos]: FAILED! => changed=false
  module_stderr: |-
    Traceback (most recent call last):
      File "/root/.ansible/tmp/ansible-tmp-1658979600.837124-54448-133560901084165/AnsiballZ_dnf.py", line 100, in <module>
        _ansiballz_main()
      File "/root/.ansible/tmp/ansible-tmp-1658979600.837124-54448-133560901084165/AnsiballZ_dnf.py", line 92, in _ansiballz_main
        invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
      File "/root/.ansible/tmp/ansible-tmp-1658979600.837124-54448-133560901084165/AnsiballZ_dnf.py", line 40, in invoke_module
        runpy.run_module(mod_name='ansible.modules.dnf', init_globals=dict(_module_fqn='ansible.modules.dnf', _modlib_path=modlib_path),
      File "/usr/lib64/python3.9/runpy.py", line 210, in run_module
        run_name -- if not None, this will be used for setting __name__;
      File "/usr/lib64/python3.9/runpy.py", line 97, in _run_module_code
        _run_code(code, mod_globals, init_globals,
      File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code
        exec(code, run_globals)
      File "/tmp/ansible_ansible.legacy.dnf_payload_upuzz8jf/ansible_ansible.legacy.dnf_payload.zip/ansible/modules/dnf.py", line 1363, in <module>
      File "/tmp/ansible_ansible.legacy.dnf_payload_upuzz8jf/ansible_ansible.legacy.dnf_payload.zip/ansible/modules/dnf.py", line 1352, in main
      File "/tmp/ansible_ansible.legacy.dnf_payload_upuzz8jf/ansible_ansible.legacy.dnf_payload.zip/ansible/modules/dnf.py", line 1326, in run
      File "/tmp/ansible_ansible.legacy.dnf_payload_upuzz8jf/ansible_ansible.legacy.dnf_payload.zip/ansible/modules/dnf.py", line 1245, in ensure
      File "/usr/lib/python3.9/site-packages/dnf/base.py", line 983, in do_transaction

      File "/usr/lib/python3.9/site-packages/dnf/lock.py", line 147, in __exit__
        os.unlink(self.target)
    FileNotFoundError: [Errno 2] No such file or directory: '/var/lib/dnf/rpmdb_lock.pid'
  module_stdout: ''
  msg: |-
    MODULE FAILURE
    See stdout/stderr for the exact error
  rc: 1
```
It might be happening due to cs8 to cs9 migration. I am adding this bug as a tracker and marking this job to non-voting to unblock the patch.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/tripleo-ansible/+/848810
Committed: https://opendev.org/openstack/tripleo-ansible/commit/b666c78954fbbe7edfd50b6b949c4921fc6c5518
Submitter: "Zuul (22348)"
Branch: stable/wallaby

commit b666c78954fbbe7edfd50b6b949c4921fc6c5518
Author: Chandan Kumar (raukadah) <email address hidden>
Date: Mon May 30 09:45:42 2022 +0530

    Use ubi9-init image as base

    CentOS-8 no longer exists and existing jobs are running
    on centos-stream-8. There will be no openstack master
    rpm packages for centos-stream-8.

    In order to use ubi9-init image in molecule jobs,
    we need to include following workarounds:
    - redhat-release package(RH specific package) is installed in ubi-init
      which does not work with centos-stream package.
      On removing redhat-release, dnf ansible breaks as they did not find
      --releasever during package installation.
      - By adding releasever in the package module fixes the issue.
    - In Molecule job, we can use any hostname either of centos or ubi both
      will work.
    - It changes the nodeset and image to ubi9.
    - Install pip < 20.3.0 to avoid https://pip.pypa.io/en/stable/topics/dependency-resolution/#backtracking
      and upstream requirements file to avoid cyclic dependency
      issue.
    - Set enable_fips: true in collectd jobs as in molecule it is not
      needed otherwise job will enter into retry_limit.

    We cannot use centos:stream9 image as, init support is removed
    from there.

    Below is the list of following jobs marked non-voting
    due to following bugs.
    - tripleo-ansible-centos-stream-molecule-tripleo_update_trusted_cas
      - https://bugs.launchpad.net/tripleo/+bug/1979807
    - tripleo-ansible-centos-stream-molecule-tripleo-modules
      - https://bugs.launchpad.net/tripleo/+bug/1979807
    - tripleo-ansible-centos-stream-molecule-tripleo_container_manage
      - https://bugs.launchpad.net/tripleo/+bug/1979810
    - tripleo-ansible-centos-stream-molecule-tripleo_redhat_enforce
      - https://bugs.launchpad.net/tripleo/+bug/1979971
    - tripleo-ansible-centos-stream-molecule-tripleo_collectd
      - https://bugs.launchpad.net/tripleo/+bug/1979972
    - tripleo-ansible-centos-stream-molecule-backup_and_restore
      - https://bugs.launchpad.net/tripleo/+bug/1980198

    Note: It is not a clean cherry pick.
    - Remove cap from molecule dependency
      to fix No module named 'ansiblelint.prerun'
    - Populate wallaby repos using tripleo-repos tool
    - This includes the commit 0bc2fa808d34235ffe7cf8d36920f6d96f0685b5
      which fixed the issue with the original change in master.
    - Marking tripleo-ansible-centos-stream-molecule-tripleo_packages
      to non-voting due to
      https://bugs.launchpad.net/tripleo/+bug/1983347

    Related-Bug: #1979807
    Related-Bug: #1979810
    Related-Bug: #1979971
    Related-Bug: #1979972
    Related-Bug: #1980198
    Related-Bug: #1983347

    Signed-off-by: Chandan Kumar (raukadah) <email address hidden>
    Change-Id: I75a27ca496fb7670ba22a6a22f62f001c8afe6a9
    (cherry picked from commit 4c9233d07c0886871719b46b2801c7b0565f26cc)