The ansible-hardening role contains in its RedHat 7 implementation of STIG a single play which both adds and removes packages:
https://github.com/openstack/ansible-hardening/blob/master/tasks/rhel7stig/packages.yml#L16
This task frequently results in RPM database corruption on the target hosts: it looks as if ansible splits the task into multiple calls to yum and triggers them simultaneously, resulting in lock conflicts that break the RPM database:
2019-11-09 18:10:01,305 p=43314 u=root | failed: [redacted_hostname] (item=absent) => {"changed": false, "item": "absent", "module_stderr": "Warning: Permanently added 'redacted_ip' (E
CDSA) to the list of known hosts.\r\n------------------------------------------------------------------------------\n* WARNING
*\n* You are accessing a secured system and your actions will be logged along *\n* with identifying information. Disconnect immediately if you are not an *\
n* authorized user of this system. *\n------------------------------------------------------------------------------\nerror: rpmdb: BDB
0113 Thread/process 17868/140288263792704 failed: BDB1507 Thread died in Berkeley DB library\nerror: db5 error(-30973) from dbenv->failchk: BDB0087 DB_RUNRECOVERY: Fatal error, r
un database recovery\nerror: cannot open Packages index using db5 - (-30973)\nerror: cannot open Packages database in /var/lib/rpm\nTraceback (most recent call last):\n File \"
/tmp/ansible_8P4c67/ansible_module_yum.py\", line 1411, in <module>\n main()\n File \"/tmp/ansible_8P4c67/ansible_module_yum.py\", line 1373, in main\n my.conf\n File \"/
usr/lib/python2.7/site-packages/yum/__init__.py\", line 1079, in <lambda>\n conf = property(fget=lambda self: self._getConfig(),\n File \"/usr/lib/python2.7/site-packages/yum
/__init__.py\", line 349, in _getConfig\n startupconf = config.readStartupConfig(fn, root, releasever)\n File \"/usr/lib/python2.7/site-packages/yum/config.py\", line 1115, i
n readStartupConfig\n startupconf.distroverpkg)\n File \"/usr/lib/python2.7/site-packages/yum/config.py\", line 1260, in _getsysver\n raise Errors.YumBaseError(\"Error: \"
+ str(e))\nyum.Errors.YumBaseError: Error: rpmdb open failed\n", "module_stdout": "", "msg": "MODULE FAILURE", "rc": 1}
In my environment, where this issue has plagued our upgrade and scaling tasks, I have broken the task up into two separate plays; one for adding packages and one for removing them. This approach does not result in RPM database corruption. I recommend adopting this approach and backporting it to supported OSA releases.
Fix proposed to branch: master
Review: https:/