Description
===========
I run a Red Hat OpenStack 13 lab cloud in "forced" FIPS mode (fips=1 kernel flag) and this is one of the few issues I encountered along the way that required patching. Basically, MD5 is disabled in OpenSSL / Python's hashlib / NSS and anything else that honors FIPS mode, regardless of MD5's intended usage (for data security or not).
When creating an instance, MD5 is being used by privsep for get 7 random characters to append to the image cache filename.
Proposed patch - https://review.openstack.org/#/c/615704/
Steps to reproduce
==================
Launch an instance using CLI/API or Horizon.
(bootstrap-venv) [root@compute-1 bootstrap]# openstack server create \
--image '54568928-d6bd-47b0-8985-faa34b8aefd2' \
--flavor '63c23b65-6e9c-4910-887a-188a7c8b61da' \
--key-name 'default' \
--security-group '59c728b5-4856-4714-a4ae-21fcb239f8e1' \
--network '8017a74e-de17-4abd-8261-55abad7fbf35' \
'test-fips'
Expected result
===============
Instance gets created and enters the "running" state.
Actual result
=============
Instance sometimes gets stuck in "scheduling" state but other times fails with a Horizon error popup of "Error: Failed to perform requested operation on instance "test-fips", the instance has an error status: Please try again later [Error: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 585f5f53-aa31-4e88-9a13-bc1d517e9d0f.]".
/var/log/nova/nova-compute.log shows a FIPS violation, stack trace -
2018-11-06 01:42:24.202 35447 ERROR nova.compute.manager [req-65312099-b42c-464e-a5d7-e46e0b07edc9 31a8c18286284697a9714cfe327f535f 3c3458e78f864f9385171fa621e5c757 - default default] [instance: 585f5f53-aa31-4e88-9a13-bc1d517e9d0f] Instance failed to spawn: ValueError: error:060800A
3:digital envelope routines:EVP_DigestInit_ex:disabled for fips
2018-11-06 01:42:24.202 35447 ERROR nova.compute.manager [instance: 585f5f53-aa31-4e88-9a13-bc1d517e9d0f] Traceback (most recent call last):
2018-11-06 01:42:24.202 35447 ERROR nova.compute.manager [instance: 585f5f53-aa31-4e88-9a13-bc1d517e9d0f] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2236, in _build_resources
2018-11-06 01:42:24.202 35447 ERROR nova.compute.manager [instance: 585f5f53-aa31-4e88-9a13-bc1d517e9d0f] yield resources
2018-11-06 01:42:24.202 35447 ERROR nova.compute.manager [instance: 585f5f53-aa31-4e88-9a13-bc1d517e9d0f] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2016, in _build_and_run_instance
2018-11-06 01:42:24.202 35447 ERROR nova.compute.manager [instance: 585f5f53-aa31-4e88-9a13-bc1d517e9d0f] block_device_info=block_device_info)
2018-11-06 01:42:24.202 35447 ERROR nova.compute.manager [instance: 585f5f53-aa31-4e88-9a13-bc1d517e9d0f] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 3084, in spawn
2018-11-06 01:42:24.202 35447 ERROR nova.compute.manager [instance: 585f5f53-aa31-4e88-9a13-bc1d517e9d0f] block_device_info=block_device_info)
2018-11-06 01:42:24.202 35447 ERROR nova.compute.manager [instance: 585f5f53-aa31-4e88-9a13-bc1d517e9d0f] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 3470, in _create_image
2018-11-06 01:42:24.202 35447 ERROR nova.compute.manager [instance: 585f5f53-aa31-4e88-9a13-bc1d517e9d0f] os_type_with_default)
2018-11-06 01:42:24.202 35447 ERROR nova.compute.manager [instance: 585f5f53-aa31-4e88-9a13-bc1d517e9d0f] File "/usr/lib/python2.7/site-packages/nova/virt/disk/api.py", line 97, in get_file_extension_for_os_type
2018-11-06 01:42:24.202 35447 ERROR nova.compute.manager [instance: 585f5f53-aa31-4e88-9a13-bc1d517e9d0f] return utils.get_hash_str(extension)[:7]
2018-11-06 01:42:24.202 35447 ERROR nova.compute.manager [instance: 585f5f53-aa31-4e88-9a13-bc1d517e9d0f] File "/usr/lib/python2.7/site-packages/nova/utils.py", line 1064, in get_hash_str
2018-11-06 01:42:24.202 35447 ERROR nova.compute.manager [instance: 585f5f53-aa31-4e88-9a13-bc1d517e9d0f] return hashlib.md5(base_str).hexdigest()
2018-11-06 01:42:24.202 35447 ERROR nova.compute.manager [instance: 585f5f53-aa31-4e88-9a13-bc1d517e9d0f] ValueError: error:060800A3:digital envelope routines:EVP_DigestInit_ex:disabled for fips
2018-11-06 01:42:24.202 35447 ERROR nova.compute.manager [instance: 585f5f53-aa31-4e88-9a13-bc1d517e9d0f]
2018-11-06 01:42:24.204 35447 INFO nova.compute.manager [req-65312099-b42c-464e-a5d7-e46e0b07edc9 31a8c18286284697a9714cfe327f535f 3c3458e78f864f9385171fa621e5c757 - default default] [instance: 585f5f53-aa31-4e88-9a13-bc1d517e9d0f] Terminating instance
/var/log/nova/nova-conductor.log also shows a FIPS error -
2018-11-06 01:42:25.728 35493 ERROR nova.scheduler.utils [req-73ae3a5e-9936-4f1d-a45e-4dc9e145cd77 5e361d86f0fa4cfc82a1a0d481538d2c 0088c9e9ed484e41be132623e4a9e780 - a7052d336b4d45b196ab457dde9b34c6 a7052d336b4d45b196ab457dde9b34c6] [instance: 585f5f53-aa31-4e88-9a13-bc1d517e9d0f] Error from last host: compute-1.us-east-1.nonplus.io (node compute-1.us-east-1.nonplus.io): [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1825, in _do_build_and_run_instance\n filter_properties, request_spec)\n', u' File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2105, in _build_and_run_instance\n instance_uuid=instance.uuid, reason=six.text_type(e))\n', u'RescheduledException: Build of instance 585f5f53-aa31-4e88-9a13-bc1d517e9d0f was re-scheduled: error:060800A3:digital envelope routines:EVP_DigestInit_ex:disabled for fips\n']
Environment
===========
(bootstrap-venv) [root@compute-1 bootstrap]# cat /etc/redhat-release
Red Hat Enterprise Linux Server release 7.5 (Maipo)
(bootstrap-venv) [root@compute-1 bootstrap]# sysctl crypto.fips_enabled && cat /proc/sys/crypto/fips_enabled
crypto.fips_enabled = 1
1
(bootstrap-venv) [root@compute-1 bootstrap]# rpm -qa | grep nova
openstack-nova-api-17.0.5-3.d7864fbgit.el7ost.noarch
openstack-nova-compute-17.0.5-3.d7864fbgit.el7ost.noarch
openstack-nova-novncproxy-17.0.5-3.d7864fbgit.el7ost.noarch
openstack-nova-placement-api-17.0.5-3.d7864fbgit.el7ost.noarch
python2-novaclient-10.1.0-1.el7ost.noarch
openstack-nova-common-17.0.5-3.d7864fbgit.el7ost.noarch
openstack-nova-conductor-17.0.5-3.d7864fbgit.el7ost.noarch
python-nova-17.0.5-3.d7864fbgit.el7ost.noarch
openstack-nova-console-17.0.5-3.d7864fbgit.el7ost.noarch
openstack-nova-scheduler-17.0.5-3.d7864fbgit.el7ost.noarch
