Thales Luna HSM Firmware above v7.4.0 doesnt support CKM_AES_CBC_PAD with FIPS Mode Enabled

[Rajiv Mucheli]

Hi,

After discussing with Thales Engineering, Thales Luna HSM Firmware above v7.4.0 doesnt support CKM_AES_CBC_PAD wrapping mechanism. Unless we fix this in Barbican, we cannot upgrade to the latest Thales HSM firmware version.

Can i setup a call with Thales Engineering to discuss this further ?

I also found SoftHSM also doesnt support CKM_AES_CBC_PAD wrapping mechanism, more details are provided here :

https://github.com/opendnssec/SoftHSMv2/issues/405
https://github.com/opendnssec/SoftHSMv2/issues/229

Please let me know if we need to test on my Thales Luna A790 network device or if further information is required.

Regards,
Rajiv

[Rajiv Mucheli (rajiv.mucheli)]

FYI : https://opendev.org/openstack/barbican/src/branch/master/barbican/plugin/crypto/pkcs11.py#L142

My production is running on Openstack Barbican Zed Release, i am planning to upgrade to Bobcat once the release is available

[Rajiv Mucheli (rajiv.mucheli)]

correct link : https://opendev.org/openstack/barbican/src/branch/master/barbican/plugin/crypto/pkcs11.py#L778

the wrap and unwrap key mechanism is CKM_AES_CBC_PAD

[Douglas Mendizábal (dougmendizabal)]

According to Thales docs, CKM_AES_CBC_PAD is not supported for Wrap only when FIPS mode is enabled:

https://thalesdocs.com/gphsm/luna/7/docs/network/Content/sdk/mechanisms/CKM_AES_CBC_PAD.htm

[Rajiv Mucheli (rajiv.mucheli)]

Hi Doug,

Thanks for looking into this, until 7.3.3 wrapping is supported with FIPS MODE ENABLED. Is it possible to support this higher hsm firmware versions like 7.7.1 or higher with FIPS MODE ON ?

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/barbican/+/933461

[Rajiv Mucheli (rajiv.mucheli)]

Hi Doug,

Thanks for the efforts, unfortunately i get the below error while post upgrading from 7.3.3 to 7.7.1 :

2024-10-28 09:59:08,910 7 INFO barbican.model.repositories [-] Setting up database engine and session factory
2024-10-28 09:59:09,056 7 INFO barbican.model.repositories [-] Not auto-creating barbican registry DB
2024-10-28 09:59:09,183 7 INFO barbican.plugin.crypto.simple_crypto [-] Software Only Crypto initialized
2024-10-28 09:59:09,540 7 ERROR barbican.plugin.util.utils [-] Problem seen creating plugin: 'p11_crypto': barbican.common.exception.P11CryptoPluginException: HSM returned response code: 0x5 CKR_GENERAL_ERROR
2024-10-28 09:59:09,540 7 ERROR barbican.plugin.util.utils Traceback (most recent call last):
2024-10-28 09:59:09,540 7 ERROR barbican.plugin.util.utils File "/var/lib/openstack/lib/python3.10/site-packages/barbican/plugin/util/utils.py", line 41, in instantiate_plugins
2024-10-28 09:59:09,540 7 ERROR barbican.plugin.util.utils plugin_instance = ext.plugin(*invoke_args, **invoke_kwargs)
2024-10-28 09:59:09,540 7 ERROR barbican.plugin.util.utils File "/var/lib/openstack/lib/python3.10/site-packages/barbican/plugin/crypto/p11_crypto.py", line 170, in __init__
2024-10-28 09:59:09,540 7 ERROR barbican.plugin.util.utils self.pkcs11 = pkcs11 or self._create_pkcs11(ffi)
2024-10-28 09:59:09,540 7 ERROR barbican.plugin.util.utils File "/var/lib/openstack/lib/python3.10/site-packages/barbican/plugin/crypto/p11_crypto.py", line 346, in _create_pkcs11
2024-10-28 09:59:09,540 7 ERROR barbican.plugin.util.utils return pkcs11.PKCS11(
2024-10-28 09:59:09,540 7 ERROR barbican.plugin.util.utils File "/var/lib/openstack/lib/python3.10/site-packages/barbican/plugin/crypto/pkcs11.py", line 463, in __init__
2024-10-28 09:59:09,540 7 ERROR barbican.plugin.util.utils self._check_error(rv)
2024-10-28 09:59:09,540 7 ERROR barbican.plugin.util.utils File "/var/lib/openstack/lib/python3.10/site-packages/barbican/plugin/crypto/pkcs11.py", line 878, in _check_error
2024-10-28 09:59:09,540 7 ERROR barbican.plugin.util.utils raise exception.P11CryptoPluginException(u._(
2024-10-28 09:59:09,540 7 ERROR barbican.plugin.util.utils barbican.common.exception.P11CryptoPluginException: HSM returned response code: 0x5 CKR_GENERAL_ERROR

2024-10-28 09:59:09,839 7 ERROR barbican.api.app [-] Failed to sync secret_stores table.: barbican.common.exception.MissingArgumentError: Must supply non-Blank name argument for SecretStores entry.
2024-10-28 09:59:09,839 7 ERROR barbican.api.app Traceback (most recent call last):
2024-10-28 09:59:09,839 7 ERROR barbican.api.app File "/var/lib/openstack/lib/python3.10/site-packages/barbican/api/app.py", line 85, in _wrapper
2024-10-28 09:59:09,839 7 ERROR barbican.api.app repositories.setup_database_engine_and_factory(
2024-10-28 09:59:09,839 7 ERROR barbican.api.app File "/var/lib/openstack/lib/python3.10/site-packages/barbican/model/repositories.py", line 116, in setup_database_engine_and_factory
2024-10-28 09:59:09,839 7 ERROR barbican.api.app _initialize_secret_stores_data()
2024-10-28 09:59:09,839 7 ERROR barbican.api.app File "/var/lib/openstack/lib/python3.10/site-package...

[Rajiv Mucheli (rajiv.mucheli)]

on disabling Secure Trust Channel on the device i can resolve 0x5 CKR_GENERAL_ERROR but i get the below while trying to create secrets on

Existing projects :
0xc0 CKR_SIGNATURE_INVALID

New projects :
0x71 CKR_MECHANISM_PARAM_INVALID

[Douglas Mendizábal (dougmendizabal)]

Rajiv, the patch is currently a WIP. You may want to wait to test it after the test is passing the gates.

[Rajiv Mucheli (rajiv.mucheli)]

Hi, looks like Patch# 5 passed the tests, right time to test ?

[Douglas Mendizábal (dougmendizabal)]

Hi @Rajiv, yeah, I'd be interested to know if this patch works with your luna device. The luna device I have access to is older, so I'm using CKM_AES_CBC_PAD as the wrapping algorithm. I'd be interested to know if this works for you with CKM_AES_KEY_WRAP_KWP. You may also want to try with and without IV-generation. This patch makes IV generation for key wrapping configurable as well.

[Rajiv Mucheli (rajiv.mucheli)]

Hi Doug,

i get :

./lunacm
lunacm (64-bit) v10.7.0-255. Copyright (c) 2023 Thales Group. All rights reserved.

Error getting PKCS #11 function pointers: 54

barbican logs show :

2024-11-07 11:25:00,834 7 ERROR barbican.plugin.util.utils return pkcs11.PKCS11(
2024-11-07 11:25:00,834 7 ERROR barbican.plugin.util.utils File "/var/lib/openstack/lib/python3.10/site-packages/barbican/plugin/crypto/pkcs11.py", line 463, in __init__
2024-11-07 11:25:00,834 7 ERROR barbican.plugin.util.utils self._check_error(rv)
2024-11-07 11:25:00,834 7 ERROR barbican.plugin.util.utils File "/var/lib/openstack/lib/python3.10/site-packages/barbican/plugin/crypto/pkcs11.py", line 878, in _check_error
2024-11-07 11:25:00,834 7 ERROR barbican.plugin.util.utils raise exception.P11CryptoPluginException(u._(
2024-11-07 11:25:00,834 7 ERROR barbican.plugin.util.utils barbican.common.exception.P11CryptoPluginException: HSM returned response code: 0x5 CKR_GENERAL_ERROR

[Douglas Mendizábal (dougmendizabal)]

Rajiv, The Barbican logs you pasted make it seem like your environment does not have the latest patch. Specifically:

2024-11-07 11:25:00,834 7 ERROR barbican.plugin.util.utils File "/var/lib/openstack/lib/python3.10/site-packages/barbican/plugin/crypto/pkcs11.py", line 878, in _check_error

line 878 in the latest patch (Patchset 5) is mech.mechanism = self.hmac_mechanism

It looks like Patchset 3 is the one with _check_error in line 878. Please ensure you're using Patchest 5 and try again.

[Rajiv Mucheli (rajiv.mucheli)]

i am unsure if its a vendor issue now, i get the below if STC is disabled on the HSM config :

root@barbican-api-7f7d79c57c-7r85v:/thales/safenet/lunaclient/bin/64# ./lunacm
lunacm (64-bit) v10.7.0-255. Copyright (c) 2023 Thales Group. All rights reserved.

Error getting PKCS #11 function pointers: 54
root@barbican-api-7f7d79c57c-7r85v:/thales/safenet/lunaclient/bin/64# exit

Barbican logs are :

2024-11-11 08:50:41,087 7 ERROR barbican.plugin.util.utils File "/var/lib/openstack/lib/python3.10/site-packages/barbican/plugin/crypto/pkcs11.py", line 582, in get_session
2024-11-11 08:50:41,087 7 ERROR barbican.plugin.util.utils session = self._open_session(self.slot_id)
2024-11-11 08:50:41,087 7 ERROR barbican.plugin.util.utils File "/var/lib/openstack/lib/python3.10/site-packages/barbican/plugin/crypto/pkcs11.py", line 972, in _open_session
2024-11-11 08:50:41,087 7 ERROR barbican.plugin.util.utils self._check_error(rv)
2024-11-11 08:50:41,087 7 ERROR barbican.plugin.util.utils File "/var/lib/openstack/lib/python3.10/site-packages/barbican/plugin/crypto/pkcs11.py", line 917, in _check_error
2024-11-11 08:50:41,087 7 ERROR barbican.plugin.util.utils raise exception.P11CryptoTokenException(slot_id=self.slot_id)
2024-11-11 08:50:41,087 7 ERROR barbican.plugin.util.utils barbican.common.exception.P11CryptoTokenException: No token was found in slot 0
2024-11-11 08:50:41,087 7 ERROR barbican.plugin.util.utils

[Douglas Mendizábal (dougmendizabal)]

@Rajiv, Check `vtl verify` to ensure you are set up to connect to the HSM correctly. It should show the relevant partition in slot O.

If `vtl verify` works then it's likely that your barbican process does not have the correct linux permissions to use the client software.

Depending on how you're running the barbican process, you may need to add theh `hsmusers` group to the user running the process or edit the devstack systemd unit to ensure barbican-svc is running with the GID corresponding to `hsmusers`.

[Rajiv Mucheli (rajiv.mucheli)]

Hi Doug,

There seem to be issues with STC configuration post upgrading to 7.7.1 version, disabling STC works well on minimal client 10.7.0 and 10.7.2. I have raised a vendor ticket to validate this.

Regards,
Rajiv

[Rajiv Mucheli (rajiv.mucheli)]

Hi Doug,

Firmware 7.7.0 and above introduced a new token format config/stc/token/001/token_v2.db, updating my config resolved the issue.

Thanks for the efforts, to confirm this patch will be compatible with firmware 7.3.3 as well right ? i am asking this since, i will need to introduce this patch first and then upgrade the HSM firmware.

Lastly, my locust and tempest tests are also green, hence i presume everything works well ? the barbican backend docu is also updated. I will ask Thales to update their documentations

Regards,
Rajiv

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/barbican/+/933461
Committed: https://opendev.org/openstack/barbican/commit/0d4101fa5da52f242ab0a52955f67769b23485a1
Submitter: "Zuul (22348)"
Branch: master

commit 0d4101fa5da52f242ab0a52955f67769b23485a1
Author: Douglas Mendizábal <email address hidden>
Date: Fri Oct 25 16:45:58 2024 -0400

    Configure mechanism for wrapping pKEKs

    The PKCS#11 backend key-wraps (encrypts) the project-specific Key
    Encryption Keys (pKEKs) using the master encryption key (MKEK).

    The mechanism for wrapping/unwrapping the keys was hard-coded to use
    CKM_AES_CBC_PAD. This patch refactors the pkcs11 module to make this
    mechanism configurable.

    This is necessary to fix Bug #2036506 because some PKCS#11 devices and
    software implementations no longer allow CKM_AES_CBC_PAD to be used for
    key wrapping.

    Supported key wrap mechanisms now include:

    * CKM_AES_CBC_PAD
    * CKM_AES_KEY_WRAP_PAD
    * CKM_AES_KEY_WRAP_KWP

    Closes-Bug: #2036506
    Change-Id: Ic2009a2a55622bb707e884d6a960c044b2248f52

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/barbican/+/935224

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/2024.2
Review: https://review.opendev.org/c/openstack/barbican/+/935225

[Rajiv Mucheli (rajiv.mucheli)]

To confirm which mechanism is Barbican supporting now, CKM_AES_CBC_PAD ?

But Thales firmware doc says mechanism this cannot wrap on FIPS mode enabled :

https://www.thalesdocs.com/gphsm/luna/7/docs/network/Content/sdk/mechanisms/CKM_AES_CBC_PAD.htm

This also needs to be updated in the Thales Openstack Barbican integration guide.

[Douglas Mendizábal (dougmendizabal)]

@Rajiv, thanks for testing the patch. Now that the change has merged barbican supports these mechanisms:

* CKM_AES_KEY_WRAP_KWP
* CKM_AES_KEY_WRAP_PAD
* CKM_AES_CBC_PAD

If available, you should use CKM_AES_KEY_WRAP_KWP, which was added to PKCS#11 in version 3.0 of the spec. If your device does not support that yet, then CKM_AES_KEY_WRAP_PAD should work. I've tested it with SoftHSM and things seem to be working as long as you set the IV generation flag to False.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/barbican/+/935224
Committed: https://opendev.org/openstack/barbican/commit/7b36764cd12781bdb1acc37dcd52dd4e6637171e
Submitter: "Zuul (22348)"
Branch: master

commit 7b36764cd12781bdb1acc37dcd52dd4e6637171e
Author: Douglas Mendizabal <email address hidden>
Date: Thu Nov 14 15:39:37 2024 -0500

    Fix typo in wrap_key function

    This patch fixes a typo in one of the mechanisms in the
    PKCS11.wrap_key() function in the pkcs11 module.

    Closes-Bug: #2036506
    Change-Id: I0b4b43cc64a2c18b8e99ab85bbcad05c46611d8d

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/barbican/+/935708

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/barbican/+/935708
Committed: https://opendev.org/openstack/barbican/commit/bae6737cb33ebe47c0655a704ff434539db3dc00
Submitter: "Zuul (22348)"
Branch: master

commit bae6737cb33ebe47c0655a704ff434539db3dc00
Author: Douglas Mendizabal <email address hidden>
Date: Tue Nov 19 14:45:18 2024 -0500

    Increase unit testing coverage for PKCS#11

    This patch adds a few tests to increase the test coverage for the
    PKCS#11 backend.

    Related-Bug: #2036506
    Change-Id: I3a95d3c1bedb42f8874be8ef622f0b9b7ae27bd7

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/barbican/+/935225
Committed: https://opendev.org/openstack/barbican/commit/b5841df387e5ab38caf173950a1d98ab37a51453
Submitter: "Zuul (22348)"
Branch: stable/2024.2

commit b5841df387e5ab38caf173950a1d98ab37a51453
Author: Douglas Mendizábal <email address hidden>
Date: Fri Oct 25 16:45:58 2024 -0400

    Configure mechanism for wrapping pKEKs

    The PKCS#11 backend key-wraps (encrypts) the project-specific Key
    Encryption Keys (pKEKs) using the master encryption key (MKEK).

    The mechanism for wrapping/unwrapping the keys was hard-coded to use
    CKM_AES_CBC_PAD. This patch refactors the pkcs11 module to make this
    mechanism configurable.

    This is necessary to fix Bug #2036506 because some PKCS#11 devices and
    software implementations no longer allow CKM_AES_CBC_PAD to be used for
    key wrapping.

    Supported key wrap mechanisms now include:

    * CKM_AES_CBC_PAD
    * CKM_AES_KEY_WRAP_PAD
    * CKM_AES_KEY_WRAP_KWP

    This patch also includes two additional patches so they can all be
    tested at the same time:

    Fix typo in wrap_key function

    This patch fixes a typo in one of the mechanisms in the
    PKCS11.wrap_key() function in the pkcs11 module.

    and

    Increase unit testing coverage for PKCS#11

    This patch adds a few tests to increase the test coverage for the
    PKCS#11 backend.

    Closes-Bug: #2036506
    Change-Id: Ic2009a2a55622bb707e884d6a960c044b2248f52
    (cherry picked from commit 0d4101fa5da52f242ab0a52955f67769b23485a1)
    (cherry picked from commit 7b36764cd12781bdb1acc37dcd52dd4e6637171e)
    (cherry picked from commit bae6737cb33ebe47c0655a704ff434539db3dc00)

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/2024.1
Review: https://review.opendev.org/c/openstack/barbican/+/935823

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/barbican/+/935823
Committed: https://opendev.org/openstack/barbican/commit/6945564c4c3c8203f779d17b41e4c38d30664d84
Submitter: "Zuul (22348)"
Branch: stable/2024.1

commit 6945564c4c3c8203f779d17b41e4c38d30664d84
Author: Douglas Mendizábal <email address hidden>
Date: Fri Oct 25 16:45:58 2024 -0400

    Configure mechanism for wrapping pKEKs

    The PKCS#11 backend key-wraps (encrypts) the project-specific Key
    Encryption Keys (pKEKs) using the master encryption key (MKEK).

    The mechanism for wrapping/unwrapping the keys was hard-coded to use
    CKM_AES_CBC_PAD. This patch refactors the pkcs11 module to make this
    mechanism configurable.

    This is necessary to fix Bug #2036506 because some PKCS#11 devices and
    software implementations no longer allow CKM_AES_CBC_PAD to be used for
    key wrapping.

    Supported key wrap mechanisms now include:

    * CKM_AES_CBC_PAD
    * CKM_AES_KEY_WRAP_PAD
    * CKM_AES_KEY_WRAP_KWP

    This patch also includes two additional patches so they can all be
    tested at the same time:

    Fix typo in wrap_key function

    This patch fixes a typo in one of the mechanisms in the
    PKCS11.wrap_key() function in the pkcs11 module.

    and

    Increase unit testing coverage for PKCS#11

    This patch adds a few tests to increase the test coverage for the
    PKCS#11 backend.

    Closes-Bug: #2036506
    Change-Id: Ic2009a2a55622bb707e884d6a960c044b2248f52
    (cherry picked from commit 0d4101fa5da52f242ab0a52955f67769b23485a1)
    (cherry picked from commit 7b36764cd12781bdb1acc37dcd52dd4e6637171e)
    (cherry picked from commit bae6737cb33ebe47c0655a704ff434539db3dc00)
    (cherry picked from commit b5841df387e5ab38caf173950a1d98ab37a51453)

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/2023.2
Review: https://review.opendev.org/c/openstack/barbican/+/936859

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/barbican/+/936859
Committed: https://opendev.org/openstack/barbican/commit/20e4946cb8ae5c3465ee033696ca5180b4c50c43
Submitter: "Zuul (22348)"
Branch: stable/2023.2

commit 20e4946cb8ae5c3465ee033696ca5180b4c50c43
Author: Douglas Mendizábal <email address hidden>
Date: Fri Oct 25 16:45:58 2024 -0400

    Configure mechanism for wrapping pKEKs

    The PKCS#11 backend key-wraps (encrypts) the project-specific Key
    Encryption Keys (pKEKs) using the master encryption key (MKEK).

    The mechanism for wrapping/unwrapping the keys was hard-coded to use
    CKM_AES_CBC_PAD. This patch refactors the pkcs11 module to make this
    mechanism configurable.

    This is necessary to fix Bug #2036506 because some PKCS#11 devices and
    software implementations no longer allow CKM_AES_CBC_PAD to be used for
    key wrapping.

    Supported key wrap mechanisms now include:

    * CKM_AES_CBC_PAD
    * CKM_AES_KEY_WRAP_PAD
    * CKM_AES_KEY_WRAP_KWP

    This patch also includes two additional patches so they can all be
    tested at the same time:

    Fix typo in wrap_key function

    This patch fixes a typo in one of the mechanisms in the
    PKCS11.wrap_key() function in the pkcs11 module.

    and

    Increase unit testing coverage for PKCS#11

    This patch adds a few tests to increase the test coverage for the
    PKCS#11 backend.

    Closes-Bug: #2036506
    Change-Id: Ic2009a2a55622bb707e884d6a960c044b2248f52
    (cherry picked from commit 0d4101fa5da52f242ab0a52955f67769b23485a1)
    (cherry picked from commit 7b36764cd12781bdb1acc37dcd52dd4e6637171e)
    (cherry picked from commit bae6737cb33ebe47c0655a704ff434539db3dc00)
    (cherry picked from commit b5841df387e5ab38caf173950a1d98ab37a51453)
    (cherry picked from commit 6945564c4c3c8203f779d17b41e4c38d30664d84)

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/2023.1
Review: https://review.opendev.org/c/openstack/barbican/+/936983

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Dr. Jens Harbott <email address hidden>" on branch: stable/2023.1
Review: https://review.opendev.org/c/openstack/barbican/+/936983
Reason: stable/2023.1 branch of openstack/barbican is about to be deleted. To be able to do that, all open patches need to be abandoned. Please cherry pick the patch to unmaintained/2023.1 if you want to further work on this patch.