Fernet keys are lost during Rocky->Stein upgrade

[Jonathan Rosser]

With the transition to smart sources the /etc/keystone/.... directory is relocated from the host to be symlinked into the keystone venv.

The os_keystone role deletes the original contents of /etc/keystone here https://github.com/openstack/openstack-ansible-os_keystone/blob/03b0aaf019d44e2fff3658a65c4f5cf503f61d35/tasks/keystone_pre_install.yml#L77-L86 and thus discards the fernet keys.

This very undesirable behaviour with existing tokens failing and credential keys being lost.

[Logan V (loganv)]

One note about this bug, this affects not only fernet keys but also credential keys.

Fernet key loss is recoverable. Your tokens are invalidated but you can always log back in and obtain another token.

Credential key loss is unrecoverable. All credential keys stored in the database are encrypted by the credential keys, and losing the credential key repository invalidates them all permanently.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/666428
Committed: https://git.openstack.org/cgit/openstack/openstack-ansible-os_keystone/commit/?id=8e1f7f4ad8918af9e467387144c7eede7f19f92a
Submitter: Zuul
Branch: master

commit 8e1f7f4ad8918af9e467387144c7eede7f19f92a
Author: Jonathan Rosser <email address hidden>
Date: Wed Jun 19 20:17:02 2019 +0100

    Fix loss of fernet and credential keys during Rocky to Stein upgrade

    This applies only to source based installations.

    The introduction of smart-sources in [1] created a code path
    which deletes the /etc/keystone directory before symlinking it
    into the keystone venv and creating the necessary config files.

    Unfortunatley this has the side effect of also deleting any fernet
    and credential keys which pre-existed in the case of an upgrade from
    Rocky. The original keys were deleted simulataneously across the whole
    keystone_all group in a way which is makes them unrecoverable in
    the absence of a backup taken by the operator.

    This change simplifies the smart-sources code to always keep the
    keystone config files and fernet keys in the host /etc/keystone.
    This ensures that the lifecycle of the fernet keys is not coupled
    to the lifecycle of the keystone venvs.

    In addition, a task is added to rescue any keys which have been
    created in the keystone venv by installations from the Stein
    release-candidate.

    [1] https://review.opendev.org/#/c/588960/

    Closes-Bug: 1833414
    Change-Id: Ide611fd3d88e352367220f05dbcf4186ac20319f

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/667201

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/667201
Committed: https://git.openstack.org/cgit/openstack/openstack-ansible-os_keystone/commit/?id=8fc9bbb88bf677ddbb5f2780e9ed2b7dcd668458
Submitter: Zuul
Branch: stable/stein

commit 8fc9bbb88bf677ddbb5f2780e9ed2b7dcd668458
Author: Jonathan Rosser <email address hidden>
Date: Wed Jun 19 20:17:02 2019 +0100

    Fix loss of fernet and credential keys during Rocky to Stein upgrade

    This applies only to source based installations.

    The introduction of smart-sources in [1] created a code path
    which deletes the /etc/keystone directory before symlinking it
    into the keystone venv and creating the necessary config files.

    Unfortunatley this has the side effect of also deleting any fernet
    and credential keys which pre-existed in the case of an upgrade from
    Rocky. The original keys were deleted simulataneously across the whole
    keystone_all group in a way which is makes them unrecoverable in
    the absence of a backup taken by the operator.

    This change simplifies the smart-sources code to always keep the
    keystone config files and fernet keys in the host /etc/keystone.
    This ensures that the lifecycle of the fernet keys is not coupled
    to the lifecycle of the keystone venvs.

    In addition, a task is added to rescue any keys which have been
    created in the keystone venv by installations from the Stein
    release-candidate.

    [1] https://review.opendev.org/#/c/588960/

    Closes-Bug: 1833414
    Change-Id: Ide611fd3d88e352367220f05dbcf4186ac20319f
    (cherry picked from commit 8e1f7f4ad8918af9e467387144c7eede7f19f92a)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/openstack-ansible-os_keystone stein-eol release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/openstack-ansible-os_keystone train-eol release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/openstack-ansible-os_keystone ussuri-eol release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/openstack-ansible-os_keystone yoga-eom release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/openstack-ansible-os_keystone victoria-eom release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/openstack-ansible-os_keystone wallaby-eom release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/openstack-ansible-os_keystone xena-eom release.