Duplicate ssh keys after every upgrade-charm hook

[Rodrigo Barbieri]

version: stable/21.04
deployment: bionic-ussuri

Upon deploying this charm and then trying to upgrade it (even if the code is exactly the same), when the upgrade-charm hook runs, the contents of:

- /etc/nova/compute_ssh/*/authorized_keys
- /etc/nova/compute_ssh/*/known_hosts
- cloud-compute relations

gets duplicated.

Starting a fresh deploying with 3 compute nodes:
- authorized_keys has 3 keys
- known_hosts has 9 keys, where they are already duplicated 3 times with only the host hash being different

Investigating this, I found:

Condition [0] always returns false. Testing with "public_key in keys.read()" works.

Command [1] exit code is always 1, therefore the function always returns None. Also, testing the command does not show "'# Host xx found: line 1 type RSA'," as the code comment implies. When the code is changed to always return the output even if RC=1 (and lines[0] instead of lines[1]) the deduplication logic at [2] is invoked and works.

[0] https://github.com/openstack/charm-nova-cloud-controller/blob/b2fe2601c96e44c07816d1c70b9c5d7e2409e2b3/hooks/nova_cc_utils.py#L1269

[1] https://github.com/openstack/charm-nova-cloud-controller/blob/b2fe2601c96e44c07816d1c70b9c5d7e2409e2b3/hooks/nova_cc_utils.py#L1163

[2] https://github.com/openstack/charm-nova-cloud-controller/blob/b2fe2601c96e44c07816d1c70b9c5d7e2409e2b3/hooks/nova_cc_utils.py#L1197

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/charm-nova-cloud-controller/+/809281

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/charm-nova-cloud-controller/+/809281
Committed: https://opendev.org/openstack/charm-nova-cloud-controller/commit/1a63d9c0b690638e3eab4df8a7666f98d2b68b7a
Submitter: "Zuul (22348)"
Branch: master

commit 1a63d9c0b690638e3eab4df8a7666f98d2b68b7a
Author: Rodrigo Barbieri <email address hidden>
Date: Wed Sep 15 16:27:14 2021 -0300

    Fix ssh keys duplication

    Upon running hooks that update ssh_keys, they
    end up duplicated in the /etc/nova/compute_ssh/* files
    and cloud-compute relations because the code that
    checks whether the keys already exist are currently
    not working.

    This change fixes the deduplication code and improves
    unit tests, while also handling a special case for
    specific ubuntu-version scenarios.

    Change-Id: I93f9418d5340e7cb599a42970d78557362c1542f
    Closes-bug: #1943753