Restoring backup on upgraded system controller fails due to non ssl_ca installed as ssl_ca

[Reinildes Oliveira]

Load Info
----------------------------------------
stx 7

System Config
Distributed Cloud

Description of failure
----------------------------------------

System controller was upgraded from stx 6 to stx 7 and backup was taken after upgrade. When the system controller was wiped and being re-deployed using the backup, it fails due to non ssl_ca being installed as ssl_ca previously.

Timestamp when failure occurred
2/21 @18:03:17

Steps to reproduce this issue
----------------------------------------

1. Upgrade system controller from stx 6 to stx 7

2. Create backup after upgrade and save it on local machine
ansible-playbook /usr/share/ansible/stx-ansible/playbooks/backup.yml -e "ansible_become_pass=${OS_PASSWORD} admin_password=${OS_PASSWORD}" -e "backup_user_images=true"

4. Wipe the system controller were wiped and install controller-0.

5. After installation, get the backup on controller-0 and initiate a restore but restore fails

ansible-playbook /usr/share/ansible/stx-ansible/playbooks/restore_platform.yml \
-e "on_box_data=true
initial_backup_dir=/opt/platform-backup/backups/22.12
ansible_become_pass=${OS_PASSWORD} admin_password=${OS_PASSWORD}
backup_filename=localhost_platform_backup_2024_02_21_00_45_23.tgz
restore_user_images=true"

***Error***
Timeout waiting for ssl_ca certificate install: /tmp/ca-cert.pem Err_code= failed_ssl_cert WARNING: Some certificates were not installed. Cannot install non-CA type certificate as SSL CA certificate

Reproducibility
----------------------------------------

Issue intermittent (Frequency of occurrence) or 100% Reproducible?
Issue was reproduced

Impact on users
----------------------------------------

Major, it breaks BnR

Logs
----------------------------------------

2024-02-21T17:43:16.224 localhost sh: info HISTORY: PID=6073 UID=42425 ansible-playbook /usr/share/ansible/stx-ansible/playbooks/restore_platform.yml -e "on_box_data=true
initial_backup_dir=/opt/platform-backup/backups/22.12
ansible_become_pass=${OS_PASSWORD} admin_password=${OS_PASSWORD}
backup_filename=localhost_platform_backup_2024_02_21_00_45_23.tgz
restore_user_images=true"
TASK [bootstrap/persist-config : Wait for certificate install] ****************************************************************************
Wednesday 21 February 2024 18:03:17 +0000 (0:00:04.797) 0:19:59.637 ****
fatal: [localhost]: FAILED! => changed=false
elapsed: 360
msg: |-
Timeout waiting for ssl_ca certificate install: /tmp/ca-cert.pem Err_code= failed_ssl_cert WARNING: Some certificates were not installed.
Cannot install non-CA type certificate as SSL CA certificate

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/starlingx/ansible-playbooks/+/910894

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Reinildes Oliveira <email address hidden>" on branch: master
Review: https://review.opendev.org/c/starlingx/ansible-playbooks/+/910894
Reason: there was a suitable existing solution

[Ubuntu QA Website (ubuntuqa)]

This bug has been reported on the Ubuntu ISO testing tracker.

A list of all reports related to this bug can be found here:
http://iso.qa.ubuntu.com/qatracker/reports/bugs/2055880