Upgrading charm causes removal of glance-api-paste.ini

[Alan Baghumian]

This is the second time this is happening, so I'm filing this as a bug. The first time I saw this, I was running focal/xena and upgrading the glance charm caused one of the HA glance units to lose glance-api-paste.ini, hence breaking the API service.

This time, I'm running focal/yoga and after upgrading the glance charm today, two of the HA units lost the glance-api-paste.ini file.

Looking at the code, these files are supposed to get reinstalled by re-installing glance-common:

def reinstall_paste_ini(force_reinstall=False):
    '''
    Re-install glance-{api,registry}-paste.ini file from packages

    Existing glance-{api,registry}-paste.ini file will be removed
    and the original files provided by the packages will be
    re-installed.

    This will only be performed once per unit unless force_reinstall
    is set to True.
    '''
    db = kv()
    reinstall = not db.get(PASTE_INI_MARKER) or force_reinstall
    cmp_release = CompareOpenStackReleases(os_release('glance-common'))

    if not os.path.exists(GLANCE_REGISTRY_PASTE) and cmp_release < 'rocky':
        # See LP: #1812972
        reinstall = True

    if reinstall:
        for paste_file in [GLANCE_REGISTRY_PASTE,
                           GLANCE_API_PASTE]:
            if os.path.exists(paste_file):
                os.remove(paste_file)
        # glance-registry is deprecated at queens but still
        # installed.
        if cmp_release < 'rocky':
            pkg_list = ['glance-api', 'glance-registry']
        # File is in glance-common for py3 packages.
        else:
            pkg_list = ['glance-common']
        apt_install(packages=pkg_list,
                    options=REINSTALL_OPTIONS,
                    fatal=True)
        db.set(PASTE_INI_MARKER, True)
        db.flush()

However if for any reasons glance-common is not re-installable (example: UCA repository not being available), the charm upgrade breaks the API services by removing these.

I think we should add a safeguard to backup and restore these ini files if apt_install() fails.

Please let me know if need any logs or extra information.

Thank you,
Alan

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/charm-glance/+/900258

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/charm-glance/+/900259

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Alan Baghumian <email address hidden>" on branch: master
Review: https://review.opendev.org/c/openstack/charm-glance/+/900259
Reason: Duplicated,

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/charm-glance/+/900258
Committed: https://opendev.org/openstack/charm-glance/commit/59efce8bc81fc7800d2cb6ba696028832063a6e0
Submitter: "Zuul (22348)"
Branch: master

commit 59efce8bc81fc7800d2cb6ba696028832063a6e0
Author: Alan Baghumian <email address hidden>
Date: Mon Nov 6 16:06:45 2023 -0800

    Restore original paste files if glance-common
    re-installation does not restore them.

    Implemented a unit test function for the above
    scenario.

    Call apt_update from upgrade_charm().

    Closes-Bug: #2042792

    Change-Id: I57c5fa293b421483b0879f24ddac260e47c7cbef

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/yoga
Review: https://review.opendev.org/c/openstack/charm-glance/+/900951

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/zed
Review: https://review.opendev.org/c/openstack/charm-glance/+/900952

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/2023.1
Review: https://review.opendev.org/c/openstack/charm-glance/+/900959

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/2023.2
Review: https://review.opendev.org/c/openstack/charm-glance/+/900960

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/yoga
Review: https://review.opendev.org/c/openstack/charm-glance/+/901618

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/zed
Review: https://review.opendev.org/c/openstack/charm-glance/+/901619

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Alan Baghumian <email address hidden>" on branch: stable/zed
Review: https://review.opendev.org/c/openstack/charm-glance/+/900952
Reason: Replaced with a clean Cherry-Pick.

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/2023.1
Review: https://review.opendev.org/c/openstack/charm-glance/+/901625

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Alan Baghumian <email address hidden>" on branch: stable/yoga
Review: https://review.opendev.org/c/openstack/charm-glance/+/900951
Reason: Re-submitted using a clean cherry-pick.

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Alan Baghumian <email address hidden>" on branch: stable/2023.1
Review: https://review.opendev.org/c/openstack/charm-glance/+/900959
Reason: Re-submitted using a clean cherry-pick.

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Alan Baghumian <email address hidden>" on branch: stable/2023.2
Review: https://review.opendev.org/c/openstack/charm-glance/+/900960
Reason: Re-submitted using a clean cherry-pick.

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/2023.2
Review: https://review.opendev.org/c/openstack/charm-glance/+/901626

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/charm-glance/+/901626
Committed: https://opendev.org/openstack/charm-glance/commit/1364f382e6631d98fc35476cbfb267e31e865ad1
Submitter: "Zuul (22348)"
Branch: stable/2023.2

commit 1364f382e6631d98fc35476cbfb267e31e865ad1
Author: Alan Baghumian <email address hidden>
Date: Tue Nov 14 15:42:29 2023 -0800

    Restore original paste files if glance-common
    re-installation does not restore them.

    Implemented a unit test function for the above
    scenario.

    Call apt_update from upgrade_charm().

    Closes-Bug: #2042792
    Change-Id: I57c5fa293b421483b0879f24ddac260e47c7cbef
    (cherry picked from commit 59efce8bc81fc7800d2cb6ba696028832063a6e0)