Images provided via helm overrides not being downloaded when applying apps

[Igor Pires Soares]

*+Brief Description+*

When a user specifies 'system helm-overide-update ...' for an image tag in an application's helm chart, the platform's logs continue to print the tag from fluxcd manifest rather than the tag specified in helm override. That is a symptom that the images provided via helm overrides are not being downloaded and pushed to the local registry.

*+Severity+*

Minor

*+Steps to Reproduce+*

# apply the application to download images
system application-upload /usr/local/share/applications/helm/vault-*.tgz
system application-apply vault

# wait for apply to complete, then remove
system application-remove vault

# prepare the test
cat <<EOF > vault_override.yaml
manager:
  image:
    repository: starlingx/stx-vault-manager
    tag: master-debian-stable-20230423T060000Z.0
EOF
system helm-override-update vault vault-manager vault --values=vault_override.yaml

# The above image does not exist in our internal registry - that's ok
# we don't need the next application-apply to pass, we only need the log

# Run the test:
system application-apply vault

*+Expected Behavior+*

Images should be downloaded by the application framework and pushed to the local registry. Logs should indicate the tag being pulled.

*+Actual Behavior+*

The tag being pulled is not found in logs (except in sysinv-api.log where the helm-override may appear in audit).  Instead the tag from fluxcd manifest is printed. Images are not downloaded beforehand and pushed to the local registry. containerd may be able to download directly from public registry.

*+Reproducibility+*

yes

*+System Configuration+*

Any probably (observed on AIO-SX)

*+Load info (eg: 2022-03-10_20-00-07)+*

Starlingx master.  2023-04-23

*+Last Pass+*

unknown

*+Timestamp/Logs+*

Collect: /folk/cgts_logs/mthebeau/CGTS-47115

From: Tue May  2 12:48:08 UTC 2023

Especially: sysinv.log and containerd.log

*+Alarms+*

No

*+Test Activity+*

Debugging vault application (feature testing)

*+Workaround+*

N/A

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/starlingx/config/+/918947
Committed: https://opendev.org/starlingx/config/commit/e692da31a26be84a5b6c2e6d954209c4359c3945
Submitter: "Zuul (22348)"
Branch: master

commit e692da31a26be84a5b6c2e6d954209c4359c3945
Author: Igor Soares <email address hidden>
Date: Thu May 9 16:58:47 2024 -0300

    Fix download of images provided via user overrides

    This commit fixes a bug that was causing container images provided via
    user overrides not to be downloaded when applying StarlingX
    applications. The problem was caused by two issues:

      * The way the chart override filename was assembled in the image
        download method got outdated due to a previous commit [1]. The
        global namespace once present in the filename is not mandatory
        anymore as it represents the namespace of FluxCD resources and not
        necessarily the namespace of all underlying resources. In order to
        fix this the download method now uses a helper function, created on
        the same referenced commit, which aims to standardize how override
        filenames are assembled.
      * The implementation assumed that override files loaded using ruamel
        always have a "data:values" section, which is not the case for
        every supported app. This was causing the overrides parsing
        function to receive an empty dictionary, thus returning no images
        to override. That assumption was removed and the whole file is now
        being parsed given that image tags can be present in different
        subsections of the yaml file.

    [1] https://review.opendev.org/c/starlingx/config/+/887430

    Test plan:
    PASS: build-pkgs -a && build-image
    PASS: AIO-SX fresh install
    PASS: Override vault-manager image using "system helm-override-update".
          Apply vault app.
          Confirm that sysinv attempted to download the image provided via
          user overrides.
    PASS: Apply vault app.
          Confirm that default images were correctly downloaded.
    PASS: Override ceph_config_helper image using
          "system helm-override-update".
          Apply platform-integ-apps.
          Confirm that sysinv attempted to download the image provided via
          user overrides.
    PASS: Apply platform-integ-apps app.
          Confirm that default images were correctly downloaded.

    Closes-bug: 2065699

    Change-Id: I9cc6b8522aaf2624377814b2249e3c00da9ec424
    Signed-off-by: Igor Soares <email address hidden>