stx-openstack fails to upload after metadata.yaml in compliance with Zuul jobs

[Thales Elero Cervi]

Brief Description
------------------
Recently all StarlingX applications were updated in order to be in compliance with new Zuul Check Jobs and the change that updated stx-openstack [1] added "app_name" and "app_version" values to the app metadata.

Without the counterpart of updates to build-tools/build-helm-charts.sh, the app change alonee introduces two problems:
1) The final meatada.yaml application name will be the generic/hybrid name "app-openstack" instead of the name we set on build time through build-helm-charts.sh "--app" argument
2) The final meatada.yaml will have duplicated "app_version" keys, which will cause the app upload to fail.

[1] https://opendev.org/starlingx/openstack-armada-app/commit/abb61c361c742571f01790a60b7023a65c78730f

Severity
---------
Major

Steps to Reproduce
-------------------
* system application-upload stx-openstack.tgz

Expected Behavior
------------------
Application is successfully uploaded

Actual Behavior
------------------
Application fails to upload (see details bellow)

Reproducibility
-----------------
Reproducible

System Configuration
--------------------
N/A

Load info
--------------------
Since BUILD_DATE="20231221T170059Z"
(https://mirror.starlingx.cengn.ca/mirror/starlingx/master/debian/openstack/20231221T170059Z/outputs/CHANGELOG.txt)

Last Pass
---------------------
Using builds prior to Dec 21st 2023

Timestamp/Logs
-----------------
/var/log/sysinv.log:
sysinv 2024-01-22 14:19:15.223 15525 ERROR sysinv.conductor.kube_app [-] while constructing a mapping
  in "/scratch/apps/app-openstack/23.09-0/metadata.yaml", line 1, column 1
found duplicate key "app_version" with value "23.09-0" (original value: "1.0-1")
  in "/scratch/apps/app-openstack/23.09-0/metadata.yaml", line 31, column 1

To suppress this check see:
    http://yaml.readthedocs.io/en/latest/api.html#duplicate-keys

Duplicate keys will become an error in future releases, and are errors
by default when using the new API.
: ruamel.yaml.constructor.DuplicateKeyError: while constructing a mapping
  in "/scratch/apps/app-openstack/23.09-0/metadata.yaml", line 1, column 1
found duplicate key "app_version" with value "23.09-0" (original value: "1.0-1")
  in "/scratch/apps/app-openstack/23.09-0/metadata.yaml", line 31, column 1

To suppress this check see:
    http://yaml.readthedocs.io/en/latest/api.html#duplicate-keys

Duplicate keys will become an error in future releases, and are errors
by default when using the new API.
2024-01-22 14:19:15.223 15525 ERROR sysinv.conductor.kube_app Traceback (most recent call last):
2024-01-22 14:19:15.223 15525 ERROR sysinv.conductor.kube_app File "/usr/lib/python3/dist-packages/sysinv/conductor/kube_app.py", line 2016, in perform_app_upload
2024-01-22 14:19:15.223 15525 ERROR sysinv.conductor.kube_app self._upload_helm_charts(app)
2024-01-22 14:19:15.223 15525 ERROR sysinv.conductor.kube_app File "/usr/lib/python3/dist-packages/sysinv/conductor/kube_app.py", line 940, in _upload_helm_charts
2024-01-22 14:19:15.223 15525 ERROR sysinv.conductor.kube_app (helm_repo, disabled_charts) = self._get_chart_data_from_metadata(app)
2024-01-22 14:19:15.223 15525 ERROR sysinv.conductor.kube_app File "/usr/lib/python3/dist-packages/sysinv/conductor/kube_app.py", line 920, in _get_chart_data_from_metadata
2024-01-22 14:19:15.223 15525 ERROR sysinv.conductor.kube_app y = yaml.safe_load(f)
2024-01-22 14:19:15.223 15525 ERROR sysinv.conductor.kube_app File "/usr/lib/python3/dist-packages/ruamel/yaml/main.py", line 1001, in safe_load
2024-01-22 14:19:15.223 15525 ERROR sysinv.conductor.kube_app return load(stream, SafeLoader, version)
2024-01-22 14:19:15.223 15525 ERROR sysinv.conductor.kube_app File "/usr/lib/python3/dist-packages/ruamel/yaml/main.py", line 956, in load
2024-01-22 14:19:15.223 15525 ERROR sysinv.conductor.kube_app return loader._constructor.get_single_data()
2024-01-22 14:19:15.223 15525 ERROR sysinv.conductor.kube_app File "/usr/lib/python3/dist-packages/ruamel/yaml/constructor.py", line 113, in get_single_data
2024-01-22 14:19:15.223 15525 ERROR sysinv.conductor.kube_app return self.construct_document(node)
2024-01-22 14:19:15.223 15525 ERROR sysinv.conductor.kube_app File "/usr/lib/python3/dist-packages/ruamel/yaml/constructor.py", line 123, in construct_document
2024-01-22 14:19:15.223 15525 ERROR sysinv.conductor.kube_app for _dummy in generator:
2024-01-22 14:19:15.223 15525 ERROR sysinv.conductor.kube_app File "/usr/lib/python3/dist-packages/ruamel/yaml/constructor.py", line 723, in construct_yaml_map
2024-01-22 14:19:15.223 15525 ERROR sysinv.conductor.kube_app value = self.construct_mapping(node)
2024-01-22 14:19:15.223 15525 ERROR sysinv.conductor.kube_app File "/usr/lib/python3/dist-packages/ruamel/yaml/constructor.py", line 440, in construct_mapping
2024-01-22 14:19:15.223 15525 ERROR sysinv.conductor.kube_app return BaseConstructor.construct_mapping(self, node, deep=deep)
2024-01-22 14:19:15.223 15525 ERROR sysinv.conductor.kube_app File "/usr/lib/python3/dist-packages/ruamel/yaml/constructor.py", line 257, in construct_mapping
2024-01-22 14:19:15.223 15525 ERROR sysinv.conductor.kube_app if self.check_mapping_key(node, key_node, mapping, key, value):
2024-01-22 14:19:15.223 15525 ERROR sysinv.conductor.kube_app File "/usr/lib/python3/dist-packages/ruamel/yaml/constructor.py", line 295, in check_mapping_key
2024-01-22 14:19:15.223 15525 ERROR sysinv.conductor.kube_app raise DuplicateKeyError(*args)
2024-01-22 14:19:15.223 15525 ERROR sysinv.conductor.kube_app ruamel.yaml.constructor.DuplicateKeyError: while constructing a mapping
2024-01-22 14:19:15.223 15525 ERROR sysinv.conductor.kube_app in "/scratch/apps/app-openstack/23.09-0/metadata.yaml", line 1, column 1
2024-01-22 14:19:15.223 15525 ERROR sysinv.conductor.kube_app found duplicate key "app_version" with value "23.09-0" (original value: "1.0-1")
2024-01-22 14:19:15.223 15525 ERROR sysinv.conductor.kube_app in "/scratch/apps/app-openstack/23.09-0/metadata.yaml", line 31, column 1
2024-01-22 14:19:15.223 15525 ERROR sysinv.conductor.kube_app
2024-01-22 14:19:15.223 15525 ERROR sysinv.conductor.kube_app To suppress this check see:
2024-01-22 14:19:15.223 15525 ERROR sysinv.conductor.kube_app http://yaml.readthedocs.io/en/latest/api.html#duplicate-keys
2024-01-22 14:19:15.223 15525 ERROR sysinv.conductor.kube_app
2024-01-22 14:19:15.223 15525 ERROR sysinv.conductor.kube_app Duplicate keys will become an error in future releases, and are errors
2024-01-22 14:19:15.223 15525 ERROR sysinv.conductor.kube_app by default when using the new API.
2024-01-22 14:19:15.223 15525 ERROR sysinv.conductor.kube_app
2024-01-22 14:19:15.223 15525 ERROR sysinv.conductor.kube_app

Alarms
---------------
[sysadmin@controller-0 ~(keystone_admin)]$ fm alarm-list
+----------+-------------------------------+--------------------------+----------+-------------------+
| Alarm ID | Reason Text | Entity ID | Severity | Time Stamp |
+----------+-------------------------------+--------------------------+----------+-------------------+
| 750.001 | Application Upload Failure | k8s_application=app- | warning | 2024-01-22T14:27: |
| | | openstack | | 51.628228 |
| | | | | |
+----------+-------------------------------+--------------------------+----------+-------------------

Test Activity
---------------
Sanity

Workaround
--------------
N/A

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/starlingx/root/+/906315

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/starlingx/root/+/906315
Committed: https://opendev.org/starlingx/root/commit/dae3b3392161e9bb231defa219c8ea3953b31e4c
Submitter: "Zuul (22348)"
Branch: master

commit dae3b3392161e9bb231defa219c8ea3953b31e4c
Author: Thales Elero Cervi <email address hidden>
Date: Mon Jan 22 16:06:13 2024 -0300

    Use script arguments to override metadata defaults

    Historically, this build-helm-charts.sh script would rely on its
    APP_NAME and APP_VERSION variables to write in the application
    metadata.yaml. The APP_NAME would be used to override in build-time the
    application final name [1] and the APP_VERSION would be initialized by
    the internal "get_app_version" functional.

    Now, after the AppFwk update on all applications Zuul jobs, the
    metadata.yaml can no longer be missing neither the "app_name" or the
    "app_version" keys/values [2].

    So, the build-helm-charts.sh needs to be updated as follows:
      1) Stop the blind write of APP_VERSION on metadata and use this
         variable to override the metadata default value instead
      2) Use APP_NAME (cli parsed argument) to override the default metadata
         app_name, instead of only using it when the metadata is missing the
         key/value.

    NOTE: Since this script is currently always used with the "--app"
    argument, this change should be safe. But if "--app" is not used, the
    app_name will always fallback to "stx-openstack"

    [1] https://opendev.org/starlingx/root/commit/31aea8e88b2f0e2fe90e3b802a2ffff3c230b212
    [2] https://review.opendev.org/c/starlingx/openstack-armada-app/+/903781

    Closes-Bug: 2050825

    TEST PLAN:
    PASS - Build stx-openstack helm-charts tarball
    PASS - system application-upload stx-openstack.tgz
    PASS - system application-apply stx-openstack

    Change-Id: Ibc37e1306ba93b911a66993f5e56c8e609291c2c
    Signed-off-by: Thales Elero Cervi <email address hidden>