Upgrade DC system controller failed with yaml loading hieradata

[Bin Qian]

During a DC system controller upgrade to StarlingX master, the following failure occurred when parsing hieradata system.yaml file.
2023-08-10T17:23:25.604 localhost upgrade_controller: info 2023-08-10 17:23:25.602 1965 INFO controllerconfig.upgrades.controller [-] Migrating hiera data
2023-08-10T17:23:26.694 localhost upgrade_controller: err 2023-08-10 17:23:26.637 1965 ERROR controllerconfig.upgrades.controller [-] could not determine a constructor for the tag 'tag:yaml.org,2002:python/tuple'
  in "/opt/platform/puppet/22.12/hieradata/system.yaml", line 468, column 38: yaml.constructor.ConstructorError: could not determine a constructor for the tag 'tag:yaml.org,2002:python/tuple'
  in "/opt/platform/puppet/22.12/hieradata/system.yaml", line 468, column 38
2023-08-10 17:23:26.637 1965 ERROR controllerconfig.upgrades.controller Traceback (most recent call last):
2023-08-10 17:23:26.637 1965 ERROR controllerconfig.upgrades.controller File "/usr/lib/python3/dist-packages/controllerconfig/upgrades/controller.py", line 1173, in main
2023-08-10 17:23:26.637 1965 ERROR controllerconfig.upgrades.controller upgrade_controller(from_release, to_release)
2023-08-10 17:23:26.637 1965 ERROR controllerconfig.upgrades.controller File "/usr/lib/python3/dist-packages/controllerconfig/upgrades/controller.py", line 989, in upgrade_controller
2023-08-10 17:23:26.637 1965 ERROR controllerconfig.upgrades.controller utils.add_upgrade_entries_to_hiera_data(from_release)
2023-08-10 17:23:26.637 1965 ERROR controllerconfig.upgrades.controller File "/usr/lib/python3/dist-packages/controllerconfig/upgrades/utils.py", line 345, in add_upgrade_entries_to_hiera_data
2023-08-10 17:23:26.637 1965 ERROR controllerconfig.upgrades.controller get_upgrade_token(from_release, config, secure_config)
2023-08-10 17:23:26.637 1965 ERROR controllerconfig.upgrades.controller File "/usr/lib/python3/dist-packages/controllerconfig/upgrades/utils.py", line 220, in get_upgrade_token
2023-08-10 17:23:26.637 1965 ERROR controllerconfig.upgrades.controller system_config = yaml.safe_load(s_file)
2023-08-10 17:23:26.637 1965 ERROR controllerconfig.upgrades.controller File "/usr/lib/python3/dist-packages/yaml/__init__.py", line 125, in safe_load
2023-08-10 17:23:26.637 1965 ERROR controllerconfig.upgrades.controller return load(stream, SafeLoader)
2023-08-10 17:23:26.637 1965 ERROR controllerconfig.upgrades.controller File "/usr/lib/python3/dist-packages/yaml/__init__.py", line 81, in load
2023-08-10 17:23:26.637 1965 ERROR controllerconfig.upgrades.controller return loader.get_single_data()
2023-08-10 17:23:26.637 1965 ERROR controllerconfig.upgrades.controller File "/usr/lib/python3/dist-packages/yaml/constructor.py", line 51, in get_single_data
2023-08-10 17:23:26.637 1965 ERROR controllerconfig.upgrades.controller return self.construct_document(node)
2023-08-10 17:23:26.637 1965 ERROR controllerconfig.upgrades.controller File "/usr/lib/python3/dist-packages/yaml/constructor.py", line 60, in construct_document
2023-08-10 17:23:26.637 1965 ERROR controllerconfig.upgrades.controller for dummy in generator:
2023-08-10 17:23:26.637 1965 ERROR controllerconfig.upgrades.controller File "/usr/lib/python3/dist-packages/yaml/constructor.py", line 413, in construct_yaml_map
2023-08-10 17:23:26.637 1965 ERROR controllerconfig.upgrades.controller value = self.construct_mapping(node)
2023-08-10 17:23:26.637 1965 ERROR controllerconfig.upgrades.controller File "/usr/lib/python3/dist-packages/yaml/constructor.py", line 218, in construct_mapping
2023-08-10 17:23:26.637 1965 ERROR controllerconfig.upgrades.controller return super().construct_mapping(node, deep=deep)
2023-08-10 17:23:26.637 1965 ERROR controllerconfig.upgrades.controller File "/usr/lib/python3/dist-packages/yaml/constructor.py", line 143, in construct_mapping
2023-08-10 17:23:26.637 1965 ERROR controllerconfig.upgrades.controller value = self.construct_object(value_node, deep=deep)
2023-08-10 17:23:26.637 1965 ERROR controllerconfig.upgrades.controller File "/usr/lib/python3/dist-packages/yaml/constructor.py", line 100, in construct_object
2023-08-10 17:23:26.637 1965 ERROR controllerconfig.upgrades.controller data = constructor(self, node)
2023-08-10 17:23:26.637 1965 ERROR controllerconfig.upgrades.controller File "/usr/lib/python3/dist-packages/yaml/constructor.py", line 427, in construct_undefined
2023-08-10 17:23:26.637 1965 ERROR controllerconfig.upgrades.controller raise ConstructorError(None, None,
2023-08-10 17:23:26.637 1965 ERROR controllerconfig.upgrades.controller yaml.constructor.ConstructorError: could not determine a constructor for the tag 'tag:yaml.org,2002:python/tuple'
2023-08-10 17:23:26.637 1965 ERROR controllerconfig.upgrades.controller in "/opt/platform/puppet/22.12/hieradata/system.yaml", line 468, column 38
2023-08-10 17:23:26.637 1965 ERROR controllerconfig.upgrades.controller

The issue is that in new release, python3.9 is used which requires a specific Loader to load the yaml with "tuple".

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/starlingx/config/+/902732

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/starlingx/config/+/902732
Committed: https://opendev.org/starlingx/config/commit/7eaf3b110cf90dc9740012d1f52ea292da6493ed
Submitter: "Zuul (22348)"
Branch: master

commit 7eaf3b110cf90dc9740012d1f52ea292da6493ed
Author: Bin Qian <email address hidden>
Date: Tue Dec 5 21:25:33 2023 +0000

    Update yaml load to use specified loader

    In STX9, python is upgraded to 3.9. Yaml load function requires specific
    loader to deal with 'tuple' type. This fix is to replace all
    yaml.safe_load(f) with yaml.load(f, Loader=yaml.FullLoader) as per [1].

    [1] https://github.com/yaml/pyyaml/blob/main/lib/yaml/loader.py#L51

    TCs:
        Passed upgrade DC system controller.
        Passed upgrade standalone DX system.

    Closes Bug: 2045695

    Change-Id: Ic5a0a51163b5bdd4e5d2c3c31c8dda4ab0de06e1
    Signed-off-by: Bin Qian <email address hidden>