Dcmanager subcloud add command returns an empty string (b' ') when using a YAML with indentation issue

Bug #2041679 reported by srana
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
StarlingX
Fix Released
Low
srana

Bug Description

Brief Description
-----------------
If the operator tries to add a subcloud using a YAML with indentation issue, a byte string is returned instead of an error message/proper string.

$ dcmanager subcloud add --bootstrap-address .... --bootstrap-values /home/sysadmin/subclouds-config/subcloud1/subcloud1-bootstrap-values-rel.yaml --sysadmin-password Li69nux*
ERROR (app) b''

Severity
--------
Minor

Steps to Reproduce
------------------
1. Modify bootstrap yaml file to insert YAML indentation issue.
2. Add the subcloud

Expected Behavior
------------------
Error message displayed

Actual Behavior
----------------
CLI outputs byte string b' ' so operator need to investigate the reason through dcmanager-api.log

Reproducibility
---------------
100%

System Configuration
--------------------
DC

Branch/Pull Time/Commit
-----------------------
Master (10/27/2023)

Last Pass
---------
This was never tested before.

Timestamp/Logs
--------------
xception in AuditLoggingHook passed to event 'on_error': mapping values are not allowed here
  in "<unicode string>", line 42, column 13:
      k8s.gcr.io:
                ^: yaml.scanner.ScannerError: mapping values are not allowed here
  in "<unicode string>", line 42, column 13:
      k8s.gcr.io:
                ^
2023-10-17 22:22:03.682 25339 ERROR dcmanager.common.context.auditor Traceback (most recent call last):
2023-10-17 22:22:03.682 25339 ERROR dcmanager.common.context.auditor File "/usr/lib/python3/dist-packages/pecan/core.py", line 683, in __call__
2023-10-17 22:22:03.682 25339 ERROR dcmanager.common.context.auditor self.invoke_controller(controller, args, kwargs, state)
2023-10-17 22:22:03.682 25339 ERROR dcmanager.common.context.auditor File "/usr/lib/python3/dist-packages/pecan/core.py", line 574, in invoke_controller
2023-10-17 22:22:03.682 25339 ERROR dcmanager.common.context.auditor result = controller(*args, **kwargs)
2023-10-17 22:22:03.682 25339 ERROR dcmanager.common.context.auditor File "/usr/lib/python3/dist-packages/dcmanager/api/controllers/v1/subclouds.py", line 1272, in post
2023-10-17 22:22:03.682 25339 ERROR dcmanager.common.context.auditor payload = self._get_request_data(request)
2023-10-17 22:22:03.682 25339 ERROR dcmanager.common.context.auditor File "/usr/lib/python3/dist-packages/dcmanager/api/controllers/v1/subclouds.py", line 198, in _get_request_data
2023-10-17 22:22:03.682 25339 ERROR dcmanager.common.context.auditor data = yaml.safe_load(file_item.file.read().decode('utf8'))
2023-10-17 22:22:03.682 25339 ERROR dcmanager.common.context.auditor File "/usr/lib/python3/dist-packages/yaml/__init__.py", line 162, in safe_load
2023-10-17 22:22:03.682 25339 ERROR dcmanager.common.context.auditor return load(stream, SafeLoader)
2023-10-17 22:22:03.682 25339 ERROR dcmanager.common.context.auditor File "/usr/lib/python3/dist-packages/yaml/__init__.py", line 114, in load
2023-10-17 22:22:03.682 25339 ERROR dcmanager.common.context.auditor return loader.get_single_data()
2023-10-17 22:22:03.682 25339 ERROR dcmanager.common.context.auditor File "/usr/lib/python3/dist-packages/yaml/constructor.py", line 49, in get_single_data
2023-10-17 22:22:03.682 25339 ERROR dcmanager.common.context.auditor node = self.get_single_node()
2023-10-17 22:22:03.682 25339 ERROR dcmanager.common.context.auditor File "/usr/lib/python3/dist-packages/yaml/composer.py", line 36, in get_single_node
2023-10-17 22:22:03.682 25339 ERROR dcmanager.common.context.auditor document = self.compose_document()
2023-10-17 22:22:03.682 25339 ERROR dcmanager.common.context.auditor File "/usr/lib/python3/dist-packages/yaml/composer.py", line 55, in compose_document
2023-10-17 22:22:03.682 25339 ERROR dcmanager.common.context.auditor node = self.compose_node(None, None)
2023-10-17 22:22:03.682 25339 ERROR dcmanager.common.context.auditor File "/usr/lib/python3/dist-packages/yaml/composer.py", line 84, in compose_node
2023-10-17 22:22:03.682 25339 ERROR dcmanager.common.context.auditor node = self.compose_mapping_node(anchor)
2023-10-17 22:22:03.682 25339 ERROR dcmanager.common.context.auditor File "/usr/lib/python3/dist-packages/yaml/composer.py", line 133, in compose_mapping_node
2023-10-17 22:22:03.682 25339 ERROR dcmanager.common.context.auditor item_value = self.compose_node(node, item_key)
2023-10-17 22:22:03.682 25339 ERROR dcmanager.common.context.auditor File "/usr/lib/python3/dist-packages/yaml/composer.py", line 84, in compose_node
2023-10-17 22:22:03.682 25339 ERROR dcmanager.common.context.auditor node = self.compose_mapping_node(anchor)
2023-10-17 22:22:03.682 25339 ERROR dcmanager.common.context.auditor File "/usr/lib/python3/dist-packages/yaml/composer.py", line 127, in compose_mapping_node
2023-10-17 22:22:03.682 25339 ERROR dcmanager.common.context.auditor while not self.check_event(MappingEndEvent):
2023-10-17 22:22:03.682 25339 ERROR dcmanager.common.context.auditor File "/usr/lib/python3/dist-packages/yaml/parser.py", line 98, in check_event
2023-10-17 22:22:03.682 25339 ERROR dcmanager.common.context.auditor self.current_event = self.state()
2023-10-17 22:22:03.682 25339 ERROR dcmanager.common.context.auditor File "/usr/lib/python3/dist-packages/yaml/parser.py", line 428, in parse_block_mapping_key
2023-10-17 22:22:03.682 25339 ERROR dcmanager.common.context.auditor if self.check_token(KeyToken):
2023-10-17 22:22:03.682 25339 ERROR dcmanager.common.context.auditor File "/usr/lib/python3/dist-packages/yaml/scanner.py", line 116, in check_token
2023-10-17 22:22:03.682 25339 ERROR dcmanager.common.context.auditor self.fetch_more_tokens()
2023-10-17 22:22:03.682 25339 ERROR dcmanager.common.context.auditor File "/usr/lib/python3/dist-packages/yaml/scanner.py", line 223, in fetch_more_tokens
2023-10-17 22:22:03.682 25339 ERROR dcmanager.common.context.auditor return self.fetch_value()
2023-10-17 22:22:03.682 25339 ERROR dcmanager.common.context.auditor File "/usr/lib/python3/dist-packages/yaml/scanner.py", line 577, in fetch_value
2023-10-17 22:22:03.682 25339 ERROR dcmanager.common.context.auditor raise ScannerError(None, None,
2023-10-17 22:22:03.682 25339 ERROR dcmanager.common.context.auditor yaml.scanner.ScannerError: mapping values are not allowed here
2023-10-17 22:22:03.682 25339 ERROR dcmanager.common.context.auditor in "<unicode string>", line 42, column 13:
2023-10-17 22:22:03.682 25339 ERROR dcmanager.common.context.auditor k8s.gcr.io:

Test Activity
-------------
System Test

Workaround
----------
Fix YAML file

Changed in starlingx:
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to distcloud (master)

Reviewed: https://review.opendev.org/c/starlingx/distcloud/+/899490
Committed: https://opendev.org/starlingx/distcloud/commit/a46055a2efd9a7acb1e37368b551979ea3d650f1
Submitter: "Zuul (22348)"
Branch: master

commit a46055a2efd9a7acb1e37368b551979ea3d650f1
Author: srana <email address hidden>
Date: Fri Oct 27 11:40:39 2023 -0400

    Fix: Error Handling for YAML Load

    Introduce yaml_safe_load, a util wrapper to invoke
    yaml.safe_load and correctly handle malformed YAML files.
    This wrapper catches exceptions thrown (e.g., incorrect syntax)
    while loading/parsing a yaml file. Currently, there's no error
    reported back to the user on such failures, rather an ambiguous
    python object string is reported and the user is unaware of the
    cause of failure. All calls for reading yaml files being uploaded
    by the user have been replaced with the wrapper for correct error
    reporting.

    Test:
      1. PASS: Ensure that the following error message is reported when
               a malformed file is passed to a command:
               "Error: Unable to load file contents.
               (problem on line: #)."
                - Add an entry in the bootstrap yaml file without a
                  space after the colon:
                    "key:val"
                - Run dcmanager subcloud add with the
                  malformed bootstrap file
                - Check the cmdline reported error
      2. PASS: Repeat test #1 with common YAML issues, for example:
                - Incorrect indentation
                - Missing new line between entries
      3. PASS: Repeat test #1 with an empty file. Ensure that the
               following error message is reported:
               "Error: Unable to load file contents.
               (empty file provided)."

    Closes-Bug: 2041679

    Change-Id: I33210a45038d6d4e98b69809ffa8d2d431b43190
    Signed-off-by: Salman Rana <email address hidden>

Changed in starlingx:
status: In Progress → Fix Released
Ghada Khalil (gkhalil)
Changed in starlingx:
importance: Undecided → Low
tags: added: stx.9.0 stx.distcloud
Changed in starlingx:
assignee: nobody → srana (salmanr)
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.