oslo-config-generator fails to generate a yaml when options are using i18n

Bug #1928582 reported by David Vallee Delisle
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
oslo.config
Fix Released
Undecided
Unassigned

Bug Description

When running oslo-config-generator with yaml format and there's some oslo.i18n fields, it fails to represent the string [1].

This is not reproducible with json format [2]

Proposed patch will add a yaml representer for this type of field and output is working now [3].

[1]
~~~
[root@overcloud-controller-0 ~]# podman exec -u0 -ti openstack-cinder-volume-podman-0 oslo-config-generator --output-file /config-test.yaml --format yaml --namespace cinder --namespace keystonemiddleware.auth_token --namespace oslo.log --namespace oslo.messaging --namespace oslo.policy --namespace oslo.privsep --namespace oslo.service.periodic_task --namespace oslo.service.service --namespace oslo.db --namespace oslo.db.concurrency --namespace oslo.middleware --namespace oslo.concurrency --namespace osprofiler
Traceback (most recent call last):
  File "/usr/bin/oslo-config-generator", line 10, in <module>
    sys.exit(main())
  File "/usr/lib/python3.6/site-packages/oslo_config/generator.py", line 824, in main
    generate(conf)
  File "/usr/lib/python3.6/site-packages/oslo_config/generator.py", line 805, in generate
    conf=conf)
  File "/usr/lib/python3.6/site-packages/oslo_config/generator.py", line 742, in _output_machine_readable
    default_flow_style=False))
  File "/usr/lib64/python3.6/site-packages/yaml/__init__.py", line 306, in safe_dump
    return dump_all([data], stream, Dumper=SafeDumper, **kwds)
  File "/usr/lib64/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all
    dumper.represent(data)
  File "/usr/lib64/python3.6/site-packages/yaml/representer.py", line 27, in represent
    node = self.represent_data(data)
  File "/usr/lib64/python3.6/site-packages/yaml/representer.py", line 48, in represent_data
    node = self.yaml_representers[data_types[0]](self, data)
  File "/usr/lib64/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict
    return self.represent_mapping('tag:yaml.org,2002:map', data)
  File "/usr/lib64/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping
    node_value = self.represent_data(item_value)
  File "/usr/lib64/python3.6/site-packages/yaml/representer.py", line 48, in represent_data
    node = self.yaml_representers[data_types[0]](self, data)
  File "/usr/lib64/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict
    return self.represent_mapping('tag:yaml.org,2002:map', data)
  File "/usr/lib64/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping
    node_value = self.represent_data(item_value)
  File "/usr/lib64/python3.6/site-packages/yaml/representer.py", line 48, in represent_data
    node = self.yaml_representers[data_types[0]](self, data)
  File "/usr/lib64/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict
    return self.represent_mapping('tag:yaml.org,2002:map', data)
  File "/usr/lib64/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping
    node_value = self.represent_data(item_value)
  File "/usr/lib64/python3.6/site-packages/yaml/representer.py", line 48, in represent_data
    node = self.yaml_representers[data_types[0]](self, data)
  File "/usr/lib64/python3.6/site-packages/yaml/representer.py", line 199, in represent_list
    return self.represent_sequence('tag:yaml.org,2002:seq', data)
  File "/usr/lib64/python3.6/site-packages/yaml/representer.py", line 92, in represent_sequence
    node_item = self.represent_data(item)
  File "/usr/lib64/python3.6/site-packages/yaml/representer.py", line 48, in represent_data
    node = self.yaml_representers[data_types[0]](self, data)
  File "/usr/lib64/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict
    return self.represent_mapping('tag:yaml.org,2002:map', data)
  File "/usr/lib64/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping
    node_value = self.represent_data(item_value)
  File "/usr/lib64/python3.6/site-packages/yaml/representer.py", line 58, in represent_data
    node = self.yaml_representers[None](self, data)
  File "/usr/lib64/python3.6/site-packages/yaml/representer.py", line 231, in represent_undefined
    raise RepresenterError("cannot represent an object", data)
yaml.representer.RepresenterError: ('cannot represent an object', 'The url where the S3 server is listening.')
~~~

[2]
~~~
        {
          "advanced": false,
          "choices": [],
          "default": null,
          "deprecated_for_removal": false,
          "deprecated_opts": [],
          "deprecated_reason": null,
          "deprecated_since": null,
          "dest": "backup_s3_endpoint_url",
          "help": "The url where the S3 server is listening.",
          "max": null,
          "metavar": null,
          "min": null,
          "mutable": false,
          "name": "backup_s3_endpoint_url",
          "namespace": "cinder",
          "positional": false,
          "required": false,
          "sample_default": null,
          "secret": false,
          "short": null,
          "type": "string value"
        },
~~~

[3]
~~~
    - advanced: false
      choices: []
      default: null
      deprecated_for_removal: false
      deprecated_opts: []
      deprecated_reason: null
      deprecated_since: null
      dest: backup_s3_endpoint_url
      help: The url where the S3 server is listening.
      max: null
      metavar: null
      min: null
      mutable: false
      name: backup_s3_endpoint_url
      namespace: cinder
      positional: false
      required: false
      sample_default: null
      secret: false
      short: null
      type: string value
~~~

Changed in oslo.config:
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to oslo.config (stable/wallaby)

Fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/oslo.config/+/794184

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to oslo.config (master)

Reviewed: https://review.opendev.org/c/openstack/oslo.config/+/790883
Committed: https://opendev.org/openstack/oslo.config/commit/e5fc313ecf47813364a2111b987ced823c0d64e3
Submitter: "Zuul (22348)"
Branch: master

commit e5fc313ecf47813364a2111b987ced823c0d64e3
Author: David Vallee Delisle <email address hidden>
Date: Tue May 11 21:49:04 2021 -0400

    config-generator yaml format doesn't work with i18n fields

    This is because there's no yaml representer for i18n Messages object.
    This patch aims to add this representer and allow the generation of
    configurations using oslo.i18n strings.

    One example of this is cinder.

    Closes-bug: #1928582
    Change-Id: I70ab87c9bed093cad883b6301b8a09753fc470d9

Changed in oslo.config:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to oslo.config (stable/victoria)

Fix proposed to branch: stable/victoria
Review: https://review.opendev.org/c/openstack/oslo.config/+/794815

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to oslo.config (stable/ussuri)

Fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/c/openstack/oslo.config/+/794816

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to oslo.config (stable/train)

Fix proposed to branch: stable/train
Review: https://review.opendev.org/c/openstack/oslo.config/+/794817

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to oslo.config (stable/wallaby)

Reviewed: https://review.opendev.org/c/openstack/oslo.config/+/794184
Committed: https://opendev.org/openstack/oslo.config/commit/de1dbeed235d2df7cf1528aef6ad4a70ddf8559f
Submitter: "Zuul (22348)"
Branch: stable/wallaby

commit de1dbeed235d2df7cf1528aef6ad4a70ddf8559f
Author: David Vallee Delisle <email address hidden>
Date: Tue May 11 21:49:04 2021 -0400

    config-generator yaml format doesn't work with i18n fields

    This is because there's no yaml representer for i18n Messages object.
    This patch aims to add this representer and allow the generation of
    configurations using oslo.i18n strings.

    One example of this is cinder.

    Closes-bug: #1928582
    Change-Id: I70ab87c9bed093cad883b6301b8a09753fc470d9
    (cherry picked from commit e5fc313ecf47813364a2111b987ced823c0d64e3)

tags: added: in-stable-wallaby
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to oslo.config (stable/victoria)

Reviewed: https://review.opendev.org/c/openstack/oslo.config/+/794815
Committed: https://opendev.org/openstack/oslo.config/commit/0950f82ec59e1b4a2af66b5c5940a651a5cb574f
Submitter: "Zuul (22348)"
Branch: stable/victoria

commit 0950f82ec59e1b4a2af66b5c5940a651a5cb574f
Author: David Vallee Delisle <email address hidden>
Date: Tue May 11 21:49:04 2021 -0400

    config-generator yaml format doesn't work with i18n fields

    This is because there's no yaml representer for i18n Messages object.
    This patch aims to add this representer and allow the generation of
    configurations using oslo.i18n strings.

    One example of this is cinder.

    Closes-bug: #1928582
    Change-Id: I70ab87c9bed093cad883b6301b8a09753fc470d9
    (cherry picked from commit e5fc313ecf47813364a2111b987ced823c0d64e3)
    (cherry picked from commit de1dbeed235d2df7cf1528aef6ad4a70ddf8559f)

tags: added: in-stable-victoria
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to oslo.config (stable/ussuri)

Reviewed: https://review.opendev.org/c/openstack/oslo.config/+/794816
Committed: https://opendev.org/openstack/oslo.config/commit/be3558bb51143c268c0e607e6189b496efbbf308
Submitter: "Zuul (22348)"
Branch: stable/ussuri

commit be3558bb51143c268c0e607e6189b496efbbf308
Author: David Vallee Delisle <email address hidden>
Date: Tue May 11 21:49:04 2021 -0400

    config-generator yaml format doesn't work with i18n fields

    This is because there's no yaml representer for i18n Messages object.
    This patch aims to add this representer and allow the generation of
    configurations using oslo.i18n strings.

    One example of this is cinder.

    Closes-bug: #1928582
    Change-Id: I70ab87c9bed093cad883b6301b8a09753fc470d9
    (cherry picked from commit e5fc313ecf47813364a2111b987ced823c0d64e3)
    (cherry picked from commit de1dbeed235d2df7cf1528aef6ad4a70ddf8559f)
    (cherry picked from commit 0950f82ec59e1b4a2af66b5c5940a651a5cb574f)

tags: added: in-stable-ussuri
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to oslo.config (stable/train)

Reviewed: https://review.opendev.org/c/openstack/oslo.config/+/794817
Committed: https://opendev.org/openstack/oslo.config/commit/149d726d2c2006cfce72c17dc366647cfea27861
Submitter: "Zuul (22348)"
Branch: stable/train

commit 149d726d2c2006cfce72c17dc366647cfea27861
Author: David Vallee Delisle <email address hidden>
Date: Tue May 11 21:49:04 2021 -0400

    config-generator yaml format doesn't work with i18n fields

    This is because there's no yaml representer for i18n Messages object.
    This patch aims to add this representer and allow the generation of
    configurations using oslo.i18n strings.

    One example of this is cinder.

    Closes-bug: #1928582
    Change-Id: I70ab87c9bed093cad883b6301b8a09753fc470d9
    (cherry picked from commit e5fc313ecf47813364a2111b987ced823c0d64e3)
    (cherry picked from commit de1dbeed235d2df7cf1528aef6ad4a70ddf8559f)
    (cherry picked from commit 0950f82ec59e1b4a2af66b5c5940a651a5cb574f)
    (cherry picked from commit 11f8a430d34ed880a02eaa59092e8c9873f37c6c)

tags: added: in-stable-train
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/oslo.config 8.7.1

This issue was fixed in the openstack/oslo.config 8.7.1 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/oslo.config 8.0.5

This issue was fixed in the openstack/oslo.config 8.0.5 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/oslo.config 8.5.1

This issue was fixed in the openstack/oslo.config 8.5.1 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/oslo.config 8.3.4

This issue was fixed in the openstack/oslo.config 8.3.4 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/oslo.config train-eol

This issue was fixed in the openstack/oslo.config train-eol release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.