keystone_bootstrap.sh fails to display error message due to unprintable characters

[Mark Goddard]

First discussed in https://bugs.launchpad.net/kolla-ansible/+bug/1797814, thanks to rowan-potgieter for investigating.

In some situations, Keystone bootstrap can fail, and then unhelpfully fails displaying the error message output by the 'keystone-manage bootstrap' command. This appears to be due to unprintable control characters in the error message which prevent the output of the script from being valid JSON.

Reproducing some of Rowan's investigation here. He ran the command manually, and captured the output in file.txt.

"
$ cat -vE file.txt
   {"failed": true, "msg": "2019-11-25 12:57:43.952 541 WARNING keystone.access_rules_config.backends.json [-] No config file found for access rules, application credential access rules will be unavailable.: FileNotFoundError: [Errno 2] No such file or directory: '/etc/keystone/access_rules.json'^[[00m /etc/keystone/fernet-keys/ does not exist", "changed": true}$

I believe the ^[[00m in the "msg" field is causing the issue. If I pipe the response to jq I receive similar errors:
   parse error: Invalid string: control characters from U+0000 through U+001F must be escaped at line 1, column 342
"

Note that this issue is simply about the error display, not the underlying issue.

[Mark Goddard (mgoddard)]

Fix proposed here: https://review.opendev.org/#/c/697982

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/697982
Committed: https://git.openstack.org/cgit/openstack/kolla/commit/?id=bcca3fabe305c1e206fdb231c31809ceff2d4774
Submitter: Zuul
Branch: master

commit bcca3fabe305c1e206fdb231c31809ceff2d4774
Author: Mark Goddard <email address hidden>
Date: Mon Dec 9 11:11:26 2019 +0000

    Fix keystone bootstrap error message display

    In some situations, Keystone bootstrap can fail, and then unhelpfully
    fails displaying the error message output by the 'keystone-manage
    bootstrap' command. This appears to be due to unprintable control
    characters in the error message which prevent the output of the script
    from being valid JSON.

    This change fixes the issue by piping the output through 'cat -v', which
    replaces unprintable characters with control codes.

    Change-Id: I82444bc2272311023cc9e92c5a298d1c4c87483b
    Closes-Bug: #1855701

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/train
Review: https://review.opendev.org/698019

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/698020

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/rocky
Review: https://review.opendev.org/698021

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/698019
Committed: https://git.openstack.org/cgit/openstack/kolla/commit/?id=17b16cde7a38f56052f4397fbd707c33b2af0555
Submitter: Zuul
Branch: stable/train

commit 17b16cde7a38f56052f4397fbd707c33b2af0555
Author: Mark Goddard <email address hidden>
Date: Mon Dec 9 11:11:26 2019 +0000

    Fix keystone bootstrap error message display

    In some situations, Keystone bootstrap can fail, and then unhelpfully
    fails displaying the error message output by the 'keystone-manage
    bootstrap' command. This appears to be due to unprintable control
    characters in the error message which prevent the output of the script
    from being valid JSON.

    This change fixes the issue by piping the output through 'cat -v', which
    replaces unprintable characters with control codes.

    Change-Id: I82444bc2272311023cc9e92c5a298d1c4c87483b
    Closes-Bug: #1855701
    (cherry picked from commit bcca3fabe305c1e206fdb231c31809ceff2d4774)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/698021
Committed: https://git.openstack.org/cgit/openstack/kolla/commit/?id=aaf34cfa2570a65fb36fea12f38edeeb3ad2a4db
Submitter: Zuul
Branch: stable/rocky

commit aaf34cfa2570a65fb36fea12f38edeeb3ad2a4db
Author: Mark Goddard <email address hidden>
Date: Mon Dec 9 11:11:26 2019 +0000

    Fix keystone bootstrap error message display

    In some situations, Keystone bootstrap can fail, and then unhelpfully
    fails displaying the error message output by the 'keystone-manage
    bootstrap' command. This appears to be due to unprintable control
    characters in the error message which prevent the output of the script
    from being valid JSON.

    This change fixes the issue by piping the output through 'cat -v', which
    replaces unprintable characters with control codes.

    Change-Id: I82444bc2272311023cc9e92c5a298d1c4c87483b
    Closes-Bug: #1855701
    (cherry picked from commit bcca3fabe305c1e206fdb231c31809ceff2d4774)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/698020
Committed: https://git.openstack.org/cgit/openstack/kolla/commit/?id=7934cf7d4a18e90be38cb4c40f39b1dfd4f49a54
Submitter: Zuul
Branch: stable/stein

commit 7934cf7d4a18e90be38cb4c40f39b1dfd4f49a54
Author: Mark Goddard <email address hidden>
Date: Mon Dec 9 11:11:26 2019 +0000

    Fix keystone bootstrap error message display

    In some situations, Keystone bootstrap can fail, and then unhelpfully
    fails displaying the error message output by the 'keystone-manage
    bootstrap' command. This appears to be due to unprintable control
    characters in the error message which prevent the output of the script
    from being valid JSON.

    This change fixes the issue by piping the output through 'cat -v', which
    replaces unprintable characters with control codes.

    Change-Id: I82444bc2272311023cc9e92c5a298d1c4c87483b
    Closes-Bug: #1855701
    (cherry picked from commit bcca3fabe305c1e206fdb231c31809ceff2d4774)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/kolla 7.1.0 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/kolla 8.0.2 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/kolla 9.0.1 release.