undercloud upgrades/scaling commands failing on commands with unicode output

[Julie Pichon]

If a user adds a project or project description with unicode characters in the name, the undercloud upgrade fails with:

2018-01-18 11:50:34,669 INFO: Error: Failed to apply catalog: Execution of '/bin/openstack project list --quiet --format csv --long' returned 1: 'ascii' codec can't encode character u'\xe9' in position 1: ordinal not in range(128)
2018-01-18 11:50:34,670 INFO: "ID","Name","Domain ID","Description","Enabled"
2018-01-18 11:50:34,670 INFO: "1c8fc0b2dd23498fba191eed5c5d6c42","admin","default","admin tenant",True
2018-01-18 11:50:34,671 INFO: "301f05e56656474d85eabe0543b681c6","service","default","Tenant for the openstack services",True (tried 34, for a total of 170 seconds)

The command used to fail in the same way when called directly on the CLI, so bug 1720115 was opened against cliff, with a fix released released in 2.10. However the command still fails now, only when it's run as part of e.g. "undercloud upgrade".

This looks very similar to bug 1722792 ("undercloud upgrades fails on non-english locale") but I have that fix in my environment and it makes no difference. This seems to be catching a number of users, mainly when running undercloud upgrades but also reported at least once in the context of scaling.

Versions
========
python2-openstackclient-3.13.0-0.20171221132212.8c5f755.el7.centos.noarch
python-cliff-2.10.0-0.20180110042800.fa2de57.el7.centos.noarch
instack-undercloud-8.1.1-0.20180117134320.cea0211.el7.centos.noarch

How to reproduce
================
1. Create a new project with an accent or some unicode character, e.g. openstack project create tést
2. Run openstack undercloud upgrade

You should also install the latest cliff from e.g. https://trunk.rdoproject.org/centos7/current/python-cliff-2.10.0-0.20180110042800.fa2de57.el7.centos.noarch.rpm to make sure "/bin/openstack project list --quiet --format csv --long" works at least when called directly.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/583675

[OpenStack Infra (hudson-openstack)]

Change abandoned by Emilien Macchi (<email address hidden>) on branch: master
Review: https://review.openstack.org/583675
Reason: feel free to restore, I was just experimenting here.

[Cyril Roelandt (cyril-roelandt)]

So, are we abandoning the "PYTHONIOENCODING" fix? What was the issue with it? Do we have a better way of fixing this?

[Emilien Macchi (emilienm)]

I was experimenting in this patch but I'm not sure this is the right approach. I'll poke around again and get some feedback, so we can move forward.

[Cédric Jeanneret (cjeanner)]

So, apparently, it's more due to puppet than python.
More specifically, there's a "puppet-openstacklib" package that gives some provider for openstack.
It also provides a small lib that parses the CSVs from openstack CLI, and map the content to some simple object.

The issue is, nothing is done with string encoding in there. And this is where puppet crashes. Since its very débuts, puppet has had issues with non-ASCII strings - I even had some template with a simple "…" in it that made the whole catalogue crash. So long for i18n support…

Therefore, we need to either ensure user can't input any non-ASCII string (bad UX in the end, can't even think about the issues/troubles this will create), or filter out the non-ASCII things within puppet.

In order to do the latter, I see two solutions :

- either use the "transliterate" from i18n package, that will work for latin-based languages such as Czech, Roman, French, German and so on.

- or make a deeper change, and hash strings in order to be able to do proper comparison if needed, while still avoiding non-ASCII.

For both solutions, the file to hit is:
https://github.com/openstack/puppet-openstacklib/blob/master/lib/puppet/provider/openstack.rb#L150-151

For the latter solution, while being longer and a bit more complicated, we might want to create a function that will iterate on the CSV object, with a provided list of "user fields" (such as "name", "description", and so on) that we need to hash (think: md5, sha1, whatever).
Doing so will allow to compare strings in case we need to base something on user input, as we can simply hash both side, compare, and act upon the comparison result.

Using the transliteration looks nice as it will allow to still read the string, but it will hit issues in those cases:
- non-latin base language such as Chinese, Japanese
- unable to do any proper string comparison

[Sofer Athlan-Guyot (sofer-athlan-guyot)]

Hi,

so the bottom of this problem is that, by default, puppet reset the
locale to C when executing commands[1].

We have to pass override_locale false in order to prevent that behavior.

The review there[2] does it.

To ease debugging/validation here's how to test it:

ssh undercloud

. stackrc

openstack project create --description "A tést" tést --domain default

cat > test.pp<<EOP
  keystone_tenant { 'test1':
  ensure => present
}
EOP

sudo puppet apply --noop --debug test.pp

[1] https://github.com/puppetlabs/puppet/blob/master/lib/puppet/util/execution.rb#L349-L357
[2] https://review.openstack.org/#/c/640997/

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/puppet-openstacklib 14.4.0 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/puppet-openstacklib 11.6.0 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/puppet-openstacklib queens-eol release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/puppet-openstacklib rocky-eol release.