Sahara can't display exception with non-ascii symbols from subprocess

[Andrew Lazarev]

stacktrace:

2014-12-04 16:24:24.784 95852 ERROR sahara.service.ops [-] Error during operating cluster 'al-test17' (reason: UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 16: ordinal not in range(128))
2014-12-04 16:24:24.784 95852 TRACE sahara.service.ops Traceback (most recent call last):
2014-12-04 16:24:24.784 95852 TRACE sahara.service.ops File "/Users/alazarev/openstack/sahara/sahara/service/ops.py", line 141, in wrapper
2014-12-04 16:24:24.784 95852 TRACE sahara.service.ops f(cluster_id, *args, **kwds)
2014-12-04 16:24:24.784 95852 TRACE sahara.service.ops File "/Users/alazarev/openstack/sahara/sahara/service/ops.py", line 272, in _provision_scaled_cluster
2014-12-04 16:24:24.784 95852 TRACE sahara.service.ops plugin.scale_cluster(cluster, instances)
2014-12-04 16:24:24.784 95852 TRACE sahara.service.ops File "/Users/alazarev/openstack/sahara/sahara/plugins/vanilla/plugin.py", line 60, in scale_cluster
2014-12-04 16:24:24.784 95852 TRACE sahara.service.ops cluster.hadoop_version).scale_cluster(cluster, instances)
2014-12-04 16:24:24.784 95852 TRACE sahara.service.ops File "/Users/alazarev/openstack/sahara/sahara/plugins/vanilla/v2_4_1/versionhandler.py", line 106, in scale_cluster
2014-12-04 16:24:24.784 95852 TRACE sahara.service.ops sc.scale_cluster(self.pctx, cluster, instances)
2014-12-04 16:24:24.784 95852 TRACE sahara.service.ops File "/Users/alazarev/openstack/sahara/sahara/plugins/vanilla/hadoop2/scaling.py", line 31, in scale_cluster
2014-12-04 16:24:24.784 95852 TRACE sahara.service.ops config.configure_instances(pctx, instances)
2014-12-04 16:24:24.784 95852 TRACE sahara.service.ops File "/Users/alazarev/openstack/sahara/sahara/plugins/vanilla/hadoop2/config.py", line 57, in configure_instances
2014-12-04 16:24:24.784 95852 TRACE sahara.service.ops _provisioning_configs(pctx, instance)
2014-12-04 16:24:24.784 95852 TRACE sahara.service.ops File "/Users/alazarev/openstack/sahara/sahara/plugins/vanilla/hadoop2/config.py", line 63, in _provisioning_configs
2014-12-04 16:24:24.784 95852 TRACE sahara.service.ops _push_xml_configs(instance, xmls)
2014-12-04 16:24:24.784 95852 TRACE sahara.service.ops File "/Users/alazarev/openstack/sahara/sahara/plugins/vanilla/hadoop2/config.py", line 261, in _push_xml_configs
2014-12-04 16:24:24.784 95852 TRACE sahara.service.ops _push_configs_to_instance(instance, xml_confs)
2014-12-04 16:24:24.784 95852 TRACE sahara.service.ops File "/Users/alazarev/openstack/sahara/sahara/plugins/vanilla/hadoop2/config.py", line 268, in _push_configs_to_instance
2014-12-04 16:24:24.784 95852 TRACE sahara.service.ops r.write_file_to(fl, data, run_as_root=True)
2014-12-04 16:24:24.784 95852 TRACE sahara.service.ops File "/Users/alazarev/openstack/sahara/sahara/utils/ssh_remote.py", line 578, in write_file_to
2014-12-04 16:24:24.784 95852 TRACE sahara.service.ops self._run_s(_write_file_to, timeout, remote_file, data, run_as_root)
2014-12-04 16:24:24.784 95852 TRACE sahara.service.ops File "/Users/alazarev/openstack/sahara/sahara/utils/ssh_remote.py", line 643, in _run_s
2014-12-04 16:24:24.784 95852 TRACE sahara.service.ops return self._run_with_log(func, timeout, *args, **kwargs)
2014-12-04 16:24:24.784 95852 TRACE sahara.service.ops File "/Users/alazarev/openstack/sahara/sahara/utils/ssh_remote.py", line 517, in _run_with_log
2014-12-04 16:24:24.784 95852 TRACE sahara.service.ops return self._run(func, *args, **kwargs)
2014-12-04 16:24:24.784 95852 TRACE sahara.service.ops File "/Users/alazarev/openstack/sahara/sahara/utils/ssh_remote.py", line 640, in _run
2014-12-04 16:24:24.784 95852 TRACE sahara.service.ops return procutils.run_in_subprocess(self.proc, func, args, kwargs)
2014-12-04 16:24:24.784 95852 TRACE sahara.service.ops File "/Users/alazarev/openstack/sahara/sahara/utils/procutils.py", line 52, in run_in_subprocess
2014-12-04 16:24:24.784 95852 TRACE sahara.service.ops raise SubprocessException(result['exception'])
2014-12-04 16:24:24.784 95852 TRACE sahara.service.ops SubprocessException: UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 16: ordinal not in range(128)
2014-12-04 16:24:24.784 95852 TRACE sahara.service.ops
2014-12-04 16:24:24.970 95852 INFO sahara.utils.general [-] Cluster status has been changed: id=844966c7-71ea-4e99-8ee6-112992fdfcbe, New status=Error

[Andrew Lazarev (alazarev)]

Have 100% repro on the lab.

[Andrew Lazarev (alazarev)]

Stacktrace from subprocess:

2014-12-05 14:01:43.031 97577 INFO sahara.utils.procutils [-] Traceback (most recent call last):
  File "/Users/alazarev/openstack/sahara/sahara/cli/sahara_subprocess.py", line 36, in main
    result['output'] = func(*args, **kwargs)
  File "/Users/alazarev/openstack/sahara/sahara/utils/ssh_remote.py", line 211, in _write_file_to
    _write_file(_ssh.open_sftp(), remote_file, data, run_as_root)
  File "/Users/alazarev/openstack/sahara/sahara/utils/ssh_remote.py", line 192, in _write_file
    'mv %s %s' % (temp_file, remote_file), run_as_root=True)
  File "/Users/alazarev/openstack/sahara/sahara/utils/ssh_remote.py", line 132, in _execute_command
    stdout=stdout, stderr=stderr)
  File "/Users/alazarev/openstack/sahara/sahara/exceptions.py", line 106, in __init__
    self.message = '%s\nSTDERR:\n%s' % (self.message, stderr)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 16: ordinal not in range(128)

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/139742

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/139742
Committed: https://git.openstack.org/cgit/openstack/sahara/commit/?id=2f9a12bcb270c5fc00b5ec3cb495155d510b5762
Submitter: Jenkins
Branch: master

commit 2f9a12bcb270c5fc00b5ec3cb495155d510b5762
Author: Andrew Lazarev <email address hidden>
Date: Fri Dec 5 14:35:36 2014 -0800

    Fixed subprocess error reporting

    Moved stdout and stderr decoding to earlier stage to prevent
    decode error.

    Note, the code is not py3 compatible. But this change doesn't
    introduce any new incompatibility, it changes order of existing
    decode calls.

    Change-Id: I23f7340e254ec249f00c6de9a980adac9e637983
    Closes-Bug: #1399490