traceback due to unencodable char in /etc/sysctl.d/10-ipv6-privacy.conf

[Jeff Lane ]

I'm doing regression testing on s390 and I just noticed the following traceback in the sysctl_attachment job when run on a s390 zKVM instance:

Exception in thread queue_worker:
Traceback (most recent call last):
  File "/usr/lib/python3.5/threading.py", line 914, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.5/threading.py", line 862, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/lib/python3/dist-packages/plainbox/vendor/extcmd/__init__.py", line 625, in _drain_queue
    self._delegate.on_line(*args)
  File "/usr/lib/python3/dist-packages/plainbox/vendor/extcmd/__init__.py", line 663, in on_line
    delegate.on_line(stream_name, line)
  File "/usr/lib/python3/dist-packages/plainbox/impl/runner.py", line 252, in on_line
    self.ui.got_program_output(stream_name, line)
  File "/usr/lib/python3/dist-packages/plainbox/impl/commands/inv_run.py", line 205, in got_program_output
    end='', file=stream)
UnicodeEncodeError: 'ascii' codec can't encode character '\u2019' in position 21: ordinal not in range(128)

plainbox-provider-checkbox 0.39.0+git201708042010+pkg60~ubuntu16.04.1

installed from the dev PPA.

I can't find a char
Since plainbox has no logs, there is not much more I can do to help with this :(

[Jeff Lane  (bladernr)]

Looking at the output of the command for that job, I think the culprit is the characters in this section of the output:

###################################################################
# Additional settings - these settings can improve the network
# security of the host and prevent against some network attacks
# including spoofing attacks and man in the middle attacks through
# redirection. Some network environments, however, require that these
# settings are disabled so review and enable them as needed.
#
# Do not accept ICMP redirects (prevent MITM attacks)
#net.ipv4.conf.all.accept_redirects = 0
#net.ipv6.conf.all.accept_redirects = 0
# _or_
# Accept ICMP redirects only for gateways listed in our default
# gateway list (enabled by default)
# net.ipv4.conf.all.secure_redirects = 1
#
# Do not send ICMP redirects (we are not a router)
#net.ipv4.conf.all.send_redirects = 0
#
# Do not accept IP source route packets (we are not a router)
#net.ipv4.conf.all.accept_source_route = 0
#net.ipv6.conf.all.accept_source_route = 0
#
# Log Martian Packets
#net.ipv4.conf.all.log_martians = 1
#
# IPv6 Privacy Extensions (RFC 4941)
# ---
# IPv6 typically uses a device's MAC address when choosing an IPv6 address
# to use in autoconfiguration. Privacy extensions allow using a randomly
# generated IPv6 address, which increases privacy.
#
# Acceptable values:
# 0 - don<E2><80><99>t use privacy extensions.
# 1 - generate privacy addresses
# 2 - prefer privacy addresses and use them over the normal addresses.
net.ipv6.conf.all.use_tempaddr = 2
net.ipv6.conf.default.use_tempaddr = 2

traceback occurs immediately after "Acceptable values" and as you can see, the definition for 0 has a weird char combo for the ' char. This is causing the traceback.

This appears in the file /etc/sysctl.d/10-ipv6-privacy.conf.
ubuntu@s1lp9g003:~$ cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=16.04
DISTRIB_CODENAME=xenial
DISTRIB_DESCRIPTION="Ubuntu 16.04.3 LTS"
ubuntu@s1lp9g003:~$ cat /etc/sysctl.d/10-ipv6-privacy.conf
# IPv6 Privacy Extensions (RFC 4941)
# ---
# IPv6 typically uses a device's MAC address when choosing an IPv6 address
# to use in autoconfiguration. Privacy extensions allow using a randomly
# generated IPv6 address, which increases privacy.
#
# Acceptable values:
# 0 - don’t use privacy extensions.
# 1 - generate privacy addresses
# 2 - prefer privacy addresses and use them over the normal addresses.
net.ipv6.conf.all.use_tempaddr = 2
net.ipv6.conf.default.use_tempaddr = 2

Interestingly, I tried the same command that checkbox is using:

find /etc/sysctl.* -name \*.conf | xargs cat in both bash and sh, and in both cases, the output is properly displayed... this only happens in the subshell that python3 is spawning to run the command:

>>> cmd = 'cat /etc/sysctl.d/10-ipv6-privacy.conf'
>>> subprocess.check_output(shlex.split(cmd),shell=False)
b"# IPv6 Privacy Extensions (RFC 4941)\n# ---\n# IPv6 typically uses a device's MAC address when choosing an IPv6 address\n# to use in autoconfiguration. Privacy extensions allow using a randomly\n# generated IPv6 address, which increases privacy.\n#\n# Acceptable values:\n# 0 - don\xe2\x80\x99t...

[Jeff Lane  (bladernr)]

Summary changed I can reproduce this on amd64 as well:

>>> subprocess.check_output(shlex.split(cmd),shell=False)
b"# IPv6 Privacy Extensions (RFC 4941)\n# ---\n# IPv6 typically uses a device's MAC address when choosing an IPv6 address\n# to use in autoconfiguration. Privacy extensions allow using a randomly\n# generated IPv6 address, which increases privacy.\n#\n# Acceptable values:\n# 0 - don\xe2\x80\x99t use privacy extensions.\n# 1 - generate privacy addresses\n# 2 - prefer privacy addresses and use them over the normal addresses.\nnet.ipv6.conf.all.use_tempaddr = 2\nnet.ipv6.conf.default.use_tempaddr = 2\n"