[2.2, 2.1] ssh key comment contains unicode character - node deployment fails

Bug #1668329 reported by Tobias Brausen
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
MAAS
Invalid
High
Unassigned
curtin
Invalid
Undecided
Unassigned

Bug Description

if one of the MAAS admin ssh keys contains in the ssh comment the unicode charakter Unicode Character 'RIGHT SINGLE QUOTATION MARK' (U+2019) (see: http://www.fileformat.info/info/unicode/char/2019/index.htm) the deployment of a node fails with:

gui-log:
---
        Node changed status - From 'Deploying' to 'Failed deployment' Mon, 27 Feb. 2017 15:52:22
        Marking node failed - Installation failed (refer to the installation log for more information). Mon, 27 Feb. 2017 15:52:22
        Node installation failure - 'cloudinit' running modules for final Mon, 27 Feb. 2017 15:52:22
        Node installation failure - 'cloudinit' running config-ssh-authkey-fingerprints with frequency once-per-instance Mon, 27 Feb. 2017 15:52:21
        Installation complete - Node disabled netboot Mon, 27 Feb. 2017 15:52:19
        PXE Request - installation Mon, 27 Feb. 2017 15:49:08
---

installation log excerpt via gui:
---
Traceback (most recent call last):
  File "/curtin/curtin/commands/apply_net.py", line 140, in _disable_ipv6_privacy_extensions
    contents = util.load_file(cfg)
  File "/curtin/curtin/util.py", line 340, in load_file
    return fp.read(read_len) if read_len else fp.read()
  File "/usr/lib/python3.5/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 277: ordinal not in range(128)
Injecting fix for ipv6 mtu settings: /tmp/tmpboc7vlo8/target/etc/network/if-pre-up.d/mtuipv6
---

when the key is removed nodes can be deployed normally again.

MAAS versions:
root@ba0b1a-mgm231maasregion-sk:/var/log/maas# dpkg -l '*maas*'|cat
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-===============================-==============================-============-=============================================
un maas <none> <none> (no description available)
ii maas-cli 2.1.3+bzr5573-0ubuntu1~16.04.1 all MAAS client and command-line interface
ii maas-common 2.1.3+bzr5573-0ubuntu1~16.04.1 all MAAS server common files
ii maas-dns 2.1.3+bzr5573-0ubuntu1~16.04.1 all MAAS DNS server
ii maas-proxy 2.1.3+bzr5573-0ubuntu1~16.04.1 all MAAS Caching Proxy
ii maas-region-api 2.1.3+bzr5573-0ubuntu1~16.04.1 all Region controller API service for MAAS
ii maas-region-controller 2.1.3+bzr5573-0ubuntu1~16.04.1 all Region Controller for MAAS
un maas-region-controller-min <none> <none> (no description available)
un python-django-maas <none> <none> (no description available)
un python-maas-client <none> <none> (no description available)
ii python3-django-maas 2.1.3+bzr5573-0ubuntu1~16.04.1 all MAAS server Django web framework (Python 3)
ii python3-maas-client 2.1.3+bzr5573-0ubuntu1~16.04.1 all MAAS python API client (Python 3)
ii python3-maas-provisioningserver 2.1.3+bzr5573-0ubuntu1~16.04.1 all MAAS server provisioning libraries (Python 3)

/var/log/maas/* is available upon request

replicate bug:

1. add key via cli:
dtadmin@ba0b1a-mgm231control-sk:~$ maas maas sshkeys create name=admin key="$(cat peter_piper.pub)"
Success.
Machine-readable output follows:
{
    "id": 22,
    "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEULSsVIINkRrV4ljo+oC0iARCM0IeJZOU5/zO3hVV+ftJvfLze63tDZLIfzLhsjnwFwNjnjGNvYxq661Ys0mYLS+W935TJu/EqLdVBY8Y32I7K0PorQzgNAZlV8XYU3rJFA9avxaJdt4fJWEiWg0/h74GJge+IXI/Pv0FS7yOvN97qPwfKobtZr5IhtH1ckGipQmBMMGVE7UPtgYoaRifSMIcf/m4SehMup0GKc++hk27+G6DAREQsby3gJQfhqDXer/cDuSEAIJAoDzYz7qKOOgslk4trRC+UpVgcYTWAqZSasJ97KxE3pzt+PLkuKwF8q8+ZbR0WIEL7m0R96bZ Pe’er Piper | Master key of <peterpiper@home>",
    "keysource": null
}

2. deploy a node

3. remove this key:
dtadmin@ba0b1a-mgm231control-sk:~$ maas maas sshkey delete 22
Success.

4. deploy node again

relevant key part:
dtadmin@ba0b1a-mgm231control-sk:~$ hexdump -C peter_piper.pub
00000000 73 73 68 2d 72 73 61 20 41 41 41 41 42 33 4e 7a |ssh-rsa AAAAB3Nz|
[...]
00000170 57 49 45 4c 37 6d 30 52 39 36 62 5a 20 50 65 e2 |WIEL7m0R96bZ Pe.|
00000180 80 99 65 72 20 50 69 70 65 72 20 7c 20 4d 61 73 |..er Piper | Mas|
[...]

Revision history for this message
Tobias Brausen (t-brausen) wrote :
Changed in maas:
milestone: none → 2.2.0
importance: Undecided → High
status: New → Triaged
summary: - ssh key comment contains unicode character - node deployment fails
+ [2.2, 2.1] ssh key comment contains unicode character - node deployment
+ fails
Revision history for this message
Mike Pontillo (mpontillo) wrote :

Sigh. It's 2017. Any time we encode text, in the absence of knowledge to the contrary, we should assume it's UTF-8. There are far too many occurrences of "ascii" in the MAAS code.

Revision history for this message
Gavin Panella (allenap) wrote :

RIGHT SINGLE QUOTATION MARK encodes to hex e2 80 99 in UTF-8 in Python
by default, so the traceback:

  Traceback (most recent call last):
    File "/curtin/curtin/commands/apply_net.py", line 140, in
    _disable_ipv6_privacy_extensions
      contents = util.load_file(cfg)
    File "/curtin/curtin/util.py", line 340, in load_file
      return fp.read(read_len) if read_len else fp.read()
    File "/usr/lib/python3.5/encodings/ascii.py", line 26, in decode
      return codecs.ascii_decode(input, self.errors)[0]
  UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position
  277: ordinal not in range(128)

certainly mentions the right char, e2, but it's very hard to see how the
code in question would be reading the file containing a public SSH key.
This is puzzling.

Changed in maas:
milestone: 2.2.0 → 2.2.0rc2
Changed in maas:
milestone: 2.2.0rc2 → 2.2.0rc3
Changed in maas:
assignee: nobody → Данило Шеган (danilo)
status: Triaged → In Progress
Revision history for this message
Данило Шеган (danilo) wrote :

I've retested this with 2.2.0~rc3+bzr6016-0ubuntu1~16.04.1. There are a few things here.

First, the fact that Curtin bails seems fixed with lp:curtin r446 (http://bazaar.launchpad.net/~curtin-dev/curtin/trunk/revision/446) from 2017-02-06 (there's no bug reference on the commit) to support UTF-8. xenial-updates has 0.1.0~bzr470-0ubuntu1~16.04.1, and in my test curtin bzr480 gets used (I am guessing from cloud archives), so that should be fine.

Second, as Gavin noted, it's a bit unclear how does SSH key data end up being passed around to *_disable_ipv6_privacy_extensions* method, but regardless, this error does not show up in the installation log anymore.

However, I still get a "failed deployment". The rsyslog output for the node on the server log has this interesting snippet: http://paste.ubuntu.com/24466799/. This suggests that the cloudinit/config/cc_ssh_authkey_fingerprints.py module can't handle UTF-8, which would make this a bug in cloudinit.

Finally, MAAS UI hangs when attempting to add an SSH key with UTF-8 in the key description: there is no reason it should hang, especially since it works via API as suggested in the original report.

Changed in maas:
milestone: 2.2.0rc3 → 2.2.0rc4
Changed in maas:
milestone: 2.2.0rc4 → 2.2.1
Changed in maas:
milestone: 2.2.1 → 2.2.x
Changed in maas:
milestone: 2.2.x → 2.3.0
Revision history for this message
Ryan Harper (raharper) wrote :

The curtin error message was non-fatal but as mentioned in comment #4, curtin has fixed the utf8 handling and it's already fixed-released. Please re-open if you believe there is a new curtin issue.

Changed in curtin:
status: New → Invalid
Changed in maas:
milestone: 2.3.0 → 2.3.x
Changed in maas:
assignee: Данило Шеган (danilo) → nobody
status: In Progress → Confirmed
Revision history for this message
Adam Collard (adam-collard) wrote :

This bug has not seen any activity in the last 6 months, so it is being automatically closed.

If you are still experiencing this issue, please feel free to re-open.

MAAS Team

Changed in maas:
status: Confirmed → Invalid
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.