ubiquity crashed with UnicodeDecodeError in decode(): 'utf-8' codec can't decode byte 0x91 in position 269: invalid start byte

[Giovanni Preda]

Trying to install

ProblemType: Crash
DistroRelease: Ubuntu 17.10
Package: ubiquity 17.10.10
ProcVersionSignature: Ubuntu 4.13.0-16.19-generic 4.13.4
Uname: Linux 4.13.0-16-generic x86_64
ApportVersion: 2.20.7-0ubuntu3
Architecture: amd64
CasperVersion: 1.387
Date: Mon Dec 25 13:19:57 2017
ExecutablePath: /usr/lib/ubiquity/bin/ubiquity
InterpreterPath: /usr/bin/python3.6
LiveMediaBuild: Lubuntu 17.10 "Artful Aardvark" - Release amd64 (20171017.1)
ProcCmdline: /usr/bin/python3 /usr/lib/ubiquity/bin/ubiquity gtk_ui
ProcEnviron:
 TERM=unknown
 PATH=(custom, no user)
 LANG=C.UTF-8
 SHELL=/bin/bash
Python3Details: /usr/bin/python3.6, Python 3.6.3, python3-minimal, 3.6.3-0ubuntu2
PythonArgs: ['/usr/lib/ubiquity/bin/ubiquity', 'gtk_ui']
PythonDetails: /root/Error: command ['which', 'python'] failed with exit code 1:, Error: [Errno 2] No such file or directory: "/root/Error: command ['which', 'python'] failed with exit code 1:": "/root/Error: command ['which', 'python'] failed with exit code 1:", unpackaged
SourcePackage: ubiquity
Title: ubiquity crashed with UnicodeDecodeError in decode(): 'utf-8' codec can't decode byte 0x91 in position 269: invalid start byte
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups:

[Giovanni Preda (preda-giovanni)]

No one? This happens with every Ubuntu-based distro

[Jean-Baptiste Lallement (jibel)]

Thanks for your report.

Could you please detail the steps to reproduce this crash? I tried with Lubuntu 17.10, Ubuntu 17.10 and 18.04 with Italian and cannot reproduce this bug.

[Giovanni Preda (preda-giovanni)]

This problem only occurs when I try to install an Ubuntu-based distro on my Teclast X80 Plus tablet (Intel Atom x5-Z8350, 2GB RAM, 32GB EMMC).
I also tried selecting another language (English) but nothing changed.
I can install Lubuntu without any problem using the alternate iso where the installation is not made in a live environment.
Installing via USB stick made with Rufus.
Thanks for your reply.

[Giovanni Preda (preda-giovanni)]

Sorry, I forgot to mention that I also tried with Ubuntu 16.04.3 LTS and I have the same problem.

[Giovanni Preda (preda-giovanni)]

Do someone need more informations?

[Launchpad Janitor (janitor)]

[Expired for ubiquity (Ubuntu) because there has been no activity for 60 days.]

[Michal J (beepbopboop)]

This issue has timed out due to inactivity, but I have recently come across it while trying to install elementary OS 5.0 Juno, as well as KDE Neon and a few other Ubuntu-based distros that all use the ubiquity installer. The issue, I think, boils down to a call to parted_devices, which returns a list of the available devices and partitions. This can be seen in the UbiquitySyslog.txt attachment above, where the Hynix MMC device returns the device name "HBG4a" suffixed by the unicode character U+FFFD, ie the replacement character when it's unable to read the rest of the string. I believe this is the cause of the crash since it is a non-UTF8 character. Hopefully someone else has run into this and can confirm it.

[Michal J (beepbopboop)]

Digging a little deeper, specifically in /usr/lib/ubiquity/plugins/ubi-prepare.py, in the method setup_sufficient_space() there is a call to a subprocess, that subprocess being parted_devices. Running parted_devices on your own in terminal with this device will report the above "HBG4a" and unicode character, but since this is not UTF8 I think the python script will fail since the default stance on errors is to fail with an error code, UnicodeDecodeError(). To bypass this, I modified that call to be as such:

proc = subprocess.Popen(['parted_devices'], stdout=subprocss.PIPE, encoding='iso-8859-1', universal_newlines=True)

This allows it to at least get passed this reported error. But subsequently, it will fail on ubi-partman.py with error code 141 (unknown what it means), and will not enter the partition manager in ubiquity (skips it entirely).

The problem seems that this unicode naming keeps coming back to haunt the python scripts as it attempts to run the partition manager. Ideally, it would be great to remove this unicode in the name entirely since I don't think it will affect the partitioning in any way (I even looked into where the hardware reports this from and if it can be manually changed, but so far no dice). Otherwise, I'm stuck with hunting through all the scripts that do these calls and reads from the name and patch them to work with the iso-8859-1 encoding.

Onward.

[Msd (msd+launchpad)]

I also experience this problem : "ubiquity crashed with UnicodeDecodeError in decode(): 'utf-8' codec can't decode byte 0xff in position 0: invalid start byte"

With latest Ubuntu 22.04 ISO, I try to install on a "Asus T100TA"

[Dominik (dominikwelke)]

I also experience the error with Ubuntu 20.04 LTS (under the hood of Zorin OS 16.3).

in my case caused by the SSD (disk model "INNOVATION__IT"; the underscore representing a non-utf character).

as described above (#9), hotfixing plugins/ubi-prepare.py gets me past the user settings, but then the partitioning wizard is skipped and a frozen installation screen is shown.
seems like plugins/ubi-partman.py has some unreported exceptions (maybe setting one_disk = True or the like?) and hence not going the right route.

correct me if I'm wrong, but I'd be surprised if the disk model (intended for humans) is necessary for the installation: is there a documentation I didnt find, or could a dev direct me to important sections and expected values in the script? I'd be up to trying to hardcode my ssd specs in but find it hard to reverse engineer the scripts..

[Andrew Dyer (ubuntumazda)]

Just had the same bug show up on lubuntu 18.05 trying to install on a small Atom computer. Same device, "H8G4a" suffixed by the unicode character.
Took a while to get around the issue. Ended up using a different PC to install lubuntu to a small partition on a portable HDD. Then using Clonezilla I imaged that drive and wrote the image to the embedded eMMC, followed by growing the partition into unallocated space.
I could not use the Atom computer to install to the other drive as the eMMC is non-removable. As soon as the installer looks for storage devices it hangs.
I could not work out a way disable the device or otherwise exclude the drive from the device list.
Clonezilla repeatably reported issues parsing the device name as well, but was able to proceed. Only impact being that in the device list the eMMC appeared as "emmcblk0 <default name>" or something like that.