iotop can raise UnicodeDecodeError because /proc/pid/status can have non-utf-8 data in Name
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
iotop (Debian) |
Fix Released
|
Unknown
|
|||
iotop (Ubuntu) |
Fix Released
|
Medium
|
Unassigned | ||
Focal |
Fix Released
|
Undecided
|
Michał Małoszewski | ||
Jammy |
Fix Released
|
Undecided
|
Michał Małoszewski |
Bug Description
[Impact]
* Users of the iotop package reported a crash on Focal and Jammy.
* The iotop fails to display processes with non-ascii characters.
* That issue is caused by the behavior of the try-catch block in the
parse_
* The fix is to add errors='replace' to the for loop which is present in the
try-catch block.
[Test Plan]
Make a container for testing:
$ lxc launch ubuntu-daily:jammy jammy-test --vm
$ lxc shell jammy-test
After performing the steps on jammy, create a focal container using the same commands as above and replacing jammy with focal.
Type in:
$ apt update && apt install -y iotop gcc
$ echo -e '#include <unistd.h>\nint main(){
$ ./Инстанц�
## <Type Control-Z here to put the program in the background>
$ iotop
Traceback (most recent call last):
File "/usr/sbin/iotop", line 17, in <module>
main()
File "/usr/lib/
main_loop()
File "/usr/lib/
main_loop = lambda: run_iotop(options)
File "/usr/lib/
return curses.
File "/usr/lib/
return func(stdscr, *args, **kwds)
File "/usr/lib/
ui.run()
File "/usr/lib/
self.
File "/usr/lib/
lines = self.get_data()
File "/usr/lib/
return list(map(format, processes))
File "/usr/lib/
cmdline = p.get_cmdline()
File "/usr/lib/
proc_status = parse_proc_
File "/usr/lib/
for line in open('/
File "/usr/lib/
(result, consumed) = self._buffer_
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xef in position 20: invalid continuation byte
[Where problems could occur]
* The patch itself modifies only the parse_proc_
data.py code, so regressions should be limited to the behavior of that
function which is responsible for the names of the programs displayed.
-------
Sometimes iotop fails with following error:
$ sudo iotop
Traceback (most recent call last):
File "/usr/sbin/iotop", line 17, in <module>
main()
File "/usr/lib/
main_loop()
File "/usr/lib/
main_loop = lambda: run_iotop(options)
File "/usr/lib/
return curses.
File "/usr/lib/
return func(stdscr, *args, **kwds)
File "/usr/lib/
ui.run()
File "/usr/lib/
self.
File "/usr/lib/
lines = self.get_data()
File "/usr/lib/
return list(map(format, processes))
File "/usr/lib/
cmdline = p.get_cmdline()
File "/usr/lib/
proc_status = parse_proc_
File "/usr/lib/
for line in open('/
File "/usr/lib/
(result, consumed) = self._buffer_
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xd0 in position 20: invalid continuation byte
I found that /proc/pid/status can have an invalid UTF-8 symbol as part of Name, for example:
$ cat /proc/112413/status
Name: Инстанц�
Umask: 0002
State: S (sleeping)
Tgid: 112291
Ngid: 0
Pid: 112413
PPid: 112287
TracerPid: 0
Uid: 1000 1000 1000 1000
Gid: 1000 1000 1000 1000
FDSize: 4096
Groups: 4 24 27 30 46 116 126 1000
NStgid: 112291
NSpid: 112413
NSpgid: 112287
NSsid: 15882
VmPeak: 11092600 kB
VmSize: 11092564 kB
VmLck: 0 kB
VmPin: 0 kB
VmHWM: 5985748 kB
VmRSS: 5971968 kB
RssAnon: 5963752 kB
RssFile: 8212 kB
RssShmem: 4 kB
VmData: 6228788 kB
VmStk: 136 kB
VmExe: 8 kB
VmLib: 23640 kB
VmPTE: 12460 kB
VmSwap: 0 kB
HugetlbPages: 0 kB
CoreDumping: 0
THP_enabled: 1
Threads: 83
SigQ: 0/126975
SigPnd: 0000000000000000
ShdPnd: 0000000000000000
SigBlk: 0000000000000004
SigIgn: 0000000000001000
SigCgt: 2000000181004ccf
CapInh: 0000000000000000
CapPrm: 0000000000000000
CapEff: 0000000000000000
CapBnd: 0000003fffffffff
CapAmb: 0000000000000000
NoNewPrivs: 0
Seccomp: 0
Speculation_
Cpus_allowed: ff
Cpus_allowed_list: 0-7
Mems_allowed: 00000000,
Mems_allowed_list: 0
voluntary_
nonvoluntary_
The attached patch fixes the problem.
ProblemType: Bug
DistroRelease: Ubuntu 20.04
Package: iotop 0.6-24-g733f3f8-1 [modified: usr/lib/
ProcVersionSign
Uname: Linux 5.4.0-75-generic x86_64
ApportVersion: 2.20.11-
Architecture: amd64
CasperMD5CheckR
CurrentDesktop: ubuntu:GNOME
Date: Fri Jun 18 15:19:11 2021
Dependencies:
InstallationDate: Installed on 2020-11-24 (205 days ago)
InstallationMedia: Ubuntu 18.04.3 LTS "Bionic Beaver" - Release amd64 (20190805)
ProcEnviron:
TERM=xterm-
PATH=(custom, no user)
XDG_RUNTIME_
LANG=ru_RU.UTF-8
SHELL=/bin/bash
SourcePackage: iotop
UpgradeStatus: Upgraded to focal on 2020-11-25 (205 days ago)
Related branches
- git-ubuntu bot: Approve
- Christian Ehrhardt (community): Approve
- Canonical Server Reporter: Pending requested
-
Diff: 61 lines (+41/-0)3 files modifieddebian/changelog (+9/-0)
debian/patches/0001-Workaround-crashes-due-to-non-UTF-8-characters-in-pr.patch (+31/-0)
debian/patches/series (+1/-0)
- git-ubuntu bot: Approve
- Christian Ehrhardt (community): Approve
- Canonical Server Reporter: Pending requested
-
Diff: 75 lines (+43/-1)4 files modifieddebian/changelog (+9/-0)
debian/control (+2/-1)
debian/patches/crash-due-to-non-UTF-8-characters-in-process.patch (+31/-0)
debian/patches/series (+1/-0)
Changed in iotop (Debian): | |
status: | Unknown → Confirmed |
Changed in iotop (Debian): | |
status: | Confirmed → Fix Released |
description: | updated |
description: | updated |
description: | updated |
tags: | added: server-todo |
The attachment "fix-iotop- encoding- errors. diff" seems to be a patch. If it isn't, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are a member of the ~ubuntu-reviewers, unsubscribe the team.
[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issues please contact him.]