iotop crashes on launch

[keepitsimpleengr]

~$ sudo iotop
Traceback (most recent call last):
  File "/usr/sbin/iotop", line 17, in <module>
    main()
  File "/usr/lib/python2.7/dist-packages/iotop/ui.py", line 620, in main
    main_loop()
  File "/usr/lib/python2.7/dist-packages/iotop/ui.py", line 610, in <lambda>
    main_loop = lambda: run_iotop(options)
  File "/usr/lib/python2.7/dist-packages/iotop/ui.py", line 508, in run_iotop
    return curses.wrapper(run_iotop_window, options)
  File "/usr/lib/python2.7/curses/wrapper.py", line 43, in wrapper
    return func(stdscr, *args, **kwds)
  File "/usr/lib/python2.7/dist-packages/iotop/ui.py", line 501, in run_iotop_window
    ui.run()
  File "/usr/lib/python2.7/dist-packages/iotop/ui.py", line 155, in run
    self.process_list.duration)
  File "/usr/lib/python2.7/dist-packages/iotop/ui.py", line 434, in refresh_display
    lines = self.get_data()
  File "/usr/lib/python2.7/dist-packages/iotop/ui.py", line 415, in get_data
    return list(map(format, processes))
  File "/usr/lib/python2.7/dist-packages/iotop/ui.py", line 388, in format
    cmdline = p.get_cmdline()
  File "/usr/lib/python2.7/dist-packages/iotop/data.py", line 292, in get_cmdline
    proc_status = parse_proc_pid_status(self.pid)
  File "/usr/lib/python2.7/dist-packages/iotop/data.py", line 196, in parse_proc_pid_status
    key, value = line.split(':\t', 1)
ValueError: need more than 1 value to unpack

~$ ls /usr/lib/python2.7/dist-packages/ | grep iotop
iotop
iotop-0.6.egg-info

~$ uname -a
Linux KISE-055 4.4.0-127-generic #153-Ubuntu SMP Sat May 19 10:58:46 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

~$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 16.04.4 LTS
Release: 16.04
Codename: xenial

[Vassy (vassy)]

I confirm, same error.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Scrabble (scrabblecy)]

Same here:

~$ sudo iotop -o
Traceback (most recent call last):
  File "/usr/sbin/iotop", line 17, in <module>
    main()
  File "/usr/lib/python2.7/dist-packages/iotop/ui.py", line 620, in main
    main_loop()
  File "/usr/lib/python2.7/dist-packages/iotop/ui.py", line 610, in <lambda>
    main_loop = lambda: run_iotop(options)
  File "/usr/lib/python2.7/dist-packages/iotop/ui.py", line 508, in run_iotop
    return curses.wrapper(run_iotop_window, options)
  File "/usr/lib/python2.7/curses/wrapper.py", line 43, in wrapper
    return func(stdscr, *args, **kwds)
  File "/usr/lib/python2.7/dist-packages/iotop/ui.py", line 501, in run_iotop_window
    ui.run()
  File "/usr/lib/python2.7/dist-packages/iotop/ui.py", line 155, in run
    self.process_list.duration)
  File "/usr/lib/python2.7/dist-packages/iotop/ui.py", line 434, in refresh_display
    lines = self.get_data()
  File "/usr/lib/python2.7/dist-packages/iotop/ui.py", line 415, in get_data
    return list(map(format, processes))
  File "/usr/lib/python2.7/dist-packages/iotop/ui.py", line 388, in format
    cmdline = p.get_cmdline()
  File "/usr/lib/python2.7/dist-packages/iotop/data.py", line 292, in get_cmdline
    proc_status = parse_proc_pid_status(self.pid)
  File "/usr/lib/python2.7/dist-packages/iotop/data.py", line 196, in parse_proc_pid_status
    key, value = line.split(':\t', 1)
ValueError: need more than 1 value to unpack

~$ dpkg -l | grep iotop
ii iotop 0.6-1 amd64

~$ uname -a
Linux ip-xxxxxx-aws #69-Ubuntu SMP Sun May 20 13:42:07 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

~$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 16.04.4 LTS
Release: 16.04
Codename: xenial

[Geza Lucz (glucz)]

This doesn't seem to be an iotop, but rather a kernel bug, but best fixed in iotop.

in the proc/pid/status there is a clear line before SpeculationStoreBypass

Seccomp: 0
<----
SpeculationStoreBypass: vulnerable

So the line.split(':\t', 1) chokes on it

I suggest using a try: before the split block. I don't know if the clear line is a bug or a feature.

[Geza Lucz (glucz)]

I'm not a python expert, but

            try:
                key, value = line.split(':\t', 1)
            except:
                break

in /usr/lib/python2.7/site-packages/iotop/data.py
around line 196

works for me

[Gregory M. Blumenthal Scharf (llameadrpc)]

Can confirm #5

[G.C. Hassink (gchassink)]

ls /usr/lib/python2.7/dist-packages/ | grep iotop
iotop
iotop-0.6.egg-info
4.#153-Ubuntu SMP Sat May 19 10:58:46 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

4.4.0-124-generic #148-Ubuntu SMP Wed May 2 13:00:18 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

[G.C. Hassink (gchassink)]

Iotop version:
iotop
iotop-0.6.egg-info

kernel
4.4.0-127-generic #153-Ubuntu SMP Sat May 19 10:58:46 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
does have the problem
kernel
4.4.0-124-generic #148-Ubuntu SMP Wed May 2 13:00:18 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
(I tested a few)
and lower don't have the problem

[Kirill Okhotnikov (kirill-okhotnikov)]

I can confirm #5, but instead of break, use pass. The reason for such bug is the status message like this on 4.4.0-127-generic #153-Ubuntu SMP Sat May 19 10:58:46 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

cat /proc/1/status
Name: systemd
State: S (sleeping)
Tgid: 1
Ngid: 0
Pid: 1
PPid: 0
TracerPid: 0
Uid: 0 0 0 0
Gid: 0 0 0 0
FDSize: 256
Groups:
NStgid: 1
NSpid: 1
NSpgid: 1
NSsid: 1
VmPeak: 185548 kB
VmSize: 120012 kB
VmLck: 0 kB
VmPin: 0 kB
VmHWM: 6276 kB
VmRSS: 5388 kB
VmData: 83736 kB
VmStk: 132 kB
VmExe: 1392 kB
VmLib: 3684 kB
VmPTE: 96 kB
VmPMD: 16 kB
VmSwap: 212 kB
HugetlbPages: 0 kB
Threads: 1
SigQ: 1/14288
SigPnd: 0000000000000000
ShdPnd: 0000000000000000
SigBlk: 7be3c0fe28014a03
SigIgn: 0000000000001000
SigCgt: 00000001800004ec
CapInh: 0000000000000000
CapPrm: 0000003fffffffff
CapEff: 0000003fffffffff
CapBnd: 0000003fffffffff
CapAmb: 0000000000000000
Seccomp: 0
                                     <<<--- EMPTY LINE !!!
Speculation_Store_Bypass: not vulnerable
Cpus_allowed: f
Cpus_allowed_list: 0-3
Mems_allowed: 00000000,00000001
Mems_allowed_list: 0
voluntary_ctxt_switches: 3069
nonvoluntary_ctxt_switches: 6316

The line should be skipped during parsing.

[Omer Ozarslan (ozars)]

Using try-catch-pass to ignore exceptions can hide other issues in the future as a side effect. For example, if status messages start using a different separator in the future (e.g. see [*]), try-catch-pass solution would silently skip all lines.

Instead, a better solution would be fixing just the particular cause of this issue, the empty line in data.py. See the attachment for the proposed fix.

My two cents.

[*] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=844173

[Ubuntu Foundations Team Bug Bot (crichton)]

The attachment "iotop-1773383.patch" seems to be a patch. If it isn't, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are a member of the ~ubuntu-reviewers, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issues please contact him.]

[undefined (undefined)]

Not to bikeshed what Omer Ozarslan (ozars) proposed, but instead to reflect what upstream implemented...

http://repo.or.cz/iotop.git/commit/0392b205b5c3973a326721c2e9f97f0fa2eefa82

http://repo.or.cz/iotop.git/commit/7c51ce0e29bd135c216f18e18f0c4ab769af0d6f

http://repo.or.cz/iotop.git/blobdiff/69107ad1bfa05d240dbeac52f7d377dbb4c29daf..7c51ce0e29bd135c216f18e18f0c4ab769af0d6f:/iotop/data.py

the attached patch applies to the iotop 0.6-1 source package and creates the iotop 0.6-1u1604+1 source package with the upstream patch(es) in iotop-0.6/debian/patches/Skip_invalid_lines_in__proc_PID_status_files-LP1773383.patch.

apt-get source iotop
patch -p1 <iotop_0.6-1u1604+1.patch
apt-get build-dep iotop
# instead of devscripts you can install dpkg-dev and "dpkg-buildpackage -D -us -uc"
apt-get install devscripts
cd iotop-0.6
debuild -i -us -uc
cd ..
dpkg -i iotop_0.6-1u1604+1_*.deb

[Omer Ozarslan (ozars)]

From the patch submitted by undefined:
[...]
# the needed fields are probably formatted correctly.
[...]

Under this assumption, it may be reasonable to ignore formatting errors. It makes more sense to use upstream solution anyway for such a little difference in detail.

This issue showed me proper way to package Ubuntu patches before submitting as well. Thanks!

[Clovis Wichoski (cwichoski)]

Same problem here

16.04 LTS kernel 4.4.0-127-generic #153-Ubuntu SMP Sat May 19 10:58:46 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

Using kernel 124 without problem and cant understand why I/O is too high too will downgrade until this got resolved.

[Andreas Hasenack (ahasenack)]

Adding a linux task for the kernel team to consider dropping that empty line, that was perhaps added by mistake.

[Andreas Hasenack (ahasenack)]

The kernel aspect of this is being tracked in another bug: #1772671

[Christian Schrötter (killerbees19)]

Fixed in Kernel v4.4.0-129, available via "proposed" archive.

(see parent bug #1772671)

[Robie Basak (racb)]

Looks like this is being treated as a kernel regression and is being fixed that way. So there's no need to treat it as a bug in iotop in Ubuntu. This should be fixed for users as soon as the kernel fix in bug 1772671 has landed.