iotop crashes when system time updates to a value in the past

[Andrei Costin]

1) lsb_release -rd
Description: Ubuntu 14.04.5 LTS
Release: 14.04

2) apt-cache policy iotop
iotop:
  Installed: 0.6-1
  Candidate: 0.6-1
  Version table:
 *** 0.6-1 0
        500 http://ubuntu.trumpetti.atm.tut.fi/ubuntu/ trusty/universe i386 Packages
        100 /var/lib/dpkg/status

3) while "iotop" or "sudo iotop" is running, when system time updates to a value in the past, "iotop" should continue running and not crash

4) iotop crashes when system time updates to a value in the past

Steps to reproduce:
a) modify system time to be set manually and set it to a future date, for example 1 year from now (i.e., just change the year to 2018). you can do this with minutes as well. this is a realistic scenario in some cases when the date is wrongly set by hand or by BIOS
b) in a terminal start iotop (e.g., "sudo iotop") and have iotop running
c) change the system to update its time from the internet/NTP and confirm the system date was updated
d) watch the iotop crash in the terminal and the Ubuntu AppPort error reporting popping-up
e) iotop traceback follows:
Traceback (most recent call last):
  File "/usr/sbin/iotop", line 17, in <module>
    main()
  File "/usr/lib/python2.7/dist-packages/iotop/ui.py", line 620, in main
    main_loop()
  File "/usr/lib/python2.7/dist-packages/iotop/ui.py", line 610, in <lambda>
    main_loop = lambda: run_iotop(options)
  File "/usr/lib/python2.7/dist-packages/iotop/ui.py", line 508, in run_iotop
    return curses.wrapper(run_iotop_window, options)
  File "/usr/lib/python2.7/curses/wrapper.py", line 43, in wrapper
    return func(stdscr, *args, **kwds)
  File "/usr/lib/python2.7/dist-packages/iotop/ui.py", line 501, in run_iotop_window
    ui.run()
  File "/usr/lib/python2.7/dist-packages/iotop/ui.py", line 155, in run
    self.process_list.duration)
  File "/usr/lib/python2.7/dist-packages/iotop/ui.py", line 421, in refresh_display
    format_bandwidth(self.options, total[1], duration).rjust(14)),
  File "/usr/lib/python2.7/dist-packages/iotop/ui.py", line 63, in format_bandwidth
    return format_size(options, size and float(size) / duration) + '/s'
  File "/usr/lib/python2.7/dist-packages/iotop/ui.py", line 60, in format_size
    return human_size(bytes)
  File "/usr/lib/python2.7/dist-packages/iotop/ui.py", line 55, in human_size
    return '%s%.2f %s' % (sign, (float(size) / (1 << (10 * expo))), UNITS[expo])
ValueError: negative shift count

[Christian Ehrhardt  (paelzer)]

First of all thank you for your report Andrei and your help to make Ubuntu better.
I can confirm the behavior - looking into what is affected and if there might be a solution already ...

[Christian Ehrhardt  (paelzer)]

I checked up to more recent versions and found that this is all over any release of iotop.
Later Ubuntu releases have timedatectl, keep that in mind when trying to reproduce to set time.

I checked and even latest git of iotop has the problem.
That said given the kind of issue I think it should be reported upstream and then tickle down into Ubuntu (and Debian) releases.

Would you mind reporting it upstream and mentioning a link to the bug here?
That would be great and IMHO the best way to tackle this issue.

[1]: http://freecode.com/projects/iotop
[2]: http://guichaz.free.fr/iotop/

[Andrei Costin (zveriu)]

Hello,

Not sure my email below reached "<email address hidden>".
Therefore resending, and adding "Paul Wise <email address hidden>" (based
on commit log git://repo.or.cz/iotop.git) and ChristianEhrhardt
<email address hidden> from original bug report on Ubuntu.

Hope this helps to notify whoever is necessary.

Many thanks.

Andrei

-------- Forwarded Message --------
Subject: iotop: upstream bug report
Date: Mon, 24 Apr 2017 20:28:01 +0300
From: Andrei C <email address hidden>
To: <email address hidden>

Hello guichaz,

First, thank you for maintaining this great tool.
Writing a direct email since I could not find where/how to report iotop
bugs directly and not through distributions (ubuntu, redhat, debian).

I wanted to report an upstream bug for iotop, as suggested by
ChristianEhrhardt <email address hidden> (see below his remarks).

I have open the bug in Ubuntu 14.04 LTS and it was confirmed, but
appears to affect any distribution and version (as per Christian's
analysis).

The original bug report is here:

https://bugs.launchpad.net/bugs/1685512

Many thanks!

Andrei

***
On 04/24/2017 03:03 PM, ChristianEhrhardt wrote:
> I checked up to more recent versions and found that this is all over
any release of iotop.
> Later Ubuntu releases have timedatectl, keep that in mind when trying
to reproduce to set time.
>
> I checked and even latest git of iotop has the problem.
> That said given the kind of issue I think it should be reported
upstream and then tickle down into Ubuntu (and Debian) releases.
>
> Would you mind reporting it upstream and mentioning a link to the bug
here?
> That would be great and IMHO the best way to tackle this issue.
>
> [1]: http://freecode.com/projects/iotop
> [2]: http://guichaz.free.fr/iotop/
>

***

[Paul Wise (Debian) (pabs)]

On Thu, 2017-05-04 at 11:17 +0300, Andrei C wrote:

> ChristianEhrhardt <email address hidden>

FYI, as you can see here this address only mails the bug report:

https://bugs.launchpad.net/ubuntu/+source/iotop/+bug/1685512/comments/3

> Hope this helps to notify whoever is necessary.

Thanks for the ping, I usually look at Ubuntu iotop bugs only when I'm
about to make an upload to Debian, which hasn't happened for a while
because Guillaume Chazarain hasn't made a release recently.

> https://bugs.launchpad.net/bugs/1685512

I think the issue is that iotop uses the current time to calculate
duration instead of using monotonically increasing time, which means
that when you change the time backwards, it gets a negative duration
and that causes some calculations to be incorrect.

Please test the attached patch and if it is correct I will include it
upstream and make an upload to Debian once stretch is released. The
patch can then be synced to Ubuntu. Please note that the patch relies
on Python 3.3 or later, so it won't work on anything older than trusty.
If it works then I'll try to make it work on more Python versions too.

 tag patch

--
bye,
pabs

https://wiki.debian.org/PaulWise

[Ubuntu Foundations Team Bug Bot (crichton)]

The attachment "0001-Use-monotonic-time-to-calculate-durations.patch" seems to be a patch. If it isn't, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are a member of the ~ubuntu-reviewers, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issues please contact him.]

[Paul Wise (Debian) (pabs)]

On Thu, 2017-05-04 at 21:31 +0800, Paul Wise wrote:

> Please test the attached patch and if it is correct I will include it
> upstream and make an upload to Debian once stretch is released. The
> patch can then be synced to Ubuntu. Please note that the patch relies
> on Python 3.3 or later, so it won't work on anything older than trusty.
> If it works then I'll try to make it work on more Python versions too.

I didn't get a response to the request to test the patch, so I have
just committed it. I went ahead and added fallbacks for old Python.

 status fixcommitted

--
bye,
pabs

https://wiki.debian.org/PaulWise

[Paul Wise (Debian) (pabs)]

The patch for iotop bug #1685512 has been released to Debian and will
soon be auto-synced to Ubuntu.

 status fixreleased

--
bye,
pabs

https://wiki.debian.org/PaulWise