System slows down to almost freeze

[Guus Ellenkamp]

For quite a while sometimes my system slows down to almost freeze where it becomes unusable. The problem seems to be related to heavy disk i/o, but I have not been able to pinpoint the cause. The problem seems to occur when I both use the bitcoin core application for Bitcoin Gold and VirtualBox. I normally use VirtualBox to run two Windows 2008 servers.

It seems in the end the system sorts itself out and starts running normal again, but it often causes to destroy the status of the Virtual Machines, sometimes resulting in a Virtual Machine going into 'guru-meditation' mode.

Normally my system is fast enough to run everything. I have 8 Gb memory, 5 hard disks and an Intel I5. I tried to spread the load over the hard disks, but that doesn't seem to have any effect on the slowing down of the system.

My feeling is that the problem is somehow related to some disk i/o system independent of the individual disks. Mostly the problem is solved by stopping the Bitcoin Core Gold program, but that is very hard to do, as even a terminal window takes minutes or up to an hour or so to respond. That also makes it very hard to identify the problem. I have several monitoring programs installed, but can't use them when the system is in this 'almost frozen' mode.

I am using Ubuntu 16.04.4. The problem is not to x.x.4 version of Ubuntu 16.04.

ProblemType: Bug
DistroRelease: Ubuntu 16.04
Package: e2fslibs 1.42.13-1ubuntu1
ProcVersionSignature: Ubuntu 4.13.0-38.43~16.04.1-generic 4.13.16
Uname: Linux 4.13.0-38-generic x86_64
ApportVersion: 2.20.1-0ubuntu2.15
Architecture: amd64
CurrentDesktop: LXDE
Date: Mon Apr 16 04:45:11 2018
Dependencies:
 gcc-7-base 7.1.0-10ubuntu1~16.04.york0 [origin: LP-PPA-jonathonf-gcc-7.1]
 libc6 2.23-0ubuntu10
 libgcc1 1:7.1.0-10ubuntu1~16.04.york0 [origin: LP-PPA-jonathonf-gcc-7.1]
InstallationDate: Installed on 2017-01-21 (449 days ago)
InstallationMedia: Lubuntu 15.10 "Wily Werewolf" - Release amd64 (20151021)
SourcePackage: e2fsprogs
UpgradeStatus: No upgrade log present (probably fresh install)

[Theodore Ts'o (tytso)]

Your system sounds like its thrashing due your processes wanting to you more memory than is available in your system. (BTW, this has nothing to do with e2fsprogs).

Some links to pages that might be helpful:

http://blog.scoutapp.com/articles/2015/04/10/understanding-page-faults-and-memory-swap-in-outs-when-should-you-worry
https://serverfault.com/questions/77461/how-do-i-measure-disk-thrashing-on-linux
https://unix.stackexchange.com/questions/259223/memory-usage-inexorably-creeping-upward

[Guus Ellenkamp (ellenkampguus)]

Hi Theodore,

Thanks for your reply.

As far as I know the system is not swapping, at least not a lot, but as
indicated, very hard to figure out what is going on without being able
to use a terminal screen. I also set the swap space to another disk than
the disk that seems to be causing the issue.

And yes, I know it has (probably) nothing to do with e2fsprogs, but I
had to choose something when reporting the error (using command line
utility).

And to me no matter what it is, it is still kind of a bug. I think an OS
virtually freezing like this is not a stable OS.

I am trying to change the disk scheduler now to see if that changes
anything.

Please note I started using ionice and nice for the suspect processes.
That doesn't seem to make any difference.

Looking forward to more info about how I can figure out what is going on,

Kind regards,

Guus

On Monday, 16 April, 2018 10:58 PM, Theodore Ts'o wrote:
> Your system sounds like its thrashing due your processes wanting to you
> more memory than is available in your system. (BTW, this has nothing to
> do with e2fsprogs).
>
> Some links to pages that might be helpful:
>
> http://blog.scoutapp.com/articles/2015/04/10/understanding-page-faults-and-memory-swap-in-outs-when-should-you-worry
> https://serverfault.com/questions/77461/how-do-i-measure-disk-thrashing-on-linux
> https://unix.stackexchange.com/questions/259223/memory-usage-inexorably-creeping-upward
>

[Guus Ellenkamp (ellenkampguus)]

I changed the scheduler to from 'cfq' to 'deadline' and I increased to commit times. That seems to have solved the problem for me as today the system did not slow down to almost freeze anymore with the same load.

To me it somehow still is a bug as I think an OS is not supposed to slow down that much, no matter what, without clear indication what is going on and without means to analyze or stop or reboot or whatever.