Tumblerd and Marlin use all available cpu and more

[Chascon]

Problem:
Tumblerd constantly crashes and slows down my computer to a snail's pace. top constantly reports 170 cpu usage while downloading in the background. It makes Ubuntu look like a slow and bloated desktop because it affects performance so adversely.

Step to reproduce:
1. download anything in the background
2. watch the system slow to a snails pace while top reports cpus usage peaking over 160 - 170

Eureka
I've noticed that it was reported in https://bugzilla.xfce.org/show_bug.cgi?id=8377 and has not been fixed. I am not using Thunar as noted in that report. I just noticed that that I was running the Marlin file browser, though, and on a hunch that it may be misbehaving as Thunar in the above linked bug report, I decided to close two instances of it (one with two tabs). If it's anything like the linked case, Marlin is constantly checking, accessing, and reproducing images for the constantly changing (downloading) file in one of its open directories. And this is what is taking up all the cpu.

This needs to be fixed, probably by both responsible parties but seems to be more an issue with Marlin (and possibly still Thunar).

Step to reproduce:
1. download anything in the background
2. Open an instance of Marlin to a dir that has the file that is downloading
3. watch top report tumblerd hogs the cpu

ProblemType: Bug
DistroRelease: Ubuntu 12.04
Package: tumbler 0.1.24-0ubuntu1
ProcVersionSignature: Ubuntu 3.2.0-27.43-generic 3.2.21
Uname: Linux 3.2.0-27-generic i686
ApportVersion: 2.0.1-0ubuntu11
Architecture: i386
Date: Tue Jul 31 22:18:36 2012
InstallationMedia: Ubuntu 11.04 "Natty Narwhal" - Release i386 (20110427.1)
SourcePackage: tumbler
UpgradeStatus: Upgraded to precise on 2012-03-23 (130 days ago)

[Kevin Brubeck Unhammer (unhammer)]

I noticed while downloading a film clip through Chromium, with the Downloads folder open in Thunar, that tumblerd was eating CPU. Since the thumbnail of the first 100 MB of the clip should be equal to the one of the first 200 MB, etc., would it be possible for Thunar to not request new thumbnails when a file has only been appended to?

I don't know how hard that is to keep track of … I'm assuming tumblerd doesn't/shouldn't be able to keep that kind of information along with the thumbnail, although it would of course be a more general solution if tumblerd itself could ignore these kinds of changes.

[fnds3000 (fnds3000)]

I had this problem right now!

My sugestion is that the model thumbnail was make only every x time, eg 2 seconds.
It's easy visa we can check the modified date of the current thumbnail.

[Ptipoika (ptipoika)]

I experienced the same problem with a CPU going crazy and my all system drastically slowed down. I had to take patience to kill the process and retrieve my system stable.

I noticed that it happened with a thunar window opened in the background. Surprisingly, the displayed directory in thunar was not the one where the video was downloading, but a parent of it. That seems to mean that it concerns not only current displayed directory, but also recently opened directories.

[Chascon (chascone)]

Apparently it's really noticeable when the file being downloaded is a large media file, as per some askubuntu page I can't find now.

[Chascon (chascone)]

Here's a complaint (from another user) and an explanation:

http://askubuntu.com/questions/89826/what-is-tumblerd

Thunar apparently still has the same issue (on Precise), but Nautilus on Precise doesn't, although on 12.10 beta it (Nautlilus) does. Apparently, this is a prevalent issue.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Cassidy James Blaede (cassidyjames)]

This also affects Pantheon Files. Downloading a few large video files, had the download folder open, and my laptop got extremely hot and unresponsive. Tumblerd was the culprit, maxing out multiple CPU cores.

[ammonkey (am-monkeyd)]

This is a bug in the filemanager not really a tumbler one.

[Neil-niekerk (neil-niekerk)]

I have disabled the "thumbnails" option in Thunar. "top" shows it using a lot of system resources to merely delete a file. Often the deletion of even 4 to 8 megabytes spins the hard drive for 20 seconds. Sometimes, much longer. Thank you for reading this.

[Cassidy James Blaede (cassidyjames)]

Why did this get marked as invalid for Files?

[Alejandro Perez (alejandro-perez-mendez)]

This is one of the most annoying bugs I have found when using XFCE. Whenever I copy a big file (e.g., MKV 8GB file) from a disk to another, I have to move Thunar out of the destination folder. Otherwise, it keeps trying to update the thumbnail every second, completely collapsing I/O bandwidth.

[Eric Koegel (eric-koegel)]

Created attachment 5602
Don't reload file info until changes are done

  Can you try this patch out and see if it fixes the issue? Thunar should wait for a set of changes to complete before refreshing the file/thumbnail. If it doesn't, we can also try rate-limiting the changed signals instead.

  When you do try this patch remember to tell the existing thunar daemon to quit (or kill it) before launching thunar again otherwise it won't be running the patched version.

Thanks!

[Alejandro Perez (alejandro-perez-mendez)]

Hi,

it does not work. It's weird, as even if I extend the patch and comment the thunar_file_reload (file) call, it still calls tumblerd for every single change.

I made sure I killed thunar before. "A" was not printed anyhow.

      switch (event_type)
        {
        case G_FILE_MONITOR_EVENT_CREATED:
        case G_FILE_MONITOR_EVENT_CHANGES_DONE_HINT:
        case G_FILE_MONITOR_EVENT_ATTRIBUTE_CHANGED:
          fprintf(stderr, "A\n");
          // thunar_file_reload (file);
          break;

        case G_FILE_MONITOR_EVENT_PRE_UNMOUNT:
        case G_FILE_MONITOR_EVENT_DELETED:
          // thunar_file_reload (file);
          break;

        default:
          break;
        }

[Steve Dodier-Lazaro (sidi)]

Eric: Will you have any chance to look again at this patch and issue in the future? Should I assign it to you?

Everyone else: could others test the patch and see what it does for them, to help Eric? There could be several issues aggregated under the same report, this happens quite often.

Thanks.

[Eric Koegel (eric-koegel)]

I have trouble finding a reliable way to trigger the bug, which is why
I thought my patch had solved it.

[Hjudt-l (hjudt-l)]

It looks like a good idea to apply that patch anyway.

[Hjudt-l (hjudt-l)]

Ok, lost this bug and found it again.

From my tests, this patch only helps when thunar is used to copy the file, triggering the CHANGES_DONE_HINT at the end of the file operation. If it is changed from outside thunar - simply try cp -, the CHANGES_DONE_HINT will be triggered more often, so often that the patch doesn't make a difference.

What will help here and should be implemented in the future:

a) Create a reload queue for rate-limiting, and maybe an extra queue for thumbnailing requests which can be rate-limited separately.
b) Signal "image-done" instead of file-"changed", so that the file does not get reloaded but instead the view only updates the row with the newly generated image, not wasting so many resources.

These are a bit ambitious plans though, and better to be delayed until after the gtk3 port.

However, I will apply a patch similar to the one attached here, because it makes thunar behave nicer to file operations on network resources, at least when performing them in thunar. Besides, on demand users can trigger a full reload manually now which reloads all file information.

[Hjudt-l (hjudt-l)]

After a few tests, it should actually work for operations outside thunar too. But probably it depends on the operation itself, e.g. how big the chunks are that are copied, how often a changed / changes_done_hint event is emitted.

The changes_done_hint is also emitted after an attribute_changed event, so it is safe to replace this by changes_done.

It seems to be also emitted after a create, though I am not sure about this because the create event is always followed by a changed or attribute_changed event. Maybe it is better to leave that alone for now.