Terrible wrestool performance with very large executables

Bug #614918 reported by Scott Ritchie on 2010-08-08
168
This bug affects 44 people
Affects Status Importance Assigned to Milestone
icoutils (Ubuntu)
High
Unassigned

Bug Description

Binary package hint: icoutils

If I use gnome-exe-thumbnailer to thumbnail a moderately old World of Warcraft folder (which contains several multi-gigabyte executables), the computer tends to thrash as the whole file is loaded into memory.

I'm not 100% sure if it's possible, but it seems like performance would be greatly improved if only the initial part of the file is needed to extract the icon.

ProblemType: Bug
DistroRelease: Ubuntu 10.04
Package: icoutils 0.26.0-4ubuntu1
ProcVersionSignature: Ubuntu 2.6.32-24.38-generic 2.6.32.15+drm33.5
Uname: Linux 2.6.32-24-generic x86_64
NonfreeKernelModules: nvidia
Architecture: amd64
Date: Sat Aug 7 22:52:44 2010
InstallationMedia: Ubuntu 10.04 "Lucid Lynx" - Release Candidate amd64 (20100419.1)
ProcEnviron:
 LANG=en_US.utf8
 SHELL=/bin/bash
SourcePackage: icoutils

Related branches

Scott Ritchie (scottritchie) wrote :
Biswarup Ray (jumbli) wrote :

I have just had a nightmare this morning involving wrestool. I was downloading vmware player for windows using opera and all of a sudden wrestool started and the machine went out of control. Conky was showing wrestool as the top memory user. I tried to kill wrestool using top(terminal & top were both extremely slow) but werstool would be respawned somehow. So I killed the x-server and logged into openbox then started the download again using opera, this time no problem whatsoever !

Scott Ritchie (scottritchie) wrote :

What likely happened was gnome-exe-thumbnailer was attempting to show you the file when it wasn't finished downloading, and then kept retrying every time you killed the process.

When you did the download the second time it may not have been to a visible folder, so gnome-exe-thumbnailer didn't look at it until it was done and you browsed to it, where it finally finished and cached the thumbnail

Biswarup Ray (jumbli) wrote :

When I attempted to download it the second time, I logged into openbox so no question of gnome-thumbnailer running.

A little while ago I logged into gnome again and the problem occurred again but this time the problem persisted only for a relatively short time as the gnome-exe-thumbnailer was successful in it's attempt to create the icon for vmware player lying on the desktop. But it delayed my login process and conky was showing wrestool as the top memory consuming process throughout this period.

Launchpad Janitor (janitor) wrote :

This bug was fixed in the package icoutils - 0.29.1-0ubuntu1

---------------
icoutils (0.29.1-0ubuntu1) maverick; urgency=low

  * debian/copyright:
    - code is now GPL3 licensed
    - new upstream authors
  * New upstream release (LP: #651845)
    - Drop package changes to config.sub and config.guess
    - Supports vista icons (LP: #622220, #643085)
    - Performance improvements (LP: #614918)
    - Other fixes (LP: #643460)
  * debian/control: add homepage
 -- Scott Ritchie <email address hidden> Thu, 30 Sep 2010 00:27:59 -0700

Changed in icoutils (Ubuntu):
status: New → Fix Released
heasou (allarvin2) on 2010-10-21
Changed in icoutils (Ubuntu):
status: Fix Released → New
status: New → Fix Released
Md. Enzam Hossain (ienzam) wrote :

I am using icoutils (0.29.1-0ubuntu1) on lucid (https://launchpad.net/ubuntu/lucid/amd64/icoutils/0.29.1-0ubuntu1~lucid), but the problem still exists. After installing the maverick version (https://launchpad.net/ubuntu/maverick/amd64/icoutils/0.29.1-0ubuntu1) the problem fixes. Will this update come to lucid?

Scott Ritchie (scottritchie) wrote :

That seems spurious, as the lucid and maverick versions are identical. Are you sure the icon didn't just get cached?

I have been witnessing this problem on an up-to-date Maverick.

Downloading a large .exe from firefox via downthemall to the desktop caused nautilus to try and thumbnail it. Watching top shows my RAM shooting through the roof and causes my HDD to start swapping madly, absolutely decimating my computer's response for a while. This will eventually stop but then start back up again within the next few minute.

I'd be pressed to revert this bug back to confirmed but I'd rather wait for further confirmation. There's a possible dupe report (bug 659617) that is claiming the same behavior.

So I spent a half-hour dealing with the thrashing-while-downloading issue. The moment the download finished wrestool started up in full force, taking an absurd amount of time (and resources) before it finally wine-thumbnailed the .exe.

I'm going to take the liberty to set this back to confirmed. I will not label bug 659617 as a dupe as it regards constant updating through incomplete/downloading .exes and this regards complete ones.

Apologies, I don't have permission to alter the bug's status. I'd appreciate if someone can confirm this behavior still exists. Sorry for the spam.

Changed in icoutils (Ubuntu):
status: Fix Released → Confirmed
dth (destotelhorus) wrote :

This is an excerpt of the latest version of icoutils (0.29.1). More specifically out of the wrestool/main.c file. This C-Code will be run for every file you pass as an argument to wrestool:

                /* get file size */
                fi.name = argv[c];
                fi.total_size = file_size(fi.name);
[...]
                /* open file */
                fi.file = fopen(fi.name, "rb");
[...]
                /* read all of file */
                fi.memory = xmalloc(fi.total_size);
                if (fread(fi.memory, fi.total_size, 1, fi.file) != 1)

As promised, it will try to read ALL of the file into memory. Doing this with large self-extracting archives for example is a no-go.

over-life (namek) wrote :

Same here ... Not solved ... (Ubuntu 10.04 Gnome 2.32.1) up to date.

Confirmed for Ubuntu 11.10 and Dolphin with previews for Microsoft executables enabled and file size limit set to 500MB. With even single 454.8 MB exe sitting in Download folder everything locks up pretty hard for a while, although this behaviour seems to be random at first glance.

Hi, does this still affect any on 12.04?

Changed in icoutils (Ubuntu):
status: Confirmed → Incomplete
James Thorrold (jthorrold) wrote :

Yes, can confirm this is still the case in 12.04.

Changed in icoutils (Ubuntu):
status: Incomplete → Triaged
importance: Undecided → High

Also confirmed for 12.04. I found this bug trying to look through some old game CDs with large Windows exes on them. To unmount and remove the CDs from my machine I had to repeatedly kill wrestool and md5sum, which were between them hogging all resources and preventing umount from working.

Definitely confirmed for 12.04.
Eating about 600 MB of RAM on a 1000MB system is something I would *not* call "fixed".

I created a preliminary patch that limits the amount of memory allocated and amount of file portion read (I set the limit to 10M, can be changed in wrestool/wrestool.h).

p.s.: the patch is for iconutils version 0.30.0

The attachment "restrain a wrestool utility to allocate and read not more than certain amount" of this bug report has been identified as being a patch. The ubuntu-reviewers team has been subscribed to the bug report so that they can review the patch. In the event that this is in fact not a patch you can resolve this situation by removing the tag 'patch' from the bug report and editing the attachment so that it is not flagged as a patch. Additionally, if you are member of the ubuntu-reviewers team please also unsubscribe the team from this bug report.

[This is an automated message performed by a Launchpad user owned by Brian Murray. Please contact him regarding any issues with the action taken in this bug report.]

tags: added: patch
Ville Ranki (ville-ranki) wrote :

This still happens in 12.10. I hope the patch is accepted.

Mark Enriquez (enriquezmark36) wrote :

Still happening in 13.04 (in development). I am going to try tumbler

enroxorz (enroxorz) wrote :

Can confirm the issue on 12.10

Transferring large GOG.com install files from external hard drive to ~/Downloads. Is there a work around? I can't find it in here (and if it is in here, I apologize for skipping the comment).

STILL active on 16.04. WTF??? Ubuntu Gnome Flashback Metacity, drag and drop a 2Gb self-extracting exe (even renamed to .ex_) and the system grinds to a halt as wrestool tries to do a completely unnecessary (and unwanted) activity.
Do not read the entire file into memory just to extract an icon. A. I really don't care and B. it's an OS function which should, therefore, be lean and mean.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Duplicates of this bug

Other bug subscribers