Exaile eats up memory as it continues playing

[ces]

Using the daily BZR builds for 0.2.11 (in the exaile gutsy repository). When it starts up, Exaile uses only about 20+ MB of memory; when left playing for extended periods of time, however, memory usage steadily climbs up, sometimes to more than 100 MB. Please tell me what information, etc. you need, such as logs, because I'm not sure where to get them :) Thanks!

[Dave (d-redfern)]

I see this too and it's even worse under Fedora 8 64bit, seems to be chronically leaking memory. I have ~8,500 tracks mostly 320Kbps MP3 with a handful of WMA files. Currently loaded 149 tracks of 8482 into a saved playlist. Disabled periodic scans for new files, splash screen and downloading of album art. No plugins enabled but 3 installed. This is using 0.2.11-1.fc8.x86_64 from the standard Fedora 8 reps.

As at 11:29:15
  PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
 7596 dave 20 0 596m 104m 17m S 4 2.6 0:12.72 python

As at 11:31:58
  PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
 7596 dave 20 0 607m 104m 17m S 3 2.6 0:18.10 python

[francois.martorello (francois-martorello)]

I have this on Ubuntu 7.10 64 bits exaile 0.2.11
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
11339 francois 15 0 1784m 564m 9m S 11 _56.3_ 187:49.31 python

[Veloce (kerry-mayes)]

Just been tracking this ready to supply a bug report. I'm using the develoment releases: currently 0.2.12devel under Ubuntu Gutsy.

Watching "virtual memory" as defined by Ubuntu's System Monitor, memory usage starts at about 90Mb before playing anything. Jumps about 10Mb for the first song then about 5Mb per song thereafter. At 400+Mb my system is unusable (256Mb of physical memory). I'm only using this machine for Exaile and as a thin client.

[Veloce (kerry-mayes)]

I'd have said this was a fairly serious bug, any idea when someone can have a look at it?

[Johannes Sasongko (sjohannes)]

Well, to be honest I don't know _where_ to start looking, as technically pure-Python apps don't leak unless there's a bug in one of the dependencies. I think there's one place where Exaile keeps a reference to previously-played songs, giving the impression of a leak, but AFAIK it shouldn't take a huge amount of memory.

Just one comment: it's expected that programs running in 64-bit systems consume more memory.

[Dave (d-redfern)]

"Just one comment: it's expected that programs running in 64-bit systems consume more memory."

Yes I expect 64bit apps to use more, but continuing to use more is the issue. For example: if I run exaile for several hours it's memory usage is up around 850MB virtual and 120MB ram. Though I have no proof yet, I am pretty sure running exaile for extended periods of times actually crashes my machine - this is a hard crash, X is totalled and it is impossible to switch terminals. A hard reset is the only option.

As I said I have no proof yet so tomorrow (Saturday 8th December 2007) I will set-up exaile running while logging out the process info every so often to a file and run it until either nothing happens, or it locks up. I will attempt my normal weekend usage (turn on, leave, check email, browse web etc).

Incidentally enabling album art fetch, library scanning, Last.fm artist fetch all contribute to the memory usage shooting up by a large amount.

[Dave (d-redfern)]

I ran exaile yesterday (Saturday) from around 9:50am until 11pm taking a snapshot of the memory usage every 5 seconds (ps waxl | grep exaile) and dumping it into a log file. I used the PC as I would normally so had a few other open apps (Pidgin, Evolution, Mail Notification, Eclipse Zend Studio Beta, Opera and serveral shells). I took a full process snapshot before starting exaile. exaile was run under Gnome desktop 2.20.2 on a standard Fedora 8 64 bit install (no KDE). At almost all times, exaile was only in the notification area, the main program window was not open. All data fetching in exaile was disabled and there were no library scans.

The results were interesting more for the point that nothing much happened other than only a handful of large jumps in virtual usage. Exaile performed fine the entire time, for whatever reason I did not see any large memory spikes that I had done previously. It seemed to settle at around 110MB ram and ~700MB virt. Highest recorded virtual memory usage was 757MB (I have seen higher, but I've been keeping Fedora 8 up-to-date and there have been a few python updates).

The major jumps in memory were the following (I've trimmed a few fields):

2007-12-08 09:51:20 0 500 3617 1 20 0 526660 103256 0:10 python /usr/lib64/exaile/exaile.py
2007-12-08 09:51:25 0 500 3617 1 20 0 663576 108248 0:11 python /usr/lib64/exaile/exaile.py

Which coincides with when I started playback of tracks

2007-12-08 09:54:55 0 500 3617 1 20 0 663576 108272 0:17 python /usr/lib64/exaile/exaile.py
2007-12-08 09:55:00 0 500 3617 1 20 0 685112 109140 0:17 python /usr/lib64/exaile/exaile.py

Small jump, not sure from what, possibly from a format change e.g. WMA -> MP3. This was then followed a short while later with a ~20MB drop in virtual memory usage and then a huge jump to the 700MB+ territory where it never left until I powered down.

2007-12-08 10:39:19 0 500 3617 1 20 0 675004 109536 1:40 python /usr/lib64/exaile/exaile.py
2007-12-08 10:39:24 0 500 3617 1 20 0 740540 109520 1:40 python /usr/lib64/exaile/exaile.py

The last recorded entry was:
2007-12-08 23:26:48 0 500 3617 1 20 0 734928 111532 32:49 python /usr/lib64/exaile/exaile.py

and the highest usage entry was:
2007-12-08 23:24:11 0 500 3617 1 20 0 750440 112448 25:50 python /usr/lib64/exaile/exaile.py

The log is available if any developer wants it (I'm not going to post it though as it contains no exaile specific debug data, only the results from ps).

[David Wagner (daw-bugzilla)]

On my F8 x86_64 machine, exaile is currently taking 1034MB virtual memory and 494MB rss (per top). It's been running for about a day. That seems excessive, even for a 64-bit machine.

If anyone has any suggestions on how to debug this, I'd be interested to hear it. I don't know Python tools, unfortunately. (I presume valgrind memcheck is not going to be terribly helpful, since it's written in Python?)

[Ubitux (ubitux-deactivatedaccount)]

The memleak seems to come from the Rescan Collection tool (See attachment)

[Johannes Sasongko (sjohannes)]

Confirmed at least the rescan collection problem. Mem use climbs up after every rescan and doesn't go down even after gc.collect().

[Veloce (kerry-mayes)]

Thanks for the diagnosis David!

Based on this I tried seeing if turning off automatic rescanning helped and it does. So the work around is to set edit - preferences - general - library - "Scan for new music every" to zero.

Memory use expands if you do a manual scan but it then just stays at that new level.

[Adam Olsen (arolsen)]

I may have fixed this problem in the latest bzr. Our main bzr repo is down atm, so if you want to try this fix out, use the temporary repo, which can be found here:

http://bazaar.launchpad.net/~exaile-devel/exaile/20080201temp

[Clifton Sammut (clifweb)]

Playing an mp3. And all of a sudden the memory jumped to more than 100Mb had to killed it. This was being repeated a lot of times in terminal because I was running the exaile from terminal to get the messages.
At Start up:
Exaile 0.2.14
which: no serpentine in (/usr/lib/qt-3.3/bin:/usr/kerberos/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/clifton/bin)
which: no brasero in (/usr/lib/qt-3.3/bin:/usr/kerberos/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/clifton/bin)
which: no k3b in (/usr/lib/qt-3.3/bin:/usr/kerberos/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/clifton/bin)
Created db for thread Thread-1
{'Thread-1': <sqlite3.Connection object at 0x9134bd8>}
A supported CD burning program was not found in $PATH, disabling burning capabilities.
Activated gnome mmkeys for gnome 2.22.x
Using multimedia keys from: gnome
loading tracks...
Closed db for thread Thread-1
done loading tracks...
loading songs
Clearing tracks cache
Importing /home/username/.exaile/saved/playlist0000.m3u
Last playlist loaded
Loading page 0
Traceback (most recent call last):
  File "/usr/lib/exaile/xl/gui/main.py", line 1241, in as_play_track
    int(track.duration), track.track)
  File "/usr/lib/exaile/lib/scrobbler.py", line 149, in now_playing
    raise AuthError("Please 'login()' first. (No session available)")
lib.scrobbler.AuthError: Please 'login()' first. (No session available)
I: caps.c: Limited capabilities successfully to CAP_SYS_NICE.
I: caps.c: Dropping root priviliges.
I: caps.c: Limited capabilities successfully to CAP_SYS_NICE.
ReplayGain support initialized.
Not using Equalizer disabled by the user

When memory was increasing:
<gst.Message GstMessageError, gerror=(GstGError)(NULL), debug=(string)"pulsesink.c\(455\):\ gst_pulsesink_write\ \(\):\ /GstPlayBin:playbin0/GstBin:abin/GstBin:bin0/GstGConfAudioSink:gconfaudiosink0/GstBin:bin9/GstAutoAudioSink:autoaudiosink7/GstPulseSink:autoaudiosink7-actual-sink-pulse"; from autoaudiosink7-actual-sink-pulse at 0xa6e6a60> ['__class__', '__cmp__', '__delattr__', '__dict__', '__doc__', '__getattribute__', '__grefcount__', '__gstminiobject_init__', '__gtype__', '__hash__', '__init__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__str__', 'copy', 'flags', 'parse_async_start', 'parse_buffering', 'parse_clock_lost', 'parse_clock_provide', 'parse_duration', 'parse_error', 'parse_info', 'parse_new_clock', 'parse_segment_done', 'parse_segment_start', 'parse_state_changed', 'parse_tag', 'parse_warning', 'src', 'structure', 'timestamp', 'type']

[impert (cwallace-free)]

I had a similar error message to this with PCLinux which I have just installed. After working well initially, a system update seemed to upset Exaile, and it would not play music from a playlist. Editing /etc/modprobe.d/sound to get PCL to use the correct sound card and rebooting has made it work - for now, at least. Strange that this should give that 'Please login first' error.

[reacocard (reacocard)]

retargeting to 0.3.0.2

[reacocard (reacocard)]

I'm closing this bug since it is very old and these reports are probably many disparate issues. bug #435338 tracks the currently-known issue that causes this, if new leaks are found they should be placed in their own reports.