Banshee very slow when interacting with track lists

[Jim Kirkpatrick]

Binary package hint: banshee

Banshee's great, but since I've added a few more songs it's become unbearably slow.

Scrolling up and down the library is ok, provided it's settled for a while, However, whenever a song finishes playing or I rate a track or skip a track there's a delay whist 2 of my 8 CPU cores are maxed out. The UI takes an age to redraw or respond to clicks. The delay can be just half a second, or sometimes around 5. This usually coincides with some network activity.

It makes interacting with the application nearly unbearable. Don't want to go back to Rhythmbox, but might have to :(

I have:
* Banshee 1.6 Beta 2 (1.5.1)
* about 5,300 songs in the library
* Ubuntu 9.10 Karmic 64bit
* 6Gb of memory and an i7 920 processor

Banshee extensions running:
* Audio CD Support
* Bookmarks
* Cover Art Fetching
* DAAP
* Digital Media Player Support
* IPod Support
* Karma Support
* Mass Storage Media Player Support
* MTP Media Player Support
* File System Queue
* Internet Radio
* Last.fm Radio & Scrobbling
* Multimedia Keys
* Play Queue
* Importers for Amarok & Rhythmbox
* Podcasts

[Jim Kirkpatrick (jim-kirkpatrick)]

I can also confirm this behaviour with ALL extensions disabled.

[Chow Loong Jin (hyperair)]

There may be something wrong with your music library. Could you try moving it aside (the file is ~/.config/banshee-1/banshee.db), re-importing your music, and seeing if that helps? If it does, please upload your ~/.config/banshee-1/banshee.db file for inspection. It may yield some useful insight to what's going wrong.

[Jim Kirkpatrick (jim-kirkpatrick)]

Hi Chow,

I moved the banshee.db and re-imported the library... The delay is still there, sorry... 2 cores maxed out, very slow GUI redraw, delays in responding to skip track, rating or re-sorting the display.

FYI I tend to have large playists (e.g. Unrated/Unplayed) as I'm slowly rating all my music on a new machine. Hence I'll restore my old database to keep the ~1000 or so ratings as that test made no difference.

Is there any further information I can provide?

Jim

[Chow Loong Jin (hyperair)]

On Monday 16,November,2009 10:45 PM, Jim Kirkpatrick wrote:
> Hi Chow,
>
> I moved the banshee.db and re-imported the library... The delay is still
> there, sorry... 2 cores maxed out, very slow GUI redraw, delays in
> responding to skip track, rating or re-sorting the display.
>
> FYI I tend to have large playists (e.g. Unrated/Unplayed) as I'm slowly
> rating all my music on a new machine. Hence I'll restore my old database
> to keep the ~1000 or so ratings as that test made no difference.
>
> Is there any further information I can provide?
>
> Jim
>
Hmm that's very weird for such a high-end machine. I've got only two cores to my
CPU and have no such issue. Also, my library is at ~6500 tracks. Perhaps
`gconftool-2 --recursive-unset /apps/banshee-1`?

--
Kind regards,
Chow Loong Jin (GPG: 0x8F02A411)
Ubuntu Contributing Developer

[Jim Kirkpatrick (jim-kirkpatrick)]

OK, tried "gconftool-2 --recursive-unset /apps/banshee-1" which clearly wiped my settings... No change I'm afraid, still slow as already outlined.

Is there no debugging information I can provide?

It seems to happen when the database is updated (last played/rated). It seems selects from the database into the track list on the screen aren't super quick, but they're acceptable. Insert/Updates seem to freeze the whole thing. Shouldn't Banshee's UI at least be on a separate thread to maintain snappy interactivity? If it is perhaps there's some weird deadlock happening? Just ideas...

[Chow Loong Jin (hyperair)]

On Tuesday 17,November,2009 02:26 AM, Jim Kirkpatrick wrote:
> OK, tried "gconftool-2 --recursive-unset /apps/banshee-1" which clearly
> wiped my settings... No change I'm afraid, still slow as already
> outlined.
>
> Is there no debugging information I can provide?
>
> It seems to happen when the database is updated (last played/rated). It
> seems selects from the database into the track list on the screen aren't
> super quick, but they're acceptable. Insert/Updates seem to freeze the
> whole thing. Shouldn't Banshee's UI at least be on a separate thread to
> maintain snappy interactivity? If it is perhaps there's some weird
> deadlock happening? Just ideas...
>
Try `banshee --debug --debug-sql` in a terminal and attaching the output
(Alternatively, `banshee --debug --debug-sql --redirect-log` and attach
~/.config/banshee-1/log).

--
Kind regards,
Chow Loong Jin (GPG: 0x8F02A411)
Ubuntu Contributing Developer

[Jim Kirkpatrick (jim-kirkpatrick)]

The debug log is attached... Started Banshee, played a track, skipped, rated a track and then did a few more skips.

There's some nasty inserts info the CoreCache table that take ~1 second and a few hefty selects too.

[Chow Loong Jin (hyperair)]

On Thursday 19,November,2009 08:01 AM, Jim Kirkpatrick wrote:
> The debug log is attached... Started Banshee, played a track, skipped,
> rated a track and then did a few more skips.
>
> There's some nasty inserts info the CoreCache table that take ~1 second
> and a few hefty selects too.
>
> ** Attachment added: "Banshee debug log file"
> http://launchpadlibrarian.net/35838499/log
>
Out of curiosity, what filesystem are you using on your /home? Could you try
mounting with the option "nobarrier" or "barrier=0" and seeing if that helps?
There were some issues somewhere around where sqlite was kicking up a fuss on
filesystems with barriers enabled, ext4 being one of them.

--
Kind regards,
Chow Loong Jin (GPG: 0x8F02A411)
Ubuntu Contributing Developer

[ajonat (ajonat)]

He said he was using ext4 on irc.
I also think this is related to sqlite (like this liferea bug: https://bugs.launchpad.net/ubuntu/+source/liferea/+bug/290666)

Jim, could you do something like in that bug report:
# strace -etrace=fsync -o banshee_session.strace banshee 2>&1

When banshee is running try repeating what you usually do that makes banshee slow.
I have a feeling that banshee is writing to banshee.db when you change songs (to update how many times it was played) or when you rate a song, and this slows down your system because of excessive calls to fsync which on ext4 hurts overall performance.

[Gabriel Burt (gabaug)]

Banshee sets sqlite's PRAGMA synchronous = OFF, so I'm not sure it's anything to do with the filesystem.

This query (among others) is way too long; it makes me think your indexes are either broken/missing or not being used correctly.

Executed in 979ms SELECT COUNT(*), SUM(CoreTracks.FileSize) FROM CoreTracks,CoreArtists,CoreAlbums, CoreSmartPlaylistEntries WHERE CoreArtists.ArtistID = CoreTracks.ArtistID AND CoreAlbums.AlbumID = CoreTracks.AlbumID AND CoreSmartPlaylistEntries.TrackID = CoreTracks.TrackID AND CoreSmartPlaylistEntries.SmartPlaylistID = 2

Can you paste here the output of running this:

sqlite3 ~/.config/banshee-1/banshee.db ".schema CoreSmartPlaylistEntries"

After doing that, backup your database again (if needed) and try running

sqlite3 ~/.config/banshee-1/banshee.db "analyze"

and restart Banshee, see if things are better.

[Jim Kirkpatrick (jim-kirkpatrick)]

Right, well lots to reply to...

Yes, I'm using EXT4 - fresh Karmic install. sqlite + ext4 problems could also explain Liferea's slowness when updating feeds, and Firefox's occasional oddities.

Chow:
I'd prefer not to disable file system integrity features, this is my work machine too. I see others (http://bbs.archlinux.org/viewtopic.php?id=65341) are discussing its relative merits... For now though, altering my file system to fix an application is not an acceptable option, sorry.

ajonat:
I'm not on IRC! But yes, I have EXT4... ran your strace and the file is attached as requested.

Gabriel:
Yes, my thoughts exactly: ~1 sec for inserts or even selects(!) implies table scans or duff indexes or just bloated tables - an i7 machine should be able easily handle Banshee's database needs.

# sqlite3 ~/.config/banshee-1/banshee.db ".schema CoreSmartPlaylistEntries" returns:
CREATE TABLE CoreSmartPlaylistEntries (
                    EntryID INTEGER PRIMARY KEY,
                    SmartPlaylistID INTEGER NOT NULL,
                    TrackID INTEGER NOT NULL
                );
CREATE INDEX CoreSmartPlaylistEntriesIndex ON CoreSmartPlaylistEntries(SmartPlaylistID, TrackID);

# sqlite3 ~/.config/banshee-1/banshee.db "analyze" returned almost immediately, but unfortunately when I ran Banshee the slowness was still there.

Jim

[Maxime (maxime-maury)]

Hi!

I moved my banshee.db from an EXT4 partition to an EXT3 (with a soft link) and have the same problem.

It takes a lot of time to access the database, see the attached log "Executed in 18649ms "
banshee-1 --debug-sql --debug

I did not have this problem on Fedora 11. It came up in Fedora 12. And I have not changed the filesystem.

[Jens (mail-jensschanz)]

Same problem here.
After importing the media library it's ok. But after the first restart it's very, very slow.
I've identified two queries against the database which took a long time:

sqlite> SELECT COUNT(*), SUM(CoreTracks.FileSize) FROM CoreTracks,CoreArtists,CoreAlbums, CoreSmartPlaylistEntries WHERE CoreArtists.ArtistID = CoreTracks.ArtistID AND CoreAlbums.AlbumID = CoreTracks.AlbumID AND CoreSmartPlaylistEntries.TrackID = CoreTracks.TrackID AND CoreSmartPlaylistEntries.SmartPlaylistID = 7 ;
14508|86984790003
CPU Time: user 28.910000 sys 8.170000

sqlite> SELECT COUNT(*), SUM(CoreTracks.FileSize) FROM CoreTracks,CoreArtists,CoreAlbums, CoreSmartPlaylistEntries WHERE CoreArtists.ArtistID = CoreTracks.ArtistID AND CoreAlbums.AlbumID = CoreTracks.AlbumID AND CoreSmartPlaylistEntries.TrackID = CoreTracks.TrackID AND CoreSmartPlaylistEntries.SmartPlaylistID = 8 AND MetadataHash NOT IN (SELECT MetadataHash FROM CoreTracks WHERE PrimarySourceId = 9);
14437|86520727025
CPU Time: user 30.410000 sys 8.380000

[Launchpad Janitor (janitor)]

This bug was fixed in the package banshee - 1.5.5-1

---------------
banshee (1.5.5-1) unstable; urgency=low

  * New upstream release
    + New Features:
      - Gapless Playback / Playbin2 (LP: #45518, #377354)
      - Grid View
      - YouTube Extension
    + Enhancements:
      - Improve search responsiveness on large libraries (LP: #474514)
      - Add icons for Nexus One and Audiobooks (LP: #532182)
      - Play Queue item count, size, duration now ignores old tracks
      - Muinshee fixes: disable Auto-DJ, allow reordering, hide previous song
    + Notable bug fixes:
      - Fix a very common, SQL-related crash in 1.5.4 (Closes: #572136)
      - Fix saving equalizer settings in culture-invariant way
      - Jumping to a source's prefs via its context menu works again
      - Usage data not submitted more than every 48 hours
      - Fix repeatedly resyncing some files to a device b/c transcoded
      - Clear the redo stack on shuffle mode change
      - Accept feeds with empty title (LP: #468323)
      - Uri encode file location queries, making them work properly
  * debian/control:
    + Bump gst-plugins-base-dev version for gapless playback
    + Add libgdata-cil-dev for youtube extenesion
  * debian/rules:
    + Add --enable-gapless-playback and --enable-gdata

banshee (1.5.4-1) unstable; urgency=low

  * New upstream release (Closes: #571734)
    + New features:
      - Opt-In Usage Data Collection
      - Default Equalizer Presets
      - Extensible Shuffle Modes
    + Enhancements:
      - Wikipedia context pane extension enabled by default
      - Add support for Nokia N900 phones
      - Add tooltip to playback error column
      - On close Internet Archive item, return to Search
      - Notify user if trying to sync missing file to DAP
      - Popup explanation of manual playlist ordering when appropriate
      - Simplify the default set of columns in Podcasts
      - Enable 'Delete From Drive' action in File System Queue
      - Coverart for unicode artist/albums now supported
      - Dropped glade-sharp dep; GNOME 3.0 ready
      - Add columns showing track sample rate and bits per sample
      - Option to sort an artist's albums by year, not title
      - If starting Banshee hidden (--hide), up to half a second of
        startup time is saved
    + Notable bug fixes:
      - Enable LibraryWatcher only for Music and Video libraries
      - Do better job preserving IsCompilation metadata
      - Store some PlayQueue settings in the db (not GConf)
      - Update to Last.fm's API change for scrobbling/recs
      - Bring back static FileNamePattern API used by some scripts
      - Fix several memory leaks
    + Other bugs fixed:
      - Fix segfault with Cairo when playing a song without album art (LP: #523913)
      - Follow symlinks when scanning the library (LP: #406667)
      - Fix issue importing mp3s with non-ASCII ID3 tags (LP: #364562)
  * debian/control:
    + Bump libtaglib-cil-dev version requirement to 2.0.3.5
    + Drop libglade2.0-cil-dev and libgnome2.0-cil-dev build-dep
    + Add libwebkit-cil-dev to build-deps
  * debian/rules:
    + Reorder configure flags to match upstream's ./configure --help listing
    + Ad...

[mark (mark-carpaij)]

I can confirm that new release of Banshee is much faster, and handles large libraries smoothly.

Thank you!

[Toby Smithe (tsmithe)]

This seems to be present as a regression in maverick with banshee 1.7.3-2ubuntu1.

[Chow Loong Jin (hyperair)]

On Saturday 31,July,2010 04:48 AM, Toby Smithe wrote:
> This seems to be present as a regression in maverick with banshee
> 1.7.3-2ubuntu1.
>
> ** Changed in: banshee (Ubuntu)
> Status: Fix Released => Incomplete
>

It is probably related to something in sqlite. Please post the output of
"banshee --debug --debug-sql" while reproducing this issue.

--
Kind regards,
Chow Loong Jin

[Toby Smithe (tsmithe)]

As you can see, more complex queries are taking thousands or tens of thousands of ms. This makes the UI pretty much unusable. To create this log, I started banshee, pressed play (shuffle mode, by track, is selected), pressed next twice, scrolled the artists list, and quit through the Media menu. My library is around 5543 tracks - I would give the precise number, but banshee is taking too long to reload! - and ironically, banshee receives this error when posting usage data, meaning that fewer people see how slow it is being for me:

"Error posting metrics - System.Net.WebException: The remote server returned an error: (503) Service Not Available. (in `System')"

[Chow Loong Jin (hyperair)]

On Saturday 31,July,2010 07:36 PM, Toby Smithe wrote:
> As you can see, more complex queries are taking thousands or tens of
> thousands of ms. This makes the UI pretty much unusable. To create this
> log, I started banshee, pressed play (shuffle mode, by track, is
> selected), pressed next twice, scrolled the artists list, and quit
> through the Media menu. My library is around 5543 tracks - I would give
> the precise number, but banshee is taking too long to reload! - and
> ironically, banshee receives this error when posting usage data, meaning
> that fewer people see how slow it is being for me:
>
> "Error posting metrics - System.Net.WebException: The remote server
> returned an error: (503) Service Not Available. (in `System')"
>
> ** Attachment added: "log"
> http://launchpadlibrarian.net/52777120/log

What do you mean by "too long to reload"? I would think that you would have to
wait for Banshee to fully load before you can do any of the actions you
mentioned, and once Banshee has fully loaded, the track count is listed both on
the bar at the bottom of the window, as well as the sidebar on the left.

Also, out of curiosity, what are the specs of the machine? Could you also
describe where, in the sequence of the aforementioned actions you performed, did
Banshee slow down, and how slow it was?

For the record, I have not noticed any slowness while interacting with Banshee,
and my library has 7085 songs.

--
Kind regards,
Chow Loong Jin

[Toby Smithe (tsmithe)]

I mean simply that I believed that the count was 5543, from memory, but I had shut banshee after performing the steps to generate the log without taking account of the actual number. When I restarted banshee, during writing the bug comment, the time the application took to reload was not worth the wait; indeed, my comment was simply a statement of my impatience (but this bug report is all about impatience!).

The machine is a 2.00GHz Core 2 Duo with 4GB of RAM and the same of swap. I am running 64-bit Ubuntu.

[Chow Loong Jin (hyperair)]

On Monday 02,August,2010 02:53 AM, Toby Smithe wrote:
> I mean simply that I believed that the count was 5543, from memory, but
> I had shut banshee after performing the steps to generate the log
> without taking account of the actual number. When I restarted banshee,
> during writing the bug comment, the time the application took to reload
> was not worth the wait; indeed, my comment was simply a statement of my
> impatience (but this bug report is all about impatience!).
>
> The machine is a 2.00GHz Core 2 Duo with 4GB of RAM and the same of
> swap. I am running 64-bit Ubuntu.
>

Are you running maverick, by any chance? From what I hear, there have been some
regressions in performance related to the new version of sqlite available there
(3.7.0). Could you perhaps try downgrading libsqlite3-0 to Lucid's version
(3.6.22-1) and seeing if the issue persists?

--
Kind regards,
Chow Loong Jin

[Toby Smithe (tsmithe)]

Ah, yes, of course - I suspected it might be something actually to do with sqlite. I'll try that - thanks!

[Toby Smithe (tsmithe)]

And, you were absolutely right. Does that mean this bug, therefore, is not the best place to report/track the issue?

[Chow Loong Jin (hyperair)]

On Monday 02,August,2010 05:22 AM, Toby Smithe wrote:
> And, you were absolutely right. Does that mean this bug, therefore, is
> not the best place to report/track the issue?

It will be, pending investigation.

--
Kind regards,
Chow Loong Jin

[Chow Loong Jin (hyperair)]

Actually on second thoughts, it's probably a different bug. I'll close this bug report and create a new one.

[Chow Loong Jin (hyperair)]

The sqlite3 regression bug has been moved to https://bugs.launchpad.net/bugs/612370. Toby, I have subscribed you to that new bug, so I think you should have received an e-mail about it.

[Toby Smithe (tsmithe)]

Thanks, Chow :)

[Myk Dowling (politas)]

I'm getting this problem in Natty; is it meant to have been fixed or not? My music library is over 19000 tracks. Rhythmbox seems to handle it OK.

[Chow Loong Jin (hyperair)]

Hi Myk,

I believe you're experiencing a different bug. Could you run "banshee --debug --debug-sql" and post the terminal output in a new bug report, please?

[Si Dedman (si-dedman)]

Hi Chow, all,
I'm getting what I think is a similar thing: 100% use of one CPU core after saving any tags edits (2.4ghz dual core E6600); generally sluggish performance. I've turned BPM detection off & podcast support off. I just ran the debug command as per your advice to Myk, made one tag edit, saved, waited til the CPU dropped back to idle (~15 seconds) then closed & pasted the (visible) terminal to a file. The file makes no mention of the file I tagged though, which seems odd?

Banshee v2.62 on xubuntu 14.10 (was the same on 14.04).

p.s. is there a daily PPA?

[Chow Loong Jin (hyperair)]

On Sun, Oct 26, 2014 at 10:04:12PM -0000, Si Dedman wrote:
> Hi Chow, all,
> I'm getting what I think is a similar thing: 100% use of one CPU core after
> saving any tags edits (2.4ghz dual core E6600); generally sluggish
> performance. I've turned BPM detection off & podcast support off. I just ran
> the debug command as per your advice to Myk, made one tag edit, saved, waited
> til the CPU dropped back to idle (~15 seconds) then closed & pasted the
> (visible) terminal to a file. The file makes no mention of the file I tagged
> though, which seems odd?

Why don't you attach the output and let me see?

> Banshee v2.62 on xubuntu 14.10 (was the same on 14.04).
>
> p.s. is there a daily PPA?

Yes there is, but it's disabled for now -- there's been a migration from Gtk2 to
Gtk3, and that's not been done for some of the intermediate libraries in the
dependency chain (libgpod).

--
Kind regards,
Loong Jin

[Si Dedman (si-dedman)]

Eesh, sorry Chow, I wasn't subscribed to notifications for this! Done now - run debug script, found a file with an errant tag, edited tag, saved, waited for CPU to drop, quit banshee, pasted all terminal to text file, attached.