searching for tracks freezes player

[Florian Demmer]

using autoqueue 0.2beta2 with quotlibet 2.1, ~40k tracks collection

usually tracks for similar artists are found quickly and the player gui remains responsive. there are cases however, when i do not have a lot of music that is similar to the one autoqueue tries to find matches for. then the search takes quite a while. during that time the player gui freezes (greyed out). in the console i can see searches for various artists going on... when the searching stops the player works again.

i am not sure if this should be handled in quodlibet or autoqueue. i guess the autoqueue plugin reacts to a certain event from quodlibet to start a search. is this search then threaded or does quod libet wait for the results to queue?
if there is no such blocking call, is it a thread? could it be a loop that is starving the gui because of the GIL? (in that case sleep(0) could help)

[Peter Schwede (peter-schwede)]

I have that freezing, too. Threading would be neat, I guess.

[Eric Casteleijn (thisfred)]

Hi, sorry I missed this bug so far: obscured by too much work related launchpad mail ;). This has been one of the things that have proven hard to solve, because autoqueue potentially has a lot of work to do, especially when analyzing new tracks with mirage, or when looking up connections on last.fm: it has to write data to sqlite, and wait for that to complete to be able to query.

I've made a few changes in trunk that seem very promising however: I've moved all database writes and analysis to a separate process that's started by autoqueue when firing up quodlibet. This means all of the heavy lifting happens outside of the plugin, and it's had a huge effect on the responsiveness in my case (~170K tracks).

I'll try and release this when it stabilizes, but I've been using it for a month or two now, and it seems to have no discernable downsides yet.

Let me know if this solves you guys' problems!

[Florian Demmer (fdemmer)]

still a little freezing, especially when starting with an empty queue and just one track. lot of work then ;)

but it definitely helped splitting off the similarity-daemon. i can see both the quodlibet and the daemon process go to 100% after track change/enqueue new tracks.

[Eric Casteleijn (thisfred)]

Yeah, depending on what you set the desired queue length to, it will have to do some work. The good news is that it's not waiting on the similarity daemon anymore. The "bad" news is that the similarity scores from the daemon will only be taken into account the next time you play the track. Any unresponsiveness in the UI now, is either QL itself, or the time it takes for the plugin to look up similar tracks on last.fm, if you have that option enabled.

[Eric Casteleijn (thisfred)]

I have made some more changes in trunk, which should help a lot with the freezing. Basically I yield control back to GTK in a lot more places. Let me know if this helps.

[Florian Demmer (fdemmer)]

thanks for your continued work on this and autoqueue in general. it really finds the right kind of music almost always!

however... still not perfect :(

consider this log output, starting with directly after track change:

libmirageaudio: decoding /.../02 - Meredith Brooks - Bitch.mp3
libmirageaudio: EOS Message received
libmirageaudio: EOS Message sent
libmirageaudio: time = 980.000000
libmirageaudio: frames=2583 (maxhops=2583), size=513
Mirage: decoded frames=c_int(2583),size=c_int(513)
Mirage: mfcc Execution Time: 0:00:00.031188
Mirage: scms created in: 0:00:01.531043
[autoqueue] Getting similar tracks from last.fm for: gladys knight & the pips - i've got to use my imagination <<--- stuck here for 3 sec
[autoqueue] looking for: 000100 the supremes - stoned love
[autoqueue] looking for: 000061 wilson pickett - land of 1000 dances
[autoqueue] looking for: 000053 the four tops - i can't help myself (sugar pie honey bunch)
[autoqueue] rating: 0.5 last played 11 days ago, suggested play: after 90.0 days
[autoqueue] looking for: 000036 aretha franklin - respect
[autoqueue] looking for: 000034 gladys knight & the pips - best thing that ever happened to me
[autoqueue] looking for: 000032 gladys knight & the pips - storms of troubled times
[autoqueue] looking for: 000027 the four tops - i can't help myself (sugar pie, honey bunch)
[autoqueue] looking for: 000026 the 5th dimension - aquarius/let the sunshine in
[autoqueue] looking for: 000016 aretha franklin - chain of fools
[autoqueue] looking for: 000016 al green - let's stay together
[autoqueue] rating: 0.5 last played 14895 days ago, suggested play: after 90.0 days <<--- stuck here for 5 sec

with "<<-- stuck here" i mean this line gets printed, then stuck, before the next line appears.

what is strange about the second one is, that the line gets logged, then stuck and nothing is logged when it becomes responsive again. no idea what is happening there. i have disabled "keep db clean".

something different:
is it on purpose, that the last track in the queue (in my example "gladys knight") is used to search for more and not the currently playing song ("meredith" in the example)? and why?

[Eric Casteleijn (thisfred)]

On 10/13/2010 04:59 PM, Florian Demmer wrote:
> thanks for your continued work on this and autoqueue in general. it
> really finds the right kind of music almost always!
>
> however... still not perfect :(
>
> consider this log output, starting with directly after track change:
>
> libmirageaudio: decoding /.../02 - Meredith Brooks - Bitch.mp3
> libmirageaudio: EOS Message received
> libmirageaudio: EOS Message sent
> libmirageaudio: time = 980.000000
> libmirageaudio: frames=2583 (maxhops=2583), size=513
> Mirage: decoded frames=c_int(2583),size=c_int(513)
> Mirage: mfcc Execution Time: 0:00:00.031188
> Mirage: scms created in: 0:00:01.531043
> [autoqueue] Getting similar tracks from last.fm for: gladys knight& the pips - i've got to use my imagination<<--- stuck here for 3 sec
> [autoqueue] looking for: 000100 the supremes - stoned love
> [autoqueue] looking for: 000061 wilson pickett - land of 1000 dances
> [autoqueue] looking for: 000053 the four tops - i can't help myself (sugar pie honey bunch)
> [autoqueue] rating: 0.5 last played 11 days ago, suggested play: after 90.0 days
> [autoqueue] looking for: 000036 aretha franklin - respect
> [autoqueue] looking for: 000034 gladys knight& the pips - best thing that ever happened to me
> [autoqueue] looking for: 000032 gladys knight& the pips - storms of troubled times
> [autoqueue] looking for: 000027 the four tops - i can't help myself (sugar pie, honey bunch)
> [autoqueue] looking for: 000026 the 5th dimension - aquarius/let the sunshine in
> [autoqueue] looking for: 000016 aretha franklin - chain of fools
> [autoqueue] looking for: 000016 al green - let's stay together
> [autoqueue] rating: 0.5 last played 14895 days ago, suggested play: after 90.0 days<<--- stuck here for 5 sec
>
> with "<<-- stuck here" i mean this line gets printed, then stuck, before
> the next line appears.

This is where autoqueue talks to last.fm, and the request + response
usually takes a little while to process. I'll see if it involves loops
that I can break out of more often. The good news is that it stores the
results of the request in the database, so the next time this song gets
played, the request to last.fm won't have to be made, and things will be
much faster.

> what is strange about the second one is, that the line gets logged, then
> stuck and nothing is logged when it becomes responsive again. no idea
> what is happening there. i have disabled "keep db clean".

This is where I store all the found relations into the database. I
postpone this until after queueing the newly found song. I'll
investigate if I can break up that work into smaller chunks too, so as
to minimize UI freezes.

--
eric casteleijn
https://code.launchpad.net/~thisfred
Canonical Ltd.

[Eric Casteleijn (thisfred)]

I have refactored autoqueue into a separate dbus service which helps a lot with UI freezes. The searching for tracks still happens within the player of necessity, but it's relatively minor, and proportional to the number of songs in a library, so likely much less of a problem for most people.