Crashes and other bad behavior while mediascanner is indexing

[Michi Henning]

The attached archive contains 7000 "songs: of 1-second silence. Untar the archive into the music folder. The mediscanner will take several minutes to index the collection.

Start the music app while the mediascanner is running. I'm seeing different kinds of errors every time I kill the music app and start it again while the mediscanner is indexing.

Sometimes, it's an instant crash.

Sometimes, I get the "No music found" page.

Sometimes, I get the song list, but the song list is empty and will not fill.

Sometimes, I get this in the application log:

Error finalising statement: Could not finalize statement: database is locked
Failed to retrieve rows: database is locked

And this:

Failed to start a new media-hub player session: org.freedesktop.DBus.Error.NoReply: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.
Failed to create a new media player backend. Video playback will not function.

Could not finish contructing new AalMediaPlayerService instance since m_hubPlayerSession is NULL.

Basically, the application is completely unusable and broken while indexing is in progress. Once mediascanner finishes indexing, things behave normally again.

This makes for a seriously bad user experience when someone inserts a flash card with their music collection and, almost certainly, will immediately go to the music app to start playing things.

[Andrew Hayzen (ahayzen)]

Yup, while mediascanner2 is scanning the app is unusable :-/

This bug is probably a duplicate of bug 1529991 (qmlscene crashes if music app is started while mediascanner is extracting) and linked to others about high CPU (bug 1442035, bug 1237065).

It took 1 hour and 40 minutes for mediascanner2 to complete its scan on my laptop [0], rendering the laptop useless due to the disk activity. Upon analysis I could see the sqlite lock file appearing as it adds every individual file to the store, which suggests it might be doing a commit as each track/video/picture is added, if this is the case could it not batch these into groups?

However, while scanning it would be better not to crash/be in a bad state. So could the mediascanner always give a valid model to the music-app, and if it doesn't exist already, is there a property to tell us when mediascanner is scanning? As we could show up a notification while this is occurring, to at least alert the user of what is happening?

This tends not to happen too often on the device, as by the time you have transferred files to it via MTP mediascanner has scanned. But in the case of the desktop where you can already have the media there or when a SD card is plugged into the device, this appears to occur quite often.

For reference Rhythmbox took around 2-3 minutes to scan the same data and didn't cause high disk usage, still allowing me to use the laptop.

0 - I have ~3600 audio files (~60GB), 10,600 photos (~7GB), ~180 videos (~30GB).

[Michi Henning (michihenning)]

Part of the problem might be the use of gstreamer to pull out the ID3 tags. The gstreamer API is brutally slow on the phone. Maybe there is a different ID3 tag library we could use?

[Michi Henning (michihenning)]

I don't think this is a duplicate of bug #1529991 because, in this case, it is the app that crashes, not qmlscene, and there are all sorts of other weird behaviours mentioned here.