When importing 7 digital albums music-app sometimes displays "Imported files not found" when all files have been imported

[Michael Sheldon]

Occasionally music-app displays a message stating "Imported file not found" after importing albums from 7 digital (currently in testing in silo 8, hopefully landing soon). However all the files are present in the ~/Music/Imported directory and are shown in the music app after dismissing the error.

The log reports:

qml: File(s) were not found ["/home/phablet/Music/Imported/2014/10/10/135211-Anais Mitchell & Rachel Ries - Come September.mp3","/home/phablet/Music/Im
ported/2014/10/10/135211-Anais Mitchell & Rachel Ries - Grace the Day.mp3","/home/phablet/Music/Imported/2014/10/10/135211-Anais Mitchell & Rachel Ries
 - MGD.mp3","/home/phablet/Music/Imported/2014/10/10/135211-Anais Mitchell & Rachel Ries - O My Star.mp3","/home/phablet/Music/Imported/2014/10/10/1352
11-Anais Mitchell & Rachel Ries - When you fall.mp3"]

However all the files have log entries prior to that along the lines of:

qml: MusicStore model from lookup {"objectName":"","filename":"/home/phablet/Music/Imported/2014/10/10/135211-Anais Mitchell & Rachel Ries - O My Star.
mp3","uri":"file:///home/phablet/Music/Imported/2014/10/10/135211-Anais%20Mitchell%20&%20Rachel%20Ries%20-%20O%20My%20Star.mp3","contentType":"audio/mp
eg","eTag":"1412952556:0","title":"O My Star","author":"Anais Mitchell & Rachel Ries","album":"Country EP","albumArtist":"Anais Mitchell & Rachel Ries"
,"date":"2008","genre":"Folk/Bluegrass","discNumber":1,"trackNumber":1,"duration":182,"width":0,"height":0,"latitude":0,"longitude":0,"hasThumbnail":tr
ue,"art":"image://thumbnailer/file:///home/phablet/Music/Imported/2014/10/10/135211-Anais%20Mitchell%20&%20Rachel%20Ries%20-%20O%20My%20Star.mp3"}

So appear to have been processed successfully.

[Andrew Hayzen (ahayzen)]

Thank you for taking the time to report this bug and helping to make Ubuntu better.

The error suggests that 1 or all of the files were not found. You have provided some of the QML log showing the values from the lookup but that is for only 1 of the 5 files. Are you able to supply the full log as this may help debugging?

Also I wonder if the moving is taking too long, did you try selecting "wait" to see if it succeeded the second time?

[Victor Thompson (vthompson)]

I'm able to replicate this using Jamendo pretty easily. It's almost certainly a string encoding issue as songs with an ampersand will not be found.

[Victor Thompson (vthompson)]

Actually, I ran a second test file from Jamendo with an "&" and it worked. So disregard the last comment, just a coincidence that it also failed.

[Michael Sheldon (michael-sheldon)]

Attached is a full log file, it seem that while it reports all 5 files as not being found it might only be having trouble with one. One of the files doesn't appears in the MusicStore model lookups ("Come September"), however the file is copied correctly in to ~/Music/Imported/. I tried pressing "Wait" 3 times, but it didn't make a difference.

If you contact me (Elleo) on IRC I can send you that specific file to see if there's anything specific to that file which might be causing issues, however it is important to note that it doesn't happen every time, sometimes the import completes without any issues.

[Michael Sheldon (michael-sheldon)]

Managed to figure out how to reproduce this, it seems it happens to the first item that gets imported after a clean install. So if you delete ~/Music/Imported and ~/.local/share/com.ubuntu.music/ it'll fail on the first file you attempt to import (whether it's a single file on its own or the first file in an album).

[Victor Thompson (vthompson)]

@michael-sheldon, that would explain why my first test case in my suspicion failed and proceeded to work the second time.

[Andrew Hayzen (ahayzen)]

That is interesting, so it could potentially be an issue with mediascanner2. Maybe we should look at the mediascanner2 logs around the time of import to see if it can see the file.

[Michael Sheldon (michael-sheldon)]

Attached my mediascanner log which includes a couple of times I deleted everything and reproduced the problem, there aren't any errors logged but mediascanner is failing to pick up on one file each time after starting fresh (e.g. The album Xao has 15 tracks but only 14 are detected following a clearing of the import directory and the music app directory).

[Andrew Hayzen (ahayzen)]

So I removed ~/Music/Imported and ~/.local/share/com.ubuntu.music/ restarted the device and then imported 1 single file from the browser. You can see in the log [0] that mediascanner finds the new folder but not the new file that can be seen in the music-app log.

Therefore I am adding mediascanner2 as also affects, as the first file in the new folder is not being indexed.

0 - http://pastebin.ubuntu.com/8576514/

[Jussi Pakkanen (jpakkane)]

This seems weird. Would it be possible for you to do a debugging test where you do the following:

- create the "Imported" directory
- wait a second or two
- then write the files

This will tell us whether the problem is in inotify timings or somewhere else.

Thanks.

[James Henstridge (jamesh)]

We discovered a bug in the handling of files created between the processing of the inotify event for creation of the new directory and when we create an inotify watch for that directory.

We were checking for subdirectories, but not files. Since the bug is reproduceable, we should be able to tell if this fixes the problem.

[James Henstridge (jamesh)]

I've marked the music-app bug task as invalid: we were able to reproduce the problem and verify that it no longer occurred with the attached mediascanner2 branch applied, so music-app is blameless.

[Launchpad Janitor (janitor)]

This bug was fixed in the package mediascanner2 - 0.105+15.04.20141030.1-0ubuntu1

---------------
mediascanner2 (0.105+15.04.20141030.1-0ubuntu1) vivid; urgency=low

  [ James Henstridge ]
  * When a new directory is added and a new inotify watch is set up,
    call fileAdded() on any regular files already in the directory so
    they are not missed. (LP: #1379817)
 -- Ubuntu daily release <email address hidden> Thu, 30 Oct 2014 11:23:12 +0000

[Pat McGowan (pat-mcgowan)]

good use case

[Launchpad Janitor (janitor)]

This bug was fixed in the package mediascanner2 - 0.105+15.04.20141030.1~rtm-0ubuntu1

---------------
mediascanner2 (0.105+15.04.20141030.1~rtm-0ubuntu1) 14.09; urgency=low

  [ James Henstridge ]
  * When a new directory is added and a new inotify watch is set up,
    call fileAdded() on any regular files already in the directory so
    they are not missed. (LP: #1379817)
 -- Ubuntu daily release <email address hidden> Thu, 30 Oct 2014 11:23:12 +0000