Impossible to continue aborted scan

[Max Linke]

When i abort a library scan it is impossible to continue at a later point. The scanner just thinks that I have scanned everything

Debug [LibraryScanner 1]: LibraryScanner::slotStartScan
Debug [LibraryScanner 1]: upgrade filename is "/home/kain88/.mixxx/DBUPGRADED"
Debug [LibraryScanner 1]: Committing transaction on "LIBRARY_SCANNER" result: true
Debug [LibraryScanner 1]: Legacy importer took 0 ms
Debug [LibraryScanner 1]: Recursively scanning library.
Debug [LibraryScanner 1]: LibraryScanner::slotFinishScan
Debug [LibraryScanner 1]: Recursive scanning finished cleanly.
Debug [LibraryScanner 1]: Committing transaction on "LIBRARY_SCANNER" result: true
Debug [LibraryScanner 1]: Marking tracks in changed directories as verified
Debug [LibraryScanner 1]: Marking unchanged directories and tracks as verified
Debug [LibraryScanner 1]: Checking remaining unverified tracks.
Debug [LibraryScanner 1]: Marking unverified tracks as deleted.
Debug [LibraryScanner 1]: Marking unverified directories as deleted.
Debug [LibraryScanner 1]: Detecting moved files.
Debug [LibraryScanner 1]: Committing transaction on "LIBRARY_SCANNER" result: true
Debug [LibraryScanner 1]: Detecting cover art for unscanned files.
Debug [LibraryScanner 1]: Scan finished cleanly
Debug [LibraryScanner 1]: Scan took: 150014269 ns. 617 unchanged directories. 0 changed/added directories. 0 tracks verified from changed/added directories. 0 new tracks.

[Max Linke (max-linke)]

I've started to look into this more. This seems to be a problem of the new threaded approach to scanning the library.

I guess the problem lies in recursiveScanDirectoryTask

recursiveScanDirectoryTask.cpp : 82

    if (prevHash != newHash) {
        // Rescan that mofo! If importing fails then the scan was cancelled so
        // we return immediately.
        if (!filesToImport.isEmpty()) {
            m_pScanner->queueTask(new ImportFilesTask(m_pScanner, m_scannerGlobal,
                                                      filesToImport, possibleCovers,
                                                      m_pToken));
        }

        // Insert or update the hash in the database.
        emit(directoryHashed(dirPath, !prevHashExists, newHash));
    } else {
        emit(directoryUnchanged(dirPath));
    }

To me this looks like the Code assumes that once I queue an ImportFilesTask it will run for no matter what happens later. But this assumption is wrong when we allow to cancel the scan.

This causes us to save directories as scanned and imported in the database while they are not in truth.

I haven't done any close checking yet. But what I assume happens is that the complete recursiveScan is finished a long time before the importFileTasks are finished.

Suprisingly I see this in the most severe form because I have a SSD. With an HDD there will be only a some tracks left unscanned.

I'm not sure what would be a good easy fix for this. It seems to me a like a design flaw and I don't have a lot of experience with debugging task queues. My current idea would be to keep the recursiveScanTask alive until the importFilesTask it has started is finished and notified of that, only then is the recursiveScanTask allowed to emit the directoryHashed signal. But that might complicate things way to much, it sounds like there should be an easier solution.

@rryan any comments?

[Max Linke (max-linke)]

Or maybe we just don't use the queue/task to importFiles but call al plain old function.

[Owen Williams (ywwg)]

When you say "inconsistent state" (in your email), do you mean the database is hosed, or just that the user needs to exit mixxx and restart it to fix the issue? I'm not sure what the consequence is.

[RJ Skerry-Ryan (rryan)]

Inconsistent here means:
1) Directory hash marked as up-to-date
2) Some files that contributed to that directory hash have not been imported (since the import task was canceled half way through).

Database isn't hosed but the files in the dir won't ever be scanned until the directory changes (so its hash changes). Reboot won't fix it.

[Swiftb0y (swiftb0y)]

Mixxx now uses GitHub for bug tracking. This bug has been migrated to:
https://github.com/mixxxdj/mixxx/issues/7875