Improvement: Multithreaded BPM analysis
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Mixxx |
Fix Released
|
Wishlist
|
Unassigned |
Bug Description
The current implementation of the BPM and graphic analysis (analysis view) is single threaded (one file at a time).
While it is good that it doesn't hog the CPU for the rest of the application, it's not that good that the CPU usage stays at 20% on a high end machine.
My proposed solution (initial, previous to verify how it all is coded) is like this:
- Get a thread-synchronized cue of tasks to do (the easiest model is simply the list of tracks selected to analyze).
- Calculate number of threads that the processor has.
There are several ways to get this information, depending on the platform, but the standard way is this: http://
Note: As a safety measure, programs that can use multiple threads tend to allow the user to specify it explicitly. It could be added as an option in the BPM preference page of Mixxx settings. 0 = automatic >0 = use that value. There would always be at least one thread.
- Prepare a stack of threads filled with "hardware_
That is, we will have at much Max-1 threads for analysis, so there's always one processor thread free for the rest of the application.
This could be improved as in: If mixxx is not playing, use all available threads. If it is playing, max-1.
- Modify the track analysis done on track load to use this stack of threads. Concretely, ensure that loading tracks use the same thread limit. If all threads are in use, discard one that is doing the backgroud analysis and assign it to the track(s) being loaded.
- When a thread finishes doing its task (i.e. finishes the track analysis), it "pop"s a new tasks from the task list and starts processing it.
If no more tasks are ready, the thread ends.
This would require also a small change in the UI, in how we report the current track being analyzed.
We could also study if it would be good to first do the bpm/grid/replaygain analysis, and when this finishes for all tracks, do the graphic analysis of those same tracks. the UI message could indicate "Part 1/2, Part 2/2)" or any other message to let the user know that this is a two step action.
Changed in mixxx: | |
assignee: | nobody → JosepMa (josepma) |
status: | Confirmed → In Progress |
Changed in mixxx: | |
milestone: | 2.2.0 → 2.3.0 |
assignee: | JosepMa (josepma) → Uwe Klotz (uklotzde) |
Changed in mixxx: | |
status: | In Progress → Fix Committed |
Changed in mixxx: | |
status: | Fix Committed → Fix Released |
I've taken a look at the code.
__This is what we have now:__
- A class AnalyzerQueue that extends from QThread and contains a QQueue of TrackPointer to process, and a QList of Analyzer* to use.
- Two instances of this class (i.e. two threads): One used by the library (playlist, crates and analyzer view, created in AnalyzerFeature) and the other used by the engine (decks, samplers and preview deck, created in PlayerManager). It probably is this way since the option to not do the waveform analysis when doing the batch analysis was added in 2.0 (player always require waveform analysis).
- When the thread runs, it loads one track, prepares all the analyzers and does the analysis in blocks of samples (currently 4096 frames).
It can be stopped in any iteration of this loop.
- There's a priorization option which makes a track to be analyzed immediately, stopping an ongoing track analysis if it needs to. It is currently unused (Because it has two independent threads), but the implementation is cumbersome. Instead of adding the new track to the front of the queue, it checks all the tracks to see if any of them is a track from a deck and use that instead of the one that is next in the queue. (a QQueue is a QList after all).
- While this runs, some signals are emited, so that different screen updates can happen.
- When it finishes, each analyzer saves its data to the track and other signals are emited. One of these signals calls analyzerqueue. stop() and the thread ends.
_
_
__This is how we could change it:__
- New class AnalyzerWorker that will have most of the functionality of the AnalyzerQueue (i.e. it will be the thread and contain a list of analyzer pointers).
- Modify the class AnalyzerQueue so that it becomes the manager of the workers. It contains the instances of the analizers, the queue of tracks to process and the pool of workers. Add the feature to calculate the number of threads of the CPU and/or read the max threads from the configuration.
-AnalyzerQueue becomes a single-instance class and so playermanager and library will use again the same instance.
-AnalyzerQueue contains a method to add to the queue, with a priority boolean parameter. If the amount of workers in the pool is less than the maximum number of threads allowed, add a new one and let it run. (We can assume that if there is any other worker, it is working already).
If the track was marked with priority, it is added in front of the the track queue, selects one worker from the pool and tells it to skip the current work and start with the next in the queue. (if one worker has just been created, do nothing, as it will just get the track once the semaphore lets it run)
- The run and analyze methods will not change much, except to cleanup part of the old priority code.
- Signals and slots need to be adapted (or we need to call analyzerqueue from the workers to emit the existing ones)
- Worker threads will die just like they do now when they have no more work to do.