Offset in waveforms, beat grid and cues after 2.1 upgrade

[Norman Meier]

Mixxx 2.0.0 on an up-to-date Arch Linux.
lscpu: Intel(R) Core(TM) i5-4210M CPU @ 2.60GHz
alsamixer: chip: Realtek ALC269VB, card: HDA Intel PCH
Grahics card: Intel® HD Graphics 4600
Using jack2 as intermediary between mixxx and the sound card.

Steps to reproduce:
Put a FLAC track on some deck and a MP3 track on another deck.
Both tracks have the same original bpm.
The decks have quantize activated.
Play the tracks.

Problem:
When both are playing, the FLAC track is always slightly ahead of the MP3 track.

Moar info:
I tried starting them manually at the same time with quantize off and the FLAC is also always slightly ahead so it seems it's not even related to quantize but only that FLAC tracks are playing earlier but that could just be
my sloppy hands.

If I quantize them by hand then resync them with the sync button they end up with the same delay.
Tested on a variety of different tracks.
It's the same with multiple tracks, the FLAC tracks will be quantized together but slightly ahead of the MP3 ones that are also quantized together.
The beatgrid and bpm are correctly configured on all tracks tested.
Having sync on or off doesn't matter.

[Uwe Klotz (uklotzde-deactivatedaccount)]

Decoders for various formats have been rewritten or modified substantially for the upcoming version 2.1. If this is actually a bug and it has been caused by decoding issues it might be solved now.

After decoding the signal is processed identically for both MP3 and FLAC.

[Norman Meier (n0iz)]

Just compiled latest version from git.

The problem is still in 2.1 but different, now the FLAC tracks are slightly behind instead of ahead.

I checked multiple times, rolled back to 2.0.0 to see if hadn't hallucinated them being ahead instead of behind but no:
In 2.0.0 FLAC is slightly ahead, in 2.1 it's slightly behind (looks like roughly the same amount of time).
I say slightly but it's clear to the ear, it's at the very least a 20ms delay.

[Uwe Klotz (uklotzde-deactivatedaccount)]

But don't you think that this is an encoder issue or even immanent in the lossy MP3 encoding? I'm pretty sure that we don't miss or skip any samples during decoding. All decoded samples are fed into the Mixxx engine as a raw PCM signal, the file format doesn't matter after decoding.

Did you compare the content of both files in a wave editor side by side?

[Norman Meier (n0iz)]

What's the point of comparing the files if I set the beatgrid afterward.

See attachment, that's how I have to "desync" the tracks from the quantize reference to have them play at the same time.

If I do that with 2 flac or 2 mp3 the white bars are aligned.

[Norman Meier (n0iz)]

Woops

[Norman Meier (n0iz)]

I did the same test setup with a WAV and MP3, there is the same problem.
No with WAV and FLAC though.

[Norman Meier (n0iz)]

Same screenshot for 2.1, the mp3 is now ahead. (of both flac and wav)

This, imho, looks like a bug in the mp3 decoder.
You said it was rewritten, this could cause the fact that the bug changed in 2.1

[Norman Meier (n0iz)]

Woops2

[Daniel Schürmann (daschuer)]

You probably see a leftover from old waveform analysis and beat detection.

Does this issue persist if you hide and the purge the effected tracks form the library and then drop them directly from a file browser to the decks?

[Uwe Klotz (uklotzde-deactivatedaccount)]

How did you obtain the various formats of the same file? We need to compare the original file and the decoded one, i.e. the MP3 file and the corresponding MP3 -> WAV file obtained by decoding with an external program. The results for both the MP3 and the decoded WAV file should be identical if decoding in Mixxx works correctly.

[Uwe Klotz (uklotzde-deactivatedaccount)]

And we need instructions on how to reproduce this for testing purposes.

[Norman Meier (n0iz)]

I produced the loop in my DAW, bounced it to WAV then used the audio-convert script which uses lame.
Then dropped them in mixxx.
I did manually set the beatgrid and bpm.
When I play the two loops with quantize and let the bars aligned, the audio is not aligned.
I'll do the purge thing for 2.1 but for the 2.0 screenshot they were just added.

The instructions are in my first post. I have this problem with any mp3 file when I mix them with any FLAC or WAV file.

So you want me to reconvert the mp3 to a wav with lame, then get a screenshot of them side by side in, say, audacity?

[Norman Meier (n0iz)]

Okay!! The problem is fixed in 2.1 after purging the mp3 files only! :)

Too bad purging removes files from playlists and deletes cue points.. This will be a pain since I have to do it with all my mp3s..
Maybe add a warning or something when you upgrade version that says that waveform analysis and beat detection data for mp3 files might be wrong.

[Uwe Klotz (uklotzde-deactivatedaccount)]

Glad to hear that the new MP3 decoder is in fact working correctly now. Thank you for you investigations!

Maybe Daniel has more ideas about further implications of the fixed decoders and how we could clean up only the dependent artifacts in Mixxx without purging tracks entirely? I'm afraid that all existing cue points and loops that have been set based on audio data decoded before version 2.1 will no longer be valid. Adding more options for selectively dropping those artifacts (not only bpm and beat grid) might be a solution.

[Norman Meier (n0iz)]

This is the first time I use launchpad so maybe this is a stupid question:
Should I rename the bug "mp3 decoder has a delay" and set it to "fix commited"?

[Daniel Schürmann (daschuer)]

Let's adopt the bug title to the remaining "migration" issue.

We have discussed the issue earlier, but as I remember without a conclusion.

The decoding issue of pre-2.1 versions effects from my experience only a minority of my mp3 tracks.
The issue, is that the track shifts under the waveform depending on the order of seeks though the track.

This means that the waveforms, cue points and beat grid, is inaccurate. Unfortunately Mixxx does not know yet which tracks are effected.

We have discussed to invalidate the < 2.1 data, but since most tracks are unaffected this will cause more pain than it helps.

A fancy solution would be to verify the waveforms or beatgrid once a track is loaded to Mixxx 2.1.
If a mismatch is found, try to correct the issue by reanalyzing the waveforms and shift cue points and beat-grid by an offset.

This sound somehow adventurous though.

Implementing:
https://bugs.launchpad.net/mixxx/+bug/1014449
Would help to keep at least the playlists and history valid.

Any ideas?

[Daniel Schürmann (daschuer)]

This is a follow up bug of:
https://bugs.launchpad.net/mixxx/+bug/1407394

[Be (be.ing)]

Any further thoughts on this?

[Uwe Klotz (uklotzde-deactivatedaccount)]

Any kind of semi-automatic detection would involve a lot of additional effort. And if this effort should not be wasted for just a single upgrade, we need to invest more and think about tracing all those dependencies in the database.

No, I don't have a simple solution and vote for a special warning in the release announcement and notes together with recommendations what needs to be done. If we can provide the functionality that allows the user to manually reset the affected analysis results ("Clear BPM and Beatgrid" + "Clear Waveform") for all or at least some of their MP3 files then this is a fair deal.

We are not able to make any guesses about the accuracy of existing cue points, hot cues, and loops at all, which is another argument for not developing a sophisticated detection algorithm.

[Uwe Klotz (uklotzde-deactivatedaccount)]

Changed to "Critical", because this needs to be addressed somehow!

[Uwe Klotz (uklotzde-deactivatedaccount)]

The advise might start like "The good news: We have put a lot of effort into providing sample accurate decoding for all supported audio formats with this release ... Unfortunately their are also some bad news: Analysis results, waveforms, beatgrids, cue points, hot cues, loops from the past have become inaccurate ... Here are our recommendation for updating your precious track meta- and performance data: ..."

Just an idea. Maybe a talented native speaker and writer is able to put this into words that spread the positive message while simultaneously encouraging and enabling users to fix those known issues themselves as far as possible instead of blaming and overwhelming us with bug reports that we need to reject, causing even more frustration.

[Be (be.ing)]

> special warning in the release announcement and notes together with recommendations what needs to be done

I think this is a reasonable way to handle the issue. I'd be happy to write a first draft for the beta release announcement and get feedback before publishing it. To be clear, does this affect all MP3 files or only some? Should we advise users to clear data for all MP3s or only ones that they notice are problematic? How would they identify a problematic file?

[Be (be.ing)]

I think we should reserve the Critical priority for bugs that make Mixxx or a major feature unusable. While this bug is nasty, it can be worked around, so I don't think it needs the Critical priority.

[Daniel Schürmann (daschuer)]

Even a warning message is a kind of fix. Going back to confirmed.

[Be (be.ing)]

Since we have decide to address this by documenting it in the upcoming release announcement, I'm assigning it to myself. However, I need answers to the questions in comment #22 to be able to write the announcement accurately.

[Uwe Klotz (uklotzde-deactivatedaccount)]

All MP3 files are potentially affected, we are not able to exclude any files.

I was expecting that the analysis results should be affected less than cue points and loops, because during the analysis files are read from the beginning 'til the end without jumping around. But we also had reports about mismatching waveforms: https://bugs.launchpad.net/mixxx/+bug/1407394

The only advice we could give is to both "Clear Waveform" and "Clear BPM and Beatgrid" for all files. Primarily all MP3 files, but when in doubt also for any other file formats.

We don't have an options to clear all cue points and loops for files as far as I know. Users should be prepared to adjust these track markers when loading files for the first time in Mixxx 2.1.

[Be (be.ing)]

Should we make an easy way to clear cue points and loops? Might that be useful for other use cases?

[Uwe Klotz (uklotzde-deactivatedaccount)]

It might be helpful to be able to clear or reset various metadata stored by Mixxx for tracks:

Transport markers:
    - Cue point
    - Hot cues
    - Loops
Analysis results:
    - BPM and beatgrid (only if not locked?)
    - Key
    - Waveform

Our 2 "Clear" functions for waveform and bpm/beatgrid are currently scattered in the context menu, one at the top level and one hidden inside "BPM Options". These functions are usually not needed very often, so maybe opening a separate dialog with some checkboxes that can be selected individually would be an option.

Rethinking how to present and implement this functionality sounds more like a post-2.1 task, although we need it right now ;)

[Be (be.ing)]

Redesigning the right click context menu is tracked in Bug #1733199

[Daniel Schürmann (daschuer)]

Here is an other user complaining:
https://bugs.launchpad.net/mixxx/+bug/1544739
It feels bad, that we do not have a solution.
A scan that identifies effected tracks would be nice, but this means to read every single track from the database ... Mmmm

[Be (be.ing)]

In the release announcements we can tell users to enter 'location: mp3' into the search bar, Ctrl + A to select all, then right click and clear the inaccurate info.

[Daniel Schürmann (daschuer)]

That does not sound like a high quality tool.

If we advance the beatgrid version, this will happen automatically, but for all files.

Problematic are all files where the auto detected beatgrid was adjusted ....

We hit really a trap with this bug.

[Be (be.ing)]

> That does not sound like a high quality tool.

No, it doesn't, but it seems we don't have a better option. :/

> If we advance the beatgrid version, this will happen automatically, but for all files.

Files that are not MP3 files shouldn't be affected. Also, I am hesitant to invalidate users' old data without explicit consent.

> Problematic are all files where the auto detected beatgrid was adjusted

Maybe for the future it could be useful to track whether the user has manipulated the beatgrid from what Mixxx analyzed.

[Be (be.ing)]

Beta release announcement: https://github.com/mixxxdj/mixxx/pull/1422
Required track context menu items to clear incorrect info from MP3 files: https://github.com/mixxxdj/mixxx/pull/1427

[Be (be.ing)]

Marking this as Fix Committed since we have done what we can do.

[Swiftb0y (swiftb0y)]

Mixxx now uses GitHub for bug tracking. This bug has been migrated to:
https://github.com/mixxxdj/mixxx/issues/8814