some FLAC files are not loaded/parsed

[Karel Vavra]

OS: Kubuntu 18.04
Kernel: 4.15.0-52-generic #56-Ubuntu SMP Tue Jun 4 22:49:08 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Architecture: x86_64
CPU: Intel(R) Core(TM) i5-4440 CPU @ 3.10GHz
Mixxx version: mixxx-2.3.0-alpha-pre-master-git6862-release-bionic-amd64 (nightly build)

Issue description:

Some FLAC files are not loaded/ analyzed by the software. Example FLAC file here: https://uloz.to/!dGxBLrXYOHS6/test-flac-zip

This file is loaded and parsed normally by Audacity and Virtual DJ.

From the application log:

Debug [Main]: BeatFactory::loadBeatsFromByteArray could not parse serialized beats.
Debug [Main]: SoundSourceProxy - SoundSourceProvider "Xiph.org libFLAC" created a SoundSource for file "file:///media/carloss/Data7/Mp3/= Incoming =/2018/salsa/Joe Cotto Orquesta - Descarga %234 Descarga .flac" of type "flac"
Debug [Main]: SoundSourceProxy - Skip importing of track metadata and embedded cover art from file "file:///media/carloss/Data7/Mp3/= Incoming =/2018/salsa/Joe Cotto Orquesta - Descarga %234 Descarga .flac"
Debug [Main]: BaseTrackCache(0x555fb6e94b50) updateIndexWithQuery took 0 ms
Debug [Main]: WCoverArt::slotCoverFound WCoverArt(0x555fbe84ed10) "CoverInfoRelative(NONE,GUESSED,,0x0)" QSize(0, 0)
Debug [Main]: dropEventFiles() true (QUrl("file:///media/carloss/Data7/Mp3/= Incoming =/2018/salsa/Joe Cotto Orquesta - Descarga %234 Descarga .flac"))
Debug [Main]: mimeData.hasText() true "library"
Debug [Main]: dropEventFiles() true (QUrl("file:///media/carloss/Data7/Mp3/= Incoming =/2018/salsa/Joe Cotto Orquesta - Descarga %234 Descarga .flac"))
Debug [Main]: mimeData.hasText() true "library"
Debug [Main]: BaseTrackPlayerImpl::slotLoadTrack "[Channel1]"
Debug [CachingReaderWorker 1]: SoundSourceProxy - SoundSourceProvider "Xiph.org libFLAC" created a SoundSource for file "file:///media/carloss/Data7/Mp3/= Incoming =/2018/salsa/Joe Cotto Orquesta - Descarga %234 Descarga .flac" of type "flac"
Debug [CachingReaderWorker 1]: SoundSourceProxy - Opening file "file:///media/carloss/Data7/Mp3/= Incoming =/2018/salsa/Joe Cotto Orquesta - Descarga %234 Descarga .flac" with provider "Xiph.org libFLAC" using mode Strict
Warning [CachingReaderWorker 1]: AudioSource - No audio data available
Warning [CachingReaderWorker 1]: SoundSourceProxy - File is empty "file:///media/carloss/Data7/Mp3/= Incoming =/2018/salsa/Joe Cotto Orquesta - Descarga %234 Descarga .flac"
Debug [Main]: BaseTrackPlayerImpl::slotTrackLoaded
Debug [Main]: TrackAnalysisScheduler - Resuming

[Uwe Klotz (uklotzde-deactivatedaccount)]

The FLAC stream metadata doesn't contain the total length (in sample frames) that Mixxx expects:

Format : FLAC
Format/Info : Free Lossless Audio Codec
File size : 15.5 MiB
Overall bit rate mode : Variable

Audio
Format : FLAC
Format/Info : Free Lossless Audio Codec
Bit rate mode : Variable
Channel(s) : 2 channels
Channel layout : L R
Sampling rate : 44.1 kHz
Bit depth : 16 bits
Compression mode : Lossless
Stream size : 15.5 MiB (100%)
Writing library : libFLAC 1.3.2 (UTC 2017-01-01)

I've never seen any file without length metadata before. We need to find out how to handle this without decoding the whole file upfront until EOF.

[Daniel Schürmann (daschuer)]

We can fix it during the analyzer and may estimate the length from the file size in the meantime

This bug can also be used as a prove of concept for possible steaming support.

A stream is a file without a fixed length.

How can we analyze that? A kind of on the fly analysis would be nice.
I think as a DJ a stream delay can be accepted if we can see what will come looking at the waveforms and beats in turn.

[Uwe Klotz (uklotzde-deactivatedaccount)]

The total length of the stream is used everywhere in Mixxx. This is the reason why files with undefined length are rejected. Supporting sources with undefined length is a major task!

[Daniel Schürmann (daschuer)]

Unfortunately yes ...

[Karel Vavra (sals3r0)]

In Virtual DJ the exact stream length is figured up by the analyzer (it has to decode the file anyway). Maybe it could work in a similar way in Mixxx?

This is not an infinite stream, it is a file with finite duration, but not known from metadata...handling this case should be easier than supporting streams with potentially infinite length (would be very nice though :)...

[Uwe Klotz (uklotzde-deactivatedaccount)]

We are not able to provide a solution other than improving the logging. The current architecture is not ready for streams with unknown or even unlimited length.

As a workaround I suggest re-encoding the FLAC files to generate the complete stream info metadata block for those files. Please check why your encoder didn't calculate and write the data and how to include it in the future.

This is the proposed solution even if Mixxx would support such files. There is no reason for excluding essential metadata from files that is known in advance.

[Daniel Schürmann (daschuer)]

Mixxx can already handle inaccurate length info in the file metadata.
Mayb I have missed something, but lenght = 0 is only a special case. So I think the fix is only a matter of making some lenght checks non fatal.

[Karel Vavra (sals3r0)]

Yeah I think for the time being I will just re-encode my FLAC collection with the official FLAC encoder. Thanks.

[Karel Vavra (sals3r0)]

FYI Someone had the same problem and posted a simple script to re-encode his entire FLAC collection:

https://forums.gentoo.org/viewtopic-t-831333-start-0.html

I am running a simplified version of this script now (skipping the meta-tags logic as I don't have any tags I need to preserve) and it works.

[Swiftb0y (swiftb0y)]

Mixxx now uses GitHub for bug tracking. This bug has been migrated to:
https://github.com/mixxxdj/mixxx/issues/9685