Mixxx stops receiving data from MIDI controllers when too many messages arrive at once

[Sean M. Pappalardo]

Testing with a Stanton SCS.1d on Debian 8 Linux where it shows up as a regular but extremely chatty MIDI controller,
if anything happens on my system to cause a slight delay (such as scrolling the library,) and too many MIDI message build up in the receive queue, Mixxx drops the ball and stops responding to the controller at all. It can still send things to the controller and the GUI remains responsive, it just stops getting data from the controller unless I disable and re-enable it.

I'm not sure if this is happening at the PortMIDI or ALSA-MIDI levels instead of Mixxx either. How can I best check?

I just got this, but it was after I stopped execution for a few seconds in gdb then continued:
Expression 'alsa_snd_pcm_prepare( stream->playback.pcm )' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 2920
Expression 'AlsaStart( stream, 0 )' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 3246
Expression 'AlsaRestart( self )' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 3313
Expression 'PaAlsaStream_HandleXrun( self )' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 3924
Expression 'PaAlsaStream_WaitForFrames( stream, &framesAvail, &xrun )' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 4253
[Thread 0x7fff8e24a700 (LWP 21672) exited]

[Sean M. Pappalardo (pegasus-renegadetech)]

Found that MIXXX_PORTMIDI_BUFFER_LEN was set too low at 64. After setting it to 1024 and letting Mixxx run for awhile and doing latency-inducing actions (like switching to a text console,) the highest number of queued events was 135, so I'm going to propose setting it to 256.

[Sean M. Pappalardo (pegasus-renegadetech)]

Actually after some stress-testing, 500 events were queued up at one point, so maybe 1024 is a good number to use.

[Sean M. Pappalardo (pegasus-renegadetech)]

https://github.com/mixxxdj/mixxx/commit/469a37f11239b8c700fac98ad5d78d0299f934c2

[Owen Williams (ywwg)]

For the record the portmidi header does describe how a caller is supposed to recover from overflow. If we aren't detecting overflow that's a bug as well:

/* Pm_Read() retrieves midi data into a buffer, and returns the number
    of events read. Result is a non-negative number unless an error occurs,
    in which case a PmError value will be returned.

    Buffer Overflow

    The problem: if an input overflow occurs, data will be lost, ultimately
    because there is no flow control all the way back to the data source.
    When data is lost, the receiver should be notified and some sort of
    graceful recovery should take place, e.g. you shouldn't resume receiving
    in the middle of a long sysex message.

    With a lock-free fifo, which is pretty much what we're stuck with to
    enable portability to the Mac, it's tricky for the producer and consumer
    to synchronously reset the buffer and resume normal operation.

    Solution: the buffer managed by PortMidi will be flushed when an overflow
    occurs. The consumer (Pm_Read()) gets an error message (pmBufferOverflow)
    and ordinary processing resumes as soon as a new message arrives. The
    remainder of a partial sysex message is not considered to be a "new
    message" and will be flushed as well. */

[Sean M. Pappalardo (pegasus-renegadetech)]

Good point, and that starts to answer my question of if this is the best fix. I suspect we're just bailing out, assuming any error is an unrecoverable one. Should we target this to 2.1 then?

[Owen Williams (ywwg)]

I think upping the buffer size is a good stop-gap fix, but yeah in 2.1 we should try to be able to handle this situation more gracefully.

[Daniel Schürmann (daschuer)]

I can confirm the issue with my RMX2 when I am reduce the buffer to 2

[Daniel Schürmann (daschuer)]

We use a redundant Pm_Poll call that prevents that we can recover from an overflow.
In case of overflow the buffer is flushed so Pm_Poll returns 0, but Pm_Read call does the buffer management.
Both calls poll telegrams from the hardware.

If I remove the Pm_Poll call and use a portmidi build without PM_CHECK_ERRORS all works like a charm and overflows are handled gracefully.

[Daniel Schürmann (daschuer)]

https://github.com/mixxxdj/mixxx/pull/825

[Swiftb0y (swiftb0y)]

Mixxx now uses GitHub for bug tracking. This bug has been migrated to:
https://github.com/mixxxdj/mixxx/issues/8381