Segmentation faults when I drag the waveform

[Jess Boerner]

Linux Mint 17, on the latest few builds I am getting random segfaults when I try to drag a waveform. Tends to be Channel1.

Really really need to be stable by next weekend! Haven't had issues with my build from a few weeks ago that is currently installed.

[Owen Williams (ywwg)]

Which version? What video hardware are you on, and with what drivers? And can you help us diagnose the issue by creating a backtrace? http://www.mixxx.org/wiki/doku.php/creating_backtraces

[Jess Boerner (jessboerner)]

I ran a pull today so I want to say most recent, but I tried a pull at the end of last week too that got me the same random segfault.

I'm on a Macbook air 2011, so intel HD3000 and should be the latest drivers for linux although those rarely update... I'll make sure I'm updated.

Not sure how to get a backtrace reliably when it's so random, but I will try tomorrow.

[Jess Boerner (jessboerner)]

GOT A BACKTRACE. WHOOP.

I was going through my normal workflow, dragged the waveform for Channel1, and boom, segfault.

Let me know if I can provide more. Seg Fault happens about 11 lines down and then I got the dump after.

p/04 Big Gigantic - Touch The Sky.mp3)" QSize(300, 300)
Debug [AnalyserQueue 1]: AnalysisDAO fetched 2 analyses, 2617441 bytes for track 3363 in 27 ms
Debug [AnalyserQueue 1]: Reading waveform from byte array: allSignalSize 270974 visualSampleRate 441 audioVisualRatio 100
Debug [AnalyserQueue 1]: Reading waveform from byte array: allSignalSize 3842 visualSampleRate 6.24947 audioVisualRatio 7056.6
Debug [AnalyserQueue 1]: AnalyserWaveform::loadStored - Stored waveform loaded
Debug [AnalyserQueue 1]: Track is BpmLocked: Beat calculation will not start
Debug [AnalyserQueue 1]: Key detection is deactivated
Debug [AnalyserQueue 1]: Skipping track analysis because no analyzer initialized.
[Thread 0x7ffee4cde700 (LWP 3852) exited]
Debug [Main]: Committing transaction on "qt_sql_default_connection" result: true

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7ffeedffe700 (LWP 3848)]
soundtouch::InterpolateCubic::transposeStereo (this=0x1f54eb0,
    pdest=<optimized out>, psrc=0x7ffee100e160, srcSamples=@0x7ffeedffd2ac: 1000)
    at lib/soundtouch-1.8.0/InterpolateCubic.cpp:139
139 pdest[2*i+1] = (SAMPLETYPE)out1;
(gdb) thread apply all bt

Thread 27 (Thread 0x7ffeedffe700 (LWP 3848)):
#0 soundtouch::InterpolateCubic::transposeStereo (this=0x1f54eb0,
    pdest=<optimized out>, psrc=0x7ffee100e160, srcSamples=@0x7ffeedffd2ac: 1000)
    at lib/soundtouch-1.8.0/InterpolateCubic.cpp:139
#1 0x00000000004ae7b6 in soundtouch::TransposerBase::transpose (this=0x1f54eb0,
    dest=..., src=...) at lib/soundtouch-1.8.0/RateTransposer.cpp:240
#2 0x00000000004aedaf in processSamples (nSamples=<optimized out>,
    src=<optimized out>, this=<optimized out>)
    at lib/soundtouch-1.8.0/RateTransposer.cpp:157
#3 soundtouch::RateTransposer::putSamples (this=0x1f51d30,
    samples=0x7ffee1047008, nSamples=3774931296)
    at lib/soundtouch-1.8.0/RateTransposer.cpp:123
#4 0x00000000004af051 in soundtouch::SoundTouch::putSamples (this=0x1f51ce0,
    samples=<optimized out>, nSamples=<optimized out>)
    at lib/soundtouch-1.8.0/SoundTouch.cpp:325
#5 0x000000000081b3f2 in EngineBufferScaleST::getScaled (this=0x1f51bd0,
    buf_size=2048) at src/engine/enginebufferscalest.cpp:152
#6 0x000000000080f582 in readToCrossfadeBuffer (iBufferSize=2048, this=0x1e442b0)
    at src/engine/enginebuffer.cpp:444
#7 EngineBuffer::enableIndependentPitchTempoScaling (this=this@entry=0x1e442b0,
    bEnable=bEnable@entry=false, iBufferSize=iBufferSize@entry=2048)
    at src/engine/enginebuffer.cpp:375
#8 0x0000000000816d3c in EngineBuffer::process (this=0x1e442b0,
    pOutput=0x7fffcd4d3010, iBufferSize=2048) at src/engine/enginebuffer.cpp:839
#9 0x000000000081cc69 in EngineDeck::process (this=0x1e20d30,
    pOut=0x7fffcd4d3010, iBufferSize=2048) at src/engine/enginedeck.cpp:90
#10 0x0000000000832fa5 in EngineMaster::processChannels (this=this@entry=
    0x13efdb0, iBufferSize=iBufferSize@entry=2048)
    at s...

[Jess Boerner (jessboerner)]

One more that I got to go on the second deck... appears to be an issue with soundtouch.

Debug [AnalyserQueue 1]: Analyzing "Rage The Night Away" "/home/jess/Music/DJ/Mixing/Trap/Soundcloud Trap/Aoki - Rage The Night Away.128.mp3"
Debug [Main]: WCoverArt::slotCoverFound WCoverArt(0xee4d5b0, name = "DeckCoverArt") "CoverInfo(NONE,GUESSED,,0,/home/jess/Music/DJ/Mixing/Trap/Soundcloud Trap/Aoki - Rage The Night Away.128.mp3)" QSize(0, 0)
Debug [Main]: Committing transaction on "qt_sql_default_connection" result: true
Debug [Main]: BaseTrackCache(0x2ee5ac0) updateIndexWithQuery took 1 ms
Debug [AnalyserQueue 1]: AnalysisDAO fetched 2 analyses, 1758414 bytes for track 3086 in 43 ms
Debug [AnalyserQueue 1]: Reading waveform from byte array: allSignalSize 190266 visualSampleRate 441 audioVisualRatio 100
Debug [AnalyserQueue 1]: Reading waveform from byte array: allSignalSize 3842 visualSampleRate 8.90046 audioVisualRatio 4954.8
Debug [AnalyserQueue 1]: AnalyserWaveform::loadStored - Stored waveform loaded
Debug [AnalyserQueue 1]: Track is BpmLocked: Beat calculation will not start
Debug [AnalyserQueue 1]: Key detection is deactivated
Debug [AnalyserQueue 1]: Skipping track analysis because no analyzer initialized.

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7ffeedffe700 (LWP 4374)]
soundtouch::InterpolateCubic::transposeStereo (this=0x1aa82f0,
    pdest=<optimized out>, psrc=0x7ffee01f2cd0, srcSamples=@0x7ffeedffd2ac: 1000)
    at lib/soundtouch-1.8.0/InterpolateCubic.cpp:139
139 pdest[2*i+1] = (SAMPLETYPE)out1;
(gdb) thread apply all bt

Thread 29 (Thread 0x7ffed9254700 (LWP 4563)):
#0 0x00007ffff1a6c12d in poll () at ../sysdeps/unix/syscall-template.S:81
#1 0x00007ffff0256fe4 in ?? () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
#2 0x00007ffff02570ec in g_main_context_iteration ()
   from /lib/x86_64-linux-gnu/libglib-2.0.so.0
#3 0x00007ffff0257129 in ?? () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
#4 0x00007ffff027bf05 in ?? () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
#5 0x00007ffff31bc182 in start_thread (arg=0x7ffed9254700)
    at pthread_create.c:312
#6 0x00007ffff1a7947d in clone ()
    at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111

Thread 28 (Thread 0x7ffee5ffe700 (LWP 4562)):
#0 0x00007ffff1a6c12d in poll () at ../sysdeps/unix/syscall-template.S:81
#1 0x00007ffff0256fe4 in ?? () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
#2 0x00007ffff025730a in g_main_loop_run ()
   from /lib/x86_64-linux-gnu/libglib-2.0.so.0
#3 0x00007fffe5520336 in ?? () from /usr/lib/x86_64-linux-gnu/libgio-2.0.so.0
#4 0x00007ffff027bf05 in ?? () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
#5 0x00007ffff31bc182 in start_thread (arg=0x7ffee5ffe700)
    at pthread_create.c:312
#6 0x00007ffff1a7947d in clone ()
    at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111

Thread 24 (Thread 0x7ffeecbec700 (LWP 4376)):
#0 0x00007ffff1a6c12d in poll () at ../sysdeps/unix/syscall-template.S:81
#1 0x00007ffff0256fe4 in ?? () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
#2 0x00007ffff02570ec in g_main_context_iteration ()
   from /lib/x86_64-linux-gnu/libglib-2.0.so.0
#3 0x00007ffff552f7be in QEventDispatche...

[Daniel Schürmann (daschuer)]

Thank you very much for your back traces.
It looks like there is an issue with readToCrossfadeBuffer, because EngineBufferScaleST::getScaled is regaular called before when you do not touch the waveform.
Once you touch the waveform we switch from SoundTouch to Mixxx's linear scaler. It crashes during the last call to EngineBufferScaleST::getScaled which is uses to collect the cross fading samples to switch to linear without a click.

What are your rate / pitch / keylock settings when the segfault happens?

Does it crash using Rubberband (in Hardware preferences)

[Jess Boerner (jessboerner)]

I got it to crash using only soundtouch. If I run into issues with rubberband in the meantime, I will update accordingly.

I have keylock enabled, I'm fairly certain that only one of the pitch/rate faders may have been at zero while the other was not at zero since I was actively using sync between two tracks.

And I have used soundtouch on my build from a few weeks ago which does not crash, so a recent change has most likely caused this to my knowledge... feel free to correct my young debugging mind if it deceives me.

[Jess Boerner (jessboerner)]

And that build that I am referencing was built March 6 at 2:38 PM MST from whatever was the latest revision at that time.

[Daniel Schürmann (daschuer)]

I cannot reproduce on Ubuntu Trusty :-/

[Jess Boerner (jessboerner)]

It took a good hour of use to get my first backtrace. I probably mixed 20-30 songs and loaded 60 between the decks in playing around with beatgrids and such... The second backtrace popped up almost instantly in comparison... It was after loading the first two tracks that it crashed.

Do we have any idea if something has changed in the function call that crashes since March 6?

[Daniel Schürmann (daschuer)]

Your good commit is probably 5a5eb68d0f7ada3c022e2a20d75dac718373e483

You can
git diff 5a5eb68d0f7ada3c022e2a20d75dac718373e483
to look at the changes.

Normally you can use
git bisect
to identify the faulty commit. But in you case it is hard to proof that a single revision is not effected, because of the random nature of the fault.

From the back trace it looks like "pdest" is pointing to an invalid location causing the segfault.
The question is why.

#3 soundtouch::RateTransposer::putSamples (this=0x1aa71b0,
    samples=0x7ffee1884008, nSamples=3760139472)

and

#3 soundtouch::RateTransposer::putSamples (this=0x1f51d30,
    samples=0x7ffee1047008, nSamples=3774931296)

is looking suspicious

[Daniel Schürmann (daschuer)]

Still not able to reproduce.
Are you using a build from our http://builds.mixxx.org/ Server?
Which one is exactly effected?
My is "git master r5324"
The latest ubuntu build on th server is r5298

You find the version inside Mixxx: Help->About

Are you able to build from source?
I may prepare a version that checks nSamples for reasonable values.
I wonder if it is really 3774931296 or just debugging noise.
When I edit it to this value, it crashes as well, but at a different address.

[Owen Williams (ywwg)]

Yeah I can't reproduce either with soundtouch and keylock on. We did make changes to that code recently, so like Daniel says please make sure you're using the latest latest version.

If you are, it might help to post a video recording of your desktop while you do what you did to reproduce the crash. It doesn't matter if it doesn't crash, but just seeing what's going on (what have you adjusted, are there any loops, etc etc) has been helpful in the past.

[Owen Williams (ywwg)]

If you're dragging the waveform around, the crossfade buffer is getting filled when the engine switches from Soundtouch to linear as your speed goes out of soundtouch's range. Again, we just did some cleanup in that code so I'd prefer that you test a newer version of trunk.

[Owen Williams (ywwg)]

reproduced!

[Owen Williams (ywwg)]

When the musical key has been modified and the user presses pause, the pitchratio is getting set to zero. When Soundtouch goes to interpolate it gets caught in an infinite while() loop because the increment is 0 so it never goes anywhere. So it happily mutates random memory until it crashes.

The particular call is enginebufferscalest.cpp:94, where we call setPitch with a value of 0.

[Owen Williams (ywwg)]

Debug [Engine]: speed 0
new rate 0mixxx: lib/soundtouch-1.8.0/RateTransposer.cpp:278: virtual void soundtouch::TransposerBase::setRate(float): Assertion `rate != 0' failed.

Program received signal SIGABRT, Aborted.
[Switching to Thread 0x7ffee8d27700 (LWP 5836)]
0x00007ffff175fe37 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
56 ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#0 0x00007ffff175fe37 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
#1 0x00007ffff1761528 in __GI_abort () at abort.c:89
#2 0x00007ffff1758ce6 in __assert_fail_base (fmt=0x7ffff18a8c08 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=assertion@entry=0xc575c8 "rate != 0",
    file=file@entry=0xc575a0 "lib/soundtouch-1.8.0/RateTransposer.cpp", line=line@entry=278,
    function=function@entry=0xc57660 <soundtouch::TransposerBase::setRate(float)::__PRETTY_FUNCTION__> "virtual void soundtouch::TransposerBase::setRate(float)")
    at assert.c:92
#3 0x00007ffff1758d92 in __GI___assert_fail (assertion=0xc575c8 "rate != 0", file=0xc575a0 "lib/soundtouch-1.8.0/RateTransposer.cpp", line=278,
    function=0xc57660 <soundtouch::TransposerBase::setRate(float)::__PRETTY_FUNCTION__> "virtual void soundtouch::TransposerBase::setRate(float)") at assert.c:101
#4 0x0000000000445338 in soundtouch::TransposerBase::setRate (newRate=0, this=<optimized out>) at lib/soundtouch-1.8.0/RateTransposer.cpp:278
#5 0x00000000004b2188 in setRate (newRate=0, this=<optimized out>) at lib/soundtouch-1.8.0/RateTransposer.cpp:105
#6 soundtouch::RateTransposer::setRate (this=0x2045720, newRate=0) at lib/soundtouch-1.8.0/RateTransposer.cpp:105
#7 0x00000000004b34ae in soundtouch::SoundTouch::calcEffectiveRateAndTempo (this=0x20456d0) at lib/soundtouch-1.8.0/SoundTouch.cpp:243
#8 0x00000000004b39ca in soundtouch::SoundTouch::setPitch (this=<optimized out>, newPitch=<optimized out>) at lib/soundtouch-1.8.0/SoundTouch.cpp:203
#9 0x00000000008742df in EngineBufferScaleST::setScaleParameters (this=0x20455c0, base_rate=<optimized out>, pTempoRatio=<optimized out>, pPitchRatio=0x7ffee8d263d0)
    at src/engine/enginebufferscalest.cpp:94
#10 0x000000000086efd8 in EngineBuffer::process (this=0x1f34070, pOutput=0x7fffb8f41010, iBufferSize=2048) at src/engine/enginebuffer.cpp:929
#11 0x000000000087665c in EngineDeck::process (this=0x1f10a60, pOut=0x7fffb8f41010, iBufferSize=2048) at src/engine/enginedeck.cpp:90
#12 0x00000000008851e6 in EngineMaster::processChannels (this=this@entry=0x14b6640, iBufferSize=iBufferSize@entry=2048) at src/engine/enginemaster.cpp:332
#13 0x0000000000889b1d in EngineMaster::process (this=0x14b6640, iBufferSize=iBufferSize@entry=2048) at src/engine/enginemaster.cpp:364
#14 0x0000000000b1097b in SoundManager::onDeviceOutputCallback (this=<optimized out>, iFramesPerBuffer=iFramesPerBuffer@entry=1024) at src/soundmanager.cpp:483
#15 0x0000000000b0a5bf in SoundDevicePortAudio::callbackProcessClkRef (this=0x1df05b0, framesPerBuffer=1024, out=0x364ce2d0, in=<optimized out>, timeInfo=<optimized out>,
    statusFlags=0) at src/sounddeviceportaudio.cpp:834
#16 0x00007ffff7bb3e63 in ?? () fr...

[Owen Williams (ywwg)]

We should probably file a bug with soundtouch that they should protect against this case

[Jess Boerner (jessboerner)]

Is there anything we can do to protect against it in the meantime? Rubberband doesn't like my setup even at 23.2 ms of latency :/

[Owen Williams (ywwg)]

I've pushed a oneliner fix that should make the problem go away.

[Owen Williams (ywwg)]

(But I'm not sure if it's a complete, or totally-correct fix, Daniel will have to take a look and see if it makes sense)

[Jess Boerner (jessboerner)]

So is that fix currently on master? If not where may I clone it from to test?

Appreciate your help. Don't know why I'm still getting xruns on a macbook air with 23.2 ms of latency with the lowlatency kernel though... is rubberband just THAT intensive?

Any of you familiar with linking with asmlib? (one of the options in scons) I can't figure out if that will help me at all.

[Owen Williams (ywwg)]

rubberband is really, really CPU-sucking. Yes, the fix is in master. Give it a whirl!

Btw the easiest way to reproduce was:

* adjust musical key
* adjust speed
* press play
* press pause
* drag waveform.

[Sean M. Pappalardo (pegasus-renegadetech)]

I'm familiar with using ASMLIB. (I added the support for it.) I never did any benchmarking to see if and/or how much it helps, so you may not see any improvement. But if you want to try it:
1) Download the latest package from http://www.agner.org/optimize/asmlib.zip
2) Unpack it to a directory called 'asmlib' one level above the Mixxx source directory
3) Run scons as usual but specifying asmlib=1

[Daniel Schürmann (daschuer)]

Some more fixes are here: https://github.com/mixxxdj/mixxx/pull/548
But this bug was already soled by Owen. Thank you for that and for finding the way to reproduce it.
So this bug goes back to Owen.

[Swiftb0y (swiftb0y)]

Mixxx now uses GitHub for bug tracking. This bug has been migrated to:
https://github.com/mixxxdj/mixxx/issues/7946