Miscompilation with gcc 4.8 (segfault in x264_cqm_init)

[Alex]

SRU Justification:

[Impact]

 * A segfault in libx264-123 has caused major regressions in vlc, kazam, avidemux, and avconv among others. Nearly any program that encodes video in H264 is probably affected.

[Test Case]

 * As this impacts many packages, there are a number of possible test cases. A simple one is to use kazam, the screen recording app:

 1) Install the kazam package.
 2) Run kazam
 3) Make sure "file > preferences > screencast > record with" is set to H264/MP4
 4) Attempt to record a screencast.

 With 2:0.123.2189+git35cf912-1ubuntu1 from saucy install, you will experience a segfault. With the version in saucy-proposed, a screencast will be correctly recorded and encoded.

[Regression Potential]

 * There is little chance of regression. In order to make the most minimal change possible, the package is just simply built with -fno-aggressive-loop-optimizations rather than attempting to backport any code changes from upstream. The most likely regression would be a typo causing a FTBFS (though I have built and tested the package on saucy).

[Other Info]

This had not been rebuilt with gcc 4.8 until one day before release when a rebuilt was triggered to fix the arm64 build.

http://gcc.gnu.org/gcc-4.8/changes.html

"GCC now uses a more aggressive analysis to derive an upper bound for the number of iterations of loops using constraints imposed by language standards. This may cause non-conforming programs to no longer work as expected, such as SPEC CPU 2006 464.h264ref and 416.gamess. A new option, -fno-aggressive-loop-optimizations, was added to disable this aggressive analysis. In some loops that have known constant number of iterations, but undefined behavior is known to occur in the loop before reaching or during the last iteration, GCC will warn about the undefined behavior in the loop instead of deriving lower upper bound of the number of iterations for the loop. The warning can be disabled with -Wno-aggressive-loop-optimizations."

The fix has already been uploaded to trusty.

-----------------------
Original bug:

Vlc crashes when encoding h264:

Thread 9 (Thread 0x7fffd9712700 (LWP 3034)):
#0 __memcmp_sse2 () at ../sysdeps/x86_64/multiarch/../memcmp.S:74
#1 0x00007fffc9ec9d9e in x264_cqm_init () from /usr/lib/x86_64-linux-gnu/libx264.so.123
#2 0x00007fffc9f34374 in x264_encoder_open_123 () from /usr/lib/x86_64-linux-gnu/libx264.so.123
#3 0x00007fffca1fef34 in Open (p_this=0x7fffd0000e08) at x264.c:1254
#4 0x00007ffff795ed00 in vlc_module_load (p_this=p_this@entry=0x7fffd0000e08,
    psz_capability=psz_capability@entry=0x7fffe8ceaa31 "encoder", psz_name=<optimized out>, b_strict=b_strict@entry=true,
    probe=probe@entry=0x7ffff795e5d0 <generic_start>) at modules/modules.c:347
#5 0x00007ffff795f1a4 in module_need (obj=obj@entry=0x7fffd0000e08, cap=cap@entry=0x7fffe8ceaa31 "encoder", name=<optimized out>,
    strict=strict@entry=true) at modules/modules.c:437
#6 0x00007fffe8ce8a34 in transcode_video_new (p_stream=p_stream@entry=0x7fffe00059d8, id=id@entry=0x7fffd00008e0) at video.c:241
#7 0x00007fffe8ce9c28 in transcode_video_add (p_stream=p_stream@entry=0x7fffe00059d8, p_fmt=p_fmt@entry=0x7fffe0526990,
    id=id@entry=0x7fffd00008e0) at video.c:832
#8 0x00007fffe8ce54a8 in Add (p_stream=0x7fffe00059d8, p_fmt=0x7fffe0526990) at transcode.c:553
#9 0x00007ffff797f64f in sout_InputNew (p_sout=0x7fffe000a6d8, p_fmt=p_fmt@entry=0x7fffe0526990)
    at stream_output/stream_output.c:184
#10 0x00007ffff791b6e6 in DecoderProcessSout (p_block=0x0, p_dec=0x7fffe0523ea8) at input/decoder.c:1812
#11 DecoderProcess (p_dec=p_dec@entry=0x7fffe0523ea8, p_block=p_block@entry=0x7fffe052e9d0) at input/decoder.c:2040
#12 0x00007ffff791bde4 in DecoderThread (p_data=0x7fffe0523ea8) at input/decoder.c:938
#13 0x00007ffff76c8f6e in start_thread (arg=0x7fffd9712700) at pthread_create.c:311
#14 0x00007ffff71ef9cd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:113

http://forum.doom9.org/showthread.php?p=1623567:
Configure x264 with "--extra-cflags=-fno-aggressive-loop-optimizations" solves this problem.
Also seems that problem solved in trunk - http://git.videolan.org/?p=x264.git;a=commit;h=89aecb440e2939be7fb72d8362eb12504711b94f

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Maarten Baert (maarten-baert)]

I can confirm this. It affects anything that uses libx264, not just VLC but also avconv (the command line tool) and any program that encodes video with libavcodec (i.e. most video editors and screencasting software).

The backtrace in avconv is mostly the same:
#0 __memcmp_ssse3 () at ../sysdeps/x86_64/multiarch/memcmp-ssse3.S:1684
#1 0x00007ffff3012d9e in x264_cqm_init () from /usr/lib/x86_64-linux-gnu/libx264.so.123
#2 0x00007ffff307d374 in x264_encoder_open_123 () from /usr/lib/x86_64-linux-gnu/libx264.so.123
#3 0x00007ffff693736c in ?? () from /usr/lib/x86_64-linux-gnu/libavcodec.so.53
#4 0x00007ffff6cc25f5 in avcodec_open2 () from /usr/lib/x86_64-linux-gnu/libavcodec.so.53
#5 0x00000000004051aa in ?? ()
#6 0x00007ffff59aede5 in __libc_start_main (main=0x404c10, argc=12, ubp_av=0x7fffffffe008, init=<optimized out>,
    fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffffffdff8) at libc-start.c:260
#7 0x0000000000407983 in ?? ()

[3vi1 (launchpad-net-eternaldusk)]

I can confirm both the problem and the workaround (--extra-cflags=-fno-aggressive-loop-optimizations) as well, after testing on 13.10 (64-bit)

[Doug McMahon (mc3man)]

On 10/19/2013 10:13 PM, 3vi1 wrote:
> I can confirm both the problem and the workaround (--extra-cflags=-fno-
> aggressive-loop-optimizations) as well, after testing on 13.10 (64-bit)
>
You all could also just use a newer version of x264, the current is a
bit stale, from late april 2013
(same goes for libav which is even older

[Maarten Baert (maarten-baert)]

@Doug: x264 can't be updated without recompiling or even updating libav as well, AFAIK.

I've added a patched version of x264 to my PPA:
https://launchpad.net/~maarten-baert/+archive/simplescreenrecorder
Normally I wouldn't mess with official packages, but since the official one is completely broken, I can't possibly make it worse, right? :) It's the original package from Ubuntu with the upstream patch applied to it.

For those who want to try it (at your own risk):

    sudo add-apt-repository ppa:maarten-baert/simplescreenrecorder
    sudo apt-get update
    sudo apt-get upgrade

[Dave Gilbert (ubuntu-treblig)]

Maarten: Do you have a pointer to some discussion about that patch? Is this really a gcc optimisation bug or is vlc doing something that's not legal/safe but didn't happen to trigger on older gcc?

[David Newman (drdrnewman)]

This affects vlc, avidemux, and command line use of avconv. All produce segfaults on my 64-bit Kubuntu 13.10 upgraded computer. It affects all ways in which avconv uses libx264.

[Maarten Baert (maarten-baert)]

I couldn't find any info either. As far as I understand, x264 was doing something that wasn't standard-compliant (array overreads), and the GCC optimization aggressive-loop-optimizations relies on the assumption that all code is standard-compliant. So this is not a GCC bug.

It may be possible to verify this with Valgrind, but I think the version of Valgrind in Ubuntu doesn't support all the instructions that x264 uses yet, so this is requires recompilation without assembly optimizations.

[Dave Gilbert (ubuntu-treblig)]

Triaged: Upstream fix found
High: Affecting multiple common packages making them unusable.

[Ubuntu Foundations Team Bug Bot (crichton)]

The attachment "Upstream patch (backported)" seems to be a patch. If it isn't, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are a member of the ~ubuntu-reviewers, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issues please contact him.]

[Torsten Römer (dode)]

Rebuilding the libx264-123_0.123.2189+git35cf912-1ubuntu2 packages with -fno-aggressive-loop-optimizations indeed solves the problem on 13.10, there is no more segfault: https://www.dropbox.com/sh/0o1bi2vzmp5p11o/vBzFd8nCgx

[Daniel T Chen (crimsun)]

In light of this being an SRU candidate for 13.10, I recommend we simply compile using -fno-aggressive-loop-optimizations.

[Lonnie Lee Best (launchpad-startport)]

Andrew Starr-Bochicchio has pointed out that his bug also affects kazam:
https://bugs.launchpad.net/ubuntu/+source/kazam/+bug/1241924

[Adam Pryor (adam-pryor-1992)]

I can confirm that Maarten Baert's ppa has resolved the issue I was having with x264 on 13.10. Kazam works fine now and mp4 playback is also working. I have had no negative side-effects so far, but I'll be sure to report back if I find any problems.

[Klaus Kettner (klaus-kettner)]

I can confirm that Maarten Baert's ppa has resolved my problems with avconv on the command-line. Tnx dude!

[Adam Conrad (adconrad)]

Hello Alex, or anyone else affected,

Accepted x264 into saucy-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/x264/2:0.123.2189+git35cf912-1ubuntu1.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Launchpad Janitor (janitor)]

This bug was fixed in the package x264 - 2:0.123.2189+git35cf912-1ubuntu2

---------------
x264 (2:0.123.2189+git35cf912-1ubuntu2) trusty; urgency=low

  * Compile using -fno-aggressive-loop-optimizations to work around
    segfault caused by changes in gcc 4.8 (LP: #1241772).
 -- Andrew Starr-Bochicchio <email address hidden> Tue, 22 Oct 2013 12:22:07 -0400

[John D (johndcarmack)]

I tried following the directions in the EnableProposed link, but the aptitude program complains there are no packages for saucy-proposed. Using apt-get specifically worked, though? At any rate, I finally got it installed and no more segfault.

[Miklos Juhasz (mjuhasz)]

The proposed package resolves this bug for me as well.

[enrico (eliboni)]

fixed for me as well.

[asgard2 (kamp000x)]

proposed sources with apt-get are also working for me

[Yuv (yuv)]

libx264-123 from saucy-proposed fixed the bug for me as well. It is not yet in the main saucy repository where the segfault still happens. What does it take to move the package over from proposed?

[Scott Kitterman (kitterman)]

It needs a week in -proposed for regression testing.

[Adam Pryor (adam-pryor-1992)]

Just activated the -proposed updates and installed them. Everything works fine, as it did with Maarten Baert's ppa, and I'm seeing no regressions at all so far.

[Pieter Hintjens (ph-imatix)]

Fixed for me too... Maarten, thank you.

[korgman (korgie)]

I have installed x86 binary: http://launchpadlibrarian.net/154931008/libx264-123_0.123.2189%2Bgit35cf912-1ubuntu1.1_i386.deb

and I am fine.

[Launchpad Janitor (janitor)]

This bug was fixed in the package x264 - 2:0.123.2189+git35cf912-1ubuntu1.1

---------------
x264 (2:0.123.2189+git35cf912-1ubuntu1.1) saucy-proposed; urgency=low

  * Compile using -fno-aggressive-loop-optimizations to work around
    segfault caused by changes in gcc 4.8 (LP: #1241772).
 -- Andrew Starr-Bochicchio <email address hidden> Tue, 22 Oct 2013 12:22:07 -0400

[Scott Kitterman (kitterman)]

The verification of this Stable Release Update has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regresssions.