VAAPI encoding is broken in Skylake

[Leonardo Müller]

When trying to do any type of encoding using any of the VAAPI encoders from FFmpeg, the system hangs entirely for some time. On dmesg, the following appears:

[ 107.557643] [drm] GPU HANG: ecode 9:0:0x8fd8ffff, in ffmpeg [2047], reason: Hang on rcs0, action: reset
[ 107.557651] [drm] GPU hangs can indicate a bug anywhere in the entire gfx stack, including userspace.
[ 107.557654] [drm] Please file a _new_ bug report on bugs.freedesktop.org against DRI -> DRM/Intel
[ 107.557657] [drm] drm/i915 developers can then reassign to the right component if it's not a kernel issue.
[ 107.557660] [drm] The gpu crash dump is required to analyze gpu hangs, so please always attach it.
[ 107.557665] [drm] GPU crash dump saved to /sys/class/drm/card0/error
[ 107.557684] i915 0000:00:02.0: Resetting rcs0 after gpu hang
[ 115.615609] i915 0000:00:02.0: Resetting rcs0 after gpu hang
[ 123.514807] i915 0000:00:02.0: Resetting rcs0 after gpu hang
[ 131.511092] i915 0000:00:02.0: Resetting rcs0 after gpu hang
[ 137.492729] i915 0000:00:02.0: Resetting vcs0 after gpu hang

Then the system returns. It shows an enormous RAM usage (more than 6 GB) for a few seconds and then reduces to lower values, but still high. Using h264_vaapi as a example: it uses 580 MB, while when VAAPI was not broken, it used 20 MB for the same task. The resulting media file from FFmpeg has no video output, only audio.

This is happening with all VAAPI encoders but vp8_vaapi, which it says the is no entry point (although Skylake has support for VP8 encoding working in Artful).

ProblemType: Bug
DistroRelease: Ubuntu 18.04
Package: i965-va-driver 2.0.0+dfsg1-1
ProcVersionSignature: Ubuntu 4.15.0-10.11-generic 4.15.3
Uname: Linux 4.15.0-10-generic x86_64
ApportVersion: 2.20.8-0ubuntu10
Architecture: amd64
CurrentDesktop: XFCE
Date: Sat Feb 24 14:51:41 2018
InstallationDate: Installed on 2017-06-13 (255 days ago)
InstallationMedia: Xubuntu 17.04 "Zesty Zapus" - Release amd64 (20170412)
SourcePackage: intel-vaapi-driver
UpgradeStatus: Upgraded to bionic on 2017-10-20 (127 days ago)

[Timo Aaltonen (tjaalton)]

please test with 2.1.0 which was uploaded earlier today

[Leonardo Müller (leozinho29-eu)]

I could encode successfully with all compatible VAAPI encoders on Skylake:

 V..... h264_vaapi H.264/AVC (VAAPI) (codec h264)
 V..... hevc_vaapi H.265/HEVC (VAAPI) (codec hevc)
 V..... mjpeg_vaapi MJPEG (VAAPI) (codec mjpeg)
 V..... mpeg2_vaapi MPEG-2 (VAAPI) (codec mpeg2video)
 V..... vp8_vaapi VP8 (VAAPI) (codec vp8)

Only VP9 I can't test, as Skylake can't encode it purely on hardware (https://bugs.launchpad.net/ubuntu/+source/intel-vaapi-driver/+bug/1591714).

[Timo Aaltonen (tjaalton)]

oh right, it'd require packaging intel-hybrid-driver:

https://github.com/01org/intel-hybrid-driver

[Timo Aaltonen (tjaalton)]

anyway, looks like your original issue is now resolved