Wrong aspect ratio with vdpau on i965

[Ian Turner]

After upgrading from Wily to Xenial, vlc scaling and zooming is now incorrect for h264 videos playing on i965. See attached screenshot "vlc-vaapi.png" for a clear depiction of the problem.

Four test cases:
1. With libvdpau-va-gl1 and i965-va-driver installed, video is displayed with the wrong aspect ratio and does not scale properly. Screenshot is vlc-vaapi.png and terminal output looks like this:
VLC media player 2.2.2 Weatherwax (revision 2.2.2-0-g6259d80)
[0000000000758148] core libvlc: Running vlc with the default interface. Use 'cvlc' to use vlc without interface.
libva info: VA-API version 0.39.0
libva info: va_getDriverName() returns 0
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/i965_drv_video.so
libva info: Found init function __vaDriverInit_0_39
libva info: va_openDriver() returns 0

2. With libvdpau-va-gl1 but not i965-va-driver installed, video is the same as #1 (vlc-vaapi.png) but terminal output is a little different:
VLC media player 2.2.2 Weatherwax (revision 2.2.2-0-g6259d80)
[0000000000758148] core libvlc: Running vlc with the default interface. Use 'cvlc' to use vlc without interface.
libva info: VA-API version 0.39.0
libva info: va_getDriverName() returns 0
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/i965_drv_video.so
libva info: Found init function __vaDriverInit_0_39
libva info: va_openDriver() returns 0

3. Without libvdpau-va-gl1 (and regardless of installation status of i965-va-driver), playback works perfectly. See vlc-vaapi-no-vdpau.png. Terminal output looks like this:
VLC media player 2.2.2 Weatherwax (revision 2.2.2-0-g6259d80)
[0000000000c29148] core libvlc: Running vlc with the default interface. Use 'cvlc' to use vlc without interface.
Failed to open VDPAU backend libvdpau_va_gl.so: cannot open shared object file: No such file or directory

Package versions used for this test:
||/ Name Version Architecture Description
+++-==============-============-============-=================================
ii ffmpeg 7:2.8.6-1ubu amd64 Tools for transcoding, streaming
ii i965-va-driver 1.7.0-1 amd64 VAAPI driver for Intel G45 & HD G
ii libva-x11-1:am 1.7.0-1 amd64 Video Acceleration (VA) API for L
ii libvdpau-va-gl 0.3.6-1 amd64 VDPAU driver with OpenGL/VAAPI ba
ii vdpau-va-drive 0.7.4-5 amd64 VDPAU-based backend for VA API
ii vlc 2.2.2-4 amd64 multimedia player and streamer

[Ian Turner (vectro)]

Because of bug 1568531, just changing the ffmpeg backend doesn't seem to help/change things.

[Colin Law (colin-law)]

I am seeing, with Xenial on i915, that zoom does absolutely nothing on any video type I try.

$ vlc Videos/motion/archive/20151106_221248-03.avi
VLC media player 2.2.2 Weatherwax (revision 2.2.2-0-g6259d80)
[000000000234a148] core libvlc: Running vlc with the default interface. Use 'cvlc' to use vlc without interface.
Failed to open VDPAU backend libvdpau_va_gl.so: cannot open shared object file: No such file or directory

Is this likely a related problem or should I open a new bug?

[Ian Turner (vectro)]

My opinion: Yours is an unrelated problem.

[Colin Law (colin-law)]

Thanks.

I have submitted a new bug #1574280

[Brandon Snider (brandonsnider)]

Since this bug was reported, VLC has been updated to 2.2.2-5, which contains a fix for one issue with VDPAU. Does this issue still happen with the -5 version?

Also, you don't need to remove the gl1 package. Just change the output module to XCB and disable hardware decoding, and VLC will stop trying to use VAAPI/VDPAU at all.

[Ian Turner (vectro)]

Resolved in 2.2.2-5! Thanks.

[Ian Turner (vectro)]

Actually it looks like bug 1568531 was fixed in vlc 2.2.2-5, not this one. If I set "hardware decoding" to "automatic", then with VLC 2.2.2-5 I get this on the terminal and no video at all:

VLC media player 2.2.2 Weatherwax (revision 2.2.2-0-g6259d80)
[0000000000c74148] core libvlc: Running vlc with the default interface. Use 'cvlc' to use vlc without interface.
libva info: VA-API version 0.39.0
libva info: va_getDriverName() returns 0
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/i965_drv_video.so
libva info: Found init function __vaDriverInit_0_39
libva info: va_openDriver() returns 0
[00007f9d3c001468] avcodec decoder: Using OpenGL/VAAPI/libswscale backend for VDPAU for hardware decoding.
[00007f9d0002beb8] core video output error: video output creation failed
[00007f9d3c001468] core decoder error: failed to create video output

Without i965-va-driver installed, video plays fine and terminal output looks like this:

✓ Sun Apr 24 16:12:35 vectro@mirian:~ $ vlc Downloads/honey\ and\ clover\[niizk\]/honey_and_clover_06\[h264.vorbis\]\[niizk\].mkv
VLC media player 2.2.2 Weatherwax (revision 2.2.2-0-g6259d80)
[0000000000d0a148] core libvlc: Running vlc with the default interface. Use 'cvlc' to use vlc without interface.
libva info: VA-API version 0.39.0
libva info: va_getDriverName() returns 0
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/i965_drv_video.so
libva info: va_openDriver() returns -1
[00007fcbac04e6a8] vdpau_avcodec generic error: decoder profile not supported: 8

There is one other change to my system since I originally reported this bug: I uninstalled the nvidia drivers (this is an Optimus machine) to work around bug 1571225.

[Brandon Snider (brandonsnider)]

What is the result of the vdpauinfo and vainfo commands? You may need to install those packages.

[Rémi Denis-Courmont (rdenis)]

In case 2, you claim to have removed the i965 driver, and yet the log clearly shows that it's loaded succesfully. Possibly a cut and paste error, but meanwhile we cannot make sense of your bug report.

[Ian Turner (vectro)]

Previous attachments were for case 1. I will attach new logs for case 2.

[Rémi Denis-Courmont (rdenis)]

It seems that the VDPAU output support in VA-GL driver is broken.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Jan Groenewald (jan-aims)]

Does it work if you remove libvdpau?

sudo apt-get purge libvdpau-va-gl1 # this is on debian stretch, and fixes autoscaling for me

[Ian Turner (vectro)]

Jan, yes, that works fine for me; it is Case 3 in the bug description.