During a running mp4 video, mpv and vlc loose video frames when switching to another workscreen in Compiz and output stuttering scaled videos

[Heinz Assmus]

When I check hardware acceleration in vlc for intel graphics drivers using intel-gpu-tools, it seems like vlc has a problem with the 3d render engine in the vaapi module.
I use Ubuntu MATE 20.04 with Compiz as a window manager (used plugins Scale, Expo, Desktop Wall, etc.). When I work with several virtual desktops, the load on the 3D rendering engine of the Intel driver drops when I switch to a different desktop than the one on which vlc is active.
In the Scale plug-in (overview of all windows on all workspaces) the graphics output then stutters in the reduced video image from vlc. In a normal HD video (1280x720) the load drops from approx. 32% to approx. 8%. The video engine (hardware acceleration) has no errors.
Occupancy continuously at 4% to 6%.

Vlc works perfectly under Nvidia graphics (no stuttering graphic image in the reduced vlc video window with an overview of all windows on all workspaces).

Gstreamer (Ver. 1.18) works fine with gst-play-1.0 on vaapi and nvidia-vdpau. There is no drop in the utilization of the 3D rendering engine with an overview of all windows on all workspaces in the Scale plug-in. (Checked with intel_gpu_top at Intelgraphics).
The graphics output runs without stuttering in the reduced vlc window and the Utilization in 3D Render Engine is continuously around 33% throughout.

This issue exist already a long time before up to now and in other distros and flavors with Compiz.

snapshot from intel_gpu_top with gst-play-1.0
Render/3D/0 Engine is continously used when switching to different workspaces from active vs.
Scaled video window shows normal (fluently) scaled video in overview of all windows of all workspaces.

intel-gpu-top - 208/ 208 MHz; 35% RC6; 0.86 Watts; 137 irqs/s

      IMC reads: 0 MiB/s
     IMC writes: 677 MiB/s

          ENGINE BUSY MI_SEMA MI_WAIT
     Render/3D/0 37.98% |█████████████▎ | 0% 0%
       Blitter/0 0.00% | | 0% 0%
         Video/0 13.00% |████▌ | 0% 0%
  VideoEnhance/0 0.00% | | 0% 0%

snapshot from intel_gpu_top with vlc
Render/3D/0 Engine breaks down when switched from active workspace to another workspace where vlc is not active.
scaled video is stuttering when all windows of all workspaces are shown in overview!

intel-gpu-top - 420/ 421 MHz; 48% RC6; 0.47 Watts; 50 irqs/s

      IMC reads: 0 MiB/s
     IMC writes: 201 MiB/s

          ENGINE BUSY MI_SEMA MI_WAIT
     Render/3D/0 3.00% |█ | 0% 0%
       Blitter/0 0.00% | | 0% 0%
         Video/0 6.49% |██▎ | 0% 0%
  VideoEnhance/0 0.00% | | 0% 0%

^C

snapshots not at the exactly same position in the video!

My machine:

tux@tux-Aspire-V5-573G:~$ neofetch
            .-/+oossssoo+/-. tux@tux-Aspire-V5-573G
        `:+ssssssssssssssssss+:` ----------------------
      -+ssssssssssssssssssyyssss+- OS: Ubuntu 20.04.2 LTS x86_64
    .ossssssssssssssssssdMMMNysssso. Host: Aspire V5-573G TBD by
   /ssssssssssshdmmNNmmyNMMMMhssssss/ Kernel: 5.4.0-74-generic
  +ssssssssshmydMMMMMMMNddddyssssssss+ Uptime: 2 hours, 10 mins
 /sssssssshNMMMyhhyyyyhmNMMMNhssssssss/ Packages: 2927 (dpkg), 12 (snap)
.ssssssssdMMMNhsssssssssshNMMMdssssssss. Shell: bash 5.1.8
+sssshhhyNMMNyssssssssssssyNMMMysssssss+ Resolution: 1920x1080
ossyNMMMNyMMhsssssssssssssshmmmhssssssso DE: MATE
ossyNMMMNyMMhsssssssssssssshmmmhssssssso WM: Compiz
+sssshhhyNMMNyssssssssssssyNMMMysssssss+ WM Theme: Radiant-MATE
.ssssssssdMMMNhsssssssssshNMMMdssssssss. Theme: Mojave-light [GTK2/3]
 /sssssssshNMMMyhhyyyyhdNMMMNhssssssss/ Icons: Radiant-MATE [GTK2/3]
  +sssssssssdmydMMMMMMMMddddyssssssss+ Terminal: mate-terminal
   /ssssssssssshdmNNNNmyNMMMMhssssss/ Terminal Font: Ubuntu Mono 13
    .ossssssssssssssssssdMMMNysssso. CPU: Intel i5-4200U (4) @ 2.600GHz
      -+sssssssssssssssssyyyssss+- GPU: NVIDIA GeForce GT 750M
        `:+ssssssssssssssssss+:` GPU: Intel Haswell-ULT
            .-/+oossssoo+/-. Memory: 2222MiB / 7845MiB

[Daniel van Vugt (vanvugt)]

Sorry I don't see the problem. Can you state concisely what you think the issue is?

[Heinz Assmus (tuxtester)]

Hi Daniel

i have made an description in pdf so hopefully you can reproduce the
failure. See in attachment please.

Am 21.06.21 um 08:42 schrieb Daniel van Vugt:
> Sorry I don't see the problem. Can you state concisely what you think
> the issue is?
>
> ** Summary changed:
>
> - vlc has a problem with 3D Render Engine in VAAPI
> + Problem with 3D Render Engine in VAAPI
>
> ** Also affects: mpv (Ubuntu)
> Importance: Undecided
> Status: New
>
> ** Changed in: mpv (Ubuntu)
> Status: New => Incomplete
>
> ** Changed in: vlc (Ubuntu)
> Status: New => Incomplete
>
> ** Tags added: focal
>

[Daniel van Vugt (vanvugt)]

Thanks for the detailed document but concise means short. Please describe the problem in one or two sentences.

As I understand it you are talking about intel_gpu_top reporting numbers you don't expect, but that in itself is not a bug.

[Heinz Assmus (tuxtester)]

During a running mp4 video in Intel graphics i965 (Ubuntu 18.04 Unity,
Ubuntu 20.04 MATE), mpv and vlc loose video frames (hardware
accelerated) when switching to another workscreen in Compiz and outputs
stuttering scaled videos (plug-in "scale" with overview "initiate window
selection for all windows") whereas gstreamer-based (1.18) Video players
(e.g. parole) do not have this problem.

The warning message (mpv) and error message (vlc) are then:

mpv:
Audio/Video desynchronisation detected! Possible reasons include too slow
hardware, temporary CPU spikes, broken drivers, and broken files. Audio
position will not match to the video (see A-V status field).

AV: 00:00:36 / 00:01:57 (31%) A-V:  0.000 Dropped: 422

vlc (after a while):
[h264 @ 0x7fc024cd9f40] co located POCs unavailable

Am 23.06.21 um 04:01 schrieb Daniel van Vugt:
> Thanks for the detailed document but concise means short. Please
> describe the problem in one or two sentences.
>
> As I understand it you are talking about intel_gpu_top reporting numbers
> you don't expect, but that in itself is not a bug.
>

[Daniel van Vugt (vanvugt)]

Thanks. It sounds like mpv and vlc have smaller buffers, or maybe a dependency on some part of the GPU that parole does not. You can probably find in the documentation for mpv and vlc ways to increase their buffering sizes so that they are not affected by other parts of the system.