Screencast only records one second

[Tobias Heider]

[Impact]
When recording a screencast with gnome on kinetic the resulting video will play for one second and then freeze. It looks like the same bug was discussed upstream at https://gitlab.gnome.org/GNOME/gnome-shell/-/issues/5585

This issue is caused by the new two patches in 0.3.48-1ubuntu2 which is fixed the Cheese preview stuck issue on jammy
  * d/p/0001-buffers-ensure-buffer-size-does-not-exceed-maxsize.patch
    d/p/0002-gst-dequeue-a-shared-buffer-instead-of-original-pool.patch
    - Camera output freeze when using pipewiresrc (LP: #1985057)

Here is a comment from https://bugs.launchpad.net/ubuntu/+source/pipewire/+bug/1985057/comments/51 .
===
So that's a regression of one of the cherrypicked commits, details are in https://gitlab.gnome.org/GNOME/gnome-shell/-/commit/d32c03488

the issue is fixed in Kinetic through a combination of the shell fix and a new pipewire.

In 22.04 the shell issue is fixed in the recent 42.5 update but we will need to cherrypick https://gitlab.freedesktop.org/pipewire/pipewire/-/commit/1ea1d525 in pipewire to be working.
===

[Test Plan]
1. Install Jammy on the hardware issue reported, and hardware didn't report the issue to avoid the regression
   hardware list:
   a. 0bda:58ff Realtek Semiconductor Corp. Integrated_Webcam_HD
   b. 0c45:6747 Microdia Integrated_Webcam_HD
   c. 0c45:6a14 Microdia Integrated_Webcam_HD
   d. 1bcf:28d0 Sunplus Innovation Technology Inc. Integrated_Webcam_5M
   e. 04f2:b76b Chicony Electronics Co., Ltd HP HD Camera
   f. 0408:545a Quanta Computer, Inc. HP 5MP Camera
   g. 0408:5483 Quanta Computer, Inc. HP HD Camera
   h. 174f:2459 Syntek Integrated Camera (ThinkBook 14 Gen 4)
   i. 5986:116d Acer, Inc Integrated Camera (ThinkCentre Neo 50a)
   j. 0bda:5556 Realtek Semiconductor Corp. Integrated_Webcam_FHD
2. try to install the updated pipewire packages (= 0.3.48-1ubuntu2)
3. $ sudo reboot
4. Check if gst-launch-1.0 work
   a. $ gst-device-monitor-1.0 Video/Source to get caps and suggest gst-launch-1.0 command
   b. $ gst-launch-1.0 pipewiresrc path=<id> ! <cap> ! decodebin ! videoconvert ! glimagesink
   c. Check if the result ok
5. Check the screencast function by pressing 'prt sc'
   a. the screenshot of all screen/selected region should work good
   b. the screenrecord of all screen/selected region should work good
6. Check that video recording in gnome-shell works
   - use Ctrl+Shift+Alt+R to start a recording, stop it from the indicator, verify that there is a new entry in ~/Video
7. Check that screen sharing is working
   - go to settings, screen sharing and enable the feature
   - try to connect using rdp/vnc from another client

do those steps under wayland and unset X

[Where problems could occur]
The patches try to dequeue the shared buffer, instead of pool buffer to prevent the pool buffer being corrupted. it might cause some camera preview failed if shared buffer is corrupted.
It is from upstream and there is no regression found, so the risk is low.

[Other Info]
Upstream commits for pipewire:
https://gitlab.freedesktop.org/pipewire/pipewire/-/commit/7cc509b117a6db66c395fb56ac4f17fb8cbd0c92
https://gitlab.freedesktop.org/pipewire/pipewire/-/commit/a1f33a99df5756c3dedd68f5ba2690819098d14f
https://gitlab.freedesktop.org/pipewire/pipewire/-/commit/1ea1d525c1ac946a915599c6bee813e88e8cee12
Upstream commits for gstreamer:
https://gitlab.freedesktop.org/gstreamer/gstreamer/-/commit/3b900e1fa4fd888012dc005fa26ae2532a89b7a7

[Daniel van Vugt (vanvugt)]

Sounds like this might be bug 1963264 so please try:

  cd ~/.cache
  rm -rf gstreamer-1.0

[Tobias Heider (tobhe)]

Look like a temporary fix for the bug landed in upstream with https://gitlab.gnome.org/skeller/gnome-shell/-/commit/398b1c6c79da0bb0630ab7448cd227d85c3985eb#249ab328def8cba907309a9d7c0ddf940776c578. I will test if this also fixes the bug for me.

[Tobias Heider (tobhe)]

Deleting .cache/gstreamer-1.0 doesn't change anything. The bug you mention also seems to be a different one because I get a video, it is just very choppy.
The upstream bug fix i shared also doesn't seem to help.

[Daniel van Vugt (vanvugt)]

I tested this on kinetic last night. For me the recorded video is valid but totem (Videos app) can't play it. I could play it with 'mpv' though.

[Daniel van Vugt (vanvugt)]

Thank you for taking the time to report this bug and helping to make Ubuntu better. Please execute the following command only once, as it will automatically gather debugging information, in a terminal:

  apport-collect 1987631

When reporting bugs in the future please use apport by using 'ubuntu-bug' and the name of the package affected. You can learn more about this functionality at https://wiki.ubuntu.com/ReportingBugs.

[Daniel van Vugt (vanvugt)]

From the upstream bug:

> I think we can close this given that the workaround that has now been
> released in 43 and 42.5 is all we can do from the shell side and the
> required pipewire 0.3.57 has been out for a while now as well.

[Robie Basak (racb)]

regression-updates based on duplicate report https://bugs.launchpad.net/ubuntu/+source/pipewire/+bug/1995358

[Robie Basak (racb)]

@vanvugt what about pipewire in Jammy? Based on the duplicate report, isn't this a regression in Jammy?

[Matthew D. Mower (mdmower)]

I posted duplicate report https://bugs.launchpad.net/ubuntu/+source/gnome-shell/+bug/1993912 which was merged into this one. In that report's comments, I noted that this issue was fixed in jammy by applying https://gitlab.gnome.org/GNOME/gnome-shell/-/commit/d32c03488fcf6cdb0ca2e99b0ed6ade078460deb locally, but I left out a key detail... I had also upgraded pipewire from https://launchpad.net/~pipewire-debian/+archive/ubuntu/pipewire-upstream . I'm sorry for omitting that detail because it's important - from the upstream bug report https://gitlab.gnome.org/GNOME/gnome-shell/-/issues/5585#note_1559914 :

> I think we can close this given that the workaround that has now been released in 43 and 42.5 is all we can do from the shell side and the required pipewire 0.3.57 has been out for a while now as well.
> The proper fix is still tracked in https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2928 .

Jammy is still using pipewire 0.3.48. So, for Jammy, perhaps this needs to be fixed by backporting https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2928 ?

[Daniel van Vugt (vanvugt)]

Yes you're both right (and I wasn't paying enough attention) that it sounds like a pipewire task is required here.

[Bin Li (binli)]

On 22.10, our customer could not reproduce this issue.

I picked the patch in pipewire, the screencast becomes stable, the fail rate is lower than before, although it would skip frames for seconds. I will let our customer do more test.

https://gitlab.freedesktop.org/pipewire/pipewire/-/commit/1ea1d525

https://people.canonical.com/~binli/pipewire/

[Robie Basak (racb)]

Possibly fixed in jammy-proposed with bug 1996148. If this fixes this for you, please let us know.

[Bin Li (binli)]

Yes, with the pipewire in proposed channel, we tested screencast 10 times, and couldn't reproduce this issue again. Thanks!

pipewire 0.3.48-1ubuntu3

[Bin Li (binli)]

Our customer did more test with my built packages in #11, and the screencast issue is still reproduced, it looks like the below patch is not enough.

https://gitlab.freedesktop.org/pipewire/pipewire/-/commit/1ea1d525

[Matthew D. Mower (mdmower)]

Following up on my earlier comment (#9), is it worth trying to apply patch https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2928 to gstreamer1.0 and gst-plugins-base1.0? I was able to confirm the patch picks cleanly and gstreamer still compiles when applied to the current jammy refs: https://git.launchpad.net/ubuntu/+source/gstreamer1.0/log/?h=ubuntu%2Fjammy and https://git.launchpad.net/ubuntu/+source/gst-plugins-base1.0/log/?h=ubuntu%2Fjammy .

Original patch file: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2928.patch

Patch file broken into...
gstreamer1.0: https://gist.github.com/mdmower/a758ec83b525ce8b1a59774782ec8abd
gst-plugins-base1.0: https://gist.github.com/mdmower/de7245603f12c3ef413c0beba1f02628

I haven't created Ubuntu packages before and I'm wary of trying to overwrite gstreamer on my computer; sorry, I haven't gone beyond verifying it compiles.

[Matthew D. Mower (mdmower)]

I took the time to figure out how ubuntu packages are built and managed to test the patch in my previous comment (#15) on Jammy. My tests showed great results; it would be great to get additional confirmation.

Sorry for the verbosity here, but this is new to me:
1. sudo apt-get source gstreamer1.0 gst-plugins-base1.0
2. sudo apt-get build-dep gstreamer1.0 gst-plugins-base1.0
3. apply patches from my previous comment (in gists) to respective source
4. add symbol '_gst_meta_tag_memory_reference@Base 1.20.0' to gstreamer1.0/debian/libgstreamer1.0-0.symbols
5. build gstreamer: dpkg-buildpackage -rfakeroot -b
6. install the built deb packages so that the new symbol (from -dev pkg) is available when building plugins
7. build gst-plugins-base: dpkg-buildpackage -rfakeroot -b
8. install all produced .deb packages

Video attached after gstreamer update to show the effect. I subsequently reinstalled stock gstreamer and confirmed the broken video recording happened again.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Matthew D. Mower (mdmower)]

Just a single bump in hopes that a maintainer will see my above comments. It would be wonderful if we could either cherry-pick that commit or get a merge of gstreamer 1.20.4 in jammy (similar to https://bugs.launchpad.net/ubuntu/+source/gstreamer1.0/+bug/1980239 but unfortunately Jeremy Bicha did not respond to my inquiry about merging 1.20.4).

Screen recording in jammy is still broken but I've tested this patch on two machines with success now. So, I'm hopeful it will be picked up.

[suoko (suoko)]

The bug is still there, can you please paste the complete procedure ?
@Matthew D. Mower (mdmower)

Thanks

[Santiago Fernández Núñez (santiagofn)]

I can confirm that screencast is still broken in Jammy.

There's a fix waiting in the 'proposed' repo: https://launchpad.net/ubuntu/+source/pipewire/0.3.48-1ubuntu3 Couldn't it be fast-tracked to the stable one? 22.04 is a LTS version and screencast has been broken for more than a month.

[Jakub Klos (9v-ka2ub-3y)]

Same here, I have been waiting for this to be fixed for more than 2 months. Can we have the fix applied to the stable repo, please? I can confirm the Proposed repo works properly. Thank you

[Rafael Weingartner (rafael-wo)]

Same here - thanks in advance for fixing :)

[Bin Li (binli)]

It's still in proposed. Missing tags?

$ rmadison pipewire
 pipewire | 0.3.48-1ubuntu2 | jammy-updates | source, amd64, arm64, armhf, i386, ppc64el, riscv64, s390x
 pipewire | 0.3.48-1ubuntu3 | jammy-proposed | source, amd64, arm64, armhf, i386, ppc64el, riscv64, s390x

[Matthew D. Mower (mdmower)]

@binli In comment #14 you wrote that the pipewire commit wasn't enough. Did something change? That comment was the reason I tested a gstreamer patch which seems to work well. See conversation following #14.

[Bin Li (binli)]

@mdmower,

 Yes, the patch for pipewire is not enough, the deep copy need another 2 packages, gstreamer1.0 and gst-plugins-base1.0. I built the deb packages in my ppa. The screencast works fine with below packages.

https://launchpad.net/~binli/+archive/ubuntu/gnome/

gstreamer1.0 - 1.20.3-0ubuntu1binli2

gst-plugins-base1.0 - 1.20.1-2binli1

pipewire - 0.3.48-1ubuntu2binli1
  * d/p/0003-gst-copy-buffer-memory-in-dequeue_buffer-using-gst_m.patch
    - Fixed Screencast only records few seconds issue (LP: #1995358)

[Matthew D. Mower (mdmower)]

@binli - I never found a need to update pipewire; the gstreamer patches (for gstreamer and gst-plugins-base) seem to be sufficient in my testing.

I tested your gstreamer & gst-plugins-base packages and they work the same as the ones I built. I think you could drop "-0ubuntu1binli1" from the symbols update though:
- _gst_meta_tag_memory_reference@Base 1.20.3-0ubuntu1binli1
+ _gst_meta_tag_memory_reference@Base 1.20.3

[Bin Li (binli)]

@mdmower,

 What's the version of pipewire in your side?
 This issue is caused by new patches in 0.3.48-1ubuntu2 which is fixed lp:1985057 .
  * d/p/0001-buffers-ensure-buffer-size-does-not-exceed-maxsize.patch
    d/p/0002-gst-dequeue-a-shared-buffer-instead-of-original-pool.patch
    - Camera output freeze when using pipewiresrc (LP: #1985057)
 In proposed channel, it's 0.3.48-1ubuntu3, just revert these two patches.

 Now with the two gstreamer's patches, we want add these two patches back, and from
https://bugs.launchpad.net/ubuntu/+source/pipewire/+bug/1985057/comments/51
 I added another patch, d/p/0003-gst-copy-buffer-memory-in-dequeue_buffer-using-gst_m.patch.

$ rmadison pipewire
 pipewire | 0.3.48-1ubuntu2 | jammy-updates | source, amd64, arm64, armhf, i386, ppc64el, riscv64, s390x
 pipewire | 0.3.48-1ubuntu3 | jammy-proposed | source, amd64, arm64, armhf, i386, ppc64el, riscv64, s390x

[Matthew D. Mower (mdmower)]

@binli - I'm still on pipewire from jammy-updates:

$ apt-cache policy pipewire
pipewire:
  Installed: 0.3.48-1ubuntu2
  Candidate: 0.3.48-1ubuntu2
  Version table:
 *** 0.3.48-1ubuntu2 500
        500 http://us.archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages
        100 /var/lib/dpkg/status
     0.3.48-1ubuntu1 500
        500 http://us.archive.ubuntu.com/ubuntu jammy/main amd64 Packages

[Bin Li (binli)]

@seb128,

 Could you help review and sponsor the 3 debdiff when you are available? Thanks!

https://bugs.launchpad.net/ubuntu/+source/gnome-shell/+bug/1987631/comments/27
https://bugs.launchpad.net/ubuntu/+source/gnome-shell/+bug/1987631/comments/28
https://bugs.launchpad.net/ubuntu/+source/gnome-shell/+bug/1987631/comments/30

[Bin Li (binli)]

@mdmower,

 Thanks, in your comment this issue could be fixed with gstreamer's patch, so the root cause is from gstreamer.

 I prefer to add the pipewire's patch, cause it's also from James Hilliard, it might help avoid the potential issue without deepcopy. Could you help upgrade the new pipewire from comments #25 and check this issue again?

BTW, we also need test the cheese's preview issue (lp:1985057) at the same time.

https://gitlab.freedesktop.org/pipewire/pipewire/-/commit/1ea1d525c1ac946a915599c6bee813e88e8cee12

https://gitlab.freedesktop.org/gstreamer/gstreamer/-/commit/3b900e1fa4fd888012dc005fa26ae2532a89b7a7

[Matthew D. Mower (mdmower)]

@binli - here are my testing results with patches:
- gstreamer1.0_1.20.3-0ubuntu2.debdiff
- gst-plugins-base1.0_1.20.1-2.debdiff
- pipewire_0.3.48-1ubuntu4.debdiff

Gnome Shell screen recorder: pass
gst-launch-1.0 video preview: pass
Gnome Shell screen recorder while gst-launch-1.0 video preview is visible: pass
Cheese video preview: pass*

Cheese has an asterisk* because I ran into other problems: 1) intermittently, selecting a filter can freeze the preview until a different filter is selected. 2) lp:1994928

Camera hardware tested (two different computers):
13d3:5405 IMC Networks Integrated Camera (Lenovo T14s Gen2 AMD Laptop)
046d:082d Logitech, Inc. HD Pro Webcam C920

---------

Details of how I built the packages and which ones I installed. Note that I only installed packages that are part of a default Ubuntu jammy installation: http://releases.ubuntu.com/jammy/ubuntu-22.04.1-desktop-amd64.manifest .

$ wget https://bugs.launchpad.net/ubuntu/+source/gnome-shell/+bug/1987631/+attachment/5638524/+files/gstreamer1.0_1.20.3-0ubuntu2.debdiff
$ wget https://bugs.launchpad.net/ubuntu/+source/gnome-shell/+bug/1987631/+attachment/5638525/+files/gst-plugins-base1.0_1.20.1-2.debdiff
$ wget https://bugs.launchpad.net/ubuntu/+source/gnome-shell/+bug/1987631/+attachment/5638528/+files/pipewire_0.3.48-1ubuntu4.debdiff

$ pull-lp-source gstreamer1.0 1.20.3-0ubuntu1
$ pull-lp-source gst-plugins-base1.0 1.20.1-1
$ pull-lp-source pipewire 0.3.48-1ubuntu3

$ patch -p1 -d gstreamer1.0-1.20.3 < gstreamer1.0_1.20.3-0ubuntu2.debdiff
$ patch -p1 -d gst-plugins-base1.0-1.20.1 < gst-plugins-base1.0_1.20.1-2.debdiff
$ patch -p1 -d pipewire-0.3.48 < pipewire_0.3.48-1ubuntu4.debdiff

$ sudo apt-get build-dep gstreamer1.0 gst-plugins-base1.0 pipewire

$ cd gstreamer1.0
$ dpkg-buildpackage -rfakeroot -b
$ cdu ..
$ sudo dpkg -i gir1.2-gstreamer-1.0_1.20.3-0ubuntu2_amd64.deb gstreamer1.0-tools_1.20.3-0ubuntu2_amd64.deb libgstreamer1.0-0_1.20.3-0ubuntu2_amd64.deb libgstreamer1.0-dev_1.20.3-0ubuntu2_amd64.deb

$ cd gst-plugins-base1.0-1.20.1
$ dpkg-buildpackage -rfakeroot -b
$ cdu ..
$ sudo dpkg -i gir1.2-gst-plugins-base-1.0_1.20.1-2_amd64.deb gstreamer1.0-alsa_1.20.1-2_amd64.deb gstreamer1.0-gl_1.20.1-2_amd64.deb gstreamer1.0-plugins-base_1.20.1-2_amd64.deb gstreamer1.0-plugins-base-apps_1.20.1-2_amd64.deb gstreamer1.0-x_1.20.1-2_amd64.deb libgstreamer-gl1.0-0_1.20.1-2_amd64.deb libgstreamer-plugins-base1.0-0_1.20.1-2_amd64.deb

$ cd pipewire-0.3.48
$ dpkg-buildpackage -rfakeroot -b
$ cd ..
$ sudo dpkg -i gstreamer1.0-pipewire_0.3.48-1ubuntu4_amd64.deb libpipewire-0.3-0_0.3.48-1ubuntu4_amd64.deb libpipewire-0.3-common_0.3.48-1ubuntu4_all.deb libpipewire-0.3-modules_0.3.48-1ubuntu4_amd64.deb libspa-0.2-modules_0.3.48-1ubuntu4_amd64.deb pipewire_0.3.48-1ubuntu4_amd64.deb pipewire-bin_0.3.48-1ubuntu4_amd64.deb

[Matthew D. Mower (mdmower)]

It seems like this has stalled again and I don't understand why. Patches were posted about 1 month ago and they got some testing feedback within 2 days of availability.

[Daniel van Vugt (vanvugt)]

We know. It's in the queue but I'm guessing there are not enough sponsors with authority to keep up with the backlog:

http://reqorts.qa.ubuntu.com/reports/sponsoring/

[Kai-Chuan Hsieh (kchsieh)]

@vanvugt

Hello Daniel,

could you help to guide me how to raise its priority? The SRU impacts multiple OEM's platform, and CE engineer has added oem-priority series. I have no idea how to get more sponsors to prioritize it.

Thanks,

[James (jameshilliard)]

To clarify the gstreamer fix should be sufficient. The pipewire always-copy change I made allows for working around the bug but isn't required as long as you have the gstreamer fix.

Fixes the bug in gstreamer:
https://gitlab.freedesktop.org/gstreamer/gstreamer/-/commit/1c8244a19dbb4ebdf972a703b55d719f2b968056

Makes it possible to work around the gstreamer bug without updating gstreamer:
https://gitlab.freedesktop.org/pipewire/pipewire/-/commit/1ea1d525c1ac946a915599c6bee813e88e8cee12

Note that gnome-shell applies the always-copy pipewire workaround based on gstreamer/pipewire version detection so this logic may need tweaking due to backports:
https://gitlab.gnome.org/GNOME/gnome-shell/-/blob/43.2/js/dbusServices/screencast/screencastService.js#L233-235

[Sebastien Bacher (seb128)]

The situation is a bit unclear there, the pipewire SRU got superseeded by a revert right, does that resolve the issue? Or is the request still trying to address the initially problem?

Which component do you request to be updated exactly? pipewire with the previous failed SRU + extra fixes and gstreamers?

Did anyone verify the impact on gnome-shell as pointed in #38?

[James (jameshilliard)]

If gstreamer is fixed then any pipewire combination should work I think.

[Bin Li (binli)]

@seb128,

 The situation is a bit unclear there, the pipewire SRU got superseeded by a revert right, does that resolve the issue?
-> Yes, the revert one the pipewrie(0.3.48-1ubuntu3) could resolve this issue.

Or is the request still trying to address the initially problem?
-> Yes, our OEM projects need the reverted SRU patchtes for Bug #1985057, so I added them back, but these patches also need change gstreamer at the same time.

Which component do you request to be updated exactly? pipewire with the previous failed SRU + extra fixes and gstreamers?
-> Yes, pipewire and gstreamers.

Did anyone verify the impact on gnome-shell as pointed in #38?
-> I could check it, reply you later.

[Bin Li (binli)]

@seb128,

I went through this issue again.

On 22.04.2, it works fine. With pipewire 0.3.48-1ubuntu3, because it revert the patches for Bug #1985057 , so currently we don't need to do SRU for this issue.

To make this issue clear, let's focus on the SRU process on Bug #1985057.

[Bin Li (binli)]

For comment #38, currently it sounds good, if we upgrade the gstreamer to 1.20.4, we need backport below patch.

commit d7b443197bcc0789305d6a8722bca1fdd182070b
Author: Sebastian Keller <email address hidden>
Date: Thu Oct 6 00:20:16 2022 +0200

    screencast: Don't force buffer copies on recent gstreamer versions

    Gstreamer 1.20.4 includes a fix in the videoconvert element that makes
    it no longer necessary to always copy buffers in pipewiresrc to have
    smooth recordings. This change now skips those otherwise unnecessary
    copies when using a new enough videoconvert.

    Related: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2928
    Part-of: <https://gitlab.gnome.org/GNOME/gnome-shell/-/merge_requests/2503>

diff --git a/js/dbusServices/screencast/screencastService.js b/js/dbusServices/screencast/screencastService.js
index 5ff5aff52..bb5387428 100644
--- a/js/dbusServices/screencast/screencastService.js
+++ b/js/dbusServices/screencast/screencastService.js
@@ -231,7 +231,8 @@ var Recorder = class {
     _ensurePipeline(nodeId) {
         const framerate = this._framerate;
         const needsCopy =
- Gst.Registry.get().check_feature_version('pipewiresrc', 0, 3, 57);
+ Gst.Registry.get().check_feature_version('pipewiresrc', 0, 3, 57) &&
+ !Gst.Registry.get().check_feature_version('videoconvert', 1, 20, 4);