mirscreencast slows down compositing and makes it very jerky

[Daniel van Vugt]

I've just tried out mirscreencast for the first time, to record the screen. It works but compositing is slowed significantly and made very jerky. This results in a video which plays back equally jerkily:

mencoder -demuxer rawvideo \
  -rawvideo fps=60:w=1920:h=1200:format=bgra /tmp/*bgra \
  -ovc lavc -lavcopts vcodec=ffv1 -o all.avi

mplayer all.avi

[Alexandros Frantzis (afrantzis)]

Can you please elaborate on which platform, graphics card, and what setup exactly are you using when taking the screencast (which server and clients). I want to try to reproduce locally. In previous experiments on intel hd graphics 1280x800 compositing wasn't affected at all (although 1900x1200 is over twice as large).

> -rawvideo fps=60:w=1920:h=1200:format=bgra

I doubt that we are capturing at 60 fps, buf if we are that could be the problem (too much for the hardware, so we may need to add support for throttling).

[Daniel van Vugt (vanvugt)]

I suspect 1920x1200x60 FPS is more bandwidth than my SSD has. It would need 553 MB/s writing, which it certainly can't. So I wouldn't expect it to keep up. But I think it would be nice to just degrade the screencasting frame rate (drop frames from the recording but not the actual screen).

Tested on amd64 with Intel i7-4770 (Haswell). It's a fast machine, but the SSD is pretty average: Samsung PM830 (max write speed 400 MB/s, likely less).

[Daniel van Vugt (vanvugt)]

Also, I was running 3 clients with transparency, blended over each other:
    mir_demo_client_eglplasma -s 800x800
    mir_demo_client_egltriangle -b 0.0
    mir_demo_client_progressbar

(transparency enabled with Alt+mousewheel)

[Daniel van Vugt (vanvugt)]

On second thoughts, if we start generating variable-framerate output that limits the potential output formats. I think we need to aim for a constant framerate output. So maybe this shouldn't be fixed at all.

I suspect existing screencasting software gets around the problem using multiple tricks:
  1. Only record to RAM (while possible) as it's much faster and can keep up.
  2. Only record changes (damage based regions) and not the whole screen each frame.
  3. Write to a semi-compressed file format, aided by #2.

On the issue of compressed file formats, we might benefit from simple and fast run-length encoding (as supported in simple image formats like TGA, BMP or PCX). Over the years I have used all three and they're pretty simple file formats to create.

[Alberto Aguirre (albaguirre)]

You can pipe the mirscreencast output to x264 with mkfifo, that should bring down the disk B/W down significantly.

I think last time I checked in my haswell laptop, I was able to sustain 60fps by piping to x264 (using eglplasma and egltriangle).

[Alberto Aguirre (albaguirre)]

The branch lp:~albaguirre/mir/screencast-capture-rate-limiter adds a capture fps option.

If the capture loop is not able to keep up with the display refresh rate then the client can lower the capture fps with mirscreencast --fps <fps> ...

[Alberto Aguirre (albaguirre)]

1. Only record to RAM (while possible) as it's much faster and can keep up.

That's already possible with --stdout option or by using mkfifo to create a pipe file and using the --file option. It's up to the consumer then to decide what to do - like buffer a big chunk.

[PS Jenkins bot (ps-jenkins)]

Fix committed into lp:mir/devel at revision None, scheduled for release in mir, milestone Unknown

[Launchpad Janitor (janitor)]

This bug was fixed in the package mir - 0.1.9+14.10.20140430.1-0ubuntu1

---------------
mir (0.1.9+14.10.20140430.1-0ubuntu1) utopic; urgency=medium

  [ Daniel van Vugt ]
  * New upstream release 0.1.9 (https://launchpad.net/mir/+milestone/0.1.9)
    - mirclient ABI unchanged, still at 7. Clients do not need rebuilding.
    - mirserver ABI bumped to 19. Shells need rebuilding.
    - More libmirserver class changes and reorganization, including;
      . Moving things from shell:: to scene::
      . Rewriting/refactoring surface factories.
    - Added an id() to Renderable.
    - Scene/Renderer interfaces:
      . Scene is no longer responsible for its own iteration (no for_each
        any more). Instead you should iterate over the list returned by
        Scene::generate_renderable_list().
    - Bugs fixed:
      . Stale socket issue. (LP: #1285215)
      . Qt render gets blocked on EGLSwapBuffers. (LP: #1292306)
      . Lock order violated found in helgrind (potential deadlock).
        (LP: #1296544)
      . [regression] SwitchingBundle in framedropping mode can hang.
        (LP: #1306464)
      . [DPMS] Display backlight turns back on almost immediately after
        being turned off. (LP: #1231857)
      . Wrong frame is seen on wake up/resume/unlock. (LP: #1233564)
      . Nested platform is not testable (LP: #1299101)
      . [regression] mir_demo_server_shell crashes on display resume.
        (LP: #1308941)
      . Multi-threaded composition is actually mostly serialized by
        SurfaceStack::guard. (LP: #1234018)
      . Mirscreencast slows down compositing and makes it very jerky.
        (LP: #1280938)
      . Mirscreencast can cause clients to render faster than the screen
        refresh rate. (LP: #1294361)
      . Screen turns on when a new session/surface appears. (LP: #1297876)
      . mir-doc package is >56MB in size, expands to >100MB of files.
        (LP: #1304998)
      . [regression] Clang: 'mir::test::doubles::MockSurface::visible'
        hides overloaded virtual function [-Woverloaded-virtual].
        (LP: #1301135)
      . [regression] GLRenderer* unit tests have recently become noisy.
        (LP: #1308905)
      . FocusController::set_focus_to() no longer seems to raise a session
        to the top. (LP: #1302689)

  [ Ubuntu daily release ]
  * New rebuild forced
 -- Ubuntu daily release <email address hidden> Wed, 30 Apr 2014 13:26:58 +0000