gnome-shell high memory and CPU usage when desktop files are constantly created or deleted

[Chris Halse Rogers]

[Impact]

gnome-shell memory usage grows rapidly each time files are created or deleted in the Desktop directory.

[Test Case]

0. Create a number of files/icons on the Desktop. The more existing icons you have, the faster the test will be.

1. Start monitoring gnome-shell's memory usage.

2. Repeatedly create and delete a new file on the Desktop every second.

3. Check that gnome-shell's memory usage doesn't quickly grow by hundreds of megabytes. It should only grow a little.

[Regression Potential]

Low. The current upstream stable release 19.01.4 is just a bug fix release on the same series as what's already in disco and eoan. The same fix has also been released early to eoan already.

[Original Bug Report]

This *may* be an nvidia-specific problem, but the RSS of gnome-shell on my system grows without limit (over a couple of hours it's increased past 10G).

At the same time, gnome-shell appears to become progressively less responsive.

Restarting gnome-shell (via <Alt><F2> restart) drops shell RSS down to ~400MB and improves system responsiveness a bit.

ProblemType: Bug
DistroRelease: Ubuntu 19.04
Package: gnome-shell 3.30.2-2ubuntu2
ProcVersionSignature: Ubuntu 4.20.0+bcachefs.git20190130.d1f70147-1.0-generic 4.20.0
Uname: Linux 4.20.0+bcachefs.git20190130.d1f70147-1-generic x86_64
NonfreeKernelModules: nvidia_modeset nvidia
ApportVersion: 2.20.10-0ubuntu20
Architecture: amd64
CurrentDesktop: ubuntu:GNOME
Date: Tue Feb 12 13:35:59 2019
DisplayManager: gdm3
SourcePackage: gnome-shell
UpgradeStatus: No upgrade log present (probably fresh install)

[Daniel van Vugt (vanvugt)]

Chris,

Please try uninstalling all of these (just disabling them sometimes isn't enough):

b'org.gnome.shell' b'enabled-extensions' b"['<email address hidden>', '<email address hidden>', '<email address hidden>', '<email address hidden>']"

Any improvement?

[Chris Halse Rogers (raof)]

Uninstalling all extensions did not seem to make a difference; the gnome-shell process is back up to 10G RSS after an hour.

[Daniel van Vugt (vanvugt)]

In that case I would agree with the theory this is likely nvidia-related (or a new regression).

The only other yardstick I have at the moment is my bionic desktop for which gnome-shell has been running for 5 days and RSS=293MB.

[Chris Halse Rogers (raof)]

Another data point: it appears that pressing <enter> in a terminal window (tested both gnome-terminal and qterminal) at a zsh shell prompt (but not a bash shell prompt!) results in GNOME Shell being unresponsive for ~4s at 100% CPU and increasing RSS by ~50MB.

[Didier Roche-Tolomelli (didrocks)]

confirming. I'm seeing the same regularly.

[Dylan Borg (borgdylan)]

Is this only on disco or is it observed on prior versions? I was planning to upgrade and have two systems with nvidia GPUs. On cosmic I cannot see the resident memory grow as described above.

[Daniel van Vugt (vanvugt)]

Chris and Didier have both told me that they use the nvidia-drm.modeset=1 kernel parameter. But instead of using it to enable Wayland support they still use Xorg sessions. So that's worth checking to see if the problem is specific to nvidia-drm.modeset=1. If so then it would nicely explain why more Nvidia users never see this bug.

[Chris Halse Rogers (raof)]

This is *not* specific to nvidia_drm.modeset=1¹. With KMS disabled gnome-shell still freezes for a second or two and its RSS grows by ~50MB each time a zsh prompt is displayed.

This is also a regression from Cosmic. I did not experience this behaviour on Cosmic.

¹: Although some other quirks are.

[Daniel van Vugt (vanvugt)]

This sounds important:

<RAOF> ```
 Apr 05 17:21:32 Behemoth gnome-shell[2390]: Couldn’t parse steam.desktop as a desktop file, will treat it as a regular file.
 Apr 05 17:21:33 Behemoth gnome-shell[2390]: Couldn’t parse examples.desktop as a desktop file, will treat it as a regular file.
 ```
 That's rather interesting, though.
 I get those two lines each time I hit enter in the terminal.

[Daniel van Vugt (vanvugt)]

Ah. The above journal entries are from gnome-shell-extension-desktop-icons.

So it sounds like something unusual about your *.desktop files in ~/Desktop are the trigger here.

[Daniel van Vugt (vanvugt)]

It appears the relationship between zsh and gnome-shell can be explained:

1. zsh writes to ~/.zsh_history after each command.

2. The Modify and Change timestamps of ~ change each time.

3. gnome-shell-extension-desktop-icons monitors ~/Desktop/* but also ~ because your home folder appears on the desktop too (unless you disable that icon in Tweaks).

4. gnome-shell has to wake up and run JavaScript code every time ~ gets touched, which is every time zsh runs a command.

I think this is the basic mechanism for this bug. Unfortunately I still cannot reproduce any measurable memory or CPU usage though. And there's no evidence this is related to Nvidia at all, yet. So the next step in debugging might have to be with Chris.

Chris, please try this:

1. Tweaks > Extensions > Desktop icons > cog wheel > Show the personal folder in the desktop = OFF

2. Log out and in again, to be sure.

[Daniel van Vugt (vanvugt)]

What we might be achieving right now is drawing a line between multiple different bugs :)

[Daniel van Vugt (vanvugt)]

I'm also mildly cautious about what difference bcachefs might have, due to Chris':

ProcVersionSignature: Ubuntu 4.20.0+bcachefs.git20190130.d1f70147-1.0-generic 4.20.0

[Daniel van Vugt (vanvugt)]

Also, it sounds like avoiding zsh might avoid the majority of the problem here?

[Daniel van Vugt (vanvugt)]

Please also try:

 1. sudo apt install gnome-session

 2. Log out and then into "GNOME" or "GNOME on Xorg".

 3. Now the desktop-icons extension is gone, do you see the same problems?

[Daniel van Vugt (vanvugt)]

BTW, Didier is now using bug 1823301 instead.

[Daniel van Vugt (vanvugt)]

Chris,

Please also do a sanity check and run:

  glxinfo

What do you see?

[Chris Halse Rogers (raof)]

Relevant bit of glxinfo:
...
server glx vendor string: NVIDIA Corporation
server glx version string: 1.4
...
client glx vendor string: NVIDIA Corporation
client glx version string: 1.4
...
Memory info (GL_NVX_gpu_memory_info):
    Dedicated video memory: 8192 MB
    Total available memory: 8192 MB
    Currently available dedicated video memory: 6393 MB
OpenGL vendor string: NVIDIA Corporation
OpenGL renderer string: GeForce GTX 1070/PCIe/SSE2
OpenGL core profile version string: 4.6.0 NVIDIA 418.56
OpenGL core profile shading language version string: 4.60 NVIDIA
OpenGL core profile context flags: (none)
OpenGL core profile profile mask: core profile
...
Yup, it's all there.

[Chris Halse Rogers (raof)]

Yup! In the stock GNOME session I do *not* see the multi-second pause on the zsh prompt, nor does gnome-shell's RSS increase each prompt.

Looks like it is indeed something wrong with gnome-shell-extension-desktop-icons.

[Daniel van Vugt (vanvugt)]

Chris, please try this:

1. Tweaks > Extensions > Desktop icons > cog wheel > Show the personal folder in the desktop = OFF

2. Log out and in again, to be sure.

[Daniel van Vugt (vanvugt)]

Finally, how many and what type of icons do you have on the desktop when the problem is happening?

[Chris Halse Rogers (raof)]

Turned off “Show the personal folder”; the problem remains.

I've got the standard set of icons: 21 folder icons (apparently $HOME contains 21 folders), a text file icon, and the trash icon.

Notably, I have previously deleted the two .desktop files that were being complained about in the journal (they were in $HOME); this did not change the behaviour.

[Chris Halse Rogers (raof)]

(Using the stock gnome session for longer suggests that there may indeed be a separate memory leak bug; its memory use was up to 1GB RSS after a couple of hours. I'll file that separately if it continues)

[Daniel van Vugt (vanvugt)]

> Turned off “Show the personal folder”; the problem remains.

Please remember to log out and log in again. I am not yet convinced the extension doesn't leak the monitors it previously had so I would want to see it still happening even with a fresh instance of gnome-shell.

> (Using the stock gnome session for longer suggests that there may indeed be a separate memory leak bug; its memory use was up to 1GB RSS after a couple of hours. I'll file that separately if it continues)

Thanks. I was thinking exactly the same but had not yet said it...

[Daniel van Vugt (vanvugt)]

> I've got the standard set of icons: 21 folder icons (apparently $HOME contains 21 folders), a text file icon, and the trash icon.

You mean $HOME/Desktop, right?

[Daniel van Vugt (vanvugt)]

Also, does:

  stat ~/Desktop/*

show any of those changing in any way while the problem is happening?

[Chris Halse Rogers (raof)]

On 8 April 2019 4:21:13 pm AEST, Daniel van Vugt <email address hidden> wrote:
>> I've got the standard set of icons: 21 folder icons (apparently $HOME
>contains 21 folders), a text file icon, and the trash icon.
>
>You mean $HOME/Desktop, right?

No, I mean $HOME. That's where it's pulling the icons from, for whatever reason.

[Daniel van Vugt (vanvugt)]

OK, that's a red flag. Those folders should not appear on your desktop.

On IRC you suggest that everything in ~/.config/user-dirs.dirs is pointing to $HOME/. So whatever leaks or CPU inefficiencies the gnome-shell-extension-desktop-icons has already will be amplified on your machine, by both:

 (a) The strange custom ~/.config/user-dirs.dirs where everything points to $HOME
 (b) The use of zsh which touches $HOME on every command.

I expect practically zero users to have (a), and larger but still small subset of users to have (b). So we can safely lower the priority. It's at least useful now we know how to stress test the extension for leaks and performance.

[Daniel van Vugt (vanvugt)]

Chris,

When the leak gets bad for you, does gnome-shell log much? If so then what does it say?

  journalctl -f /usr/bin/gnome-shell

I suspect I'm running into a second big leak in my investigations but that one is only accompanied by certain log messages.

[Aaron (armahillo)]

Can confirm this bug also affects me.

`gnome-shell` is currently consistently the top program in `top` -- ~5454092 (Virt) / 2.4g (Res) 15.2% Mem

Other behaviors I've noticed:

 - Nautilus is slow at times
 - When browsing the filesystem from a web browser (ie. attaching a file, or any "file dialog" moments), selecting the file causes it to hang for 5-10s

Did not have any of these problems with 18.10.

Some system-specific details:
 - I use an NVidia card (v418.56 of drivers, GTX 960 card)
 - Steam is running (I saw a few other people mention it; don't know if this would be salient or not)
 - I had to disable Wayland so that my system would boot correctly (This seems to be an NVidia thing)

[Vadym Stupakov (red-eyed)]

Hello!

Have same problem:
With zsh as default, every time I press "Enter" in terminal there is 4s delay in whole system.
I monitored CPU and CPU utilization is 100% every time I press enter in the terminal.

So, the problem was that I removed useless folders in $HOME: Desktop, Music, Templates, etc.

After, I noticed that value of $XDG_DESKTOP_DIR is "$HOME/" (But it was XDG_DESKTOP_DIR="$HOME/Desktop" before deletion of that directories)

WORKAROUND:
I created all that stuff again and changed ~/.config/user-dirs.dirs

Now it looks like that:
XDG_DESKTOP_DIR="$HOME/Desktop"
XDG_DOWNLOAD_DIR="$HOME/Downloads"
XDG_TEMPLATES_DIR="$HOME/Templates"
XDG_PUBLICSHARE_DIR="$HOME/Public"
XDG_DOCUMENTS_DIR="$HOME/Documents"
XDG_MUSIC_DIR="$HOME/Music"
XDG_PICTURES_DIR="$HOME/Pictures"
XDG_VIDEOS_DIR="$HOME/Videos"

Then re-login.

[Daniel van Vugt (vanvugt)]

It looks like most (not all) of the leak is fixed in the next version 19.01.4:

https://gitlab.gnome.org/World/ShellExtensions/desktop-icons/issues/104#note_546483

[Daniel van Vugt (vanvugt)]

Fix proposed:

https://salsa.debian.org/gnome-team/shell-extensions/gnome-shell-extension-desktop-icons/merge_requests/1

[Daniel van Vugt (vanvugt)]

Fixed in Debian (Ubuntu coming soon?):

gnome-shell-extension-desktop-icons (19.01.3+git20190814.1-1) experimental; urgency=medium

  * New upstream git snapshot up to fa21e66c5bd67bcbaf5bec3f7a31358169bb7f1b
    with memory-leak fixes.
  * Take proposed upstream patches to fix desktop-icons w/Shell 3.34. Bump
    dependency version accordingly.

 -- Iain Lane <email address hidden> Wed, 14 Aug 2019 10:41:19 +0100

[Daniel van Vugt (vanvugt)]

gnome-shell-extension-desktop-icons (19.01.3+git20190814.1-1) experimental; urgency=medium

  * New upstream git snapshot up to fa21e66c5bd67bcbaf5bec3f7a31358169bb7f1b
    with memory-leak fixes.
  * Take proposed upstream patches to fix desktop-icons w/Shell 3.34. Bump
    dependency version accordingly.

 -- Iain Lane <email address hidden> Wed, 14 Aug 2019 10:41:19 +0100

[Daniel van Vugt (vanvugt)]

I think we might go via bug 1842904 before fixing this in disco.

[Sebastien Bacher (seb128)]

Closing as wontfix for Disco, that's not a LTS serie and 19.10 is about to turn stable, for those who need the fix it's recommended to upgrade to the new serie

[Daniel van Vugt (vanvugt)]

Just retested the pathological case of this leak, which involves creating and deleting desktop files as fast as you can (thousands of times) via a script. In that case even the latest gnome-shell-extension-desktop-icons still leaks badly. So the fix I had hopes for is not enough:

  https://gitlab.gnome.org/World/ShellExtensions/desktop-icons/-/commit/16447ab5bebcd4efb792e25335438f43b503c3e6

Of course this bug was already closed because a bigger leak was successfully fixed. If anyone finds the pathological case in the wild then we should handle it in a new bug.