Nvidia drivers lead to extra memory usage for each process using libGL

[Doug McMahon]

the new nvidia-current is causing a 300% increase in memory use at login as compared to previously and or as currently with nouveau 3d drivers.
While not a big deal on machines with 2+GB ram on older machines with 1 GB it does represent a bit of an issue.
The increases are pretty much seen with all process, ie. 'across the board'
Attached are single report logs, each from fresh boot
The 3GB are from this reporting hardware, the 1GB are from a P4 with nvidia 7800 AGP and 1GB ram

ProblemType: Bug
DistroRelease: Ubuntu 11.04
Package: nvidia-current 270.29-0ubuntu3
ProcVersionSignature: Ubuntu 2.6.38-5.32-generic 2.6.38-rc6
Uname: Linux 2.6.38-5-generic i686
NonfreeKernelModules: nvidia
.proc.driver.nvidia.gpus.0: Error: [Errno 21] Is a directory: '/proc/driver/nvidia/gpus/0'
.proc.driver.nvidia.registry: Binary: ""
.proc.driver.nvidia.version:
 NVRM version: NVIDIA UNIX x86 Kernel Module 270.29 Wed Feb 23 16:16:53 PST 2011
 GCC version: gcc version 4.5.2 (Ubuntu/Linaro 4.5.2-3ubuntu3)
Architecture: i386
CompizPlugins: [core,bailer,detection,composite,opengl,decor,mousepoll,vpswitch,regex,animation,snap,expo,move,compiztoolbox,place,grid,gnomecompat,wall,ezoom,workarounds,staticswitcher,resize,fade,scale,session,unityshell]
CompositorRunning: compiz
Date: Sat Feb 26 01:37:02 2011
DistUpgraded: Fresh install
DistroCodename: natty
DistroVariant: ubuntu
DkmsStatus: nvidia-current, 270.29, 2.6.38-5-generic, i686: installed
GraphicsCard:
 nVidia Corporation G86 [GeForce 8400M GS] [10de:0427] (rev a1) (prog-if 00 [VGA controller])
   Subsystem: Dell Device [1028:0209]
InstallationMedia: Ubuntu 11.04 "Natty Narwhal" - Alpha i386 (20110202)
MachineType: Dell Inc. XPS M1330
ProcEnviron:
 LANGUAGE=C:en_US:en
 LANG=en_US.UTF-8
 LC_MESSAGES=en_GB.utf8
 SHELL=/bin/bash
ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-2.6.38-5-generic root=UUID=9aabd666-add7-49fd-b17c-780c59823b85 ro quiet splash vt.handoff=7
Renderer: Unknown
SourcePackage: nvidia-graphics-drivers
dmi.bios.date: 12/26/2008
dmi.bios.vendor: Dell Inc.
dmi.bios.version: A15
dmi.board.vendor: Dell Inc.
dmi.chassis.type: 8
dmi.chassis.vendor: Dell Inc.
dmi.modalias: dmi:bvnDellInc.:bvrA15:bd12/26/2008:svnDellInc.:pnXPSM1330:pvr:rvnDellInc.:rn:rvr:cvnDellInc.:ct8:cvr:
dmi.product.name: XPS M1330
dmi.sys.vendor: Dell Inc.
version.compiz: compiz 1:0.9.4-0ubuntu1
version.libdrm2: libdrm2 2.4.23-1ubuntu3
version.libgl1-mesa-glx: libgl1-mesa-glx 7.10.1~git20110215.cc1636b6-0ubuntu2
version.nvidia-graphics-drivers: nvidia-graphics-drivers N/A
version.xserver-xorg: xserver-xorg 1:7.6~3ubuntu8
version.xserver-xorg-video-ati: xserver-xorg-video-ati 1:6.14.0-0ubuntu2
version.xserver-xorg-video-intel: xserver-xorg-video-intel 2:2.14.0-1ubuntu11
version.xserver-xorg-video-nouveau: xserver-xorg-video-nouveau 1:0.0.16+git20110107+b795ca6e-0ubuntu5

[Bryce Harrington (bryce)]

Hi Doug,

Thanks for reproducing the issue on two computers. I could understand the X process consuming more memory but it's a bit weird that even client apps take more memory.

Do you happen to know if this is a regression from the previous -nvidia (from the xserver 1.9 days)?

If you uninstall nvidia and go back to nouveau, does the memory go back down to the low levels?

[Doug McMahon (mc3man)]

 >If you uninstall nvidia and go back to nouveau, does the memory go back down to the low levels?

Yes - the 2 mesa3d "1GB" logs attached are reflective of both prior to installing nvidia-current and removing nvidia-current and returning to mesa3d drivers (the "mesa3d_1GB log orig. attached to report itself

(On my 1GB machine have returned to mesa3d for the moment and all is well in these regards

> Do you happen to know if this is a regression from the previous -nvidia (from the xserver 1.9 days)?

I'm fairly sure that use at boot was normal then, though to maintain testing 'integrity', I updated X when it came so can't say 'for sure'

If helpful I can install an earlier .iso, revert X if need be and install prior nvidia-current from the archive.
(could also try the current nvidia-current if it accepts the 1.9 X, thought I saw mention that it could

The other possibility is I'd assume 1 more nvidia-current upgrade, could test that. (latest 270.30 +

[Doug McMahon (mc3man)]

Went ahead and did an A2 install, reverted xserver to 1.9, installed nvidia-current 260.19.21-0ubuntu2
(only upgrade was to the kernel to get away from the udev - kernel panic that was quite likely with the hardware here.
Memory use at login was pretty much normal, both for mesa3d and then with nvidia-current installed

Upgraded unity, compiz,nux and deps, no change
Then upgraded all but xserver and nvidia-current, levels at login went up to basically what was shown in logs previously attached for nvidia.

So - can pretty much say this is NOT the current xserver ,nvidia-current, unity, compiz , nux and deps that is causing this, at least by themselves.
No sense attaching process logs, will attach a free -m set w/ some description, all were from restarts, - (using the machine w/ 3 GB because it was easier to do this on.
The difference between steps 2 and 3 was around 400+ packages, if there was any reason to do so could probably narrow down the cause/trigger to a small set of packages,.

Note that w/ fresh A3 installs the numbers are a little bit higher, (5% or so ), than what was seen previously in attached logs for logins with nvidia-current, ie. A3 doesn't represent any improvement

For the most part have decided to just use nouveau 3d on a 1GB machine, on machines w/ 2GB or more this whole deal is pretty much irrelevant anyway

[Bryce Harrington (bryce)]

Wow, thanks for all the additional testing.

This suggests there is some other plumbing-layer component that changed between A2 and A3 which causes the increased memory when nvidia is loaded. That actually makes a bit more sense, because it is a little hard to imagine how -nvidia by itself could increase memory usage of everything else in the system (I can sort of imagine it increasing memory loads for GUI apps, but even that is a bit weird.)

[Doug McMahon (mc3man)]

This is the grouping that after install causes the increased mem at login (current xserver, nvidia-current, ect.
Don't have time to ck. reverting individually and can't be sure if there is any other relationship to upgrades previous to this group. (wasn't a group that seemed clearly obvious

Upgraded the following packages:
banshee (1.9.2-1ubuntu1) to 1.9.4-1ubuntu2
banshee-extension-soundmenu (1.9.2-1ubuntu1) to 1.9.4-1ubuntu2
banshee-extension-ubuntuonemusicstore (1.9.2-1ubuntu1) to 1.9.4-1ubuntu2
libcairo2 (1.10.2-1ubuntu1) to 1.10.2-2ubuntu1
libubuntuone1.0-cil (0.3.9-0ubuntu2) to 0.3.11-0ubuntu1

Installed the following packages:
libgdata1.7-cil (1.7.0.1-1)

[Doug McMahon (mc3man)]

confirmed here that the libcairo2 upgrade is causing the mem increase w/ nvidia

[Bryce Harrington (bryce)]

Thanks for narrowing down the cause of the bug to cairo. I'll close out the -nvidia portion of the bug.

[Sebastien Bacher (seb128)]

the difference in that version of cairo is that the gl backend required for wayland has been enabled, seems an issue that should be raise upstream then

[Doug McMahon (mc3man)]

Just to see did go ahead and package/install the current 1.10.2 source without gl enabled, it brought the mem use at log in back to the expected amount. (246 vs 602MB, unity login on this machine, slightly greater diff on the affected p4 1GB

As it stands now, machines w/ 2GB or more doesn't much matter, w/ 1GB can be managed though usage, time online, mem leaks, what gets swapped, ect. will be a factor.
Those w/ less than 1GB should just use nouveau (mesa3d

[Chris Halse Rogers (raof)]

Ok. So, a little bit more digging shows that the mere act of loading nVidia's libGL dirties ~5MiB of memory on AMD64. Multiplied by each process linking to cairo that's going to add up to real money pretty soon.

Unless we get cairo to dynamically load libGL when an application requests a GL context this is not going away without nvidia changing their drivers.

[Chris Halse Rogers (raof)]

And looking at cairo-gl, dynamic loading libGL is not a week-before-beta change to be making.

[Martin Pitt (pitti)]

Sebastien is currently working on splitting out the GL backend for natty, to work around this. Therefore this isn't release critical any more for the nvidia driver.

[Doug McMahon (mc3man)]

I've been using the cairo -1.10.2 redone with '--enable-gl' removed and adjusted symbols for 4 or 5 days now and can say absolutely no issues seen
And as noted before mem use is now as expected

[Sebastien Bacher (seb128)]

So I've been working for a bit on trying to make libcairo built with gl co-installable with the standard libcairo in a way that would allow tweaking the ld paths to use the gl version but hit quite some issues on the way.

I'm attaching a debdiff of the tweaked version with some comments of the limitations of the approch, if someone wants to pick up feel free, to me it seems to start being hackish and work over what is reasonable to spend to just support wayland in universe for natty.

[Sebastien Bacher (seb128)]

So I've been working for a bit on trying to make libcairo built with gl co-installable with the standard libcairo in a way that would allow tweaking the ld paths to use the gl version but hit quite some issues on the way.

I'm attaching a debdiff of the tweaked version with some comments of the limitations of the approch, if someone wants to pick up feel free, to me it seems to start being hackish and work over what is reasonable to spend to just support wayland in universe for natty.

[Sebastien Bacher (seb128)]

Summary of what was discussed and tested:

* shipping a libcairo2-gl which would conflicts,replaces,provides libcairo2 wouldn't work since provides are not versioned and that wouldn't satisfy the dependencies of most of binaries dependy currently on libcairo2
-> we need a new binary installable at the same time as libcairo2 and tweak ld paths

* the rules was updated to add a "gl" build flavor, it means cairo get built without gl and and an extra time with gl

* copying the gl build library in a new directory, that would work for runtime but is not enough for builds since the .pc and the .so don't point to the right variant, building with gl also creates some new cairo-gl.pc which need to be installed
-> we need a libcairo-gl-dev binary, the binary needs to be installable at the same time than libcairo-dev due to the same reason than the library (the provides are not versioned which would mean it can't be installed as a replacement)

* installing only the extra .pc in a different path wouldn't work since that would lead to inconsistencies in the cairo installation
-> the standard path needs to be used for those .pc

* If the .pc don't point to the right directory to build with gl it would mean wayland softwares that need cairo-gl would need hacks to point to the right directory

* The libcairo-gobject library should build with the correct variant to avoid have 2 different libcairo be loaded by libcairo-gobject users so we would probably need a libcairo-gobject-gl as well

The work starts being non trivial from there and it will lead to a solution that need hacks in softwares that want to use cairo-gl, does anybody has a better suggestion on what to do next?

[Sebastien Bacher (seb128)]

one hackish way that would be easier would be to just copy the library built with gl in a different path and dpkg-divert the library to that one, not sure it's something we should do though

[macstevejb (macstevejb)]

Still an enormous RAM increase here using the nvidia proprietary driver (the latest nvidia-current) on an older P4 machine running 2 Gb Ram with an nvidia Geforce 6100 graphics card.

It's really annoying, as I would imagine it is for anyone using nvidia graphics with Ubuntu 11.04...any idea when it will be fixed bearing in mind final release is now only weeks away?

regards

[Alessandro Rinaldi (alerinaldi)]

It's really worrying that noone is the assigner of this bug, when we are nearly reaching the release day...
For me, installing package libcairo2 from Maverick (after modifying the version number in the DEB to avoid automatic upgrading) fixed the problem without evident regressions... But I didn't try Wayland, and I know it probably won't load...
But I think that having this problem fixed in Xorg is more urgent than making Wayland working, at this point of the release schedule!
So, I think, if noone wants to fix this in time, that it would be better to bring back the Maverick library in Natty... At least until the problem will be fixed!
Trust me, noone will use Wayland atm!

[sarton85 (sarton85)]

I also have an enormous RAM increase since I updated my 10.10 system to 11.04 beta. Whenever I did a fresh reboot of my system it was using under 300MB RAM from Ubuntu 6.06 to 10.10. Now it is using around 800MB RAM after a fresh reboot and is going up to 1.3 GB RAM with just Firefox 4 and Empathy open. As I am on a 1.5 GB RAM system it is also starting to use SWAP space to cope with the increase.

This memory increase is causing the whole system to slow down enormously and applications to crash often.

I am using the Nvidia Proprietary drivers and have a Nvidia Geforce 6100 graphics card.

Hope this bug will be released before the final release. Otherwise it is going to affect a lot of users.

[sarton85 (sarton85)]

* Hope this bug will be resolved before the final release.

[sarton85 (sarton85)]

A link to a bug report where I posted about this bug before; https://bugs.launchpad.net/ubuntu/+source/unity/+bug/735482

[Edmond Thiebaud (thiebaude-hotmail)]

I also would like to confirm this bug. Im using a AMD Opteron Dual-core and I have a Nvidia 8400 GS 512mb video ram card. Let me know if you need any other information that I can provide, so we can squash this before release date, thanks everyone.

[Justin (deadite81)]

I got tired of nouveau freezing my desktop (at least I assume that's what it was, maybe it's just compiz itself) and downgraded libcairo2 to the "maverick-updates" version ( http://packages.ubuntu.com/maverick-updates/libcairo2 ). It's been about an hour and my computer hasn't exploded yet. I'm using the proprietary Nvidia driver with an onboard 6150SE.

I have 2gb RAM, and contrary to other opinions, it affected my system very noticeably and forced much swapping whereas none was needed before. I have a 2.5Ghz dual core (albeit a bit dated) and programs that normally open instantly were, well, not. Hope this gets fixed soon!

[Martin Pitt (pitti)]

At this point I'm afraid we'll disable the EGL support in cairo and move the EGL enabled one to a PPA.

[Launchpad Janitor (janitor)]

This bug was fixed in the package cairo - 1.10.2-2ubuntu2

---------------
cairo (1.10.2-2ubuntu2) natty; urgency=low

  * Don't turn the gl backend on for natty since it creates issues for nvidia
    users and is only used for wayland (lp: #725434)
  * debian/control,
    debian/libcairo2.symbols,
    debian/rules:
    - updated for the non-gl build
  * debian/control: Breaks on wayland
 -- Sebastien Bacher <email address hidden> Fri, 08 Apr 2011 18:27:43 +0200

[Edmond Thiebaud (thiebaude-hotmail)]

Yes thank you very much, i just got the latest updates, and can confirm it.Ram usage is normal for me now and i can enjoy unity now. :)

[sarton85 (sarton85)]

After updating the problem also disappeared from my system. I can also run Unity 3D as normal. Thanks, great work!

[VinDSL (perfect-pecker)]

Me too!

RAM (and CPU) usage is back to normal.

Good job! And, thanks!!!

[macstevejb (macstevejb)]

Ditto..this is the one bug that held me back from making Natty my default distro.

RAM usage back to normal here too.

Thanks for fixing! :)

[sarton85 (sarton85)]

I just said that the problem disappeared from my system, but this turns out only to be partly true. When the Unity desktop is on for a few hours it seems like some processes are slowly increasing memory. Like compiz started at 70 MB Ram, but increased to 160 MB Ram in a few hours.

I have just restarted my system because it was again using more than 800 MB Ram, with just Firefox open. Not 1.3 GiB Ram like before, but still to much, which is making my system getting slower and slower the longer it is running.

[Justin (deadite81)]

I too am experiencing the compiz memory leak now. I left my computer on over night and when I came back compiz was taking 341MB of RAM. After doing a compiz --replace it went back down to 24MBs. It's seems directly related to this, but should it be a new bug?

[Justin (deadite81)]

Oh, and this was in the "Ubuntu Classic" session, so I guess it has nothing to do with Unity.

[Jeremy Bícha (jbicha)]

Any unity/compiz memory leaks are different issues and should be a new bug (if a bug for it has not already been reported).

[Doug McMahon (mc3man)]

Jeremy Bicha wrote
> Any unity/compiz memory leaks are different issues and should be a new bug ...
bug #720446

[ccaaatt (ccaaatt)]

http://wayland.freedesktop.org/building.html :
"
cairo-gl
The Wayland clients can render using cairo-gl, but fall back to software when cairo-gl is not available.
"

So cairo (1.10.2-2ubuntu2) may didn't break wayland, I guess

[komandan (muamalhamidy)]

saya juga menggunakan nvidia
kira-kira masalhnya dimana yaa....

[Bryce Harrington (bryce)]

"So cairo (1.10.2-2ubuntu2) may didn't break wayland, I guess"

Regardless, seb128 added a 'breaks' in the packaging against wayland, so having cairo installed now makes wayland uninstallable.

[Doug McMahon (mc3man)]

Well , 11.10 has gone back today to using the --enable-gl option in cairo, so this again is a concern for nvidia users , particularly 1GB machines
Add in the still active small per use mem leaks in unity/compiz and these machines will quickly become unusable.

[Dan Andreșan (danyer)]

I have 11.10 on AMD Athlon 3500+, 1GB RAM and Nvidia 6200.

Seems this is not enough for Oneiric :(

Without X my usage is 134Mb (or something).
With X at lightdm-greeter 230 Mb
With a session started, 700 Mb.

If I start another app or two, I starts trashing.... swap memory usage increases, etc.

I read all comment, NVIDIA is guily, I know.
Don't forget to update the minimum requirements for Oneiric then when using it with the proprietary drivers :(

[Doug McMahon (mc3man)]

To add - with the latest 11.10 nvidia-current this is another consequence (apparent) of the gl enabled cairo
Bug #821702

[Alberto Milone (albertomilone)]

The problem is solved in “cairo” 1.10.2-6ubuntu3 where we disabled the gl backend again.

[Dàrent (animaletdesequia)]

I know this is old and most of you must be running 11.10 now, but I just installed 11.04 and I'm having this problem. The last version of libcairo2 in the repos is 1.11.3, which gives me a ram usage about 1.5Gb just after login. I've forced version 1.10.2 and everything is fine now, ram under 500Mb.

Another problem I noticed is some apps don't close properly with the latest libcairo2. I have a transparent gnome-terminal set as desktop background and if I close it (with right click, exit command, or even xkill) the process remains in top using ram). Also that prevents me from logout or reboot the system (it keeps telling me than some apps refuse to close). Don't know how is this related to the graphic drivers but forcing the older libcairo2 solved it.

If somebody is still taking care of this bug, can you tell me if there is a more stable solution other than maintain an older package frozen?

Sorry for my bad English :)

[Mantas Kriaučiūnas (mantas)]

Dàrent (animaletdesequia) wrote on 2011-11-14:
> I know most of you must be running 11.10 now, but I just installed 11.04 and I'm having this problem.
> The last version of libcairo2 in the repos is 1.11.3, which gives me a ram usage about 1.5Gb just after login.
> I've forced version 1.10.2 and everything is fine now, ram under 500Mb.

Last libcairo2 version in Ubuntu 11.04, 11.10 and even 12.04 is 1.10.2 - see https://launchpad.net/ubuntu/+source/cairo or http://packages.ubuntu.com/libcairo2

It seems you installed unofficial libcairo2 package from other repositories, not from Ubuntu if you had 1.11.3 version.

[Darxus (darxus)]

Nvidia proposed a solution to this problem yesterday, to reduce the memory usage of their driver:
http://lists.freedesktop.org/archives/wayland-devel/2012-March/002798.html

Opened as bug #967149.

Related bug to enable the wayland backend in GTK+: bug #954352.

[Stu (stu-axon)]

The fix for this has been available for a long time, if it works - can the cairo gl backend be re-enabled? It's been disabled for nearly 3 years now, in the meantime it is available on fedora.

This is affecting an app I maintain as I can't use it with cairo-gl for now.