ldconfig problem with 64-bit nvidia driver packages

[Frans Elliott]

On Ubuntu 13.04 64-bit, the nvidia-319 package as well as other nvidia driver pakcages (including nvidia-current and nvidia-current-updates from the main ubuntu package repositories) don't play nice with applications that use 32-bit OpenGL libraries. When such an application tries to load the nvidia 32-bit libGL.so library, they don't load the nvidia libGL.so but some other non-nvidia libGL.so. This means a 32-bit application running on Ubuntu Raring 64-bit will never use the nvidia libGL.so library.

After a lot of monkey business, I figured out what the problem is. Which libGL.so a 32-bit application uses on a 64-bit system is controlled by the file /etc/ld.so.conf.d/i386-linux-gnu_GL.conf which is just a symlink to /etc/alternatives/i386-linux-gnu_gl_conf which is another symlink to /usr/lib/nvidia-319/alt_ld.so.conf or something similar for other versions of the nvidia drivers. This file is blank in a clean install, so ldconfig doesn't know that 32-bit applications asking for libGL.so should get linked with the version in /usr/lib32/nvidia-319, so it links them with some other version. This breaks 32-bit programs like Steam (which throws a warning about not using direct rendering when it starts up) and 32-bit games like Team Fortress 2 (which fails to start altogether). Of course, this isn't specific to Steam and Team Fortress 2 but will affect any 32-bit program trying to use libGL.so and its associated libraries on a 64-bit machine.

fortunately, the fix for this is pretty simple: in the file /usr/lib/nvidia-319/alt_ld.so.conf in the nvidia-319 package add the lines:

/usr/lib32/nvidia-319
/usr/lib/nvidia-319

This will tell ldconfig to use the nvidia libGL.so and associated libraries for 32-bit applications. For the other versions of the nvidia driver (such as nvidia-304 and nvidia-304-updates in the main ubuntu repositories) it's a simple matter of replacing nvidia-319 with nvidia-304, nvidia-304-updates, nvidia-313, etc. in the directory names.

Although I reported this bug for the nvidia-319 package, I have checked and this bug applies at least to the nvidia-304 and nvidia-304-updates packages for Ubuntu 13.04 amd64 in the main ubuntu package repositories as well. It probably also applies to other nvidia driver packages as well, but I haven't checked those. I suppose all you'd have to look for is a blank /usr/lib/<nvidia driver package name>/alt_ld.so.conf file to check for the bug.

The file I've included is the output of ldd (which prints the dynamic library dependencies of a program) when it's used on a 32-bit program that needs the 32-bit libGL.so on an Ubuntu 13.04 64-bit machine. In this case, the program I used ldd on is the 32-bit version of glxinfo from mesa-utils:i386. The nvidia driver version I had installed when I did this is nvidia-319. However, you will get a very similar result for other nvidia driver versions/packages.

Also, I said "I don't know" for the package because 1.) it affects multiple packages and 2.) it didn't accept the package name nvidia-319.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Mitten Orvan (mitten-o)]

This issue appeared for me after upgrading to 14.04.
I had some driver issues after the upgrade and had to reinstall them.
After that, steam gave me the no direct glx error and games would not start.
The procedure described solved the issue.

[Bruno Nova (brunonova)]

I've looked at the source code of nvidia-graphics-drivers.
At around line 300 of debian/rules there is this code:

 # ld.so.conf
 dh_installdirs -p$(PKG_driver) "$(ld_so_conf_dir)"
 echo "$(PKG_libdir)" > "$(CURDIR)/debian/$(PKG_driver)$(ld_so_conf_path)"
ifeq ($(DEB_BUILD_ARCH),amd64)
 echo "$(PKG_libdir32)" >> $(CURDIR)/debian/$(PKG_driver)$(ld_so_conf_path)
endif
 # empty ld.so.conf for the fake multi-arch alternative
 $(shell touch "$(CURDIR)/debian/$(PKG_driver)$(alt_ld_so_conf_path)")

The "touch" in the last line is probably why the alt_ld.so.conf file exists but is empty.
But why make it empty?
Changing these lines to:

 # ld.so.conf
 dh_installdirs -p$(PKG_driver) "$(ld_so_conf_dir)"
 echo "$(PKG_libdir)" > "$(CURDIR)/debian/$(PKG_driver)$(ld_so_conf_path)"
ifeq ($(DEB_BUILD_ARCH),amd64)
 # echo "$(PKG_libdir32)" >> $(CURDIR)/debian/$(PKG_driver)$(ld_so_conf_path)
 echo "$(PKG_libdir32)" > "$(CURDIR)/debian/$(PKG_driver)$(alt_ld_so_conf_path)"
 # echo "$(PKG_libdir)" >> "$(CURDIR)/debian/$(PKG_driver)$(alt_ld_so_conf_path)"
endif

would probably fix this bug. (I've commented 2 of the lines in the "if" block because I don't know if they are really necessary)
At least, after compiling the package with "debuild", the alt_ld.so.conf file inside isn't empty.

Also, in line 122 of file debian/nvidia-current.postinst.in:

        # Deal with multi-arch ugliness until dpkg supports multi-arch:

Doesn't Debian (and dpkg) support multi-arch now?

This bug should have a high importance, as it prevents 32 bits programs (Skype, Steam, etc.) from working in 64 bit systems (at least those using the NVIDIA drivers).
Sure, there are workarounds, but average users aren't expected to find and apply them.

[Bruno Nova (brunonova)]

I've tested this a bit more.
The issue is a lot more serious than I thought!

The workaround provided by the bug reporter probably works for computers using an NVIDIA card. And my fix would probably work too.

However, for computers with Hybrid graphics/NVIDIA Optimus (Intel+NDIVIA), like my laptop, the workaround has terrible consequences:
If I add both lines to alt_ld.so.conf, Unity WON'T START when I login (I get the desktop without Unity).
If I add only the "lib32" line to it, Unity starts and I can start Skype and Steam, but I get errors, at least in Steam, about missing things in display :0 or :0.0 and other types of errors.
But if I start steam with optirun (I use bumblebee), Steam starts fine.

So, the NVIDIA libGL.so.1 version is for NVIDIA graphics ONLY.
People like me who use Hybrid Graphics, and Intel card by default, have to use the Mesa version of the library.
To fix this execute:
$ sudo update-alternatives --config i386-linux-gnu_gl_conf
and select the option that contains "mesa".
Then run "sudo ldconfig".

This issue should definitely be looked into by more experienced people!

[Bruno Nova (brunonova)]

I forgot to tell that I'm using Ubuntu 13.10 and NVIDIA 304 driver (plus bumblebee).

Also, if I run:
$ sudo update-alternatives --config x86_64-linux-gnu_gl_conf
Then select the nvidia option. Then:
$ sudo ldconfig
And then reboot, I can't login. It stays in a black screen after logging in.
Fortunately, the default option for x86_64 is mesa (but not for i386). At least for me.

[Bruno Nova (brunonova)]

The default GL option for x86_64 is mesa because bumblebee selects it during install, using update-alternatives.
But, for some reason, it doesn't do the same for i386.

[Bruno Nova (brunonova)]

@Alberto Milone, sorry to bother you, could you check this bug?

I was trying to understand why the NVIDIA driver package installs an empty alt_ld.so.conf file.
Then I found the commit that added this change by you in Oneiric: http://bazaar.launchpad.net/~ubuntu-branches/ubuntu/utopic/nvidia-graphics-drivers/utopic/revision/44

I still don't understand the reason.
The changelog says:
    Use an empty ld.so.conf when installing alternative for the
    non-native arch, otherwise it would be impossible to switch
    to mesa without installing mesa for the same architecture.
And the subject of the added patch:
    Subject: [PATCH 1/1] Install an empty ld.so.conf for the fake alternative
    This makes sure that we don't point at Nvidia's libraries and
    only prevent from using mesa's ld.so.conf

Is this still necessary?
Right now, users running 64-bit systems who install the NVIDIA driver are not able to run 32-bit programs (like Steam and Skype) because of that empty alt_ld.so.conf.

Maybe that patch should be dropped or corrected?

Also, I saw a comment in debian/nvidia-current.postinst.in saying:
    # Deal with multi-arch ugliness until dpkg supports multi-arch:
Doesn't Debian (and dpkg) support multi-arch now?

[Alberto Milone (albertomilone)]

Driver packages are not multi-arch. The 64 bit package provides both 64 bit and 32 bit libraries. The empty alternative is meant to avoid that, when nvidia is installed, users end up using nvidia's 64 bit libraries and mesa's 32 bit libraries.

As for the NVIDIA Optimus case, when in power saving mode (which you can select from nvidia-settings), the alternatives will be set to the "prime" option, which will make sure that only Mesa's GL libraries for both 64 bit and 32 bit are used, provided that you have the 32 bit version of mesa installed.

This is dealt with automatically, and users shouldn't select alternatives manually.

For example, when /usr/lib/nvidia-331-prime/ld.so.conf is selected, it will add the following:
/usr/lib/x86_64-linux-gnu/mesa
/usr/lib/i386-linux-gnu/mesa

This is what happens when /usr/lib/nvidia-331/ld.so.conf and /usr/lib/nvidia-331/alt_ld.so.conf are in use:

ldconfig -p | grep GL
        libQt5OpenGL.so.5 (libc6,x86-64) => /usr/lib/x86_64-linux-gnu/libQt5OpenGL.so.5
        libGLU.so.1 (libc6,x86-64) => /usr/lib/x86_64-linux-gnu/libGLU.so.1
        libGLESv2.so.2 (libc6,x86-64) => /usr/lib/x86_64-linux-gnu/mesa-egl/libGLESv2.so.2
        libGL.so.1 (libc6,x86-64) => /usr/lib/nvidia-331/libGL.so.1
        libGL.so.1 (libc6) => /usr/lib32/nvidia-331/libGL.so.1
        libGL.so (libc6,x86-64) => /usr/lib/nvidia-331/libGL.so
        libGL.so (libc6) => /usr/lib32/nvidia-331/libGL.so
        libEGL.so.1 (libc6,x86-64) => /usr/lib/x86_64-linux-gnu/mesa-egl/libEGL.so.1

This is what happens when /usr/lib/nvidia-331-prime/ld.so.conf and /usr/lib/nvidia-331-prime/alt_ld.so.conf are in use:

when 32 bit mesa is not installed:

# ldconfig -p | grep GL
        libQt5OpenGL.so.5 (libc6,x86-64) => /usr/lib/x86_64-linux-gnu/libQt5OpenGL.so.5
        libGLU.so.1 (libc6,x86-64) => /usr/lib/x86_64-linux-gnu/libGLU.so.1
        libGLESv2.so.2 (libc6,x86-64) => /usr/lib/x86_64-linux-gnu/mesa-egl/libGLESv2.so.2
        libGL.so.1 (libc6,x86-64) => /usr/lib/x86_64-linux-gnu/mesa/libGL.so.1
        libEGL.so.1 (libc6,x86-64) => /usr/lib/x86_64-linux-gnu/mesa-egl/libEGL.so.1

when 32 bit mesa is installed:

# ldconfig -p | grep GL
        libQt5OpenGL.so.5 (libc6,x86-64) => /usr/lib/x86_64-linux-gnu/libQt5OpenGL.so.5
        libGLU.so.1 (libc6,x86-64) => /usr/lib/x86_64-linux-gnu/libGLU.so.1
        libGLESv2.so.2 (libc6,x86-64) => /usr/lib/x86_64-linux-gnu/mesa-egl/libGLESv2.so.2
        libGL.so.1 (libc6,x86-64) => /usr/lib/x86_64-linux-gnu/mesa/libGL.so.1
        libGL.so.1 (libc6) => /usr/lib/i386-linux-gnu/mesa/libGL.so.1
        libEGL.so.1 (libc6,x86-64) => /usr/lib/x86_64-linux-gnu/mesa-egl/libEGL.so.1

Maybe you're missing the libgl1-mesa-glx:i386 package?

NOTE: we do not support Optimus systems with the 304 nvidia driver and we do not support Bumblebee at all.

[Bruno Nova (brunonova)]

@Alberto, sorry to confuse you.
I am using Bumblebee, but I have everything working now. The alternatives are pointing to the Mesa GL libraries, nvidia-331 installed and libgl1-mesa-glx:i386 installed (though it's not installed by default nor by bumblebee, so I reported Bug #1383909 there).
But forget what I said in the other comments about Bumblebee. I was referring to the NVIDIA drivers only.

As I don't have a working computer with NVIDIA only, I can't test this myself.
But the bug reporter and another commenter are/were affected. There are also several people marked as "affected".
And I can see other people in the Web using an NVIDIA card complaining about Steam and Skype not finding the 32-bit libGL.so.1 or that it has "wrong ELF class" (because /usr/lib is listed before /usr/lib32, they say), in Trusty or previous releases.

Both /usr/lib/nvidia-XXX/alt_ld.so.conf and /usr/lib/nvidia-XXX-prime/alt_ld.so.conf are empty in my system.
If this is to avoid using 32-bit Mesa libraries, couldn't /usr/lib/nvidia-XXX/alt_ld.so.conf be pointing to the NVIDIA 32-bit libraries (like the bug reporter suggested)?
Also, /usr/lib/nvidia-XXX-prime/alt_ld.so.conf could point to the 32-bit Mesa libraries (but they are not installed by default, so the nvidia-prime could recommend libgl1-mesa-glx:i386).

But you know more about this than me.

I tried configuring the alternatives to use NVIDIA's ld.so.conf and alt_ld.so.conf, and "ldd ~/.steam/bin/steamui.so | grep GL" actually finds the NVIDIA's 32-bit libGL.so.1. Maybe it's working correctly after all.
Is any Ubuntu 14.04+ user with an NVIDIA card (no Optimus) still facing this issue?

[Alberto Milone (albertomilone)]

On 22-10-14 10:28:15, Bruno Nova wrote:
> As I don't have a working computer with NVIDIA only, I can't test this
> myself.
> But the bug reporter and another commenter are/were affected. There are also several people marked as "affected".
> And I can see other people in the Web using an NVIDIA card complaining about Steam and Skype not finding the 32-bit libGL.so.1 or that it has "wrong ELF class" (because /usr/lib is listed before /usr/lib32, they say), in Trusty or previous releases.
>
> Both /usr/lib/nvidia-XXX/alt_ld.so.conf and /usr/lib/nvidia-XXX-prime/alt_ld.so.conf are empty in my system.
> If this is to avoid using 32-bit Mesa libraries, couldn't /usr/lib/nvidia-XXX/alt_ld.so.conf be pointing to the NVIDIA 32-bit libraries (like the bug reporter suggested)?

It wouldn't change anything, since /usr/lib/nvidia-331-prime/ld.so.conf
already includes both the 32 bit and the 64 bit libraries. Splitting the
two paths into two separate files won't make any difference.

> Also, /usr/lib/nvidia-XXX-prime/alt_ld.so.conf could point to the 32-bit Mesa libraries (but they are not installed by default, so the nvidia-prime could recommend libgl1-mesa-glx:i386).
>

I don't think I can make a 64 bit package depend on (or recommend) a 32
bit package.

[Bruno Nova (brunonova)]

Ok, you're right.
I thought that /usr/lib/nvidia-331-prime/ld.so.conf was used only for 64-bit, but it is indeed used for 32-bit as well.
Same thing for /usr/lib/nvidia-331-prime/ld.so.conf, which point to both 64 and 32 bit mesa libraries.

As I said, you know more about this than me. :)
Sorry to bother you then. You may want to unsubscribe yourself from this bug.

[HitmanNumber86 (hitmannumber86)]

Had a problem with this bug. Found a fix here, http://steamcommunity.com/app/221410/discussions/5/828939163894026239/?l=german

[Lars (ak-lars)]

This did help. It got rid of the error! No guarantees as of yet as to whether my video games work or not, but doing the procedure as indicated in the helpfile got rid of the OpenGL error message on the startup of STEAM. It is gone now.