Using pyopencl doesn't work and spits our ImportError

[Adrian Moisey]

Pyopencl doesn't work on raging, the following error message is generated:

Python 2.7.4 (default, Apr 19 2013, 18:32:33)
[GCC 4.7.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pyopencl
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/dist-packages/pyopencl/__init__.py", line 4, in <module>
    import pyopencl._cl as _cl
ImportError: /usr/lib/python2.7/dist-packages/pyopencl/_cl.so: symbol clRetainDevice, version OPENCL_1.2 not defined in file libOpenCL.so.1 with link time reference

I am using the following drivers:
adrian@toad:~$ dpkg -l | grep nvidia
ii nvidia-304-updates 304.88-0ubuntu2 i386 NVIDIA binary Xorg driver, kernel module and VDPAU library
ii nvidia-cg-dev:i386 3.1.0013-1 i386 Cg Toolkit - GPU Shader Authoring Language (headers)
ii nvidia-cg-toolkit 3.1.0013-1 i386 Cg Toolkit - GPU Shader Authoring Language
ii nvidia-common 1:0.2.76 i386 transitional package for ubuntu-drivers-common
ii nvidia-settings-304-updates 304.88-0ubuntu1 i386 Tool for configuring the NVIDIA graphics driver

ProblemType: Bug
DistroRelease: Ubuntu 13.04
Package: python-pyopencl 2012.1-1ubuntu4
ProcVersionSignature: Ubuntu 3.8.0-19.29-generic 3.8.8
Uname: Linux 3.8.0-19-generic i686
NonfreeKernelModules: nvidia
ApportVersion: 2.9.2-0ubuntu8
Architecture: i386
Date: Mon Apr 29 09:56:38 2013
MarkForUpload: True
ProcEnviron:
 TERM=xterm
 PATH=(custom, no user)
 XDG_RUNTIME_DIR=<set>
 LANG=en_ZA.UTF-8
 SHELL=/bin/bash
SourcePackage: pyopencl
UpgradeStatus: Upgraded to raring on 2013-04-27 (1 days ago)

[Graham Inggs (ginggs)]

I am able to import pyopencl on raring without error, although I do have nvidia-313-updates and nvidia-cuda-toolkit installed.

$ python
Python 2.7.4 (default, Apr 19 2013, 18:28:01)
[GCC 4.7.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pyopencl._cl as _cl
>>>

$ dpkg -l | grep nvidia | grep ii
ii nvidia-313-updates 313.30-0ubuntu1 amd64 NVIDIA binary Xorg driver, kernel module and VDPAU library
ii nvidia-common 1:0.2.76 amd64 transitional package for ubuntu-drivers-common
ii nvidia-cuda-dev 5.0.35-4ubuntu1 amd64 NVIDIA CUDA development files
ii nvidia-cuda-doc 5.0.35-4ubuntu1 all NVIDIA CUDA and OpenCL documentation
ii nvidia-cuda-gdb 5.0.35-4ubuntu1 amd64 NVIDIA CUDA GDB
ii nvidia-cuda-toolkit 5.0.35-4ubuntu1 amd64 NVIDIA CUDA toolkit
ii nvidia-nsight 5.0.35-4ubuntu1 amd64 NVIDIA Nsight Eclipse Edition
ii nvidia-opencl-dev 5.0.35-4ubuntu1 amd64 NVIDIA OpenCL development files
ii nvidia-profiler 5.0.35-4ubuntu1 amd64 NVIDIA Profiler for CUDA and OpenCL
ii nvidia-settings-313-updates 313.30-0ubuntu1 amd64 Tool for configuring the NVIDIA graphics driver
ii nvidia-visual-profiler 5.0.35-4ubuntu1 amd64 NVIDIA Visual Profiler

[Stefano Rivera (stefanor)]

This appears to be i386-specific

[Stefano Rivera (stefanor)]

So, on i386, the linker isn't finding ocl-icd-libopencl1 before the Nvidia driver, because i386-linux-gnu_GL.conf sorts before i686-linux-gnu.conf

[Graham Inggs (ginggs)]

I see there is a Debian bug [1] about explicit ordering of /etc/ld.so.conf.d/*.conf files.
Maybe we can change the name of the file to i386-linux-gnu.conf in libc6-i386 as a workaround?

[1] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=685706

[Graham Inggs (ginggs)]

I think the problem here is that we were allowing ocl-icd-libopencl1 and the Nvidia libopencl1 to be installed together.
There should only be one libopencl1 (OpenCL management library) installed on the system, but there can be more than one opencl-icd (Installable Client Driver).

This has been fixed to a certain extent in the current Nvidia drivers in Saucy as the Nvidia drivers now provides and conflicts on libopencl1. However, since the Nvidia driver includes libopencl1 and opencl-icd in one package, ocl-ic-libopencl, which pyopencl is built against, is not installable.

We need to separate libopencl1 out of the Nvidia (and AMD) drivers, or remove them completely.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Gijs Molenaar (gijzelaar)]

I'm having the same problem on 13.10 64 bit. I have nvidia-319 installed (and I have a Nvidia card).

[Rebecca Palmer (rebecca-palmer)]

Confirmed that on 13.10 this also affects amd64 (probably because the Conflicts: mentioned above caused ocl-icd-libopencl1 to be removed), and that it can be fixed by deleting the nvidia libopenCL.so* files and installing the ocl-icd-libopencl1 ones (I manually copied them to /usr/local/lib and ran ldconfig, but suspect force-installing the package would also work).

I agree that taking libopencl1 out of nvidia-319 (already done in Debian) would be a good way to fix this.

[sfc (sfc-0)]

13.10 amd64 here, also the same problem.
After force installed ocl-icd-libopencl1 and replaced libopenCL.so.* in nvidia-319-updates, pyopencl benchmark shows:

Execution time of test without OpenCL: 0.0963661670685 s
Traceback (most recent call last):
  File "benchmark.py", line 29, in <module>
    for platform in cl.get_platforms():
pyopencl.LogicError: clGetPlatformIDs failed: platform not found khr

[Rebecca Palmer (rebecca-palmer)]

Did you delete the nvidia libopenCL* from both /usr/lib/nvidia-319-updates and /usr/lib32/nvidia-319-updates, and are they still absent (an update of nvidia-319-updates might have put them back)? Does the problem persist after restarting the machine?

My setup of copying libOpenCL.so.1.0.0 to /usr/local/lib and there creating libOpenCL.so, libOpenCL.so.1 symlinks doesn't require deleting Nvidia's libopenCL (but does require running ldconfig) so will survive Nvidia updates, but will need to be manually removed to revert to the package when this bug is fixed.

[Rebecca Palmer (rebecca-palmer)]

This is partially fixed in Trusty (nvidia-graphics-drivers-331): nvidia-libopencl1-331 is now a separate package which the main driver only Recommends:, so co-installing ocl-icd-libopencl1 and nvidia-331 is now allowed, but as virtual packages aren't versioned, pyopencl will still think nvidia-libopencl1-331 satisfies its libopencl1 dependency.

There are a few obvious solutions, but none of them are neat:
-Have pyopencl Depend: specifically on ocl-icd-libopencl1. The maintainer has previously said he doesn't want to do that: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=682435
-Have nvidia-libopencl1-* not Provide: libopencl1 (effectively defining the libopencl1 virtual package to be "at least OpenCL 1.2").
-Remove the Recommends: nvidia-libopencl1-331 from nvidia-331, allowing pyopencl's preference for ocl-icd-libopencl1 (Depends: ocl-icd-libopencl1 | libopencl1) to take effect. Possibly replace it with a Recommends: ocl-icd-libopencl1 to retain "has OpenCL by default".

[Rebecca Palmer (rebecca-palmer)]

Fixed in Trusty: pyopencl now depends on a new virtual libopencl-1.2-1, which forces selection of a libopencl1 it will work with. (opencl-icd still needs to be chosen manually to match the hardware, but that's bug 1264844)