It seems that CUDA does not work in Wily (at least for Quadro FX 570M)

Bug #1497663 reported by DenRaskovalov
10
This bug affects 2 people
Affects Status Importance Assigned to Milestone
nvidia-cuda-toolkit (Ubuntu)
Fix Released
High
Graham Inggs

Bug Description

When I am running a simple Theano application (it uses CUDA), I get:
mod.cu(3038): warning: conversion from a string literal to "char *" is deprecated
mod.cu(3041): warning: conversion from a string literal to "char *" is deprecated
/usr/include/c++/5/bits/locale_classes.h(789) (col. 20): internal error: assertion failed at: "/dvs/p4/build/sw/rel/gpu_drv/r340/r340_00/drivers/compiler_CUREL/edg/EDG_4.8/src/cp_gen_be.c", line 8408
1 catastrophic error detected in the compilation of "/tmp/tmpxft_0000085a_00000000-4_mod.cpp4.ii".
Compilation aborted.
Aborted (core dumped)

['nvcc', '-shared', '-O3', '-use_fast_math', '-m64', '-Xcompiler', '-DCUDA_NDARRAY_CUH=mc72d035fdf91890f3b36710688069b2e,-DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION,-fPIC,-fvisibility=hidden', '-Xlinker', '-rpath,/home/denplusplus/.theano/compiledir_Linux-3.19--generic-x86_64-with-Ubuntu-15.10-wily-x86_64-3.4.3+-64/cuda_ndarray', '-I/usr/local/lib/python3.4/dist-packages/Theano-0.7.0-py3.4.egg/theano/sandbox/cuda', '-I/usr/lib/python3/dist-packages/numpy/core/include', '-I/usr/include/python3.4m', '-I/usr/local/lib/python3.4/dist-packages/Theano-0.7.0-py3.4.egg/theano/gof', '-o', '/home/denplusplus/.theano/compiledir_Linux-3.19--generic-x86_64-with-Ubuntu-15.10-wily-x86_64-3.4.3+-64/cuda_ndarray/cuda_ndarray.so', 'mod.cu', '-L/usr/lib', '-lpython3.4m', '-lcublas', '-lcudart']
ERROR (theano.sandbox.cuda): Failed to compile cuda_ndarray.cu: ('nvcc return status', 134, 'for cmd', 'nvcc -shared -O3 -use_fast_math -m64 -Xcompiler -DCUDA_NDARRAY_CUH=mc72d035fdf91890f3b36710688069b2e,-DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION,-fPIC,-fvisibility=hidden -Xlinker -rpath,/home/denplusplus/.theano/compiledir_Linux-3.19--generic-x86_64-with-Ubuntu-15.10-wily-x86_64-3.4.3+-64/cuda_ndarray -I/usr/local/lib/python3.4/dist-packages/Theano-0.7.0-py3.4.egg/theano/sandbox/cuda -I/usr/lib/python3/dist-packages/numpy/core/include -I/usr/include/python3.4m -I/usr/local/lib/python3.4/dist-packages/Theano-0.7.0-py3.4.egg/theano/gof -o /home/denplusplus/.theano/compiledir_Linux-3.19--generic-x86_64-with-Ubuntu-15.10-wily-x86_64-3.4.3+-64/cuda_ndarray/cuda_ndarray.so mod.cu -L/usr/lib -lpython3.4m -lcublas -lcudart')

The program itself:
#!/usr/bin/env python3
import theano
from theano import function, config, shared, sandbox
import theano.tensor as T
import numpy
import time

vlen = 10 * 30 * 768 # 10 x #cores x # threads per core
iters = 1000

rng = numpy.random.RandomState(22)
x = shared(numpy.asarray(rng.rand(vlen), config.floatX))
f = function([], T.exp(x))
print(f.maker.fgraph.toposort())
t0 = time.time()
for i in range(iters):
    r = f()
t1 = time.time()
print('Looping %d times took' % iters, t1 - t0, 'seconds')
print('Result is', r)
if numpy.any([isinstance(x.op, T.Elemwise) for x in f.maker.fgraph.toposort()]):
    print('Used the cpu')
else:
    print('Used the gpu')

Revision history for this message
DenRaskovalov (fangbox) wrote :

It seems that CUDA 6.5 is the latest available for by GPU. It does not support GCC 5. Can you actually ask NVIDIA to update the 340th family of drivers to support GCC 5?

Graham Inggs (ginggs)
Changed in nvidia-cuda-toolkit (Ubuntu):
status: New → Confirmed
importance: Undecided → High
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package nvidia-cuda-toolkit - 6.5.14-1ubuntu2

---------------
nvidia-cuda-toolkit (6.5.14-1ubuntu2) wily; urgency=medium

  * Re-enable the gcc/g++ wrappers, depend on gcc/g++ 4.9 | 4.8.
    (LP: #1497663)

 -- Graham Inggs <email address hidden> Mon, 21 Sep 2015 13:38:29 +0200

Changed in nvidia-cuda-toolkit (Ubuntu):
status: Confirmed → Fix Released
Revision history for this message
Boyuan Deng (bryandeng) wrote :

/usr/lib/nvidia-cuda-toolkit/bin/gcc: 12: exec: gcc-4.8: not found

It seems that g++-4.9.real doesn't exist after the updated CUDA packages installed g++-4.9.

Revision history for this message
Graham Inggs (ginggs) wrote :

@bryandeng: please post the output of:

ls -l /usr/bin/gcc*

and

ls -l /usr/bin/g++*

Revision history for this message
Boyuan Deng (bryandeng) wrote :

@ginggs g++-4.9.real belongs to package "hardening-wrapper", which is not installed by default nowadays. I installed it and then it works.

ls results before installing "hardening-wrapper"

$ ls -l /usr/bin/gcc*
lrwxrwxrwx 1 root root 5 Aug 1 21:32 /usr/bin/gcc -> gcc-5
-rwxr-xr-x 1 root root 837944 Aug 27 00:25 /usr/bin/gcc-4.9
-rwxr-xr-x 1 root root 886968 Sep 15 22:34 /usr/bin/gcc-5
lrwxrwxrwx 1 root root 8 Aug 1 21:32 /usr/bin/gcc-ar -> gcc-ar-5
-rwxr-xr-x 1 root root 26968 Aug 27 00:25 /usr/bin/gcc-ar-4.9
-rwxr-xr-x 1 root root 26976 Sep 15 22:34 /usr/bin/gcc-ar-5
lrwxrwxrwx 1 root root 8 Aug 1 21:32 /usr/bin/gcc-nm -> gcc-nm-5
-rwxr-xr-x 1 root root 26968 Aug 27 00:25 /usr/bin/gcc-nm-4.9
-rwxr-xr-x 1 root root 26976 Sep 15 22:34 /usr/bin/gcc-nm-5
lrwxrwxrwx 1 root root 12 Aug 1 21:32 /usr/bin/gcc-ranlib -> gcc-ranlib-5
-rwxr-xr-x 1 root root 26968 Aug 27 00:25 /usr/bin/gcc-ranlib-4.9
-rwxr-xr-x 1 root root 26976 Sep 15 22:34 /usr/bin/gcc-ranlib-5

$ ls -l /usr/bin/g++*
lrwxrwxrwx 1 root root 5 Aug 1 21:32 /usr/bin/g++ -> g++-5
-rwxr-xr-x 1 root root 837944 Aug 27 00:16 /usr/bin/g++-4.9
-rwxr-xr-x 1 root root 886968 Sep 15 21:40 /usr/bin/g++-5
-rwxr-xr-x 1 root root 288 Sep 28 2014 /usr/bin/g++-libc++

Revision history for this message
Graham Inggs (ginggs) wrote :

@bryandeng: thanks!

Changed in nvidia-cuda-toolkit (Ubuntu):
status: Fix Released → In Progress
assignee: nobody → Graham Inggs (ginggs)
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package nvidia-cuda-toolkit - 6.5.14-1ubuntu3

---------------
nvidia-cuda-toolkit (6.5.14-1ubuntu3) wily; urgency=medium

  * Tweak gcc/g++ wrappers to work with and without hardening-wrapper.
    (LP: #1497663)

 -- Graham Inggs <email address hidden> Wed, 23 Sep 2015 10:32:47 +0200

Changed in nvidia-cuda-toolkit (Ubuntu):
status: In Progress → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.