Openblas breaks openmp

[Bruno Chareyre]

Linking libopenblas as it is in ubuntu 12.04 disable multithread tasks (known issue with other applications to).
Yade -jN run the parallel tasks in a sequential way, and therefore it actually slows down for increasing N due to overheads, as noticed by [1].
The bug affects yade-daily and every build that links vs. openblas

There is a simple fix for compiled versions:
- uninstall openblas
- git clone git://github.com/xianyi/OpenBLAS
- edit Makefile.rule to enable openmp at line 44 in (https://github.com/xianyi/OpenBLAS/blob/develop/Makefile.rule#L43):
# If you're going to use this library with OpenMP, please comment it in.
# USE_OPENMP = 1 (the problem with debian/openblas is, obviously, that it is built with USE_OPENMP = 0)
- make PREFIX=/usr/lib
- sudo make install
- run yade (no need to recompile)

This fix should also work for prebuilt yadedaily. The only problem is it has openblas as a dependency, so basically the first step above needs to also uninstall yadedaily (you could hack it somehow, of course...)

I'm not sure how to best fix yadedaily. I noticed openblas-omp packages available for Fedora and others, but not for debian.
One option is to replace libopenblas dependency by libatlas (it will also eliminate the problems with scipy). The PFV code will be suboptimal and non-parallel, but people seeking performance can always recompile with openblas.

What do you think Anton?

[1] https://answers.launchpad.net/yade/+question/246489

[Anton Gladky (gladky-anton)]

> I'm not sure how to best fix yadedaily. I noticed openblas-omp packages
> available for Fedora and others, but not for debian.
>

See [1], it seems there are good reasons not to use Openblas+OpenMP.

[1] https://bugs.debian.org/737675

> One option is to replace libopenblas dependency by libatlas (it will also
> eliminate the problems with scipy). The PFV code will be suboptimal and
> non-parallel, but people seeking performance can always recompile with
> openblas.
>

Sure, we can switch to atlas, if you think it will fix (at least partially)
the problem.

Anton

[Anton Gladky (gladky-anton)]

Actually, I do not see any problems with parallelisation on Debian Jessie.
We can probably switch to atlas only on 12.04, which is pretty old already.

Anton

[Bruno Chareyre (bruno-chareyre)]

>it seems there are good reasons not to use Openblas+OpenMP.

Well, we do use openblas+openmp for years. I was always compiling openblas (with, namely, USE_OPENMP = 1), that is why I never noticed the problem we see now. When running yade, PFV uses openblas while python libs use atlas (see [1] #4), so the bug you mention is not affecting us.

>Sure, we can switch to atlas, if you think it will fix (at least partially)

It should fix completely.
Could we have alternatives in the dependencies? I think I've seen that in deb packages sometimes. E.g. installing yade would need to have at least one of blas/atlas/openblas, but you have a choice (maybe even cublas for gpu?). It would apply for 12.04 and Jessie as well. It is not bad for benchmarking the libs.

[1] https://answers.launchpad.net/yade/+question/246244

[Bruno Chareyre (bruno-chareyre)]

>I do not see any problems with parallelisation on Debian Jessie

Good to know that it stabilizes after 6 years of struggling with libs, just for solving a linear system!

[Anton Gladky (gladky-anton)]

2014-04-09 12:48 GMT+02:00 Bruno Chareyre <email address hidden>:

> >Sure, we can switch to atlas, if you think it will fix (at least
> partially)
>
> It should fix completely.
> Could we have alternatives in the dependencies? I think I've seen that in
> deb packages sometimes. E.g. installing yade would need to have at least
> one of blas/atlas/openblas, but you have a choice (maybe even cublas for
> gpu?). It would apply for 12.04 and Jessie as well. It is not bad for
> benchmarking the libs.
>

As far as I understand, the problem now is yadedaily on 12.04, yes? If so,
we have 2 opportunities:
a) use atlas only for 12.04
b) switch to atlas in all distros, updating even dependencies in our
installation page.

> Could we have alternatives in the dependencies?

Not sure it will work. There is an opportunity to point additional
dependencies during
the build process, but they still have a priority, and the lower will be
used if higher is
not available. It is not our case.

Anton

[Bruno Chareyre (bruno-chareyre)]

So a) is a lot better.
I can update the "compile" part of the doc to explain how to compile with openblas+openmp on 12.04.

[Anton Gladky (gladky-anton)]

Bruno, I was trying to build Yade on 12.04 against
atlas instead of OpenBlas and got problem. It seems
yade-batch is not working in this case. Not sure yet why.

Anyway, I think 12.04 is relatively old, so we should not
spend too much efforts on this problem.

Anton

2014-04-09 14:31 GMT+02:00 Bruno Chareyre <email address hidden>:
> So a) is a lot better.
> I can update the "compile" part of the doc to explain how to compile with openblas+openmp on 12.04.
>
> --
> You received this bug notification because you are subscribed to Yade.
> https://bugs.launchpad.net/bugs/1304878
>
> Title:
> Openblas breaks openmp
>
> Status in Yet Another Dynamic Engine:
> New
>
> Bug description:
> Linking libopenblas as it is in ubuntu 12.04 disable multithread tasks (known issue with other applications to).
> Yade -jN run the parallel tasks in a sequential way, and therefore it actually slows down for increasing N due to overheads, as noticed by [1].
> The bug affects yade-daily and every build that links vs. openblas
>
> There is a simple fix for compiled versions:
> - uninstall openblas
> - git clone git://github.com/xianyi/OpenBLAS
> - edit Makefile.rule to enable openmp at line 44 in (https://github.com/xianyi/OpenBLAS/blob/develop/Makefile.rule#L43):
> # If you're going to use this library with OpenMP, please comment it in.
> # USE_OPENMP = 1 (the problem with debian/openblas is, obviously, that it is built with USE_OPENMP = 0)
> - make PREFIX=/usr/lib
> - sudo make install
> - run yade (no need to recompile)
>
> This fix should also work for prebuilt yadedaily. The only problem is
> it has openblas as a dependency, so basically the first step above
> needs to also uninstall yadedaily (you could hack it somehow, of
> course...)
>
> I'm not sure how to best fix yadedaily. I noticed openblas-omp packages available for Fedora and others, but not for debian.
> One option is to replace libopenblas dependency by libatlas (it will also eliminate the problems with scipy). The PFV code will be suboptimal and non-parallel, but people seeking performance can always recompile with openblas.
>
> What do you think Anton?
>
> [1] https://answers.launchpad.net/yade/+question/246489
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/yade/+bug/1304878/+subscriptions

[Alexander Eulitz [Eugen] (kubeu)]

concerning post #1:
for removing single packages without dependencies use
sudo dpkg -r --force-depends libopenblas-base
sudo dpkg -r --force-depends libopenblas-dev
This worked for me on 12.04LTS for reenabling multicore calculation.
Nevertheless the now missing dependencies will be reinstalled with apt-get update.
To avoid this see http://askubuntu.com/questions/17745/how-to-remove-a-deb-without-removing-its-dependencies

[Bruno Chareyre (bruno-chareyre)]

#7 agreed -> won't fix