Ubuntu

mpicc (openmpi) broken in Ubuntu 7.10 RC

Reported by Michael Madden on 2007-10-13
40
Affects Status Importance Assigned to Milestone
openmpi (Ubuntu)
Undecided
Unassigned
Gutsy
Undecided
Unassigned
Hardy
Undecided
Unassigned

Bug Description

Binary package hint: openmpi-bin

package does not build correctly :

the problem is linked to one line in the 99_autoconf patch

these are the changes around line 1760 in the current 99_autoconf.patch file

1759 -datadir='${prefix}/share'
1760 +datarootdir='${prefix}/share'
1761 +datadir='${datarootdir}'

line 1761 should be +datadir='${prefix}/share'

it seems that the datarootdir does not get expanded correctly

On Ubuntu 7.04, mpicc from the openmpi-bin package works and produces the normal lines to compile a C program that uses MPI:

$ cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=7.04
DISTRIB_CODENAME=feisty
DISTRIB_DESCRIPTION="Ubuntu 7.04"
$ dpkg -l | grep openmpi
ii openmpi-bin 1.1-2.3 high performance message passing library
ii openmpi-common 1.1-2.3 high performance message passing library
ii openmpi-dev 1.1-2.3 high performance message passing library
ii openmpi-libs0 1.1-2.3 high performance message passing library
$ mpicc -showme
gcc -I/usr/include/openmpi -pthread -lmpi -lorte -lopal -ldl -Wl,--export-dynamic -lnsl -lutil -lm -ldl
$ mpicc -showme:compile
-I/usr/include/openmpi -pthread
$ mpicc -showme:link
-pthread -lmpi -lorte -lopal -ldl -Wl,--export-dynamic -lnsl -lutil -lm -ldl
$ ls hello*
hello.c
$ mpicc -Wall -O2 hello.c -o hello
$ ls hello*
hello hello.c

TEST CASE:
However on Ubuntu 7.10 RC, the mpicc command produces no output and will not build a MPI C program:

$ cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=7.10
DISTRIB_CODENAME=gutsy
DISTRIB_DESCRIPTION="Ubuntu 7.10"
$ dpkg -l | grep openmpi
ii openmpi-bin 1.1-2.5 high performance message passing library
ii openmpi-common 1.1-2.5 high performance message passing library
ii openmpi-dev 1.1-2.5 high performance message passing library
ii openmpi-libs0 1.1-2.5 high performance message passing library
$ mpicc -showme
$ mpicc -showme:compile
$ mpicc -showme:link
$ ls hello*
hello.c
$ mpicc -Wall -O2 hello.c -o hello
$ ls hello*
hello.c

Brian Barnes (bcbarnes) wrote :

I can add that mpif77 does nothing when trying to compile any sort of fortran file using 7.10 (official release) fresh install, and all the openmpi* packages installed with gcc/gfortran 4.2 working okay. AMD64. This is horrible.

Sean Fritz (sean-fritz) wrote :

This is still broken in 7.10 release.

Changed in openmpi:
status: New → Confirmed
Sam Yates (sam-quux) wrote :

I can confirm that this problem persists in 7.10. It looks like something went wrong with the configure process at build time: strace reveals that mpicc and friends attempt to read "${prefix}/share/openmpi/mpicc-wrapper-data.txt". The work-around I've used is to open up these wrapper programs with a hex editor and edit the offending string in place, replacing it with "/usr/share/openmpi/mpicc-wrapper-data.txt\0\0\0\0\0" in the mpicc case for example.

Marko Kevac (mkevac) wrote :

I can confirm that this is broken in 7.10

Marko Kevac (mkevac) wrote :

mpirun has same bug. It is unable to find files from /usr/share/openmpi/
Please fix.

Kay (noiq) wrote :

Confirmed for Xubuntu 7.10 amd64.
mpif90, mpicc, mpicxx, all do not work and strace shows the "${prefix}".
I can not confirm russian_knight's bug of mpirun, as it runs all right here after fixing the compilers.

halfflat's workaround worked fine for me. This way I could fix the 3 mpi* and I am sure this should work for other versions, too.
I had to try a bit to get it working, so here is a bit more detailed description of the workaround for those who are as unexperienced as me:

First of all it is important to use an editor that really just manipulates single bytes, as a hex editor does.
strace showed

"${prefix}/share/openmpi/mpicc-wrapper-data.txt"

but the file contains

${prefix}/share/openmpi.%s%s%s-wrapper-data.txt

I just had to change the first part and make it look like

/usr/share/openmpi

Because the "${prefix}" takes 9 Bytes and "/usr" just 4 I had to overwrite the 5 following bytes after the "openmpi" with "00".
Thanks halfflat for that hint.

Marko Kevac (mkevac) wrote :

About mpirun. Catch:

$ strace mpirun 2>&1 | grep open
open("/etc/ld.so.cache", O_RDONLY) = 3
open("/usr/lib/liborte.so.0", O_RDONLY) = 3
open("/lib/tls/i686/cmov/libdl.so.2", O_RDONLY) = 3
open("/lib/tls/i686/cmov/libnsl.so.1", O_RDONLY) = 3
open("/lib/tls/i686/cmov/libutil.so.1", O_RDONLY) = 3
open("/lib/tls/i686/cmov/libm.so.6", O_RDONLY) = 3
open("/lib/tls/i686/cmov/libpthread.so.0", O_RDONLY) = 3
open("/lib/tls/i686/cmov/libc.so.6", O_RDONLY) = 3
open("/usr/lib/libopal.so.0", O_RDONLY) = 3
open("/etc/openmpi/openmpi-mca-params.conf", O_RDONLY) = 3
open("/home/knight/.openmpi/mca-params.conf", O_RDONLY) = -1 ENOENT (No such file or directory)
open("${prefix}/share/openmpi/help-orterun.txt", O_RDONLY) = -1 ENOENT (No such file or directory)
open("${prefix}/share/openmpi/help-orterun.txt.txt", O_RDONLY) = -1 ENOENT (No such file or directory)

So same "prefix" error here.

Kay (noiq) wrote :

You are right, sorry.

I thought your mpirun does not work, so I just tried if does. strace shows me the same prefix error, but mpirun works fine, though.

csauer (caetanosauer) wrote :

The bug is confirmed.

I fixed it with an hex editor (ghex2), but there is no problem with the ${prefix} thing.
The binariy contains:

${prefix}/share/openmpi.%s%s%s-wrapper-data.txt

i just changed that dot after the "share/openmpi" to a slash, so now it looks like:

${prefix}/share/openmpi/%s%s%s-wrapper-data.txt

And it works fine:
$ mpicc.openmpi
$ gcc: no input files

csauer (caetanosauer) wrote :

Sorry, i was wrong in my last post.

The solution mentioned halfflat did not work for me. I have to do the following.

Replace:

${prefix}/share/openmpi.%s%s%s-wrapper-data.txt

for:

/usr/share/openmpi......%s%s%s-wrapper-data.txt

the dot means a zero byte, so there is two strings in that sequence, and the extra 5 bytes have to be added to the end of the first one.
But, the problem continued, because now the binary tried to open /usr/share/openmpi/mpicc.openmpi-wrapper-data.txt
So i smply made a symlink to the real file, without the ".openmpi".
If you use fortran or c++ compilers, the same thing has to be done for them.

Regards and sorry for the mistake before.

And now, it works for real ;-)

Yavor Vutov (yavor-vutov) wrote :

Guys, what's wrong with you?
Why hexedit the file, if it is possible for the package maintainer just to fix the damn package.
How many months should it take?

Brian Barnes (bcbarnes) wrote :

Apparently, at least two months. And it's still not even triaged. Why bother? If you needed MPI by now, you've recompiled openmpi yourself, hex edited it, or switched to a distro that makes sure these things work, like Fedora. I chose the latter route.

On 12/12/07, Brian Barnes <email address hidden> wrote:
>
> Apparently, at least two months. And it's still not even triaged. Why
> bother? If you needed MPI by now, you've recompiled openmpi yourself,
> hex edited it, or switched to a distro that makes sure these things
> work, like Fedora. I chose the latter route.

You are right. There was a time I loved Ubuntu, because it just worked.
Now it is getting more and more like windows. Nice to use for a while, but
afterwards - problem after problem.

Unfortunately I don't like most linux distros and other OSes I've tried
recently.
Everithing becomes worse and worse.

Yavor

Koen Beek (koen-beek) on 2007-12-13
Changed in openmpi:
assignee: nobody → koen-beek
status: Confirmed → In Progress
Koen Beek (koen-beek) wrote :

The problem seems to be caused by the autoconf patch that has been added by debian
changing debian/patches/00list to remove the mention of the 99_autoconf and removing the 99_autoconf.dpatch file creates a source package that builds correctly

Koen Beek (koen-beek) wrote :

This is a debdiff to remove the 99_autoconf patch

Koen Beek (koen-beek) wrote :

the hardy version of openmpi works correctly, so this debdiff is only to be applied to gutsy

Changed in openmpi:
assignee: koen-beek → nobody
status: In Progress → Confirmed
Daniel Holbach (dholbach) wrote :

http://wiki.ubuntu.com/StableReleaseUpdates is the process to get changes into a stable release.

Unsubscribing ubuntu-universe-sponsors for now.

Koen Beek (koen-beek) wrote :

Impact of bug : mpicc, mpirun etc are unusable
Already corrected in hardy as hardy uses a different debian version of the openmpi package (1.2-4.3 and not 1.1-2.5)

Koen Beek (koen-beek) wrote :

I've changed the debdiff so that the target is gutsy-proposed in stead of gutsy

Koen Beek (koen-beek) on 2007-12-20
description: updated
John Dong (jdong) wrote :

Hi,

Can you please provide a justification for the removal of 99_autoconf and why its presence breaks the package?

Thanks,

MOTU SRU team

Koen Beek (koen-beek) wrote :

I've pinpointed the problem to one line in the 99_autoconf patch

these are the changes around line 1760 in the current 99_autoconf.patch file

1759 -datadir='${prefix}/share'
1760 +datarootdir='${prefix}/share'
1761 +datadir='${datarootdir}'

line 1761 should be +datadir='${prefix}/share'

it seems that the dataroordir does not get expanded correctly

I'll attach two new debdiffs with this minor change only

Koen Beek (koen-beek) wrote :

result of debdiff openmpi_1.1-2.5.dsc openmpi_1.1-2.5ubuntu1.dsc > openmpi-1.1-2.5--1.1-2.5ubuntu1.debdiff

Koen Beek (koen-beek) wrote :

changes file

Koen Beek (koen-beek) wrote :

dsc file

John Dong (jdong) wrote :

Ok, this debdiff looks a lot better. Please update the original description to reflect your discovery in comment #22 and then I'd say it's good enough for upload.

John Dong (jdong) wrote :

Also, is this fixed in Hardy? If not, debdiffs against Hardy would be needed before Gutsy can be SRU'ed.

Koen Beek (koen-beek) wrote :

Hi John,

  Hardy is OK already as hardy has been synced with a newer debian version that does not have this problem (I believe the 99_autoconf patch disappeared in subsequent versions)

    Koen

description: updated
John Dong (jdong) wrote :

Marking Hardy fixed release due to last comment. The description and debdiff proposed here looks reasonable to me, so please proceed to find a sponsor to get this into gutsy-proposed.

Changed in openmpi:
status: Confirmed → Fix Released
Koen Beek (koen-beek) on 2008-01-14
Changed in openmpi:
status: New → Confirmed
Koen Beek (koen-beek) wrote :

How do I find a sponsor ?

John Dong (jdong) wrote :

Subscribe the "ubuntu-universe-sponsors" team to this bug report and leave it unassigned and Confirmed (as it currently is)

Luca Falavigna (dktrkranz) wrote :

What about Feisty? Could you please check if it is affected too and eventually prepare a debdiff for feisty-proposed?

Koen Beek (koen-beek) wrote :

feisty uses version 1.1-2.3 which doesn't have the 99_autoconf patch that causes problems in gutsy

as far as I know the only ubuntu version that has this problem is gutsy (it's the only version in ubuntu that has this 99_autoconf patch)

Changed in openmpi:
assignee: nobody → dktrkranz
status: Confirmed → In Progress
Luca Falavigna (dktrkranz) wrote :

Uploaded to gutsy-proposed, now waiting for an archive-admin to accept it.

Changed in openmpi:
assignee: dktrkranz → nobody
status: In Progress → Confirmed
Martin Pitt (pitti) wrote :

Accepted into gutsy-proposed, please test.

Changed in openmpi:
status: Confirmed → Fix Committed
Johan Nylander (jnylander) wrote :

Hi,
Seems to work using openmpi 1.1-2.5ubuntu1 packages from gutsy-proposed (I'm running Gutsy 7.10):

$ mpicc
gcc: no input files

I did this:
System -> Administration -> Software Sources
Go to Updates tab and check Pre-released updates (gutsy-proposed)
Click on Close and the Reload
System -> Administration -> Synaptic Package Manager
Search for openmpi and then I did a complete removal (Mark for Complete Removal) of all openmpi packages
Apply changes
And then (re)install (I used Synaptic) openmpi-bin, openmpi-common, openmpi-libs0, openmpi-dbg, openmpi-dev

There is probably a cute one-liner for these steps as well.

Cheers,
Johan

Sang Chul (goshng-yahoo) wrote :

It works.

Luca Falavigna (dktrkranz) wrote :

GUTSY VERIFICATION

1.1-2.5:
$ mpicc -showme
$ mpicc -showme:compile
$ mpicc -showme:link
$ ls hello*
hello.c
$ mpicc -Wall -O2 hello.c -o hello
$ ls hello*
hello.c
$

1.1-2.5ubuntu1:
$ mpicc -showme
gcc -I/usr/include/openmpi -pthread -lmpi -lorte -lopal -ldl -Wl,--export-dynamic -lnsl -lutil -lm -ldl
$ mpicc -showme:compile
-I/usr/include/openmpi -pthread
$ mpicc -showme:link
-pthread -lmpi -lorte -lopal -ldl -Wl,--export-dynamic -lnsl -lutil -lm -ldl
$ ls hello*
hello.c
$ mpicc -Wall -O2 hello.c -o hello
$ ls hello*
hello hello.c
$ ./hello
SRU candidate works!
$

It runs fine now.

Martin Pitt (pitti) wrote :

Copied to gutsy-updates. Thank you!

Changed in openmpi:
status: Fix Committed → Fix Released
ilmarw (ilmar-wilbers) wrote :

Hi,

I have updated to the latest version, but I still have problems. After installing openmpi-bin, there is no executable mpicc, only mpicc.openmpi. This executable outputs nothing whatsoever.

ilmar

ilmarw (ilmar-wilbers) wrote :

$dpkg -l | grep openmpi
ii openmpi-bin 1.1-2.5ubuntu1 high performance message passing library
ii openmpi-common 1.1-2.5ubuntu1 high performance message passing library
ii openmpi-dbg 1.1-2.5ubuntu1 high performance message passing library
ii openmpi-dev 1.1-2.5ubuntu1 high performance message passing library
ii openmpi-libs0 1.1-2.5ubuntu1 high performance message passing library

$ mpicc.openmpi
$ mpicc.openmpi -showme
$ mpicc.openmpi -showme:link
$ mpicc.openmpi -showme:compile

ilmarw (ilmar-wilbers) wrote :

Update:
When creating the link
$ cd /usr/bin/
$ sudo ln -s mpicc.openmpi mpicc
 it works as it should. So the problem seems to be that mpicc is not linked to mpicc.openmpi under /usr/bin

ilmar

ilmarw (ilmar-wilbers) wrote :

Sorry for all the posts, but I figured out what the problem was. Or at least how to solve it. Even though I made sure that no other MPI-libraries were installed (LAM, MPICH), they earlier presence contributed to this error. After removing the file /var/lib/dpkg/alternatives/mpicc the installation worked fine.

ilmar

gcordoba (glgcg) wrote :

Hi,
yes it works fine.
However, I am trying to install a software for modeling that needs both mpi and hdf5. The last reported lines in the configure file were:
checking for pthread_create in -lpthread... yes
checking for MPI_Init in -lmpich... no
checking for MPI_Init in -lmpi... yes
checking for H5open in -lhdf5... no
configure: error: HDF5 is strongly recommended for paraview support.
                  Re-run configure --with-hdf5=/path/to/hdf5.
                  Re-run configure --without-hdf5 to get rid of this message

This, despite I installed hdf5 after your tip. Perhaps there is a similar bug?
Thanks and sorry for the cross posting

I confirm that tis package stills broken in Hardy.

gcordoba (glgcg) wrote :

This is mad. It seems that newer versions of ubuntu are based on previous released versions plenty of bugs. In this way we never will obtain a stable version that make us proud of Linux.
However, and despite of that, I always prefer Linux! I never loss my hopes.
 Gustavo
PS.: the package mpich has the same problem...

ilmarw (ilmar-wilbers) wrote :

With every release, we hope that the problems we are having are solved, but they never are. We keep creating our own packages of, amongst others, openmpi based on Debias Sid. I see now that Gutsy will not see version 1.2.6. This goes for openmpi in general, not any specific bugs.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Duplicates of this bug

Other bug subscribers