JIT should be given/use proper flags

[Garth Wells]

The flags used in JIT are a mess. I'm seeing this first-hand when using specialised BLAS. The full lib path is not being used.

[Johan Hake (johan-hake)]

On Monday November 7 2011 13:10:35 Garth Wells wrote:
> Public bug reported:
>
> The flags used in JIT are a mess. I'm seeing this first-hand when using
> specialised BLAS. The full lib path is not being used.
>
> ** Affects: dolfin
> Importance: Undecided
> Status: New

Could you please hand a more descriptive report? What are you doing and what
are you expecting JIT to do. Can now the generated ufc module be dependent on
blas?

Johan

[Garth Wells (garth-wells)]

On 7 November 2011 21:28, Johan Hake <email address hidden> wrote:
> On Monday November 7 2011 13:10:35 Garth Wells wrote:
>> Public bug reported:
>>
>> The flags used in JIT are a mess. I'm seeing this first-hand when using
>> specialised BLAS. The full lib path is not being used.
>>
>> ** Affects: dolfin
>>      Importance: Undecided
>>          Status: New
>
> Could you please hand a more descriptive report? What are you doing and what
> are you expecting JIT to do. Can now the generated ufc module be dependent on
> blas?
>

No. But JIT throes in the blas libs without providing the path to the lib.

Here is just some of the junk used during JIT:

-ldl -lm -lutil -lnsl -lopen-pal -lopen-rte -lmpi -lmpi_cxx -lcppunit
-lhdf5 -lz -lboost_thread-mt -lgmp -lmpfr -lgmpxx -lCGAL -lmetis
-lparmetis -lptscotcherr -lptscotch -latlas.3gf -lf77blas.3gf
-lcblas.3gf -llapack.3gf -lccolamd -lcolamd -lcamd -lamd -lcholmod
-lumfpack -lpetsc -lslepc -lteuchos -ltpi -lsacado -lrtop -lkokkos
-lkokkosnodeapi -lkokkoslinalg -lepetra -lzoltan -lshards -lglobipack
-ltriutils -ltpetra -ltpetrainout -ltpetraext -lepetraext -lthyracore
-lthyraepetra -lthyraepetraext -lthyratpetra -lthyra -loptipack
-lisorropia -ldpliris -laztecoo -lgaleri -lamesos -lpamgen
-lpamgen_extras -lifpack -lkomplex -lml -lbelos -lbelosepetra
-lbelostpetra -lifpack2 -lstratimikosifpack -lstratimikosml
-lstratimikosamesos -lstratimikosaztecoo -lstratimikosbelos
-lstratimikos -lfei_base -lfei_trilinos -lanasazi -lanasaziepetra
-lModeLaplace -lanasazitpetra -lintrepid -lnox -lnoxepetra -lnoxthyra
-lloca -llocaepetra -llocathyra -lmoertel -lrythmos -lmoocho
-lmoochothyra -lmesquite -lboost_serialization-mt -lboost_mpi-mt
-lboost_math_tr1-mt -lboost_iostreams-mt -lboost_system-mt
-lboost_program_options-mt -lboost_filesystem-mt -larmadillo -ldolfin
-lxml2

A problem is that Cmake allows C++ code to run with libs that are not
in LD_LIBRARY_PATH by using the full path, but JIT is getting flags
from somewhere but using the full path.

JIT should either use all DOLFIN flags, but done properly (i.e. not
with pkg-config), or with a bare minimum of libs.

Garth

> Johan
>
> --
> You received this bug notification because you are a member of DOLFIN
> Core Team, which is subscribed to DOLFIN.
> https://bugs.launchpad.net/bugs/887324
>
> Title:
>  JIT should be given/use proper flags
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/dolfin/+bug/887324/+subscriptions
>

[Johan Hake (johan-hake)]

On Monday November 7 2011 13:51:46 Garth Wells wrote:
> On 7 November 2011 21:28, Johan Hake <email address hidden> wrote:
> > On Monday November 7 2011 13:10:35 Garth Wells wrote:
> >> Public bug reported:
> >>
> >> The flags used in JIT are a mess. I'm seeing this first-hand when using
> >> specialised BLAS. The full lib path is not being used.
> >>
> >> ** Affects: dolfin
> >> Importance: Undecided
> >> Status: New
> >
> > Could you please hand a more descriptive report? What are you doing and
> > what are you expecting JIT to do. Can now the generated ufc module be
> > dependent on blas?
>
> No. But JIT throes in the blas libs without providing the path to the
> lib.

So you are refering to the dolfin extension modules (expressions,
compile_subdomains) not ufc extension modules (from ufl forms)?

> Here is just some of the junk used during JIT:
>
> -ldl -lm -lutil -lnsl -lopen-pal -lopen-rte -lmpi -lmpi_cxx -lcppunit
> -lhdf5 -lz -lboost_thread-mt -lgmp -lmpfr -lgmpxx -lCGAL -lmetis
> -lparmetis -lptscotcherr -lptscotch -latlas.3gf -lf77blas.3gf
> -lcblas.3gf -llapack.3gf -lccolamd -lcolamd -lcamd -lamd -lcholmod
> -lumfpack -lpetsc -lslepc -lteuchos -ltpi -lsacado -lrtop -lkokkos
> -lkokkosnodeapi -lkokkoslinalg -lepetra -lzoltan -lshards -lglobipack
> -ltriutils -ltpetra -ltpetrainout -ltpetraext -lepetraext -lthyracore
> -lthyraepetra -lthyraepetraext -lthyratpetra -lthyra -loptipack
> -lisorropia -ldpliris -laztecoo -lgaleri -lamesos -lpamgen
> -lpamgen_extras -lifpack -lkomplex -lml -lbelos -lbelosepetra
> -lbelostpetra -lifpack2 -lstratimikosifpack -lstratimikosml
> -lstratimikosamesos -lstratimikosaztecoo -lstratimikosbelos
> -lstratimikos -lfei_base -lfei_trilinos -lanasazi -lanasaziepetra
> -lModeLaplace -lanasazitpetra -lintrepid -lnox -lnoxepetra -lnoxthyra
> -lloca -llocaepetra -llocathyra -lmoertel -lrythmos -lmoocho
> -lmoochothyra -lmesquite -lboost_serialization-mt -lboost_mpi-mt
> -lboost_math_tr1-mt -lboost_iostreams-mt -lboost_system-mt
> -lboost_program_options-mt -lboost_filesystem-mt -larmadillo -ldolfin
> -lxml2
>
> A problem is that Cmake allows C++ code to run with libs that are not
> in LD_LIBRARY_PATH by using the full path, but JIT is getting flags
> from somewhere but using the full path.

jit compilation of dolfin extension modules get the dolfin flags from pkg-
config. Everything that is in dolfin.pc gets included in the appropriate
flags. If the path to the blas library was included there it should just work.

> JIT should either use all DOLFIN flags, but done properly (i.e. not
> with pkg-config), or with a bare minimum of libs.

Any suggestion on how this can be done properly? CMake lack a good way to get
the flags from the command prompt, and hence from within instant. Here pkg-
config is nice.

What is problematic with pkg-config? Sure it is not the generation of the pkg-
config file that is flawed?

Johan

> Garth
>
> > Johan
> >
> > --
> > You received this bug notification because you are a member of DOLFIN
> > Core Team, which is subscribed to DOLFIN.
> > https://bugs.launchpad.net/bugs/887324
> >
> > Title:
> > JIT should be given/use proper flags
> >
> > To manag...

[Garth Wells (garth-wells)]

On 7 November 2011 22:25, Johan Hake <email address hidden> wrote:
> On Monday November 7 2011 13:51:46 Garth Wells wrote:
>> On 7 November 2011 21:28, Johan Hake <email address hidden> wrote:
>> > On Monday November 7 2011 13:10:35 Garth Wells wrote:
>> >> Public bug reported:
>> >>
>> >> The flags used in JIT are a mess. I'm seeing this first-hand when using
>> >> specialised BLAS. The full lib path is not being used.
>> >>
>> >> ** Affects: dolfin
>> >>      Importance: Undecided
>> >>          Status: New
>> >
>> > Could you please hand a more descriptive report? What are you doing and
>> > what are you expecting JIT to do. Can now the generated ufc module be
>> > dependent on blas?
>>
>> No. But JIT throes in the blas libs without providing the path to the
>> lib.
>
> So you are refering to the dolfin extension modules (expressions,
> compile_subdomains) not ufc extension modules (from ufl forms)?
>
>> Here is just some of the junk used during JIT:
>>
>> -ldl -lm -lutil -lnsl -lopen-pal -lopen-rte -lmpi -lmpi_cxx -lcppunit
>> -lhdf5 -lz -lboost_thread-mt -lgmp -lmpfr -lgmpxx -lCGAL -lmetis
>> -lparmetis -lptscotcherr -lptscotch -latlas.3gf -lf77blas.3gf
>> -lcblas.3gf -llapack.3gf -lccolamd -lcolamd -lcamd -lamd -lcholmod
>> -lumfpack -lpetsc -lslepc -lteuchos -ltpi -lsacado -lrtop -lkokkos
>> -lkokkosnodeapi -lkokkoslinalg -lepetra -lzoltan -lshards -lglobipack
>> -ltriutils -ltpetra -ltpetrainout -ltpetraext -lepetraext -lthyracore
>> -lthyraepetra -lthyraepetraext -lthyratpetra -lthyra -loptipack
>> -lisorropia -ldpliris -laztecoo -lgaleri -lamesos -lpamgen
>> -lpamgen_extras -lifpack -lkomplex -lml -lbelos -lbelosepetra
>> -lbelostpetra -lifpack2 -lstratimikosifpack -lstratimikosml
>> -lstratimikosamesos -lstratimikosaztecoo -lstratimikosbelos
>> -lstratimikos -lfei_base -lfei_trilinos -lanasazi -lanasaziepetra
>> -lModeLaplace -lanasazitpetra -lintrepid -lnox -lnoxepetra -lnoxthyra
>> -lloca -llocaepetra -llocathyra -lmoertel -lrythmos -lmoocho
>> -lmoochothyra -lmesquite -lboost_serialization-mt -lboost_mpi-mt
>> -lboost_math_tr1-mt -lboost_iostreams-mt -lboost_system-mt
>> -lboost_program_options-mt -lboost_filesystem-mt -larmadillo -ldolfin
>> -lxml2
>>
>> A problem is that Cmake allows C++ code to run with libs that are not
>> in LD_LIBRARY_PATH by using the full path,  but JIT is getting flags
>> from somewhere but using the full path.
>
> jit compilation of dolfin extension modules get the dolfin flags from pkg-
> config. Everything that is in dolfin.pc gets included in the appropriate
> flags. If the path to the blas library was included there it should just work.
>
>> JIT should either use all DOLFIN flags, but done properly (i.e. not
>> with pkg-config), or with a bare minimum of libs.
>
> Any suggestion on how this can be done properly? CMake lack a good way to get
> the flags from the command prompt, and hence from within instant. Here pkg-
> config is nice.
>
> What is problematic with pkg-config? Sure it is not the generation of the pkg-
> config file that is flawed?
>

Either pkg-config is crap, or our dolfin-conf.pc is crap. Probably
both. The proper library paths are not provided.

Garth

> Johan
>
>> Garth...

[Johan Hake (johan-hake)]

On Monday November 7 2011 14:38:24 Garth Wells wrote:
> On 7 November 2011 22:25, Johan Hake <email address hidden> wrote:
> > On Monday November 7 2011 13:51:46 Garth Wells wrote:
> >> On 7 November 2011 21:28, Johan Hake <email address hidden> wrote:
> >> > On Monday November 7 2011 13:10:35 Garth Wells wrote:
> >> >> Public bug reported:
> >> >>
> >> >> The flags used in JIT are a mess. I'm seeing this first-hand when
> >> >> using specialised BLAS. The full lib path is not being used.
> >> >>
> >> >> ** Affects: dolfin
> >> >> Importance: Undecided
> >> >> Status: New
> >> >
> >> > Could you please hand a more descriptive report? What are you doing
> >> > and what are you expecting JIT to do. Can now the generated ufc
> >> > module be dependent on blas?
> >>
> >> No. But JIT throes in the blas libs without providing the path to the
> >> lib.
> >
> > So you are refering to the dolfin extension modules (expressions,
> > compile_subdomains) not ufc extension modules (from ufl forms)?
> >
> >> Here is just some of the junk used during JIT:
> >>
> >> -ldl -lm -lutil -lnsl -lopen-pal -lopen-rte -lmpi -lmpi_cxx -lcppunit
> >> -lhdf5 -lz -lboost_thread-mt -lgmp -lmpfr -lgmpxx -lCGAL -lmetis
> >> -lparmetis -lptscotcherr -lptscotch -latlas.3gf -lf77blas.3gf
> >> -lcblas.3gf -llapack.3gf -lccolamd -lcolamd -lcamd -lamd -lcholmod
> >> -lumfpack -lpetsc -lslepc -lteuchos -ltpi -lsacado -lrtop -lkokkos
> >> -lkokkosnodeapi -lkokkoslinalg -lepetra -lzoltan -lshards -lglobipack
> >> -ltriutils -ltpetra -ltpetrainout -ltpetraext -lepetraext -lthyracore
> >> -lthyraepetra -lthyraepetraext -lthyratpetra -lthyra -loptipack
> >> -lisorropia -ldpliris -laztecoo -lgaleri -lamesos -lpamgen
> >> -lpamgen_extras -lifpack -lkomplex -lml -lbelos -lbelosepetra
> >> -lbelostpetra -lifpack2 -lstratimikosifpack -lstratimikosml
> >> -lstratimikosamesos -lstratimikosaztecoo -lstratimikosbelos
> >> -lstratimikos -lfei_base -lfei_trilinos -lanasazi -lanasaziepetra
> >> -lModeLaplace -lanasazitpetra -lintrepid -lnox -lnoxepetra -lnoxthyra
> >> -lloca -llocaepetra -llocathyra -lmoertel -lrythmos -lmoocho
> >> -lmoochothyra -lmesquite -lboost_serialization-mt -lboost_mpi-mt
> >> -lboost_math_tr1-mt -lboost_iostreams-mt -lboost_system-mt
> >> -lboost_program_options-mt -lboost_filesystem-mt -larmadillo -ldolfin
> >> -lxml2
> >>
> >> A problem is that Cmake allows C++ code to run with libs that are not
> >> in LD_LIBRARY_PATH by using the full path, but JIT is getting flags
> >> from somewhere but using the full path.
> >
> > jit compilation of dolfin extension modules get the dolfin flags from
> > pkg- config. Everything that is in dolfin.pc gets included in the
> > appropriate flags. If the path to the blas library was included there it
> > should just work.
> >
> >> JIT should either use all DOLFIN flags, but done properly (i.e. not
> >> with pkg-config), or with a bare minimum of libs.
> >
> > Any suggestion on how this can be done properly? CMake lack a good way to
> > get the flags from the command prompt, and hence from within instant.
> > Here pkg- config is nice.
> >
> > What is problematic with pkg-config? Sure it is not the genera...

[Johan Hake (johan-hake)]

Is this fixed with the improved FindPackage files for blas?

Johan

[Anders Logg (logg)]

Garth, does this work now?

[Garth Wells (garth-wells)]

It seems to work, but should be looked at later. We need to take care the full paths are being used.

[Johan Hake (johan-hake)]

On Wednesday November 16 2011 06:55:27 Garth Wells wrote:
> It seems to work, but should be looked at later. We need to take care
> the full paths are being used.

That is a good point, and it is adressed in the recently added --find-package
mode for cmake. If we require CMake 2.8.6 we should be able to explore this
interface.

Johan

> ** Changed in: dolfin
> Importance: Critical => Medium
>
> ** Changed in: dolfin
> Milestone: 1.0-rc1 => 1.1.0