Hi Zach, Olivier,

On Sat, Feb 18, 2017 at 2:29 PM, Olivier Mattelaer <
<email address hidden>> wrote:

> Hi Zach,
>
> > I don't love the idea of picking at random a process to run;
>
> Our code is actually designed at multiple place to configure itself at the
> time of the first process/loading of the code.
> Such behaviour is actually especially useful when combine with the
> auto-update feature of MG5aMC (feature that you do not use).
> This is perfectly fine in general but in the case of filesystem without
> write disk access since the code can not configure itself in that case.
>
> This force you/us to find some work-around from each of the auto-configure
> step in order to have those working with your setup. (Copy-file, modifying
> configuration file,…)
> I think that running one LO and one NLO process (you do not need to run
> them, just to “output” them)  is actually the best way to handle such
> difficulty in the long term.
> But this is obviously your call if you prefer to catch those problems one
> by one at each new release.
>
> By the way, do you use the script compile.py? If yes, one correct
> solution would be that I run one LO and NLO generation from that script
> (and then clean the associate output obviously).
>
> > Could you confirm that
> > these are the right ones to cover most LO and NLO use cases?
> > install SysCalc Golem95 PJFry QCDLoop maddm collier ninja oneloop
>
> I do not know which of those library " Golem95 PJFry QCDLoop collier ninja
> oneloop” are compatible with your “set dependencies “ options.
> Valentin will give more details on that.
>

Ok, so first of all the only dependencies that make sense to install for
production are:

CutTools, IREGI, Ninja and COLLIER

The first two (CutTools and IREGI) are not so useful but there mainly
because one cannot turned them off. This is a technical limitation that is
inherited from the history of the code development and that we should
eventually lift; but for now they are mandatory.
The option 'output_dependencies' basically only applies to them, and for
them two useful options are:

> internal: The whole source code of CutTools and IREGI is copied in the
output process directory and automatically recompiled when necessary when
launching an aMC@NLO run.

> external: The compiled fortran90 modules and static libraries are linked
'libcts.a libiregi.a mpmodule.mod' to the lib directory of the
process_output and therefore the target of the links must be available at
all time when launching/running the aMC@NLO code.

For all other loop-related dependencies, the option 'output_dependencies'
has no impact (or another way of saying it is: I have not implemented the
'internal' mode for them).
This means that the two additional relevant dependencies Ninja and COLLIER
(which are more powerful implementations of the CutTools and IREGI
counterparts), there is only two options of the user (assuming the other
three dependencies Golem95, PJFry and Samurai have been turned off in
mg5_configuration.txt, which I recommend you do anyway):

a) Disable them all together by specifying 'None' or '' for their
corresponding install path in mg5_configuration.txt. MadLoop will be able
to function correctly without them, but will simply be less optimal.

b) Specify a path to a valid installation (as it is done automatically when
you install them with the corresponding 'install' functions in MG5_aMC).
When doing that, these two dependencies (Ninja and COLLIER) will be linked
in either of the following to ways to the output process:

  --> b.1] If the output process is a standalone MadLoop output (obtained
with [virt=...] and which only outputs the code to compute the virtual for
specific PS points) or an output for a loop-induced process, then the
following soft-links will be added to the lib directory of the process
output:

collier_lib
collier_include
ninja_lib
ninja_include

which will by default point to the corresponding directories specified via
the path indicated in MG5aMC options (i.e. mg5_configurations.txt). Let me
stress here that when this path comes from an installation of Collier and
NINJA that was steered by MG5aMC, the lib directory soft-linked will only
contain the '.a' library so that static-linking is guaranteed, which makes
things easier for job submission for high throughput computing.

  --> b.2] When doing a complete NLO computation with aMC@NLO, there's
unfortunately no such links in the lib directory, and instead the path of
these two additional dependencies will be directly hardcoded in the
following makefile include:
<PROC_OUTPUT_DIRECTORY>/SubProcesses/makefile_loop

Now in either case b.1' or b.2] above, what I think would be the best
strategy to make the output directory "portable" is to simply update the
collier and ninja links (easy to do) or the paths in makefile_loop
(unfortunately more cumbersome to do) whenever copying the process
directory on a different machine (or make sure that these libraries are
available via the system wide LD_LIBRARY_PATH).

And again, in any case, it is useful to keep the policy of static linking
of these dependencies, so as to avoid extra hassle for job submission on
clusters.

I hope the helps,

cheers



> Concerning SysCalc, we have now an internal python module handling the
> same functionalities (and more like you can change the dynamical scale
> definition).
> Since the user support for SysCalc is not that great, I would suggest to
> use that one instead. It only requires to have lhapdf6 with python
> integration activated.
>
> Additionally, Cuttools and iregi needs to be compile (we do not have any
> install for such tools which are automatically compiled when you
> “output” your first process with loop matrix-element.
>
> > I suppose the first point that you raised indicates that the default
> > path in the config file points to a path that doesn't exist.
>
> Correct that path point to the default position where that program is
> expected to be installed.
> In most of the case this is therefore not valid
>
> > Why
> > wouldn't the default be set to none, and only if the user chooses to
> > install one of these externals would the default config card be changed
> > to use that location?
>
>
> Actually we use the fact that the “ninja” path is not valid to trigger the
> question on which reduction tools.
> If the user specify that he did not want to install ninja then this path
> is set to None such that we do not ask the question again.
>
> I can obviously prevent the asking of such question if I’m not in a
> write system. But in this context this does not sounds a good idea.
>
> Cheers,
>
> Olivier
>
>
> > On 18 Feb 2017, at 04:40, Zachary Marshall <email address hidden>
> wrote:
> >
> > Hi Olivier,
> >
> > I don't love the idea of picking at random a process to run; I'd rather
> > actually run the install commands if possible.  Could you confirm that
> > these are the right ones to cover most LO and NLO use cases?
> >
> > install SysCalc Golem95 PJFry QCDLoop maddm collier ninja oneloop
> >
> > I suppose the first point that you raised indicates that the default
> > path in the config file points to a path that doesn't exist.  Why
> > wouldn't the default be set to none, and only if the user chooses to
> > install one of these externals would the default config card be changed
> > to use that location?  If the user is installing an external then we
> > know read/write is true, so it's safe to update the config card.
> >
> > Thanks again,
> > Zach
> >
> > --
> > You received this bug notification because you are subscribed to
> > MadGraph5_aMC@NLO.
> > https://bugs.launchpad.net/bugs/1657615
> >
> > Title:
> >  Configuration writing to a read-only file system in 2.5.1
> >
> > Status in MadGraph5_aMC@NLO:
> >  New
> >
> > Bug description:
> >  Hello,
> >
> >  In 2.5.1, we can run:
> >
> >  $MADGRAPH_INSTALLATION_DIR/bin/mg5_aMC
> >
> >  generate p p > t t~ [QCD]
> >  output
> >
> >  where the installation directory is on a read-only file system, but
> >  `pwd` is read/write (e.g. a scratch area or tmp directory).  This now
> >  asks about which programs to install, and then attempts two write a
> >  configuration file back to the read-only area:
> >
> >  set golem ''
> >  save options --auto
> >  save configuration file to
> >  /cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-
> opt/19.2.5/sw/lcg/external/MCGenerators_lcgcmt67c/
> madgraph5amc/2.5.1.atlas/x86_64-slc6-gcc47-opt/input/mg5_configuration.txt
> >  Command "output tt" interrupted with error:
> >  IOError : [Errno 30] Read-only file system:
> >  '/cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-
> opt/19.2.5/sw/lcg/external/MCGenerators_lcgcmt67c/
> madgraph5amc/2.5.1.atlas/x86_64-slc6-gcc47-opt/input/mg5_configuration.txt
> '
> >
> >  I believe this is because of a change in
> >
> >  madgraph/interface/extended_cmd.py
> >
> >  which now tries to save a configuration file via the
> >  write_configuration function when you try to do almost anything.  I
> >  think, since we have the dependencies set to save locally, the correct
> >  behavior would be to make a locally directory and write there (or
> >  simply not update the config card at all).
> >
> >  Thanks,
> >  Zach
> >
> > To manage notifications about this bug go to:
> > https://bugs.launchpad.net/mg5amcnlo/+bug/1657615/+subscriptions
>
> --
> You received this bug notification because you are a bug assignee.
> https://bugs.launchpad.net/bugs/1657615
>
> Title:
>   Configuration writing to a read-only file system in 2.5.1
>
> Status in MadGraph5_aMC@NLO:
>   New
>
> Bug description:
>   Hello,
>
>   In 2.5.1, we can run:
>
>   $MADGRAPH_INSTALLATION_DIR/bin/mg5_aMC
>
>   generate p p > t t~ [QCD]
>   output
>
>   where the installation directory is on a read-only file system, but
>   `pwd` is read/write (e.g. a scratch area or tmp directory).  This now
>   asks about which programs to install, and then attempts two write a
>   configuration file back to the read-only area:
>
>   set golem ''
>   save options --auto
>   save configuration file to
>   /cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-
> opt/19.2.5/sw/lcg/external/MCGenerators_lcgcmt67c/
> madgraph5amc/2.5.1.atlas/x86_64-slc6-gcc47-opt/input/mg5_configuration.txt
>   Command "output tt" interrupted with error:
>   IOError : [Errno 30] Read-only file system:
>   '/cvmfs/atlas.cern.ch/repo/sw/software/x86_64-slc6-gcc47-
> opt/19.2.5/sw/lcg/external/MCGenerators_lcgcmt67c/
> madgraph5amc/2.5.1.atlas/x86_64-slc6-gcc47-opt/input/mg5_configuration.txt
> '
>
>   I believe this is because of a change in
>
>   madgraph/interface/extended_cmd.py
>
>   which now tries to save a configuration file via the
>   write_configuration function when you try to do almost anything.  I
>   think, since we have the dependencies set to save locally, the correct
>   behavior would be to make a locally directory and write there (or
>   simply not update the config card at all).
>
>   Thanks,
>   Zach
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/mg5amcnlo/+bug/1657615/+subscriptions
>



-- 
Valentin