MadSpin import not working for model from path

[Zachary Marshall]

Thank you for setting up the import of models from a path in MG5_aMC! When we run with MadSpin, I see that in MadSpin/decay.py around L1640 the model is imported like:

        cmd.exec_cmd('import model %s' % model.get('name'))

When the model is imported from a path or has some path included in it, this statement seems to fail. In my particular case, the model name is MadGraphModels/some_model_name, and the model's name is reported as just some_model_name . I can get this to work by changing that to:

        cmd.exec_cmd('import model %s' % model.path)

and it seems to work. Is that a patch that makes sense and could be applied?

Thanks,
Zach

[Olivier Mattelaer (olivier-mattelaer)]

Hi Zach,

model.path is not always defined and is not good enough in general.

The recomend way is to use:
> cmd.exec_cmd(‘import model %s’ % model.get(‘modelpath+restriction’))

which not only returns the path but also the restriction applied to the model (if any)
I will push this patch directly in our development version but can you confirm if this is indeed solves your problem?

Thanks,

Olivier

> On Jul 5, 2016, at 06:18, Zachary Marshall <email address hidden> wrote:
>
> Public bug reported:
>
> Thank you for setting up the import of models from a path in MG5_aMC!
> When we run with MadSpin, I see that in MadSpin/decay.py around L1640
> the model is imported like:
>
> cmd.exec_cmd('import model %s' % model.get('name'))
>
> When the model is imported from a path or has some path included in it,
> this statement seems to fail. In my particular case, the model name is
> MadGraphModels/some_model_name, and the model's name is reported as just
> some_model_name . I can get this to work by changing that to:
>
> cmd.exec_cmd('import model %s' % model.path)
>
> and it seems to work. Is that a patch that makes sense and could be
> applied?
>
> Thanks,
> Zach
>
> ** Affects: mg5amcnlo
> Importance: Undecided
> Status: New
>
> --
> You received this bug notification because you are subscribed to
> MadGraph5_aMC@NLO.
> https://bugs.launchpad.net/bugs/1599014
>
> Title:
> MadSpin import not working for model from path
>
> Status in MadGraph5_aMC@NLO:
> New
>
> Bug description:
> Thank you for setting up the import of models from a path in MG5_aMC!
> When we run with MadSpin, I see that in MadSpin/decay.py around L1640
> the model is imported like:
>
> cmd.exec_cmd('import model %s' % model.get('name'))
>
> When the model is imported from a path or has some path included in
> it, this statement seems to fail. In my particular case, the model
> name is MadGraphModels/some_model_name, and the model's name is
> reported as just some_model_name . I can get this to work by changing
> that to:
>
> cmd.exec_cmd('import model %s' % model.path)
>
> and it seems to work. Is that a patch that makes sense and could be
> applied?
>
> Thanks,
> Zach
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/mg5amcnlo/+bug/1599014/+subscriptions

[Zachary Marshall (zach-marshall)]

That seems to do the trick for me - thanks Olivier!

Best,
Zach

[Zachary Marshall (zach-marshall)]

Sorry Olivier, one additional question. It looks like there might be a similar problem in create_aloha.py on L629 -- is that right? Or should this one work out of the box?

Thanks,
Zach

[Olivier Mattelaer (olivier-mattelaer)]

Hi Zach,

In that case, using either
model.get(‘modelpath’) or model.get(‘modelpath+restriction’)
is certainly wrong.

That line seems actually correct not 100% safe (that line relies that the path to the model is already part of the path where to look at which is indeed done when the user enter “import model XXXX/YYYY/ZZZ”

Cheers,

Olivier

> On Jul 6, 2016, at 03:48, Zachary Marshall <email address hidden> wrote:
>
> Sorry Olivier, one additional question. It looks like there might be a
> similar problem in create_aloha.py on L629 -- is that right? Or should
> this one work out of the box?
>
> Thanks,
> Zach
>
> --
> You received this bug notification because you are subscribed to
> MadGraph5_aMC@NLO.
> https://bugs.launchpad.net/bugs/1599014
>
> Title:
> MadSpin import not working for model from path
>
> Status in MadGraph5_aMC@NLO:
> New
>
> Bug description:
> Thank you for setting up the import of models from a path in MG5_aMC!
> When we run with MadSpin, I see that in MadSpin/decay.py around L1640
> the model is imported like:
>
> cmd.exec_cmd('import model %s' % model.get('name'))
>
> When the model is imported from a path or has some path included in
> it, this statement seems to fail. In my particular case, the model
> name is MadGraphModels/some_model_name, and the model's name is
> reported as just some_model_name . I can get this to work by changing
> that to:
>
> cmd.exec_cmd('import model %s' % model.path)
>
> and it seems to work. Is that a patch that makes sense and could be
> applied?
>
> Thanks,
> Zach
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/mg5amcnlo/+bug/1599014/+subscriptions

[Zachary Marshall (zach-marshall)]

Hi Olivier,

I'd like to think it'd be ok, but it's not actually importing from that path. In this case, this is during the export. The caller is madgraph/loop/loop_exporters.py get_aloha_model(), and it looks like this:

        if not hasattr(self, 'aloha_model'):
            self.aloha_model = create_aloha.AbstractALOHAModel(model.get('name'))

So again this thing gets the model name, which doesn't give it the remainder of the path that was handed it. Right?

Thanks,
Zach

[Olivier Mattelaer (olivier-mattelaer)]

Hi Zach,

This sound perfectly fine as well.
The method used by ALOHA to import the model is a via the standard import module of python.
It therefore relies on the fact that the path of the directory containing the model is in the sys.path variable.
This is why, it does not need to have the full path since it will simply do “import MODELNAME”

Obviously, this can sometimes creates a problem:
1) if for some reason sys.path is reset or not updated as it should
2) if the name of the model is the same as an another valid python module (for example if you call your module subprocess, sys, os, numpy, …)
But those are reasonable limitation
The only annoying point is if you try to download two model with the same name in two different directory. in principle, I have put some security to detect such type of problem and have a “nice” crash indicating to close and restart MG5aMC

Cheers,

Olivier

> On Jul 6, 2016, at 14:49, Zachary Marshall <email address hidden> wrote:
>
> Hi Olivier,
>
> I'd like to think it'd be ok, but it's not actually importing from that
> path. In this case, this is during the export. The caller is
> madgraph/loop/loop_exporters.py get_aloha_model(), and it looks like
> this:
>
> if not hasattr(self, 'aloha_model'):
> self.aloha_model = create_aloha.AbstractALOHAModel(model.get('name'))
>
> So again this thing gets the model name, which doesn't give it the
> remainder of the path that was handed it. Right?
>
> Thanks,
> Zach
>
> --
> You received this bug notification because you are subscribed to
> MadGraph5_aMC@NLO.
> https://bugs.launchpad.net/bugs/1599014
>
> Title:
> MadSpin import not working for model from path
>
> Status in MadGraph5_aMC@NLO:
> New
>
> Bug description:
> Thank you for setting up the import of models from a path in MG5_aMC!
> When we run with MadSpin, I see that in MadSpin/decay.py around L1640
> the model is imported like:
>
> cmd.exec_cmd('import model %s' % model.get('name'))
>
> When the model is imported from a path or has some path included in
> it, this statement seems to fail. In my particular case, the model
> name is MadGraphModels/some_model_name, and the model's name is
> reported as just some_model_name . I can get this to work by changing
> that to:
>
> cmd.exec_cmd('import model %s' % model.path)
>
> and it seems to work. Is that a patch that makes sense and could be
> applied?
>
> Thanks,
> Zach
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/mg5amcnlo/+bug/1599014/+subscriptions

[Zachary Marshall (zach-marshall)]

Hi Olivier,

I mean, I'm staring at an example right now that does:

import model MadGraphModels/models/TopFCNC-onlyh

where the module is not directly in the python path. The aloha call gets only the name and tries to import based on the model name, and then we get a crash:

generate 14:40:14 ImportError : No module named TopFCNC-onlyh

So this is for sure not working, and not because 'os' or something else is in the path name (I agree with you that would be a reasonable restriction). If we're allowed to import from a relative path in the rest of MadGraph, shouldn't aloha support the same?

Best,
Zach

[Olivier Mattelaer (olivier-mattelaer)]

Hi Zach,

Ah ok, I was not sure that you were referring to an actual bug or just notice that sometimes get(’name’) was used.
Thanks for the example, it allow me to understand what the problem was (without the exact line, I was not able to spot any problem at all)
Please in future try to always provide the debug file, this helps so much.

So the correct patch is here to replace the line by:
aloha_model = create_aloha.AbstractALOHAModel(os.path.basename(model.get('modelpath')))
(which return only the directory name and not the full path)

Cheers and thanks,

Olivier

> On Jul 6, 2016, at 15:36, Zachary Marshall <email address hidden> wrote:
>
> Hi Olivier,
>
> I mean, I'm staring at an example right now that does:
>
> import model MadGraphModels/models/TopFCNC-onlyh
>
> where the module is not directly in the python path. The aloha call
> gets only the name and tries to import based on the model name, and then
> we get a crash:
>
> generate 14:40:14 ImportError : No module named TopFCNC-onlyh
>
> So this is for sure not working, and not because 'os' or something else
> is in the path name (I agree with you that would be a reasonable
> restriction). If we're allowed to import from a relative path in the
> rest of MadGraph, shouldn't aloha support the same?
>
> Best,
> Zach
>
> --
> You received this bug notification because you are subscribed to
> MadGraph5_aMC@NLO.
> https://bugs.launchpad.net/bugs/1599014
>
> Title:
> MadSpin import not working for model from path
>
> Status in MadGraph5_aMC@NLO:
> New
>
> Bug description:
> Thank you for setting up the import of models from a path in MG5_aMC!
> When we run with MadSpin, I see that in MadSpin/decay.py around L1640
> the model is imported like:
>
> cmd.exec_cmd('import model %s' % model.get('name'))
>
> When the model is imported from a path or has some path included in
> it, this statement seems to fail. In my particular case, the model
> name is MadGraphModels/some_model_name, and the model's name is
> reported as just some_model_name . I can get this to work by changing
> that to:
>
> cmd.exec_cmd('import model %s' % model.path)
>
> and it seems to work. Is that a patch that makes sense and could be
> applied?
>
> Thanks,
> Zach
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/mg5amcnlo/+bug/1599014/+subscriptions

[Zachary Marshall (zach-marshall)]

Thanks Olivier, that patch does indeed seem to do the job! And sorry for not being clear that I was looking at a crash; I'll be sure to send the full report next time.

Best,
Zach

[Zachary Marshall (zach-marshall)]

Ahhh, spoke too soon. Looks like exactly the same issue on L771 of madgraph/iolibs/export_v4.py , in convert_model_to_mg4. Full stack below. With the same patch there it seems to get to the end of the job properly.

Thanks again,
Zach

launch --parton --nocompile --only_generation -f --name=mcttqHWbNLO5FSNewMod
Traceback (most recent call last):
  File "/tmp/zmarshal/run5/PROCNLO_TopFCNC-onlyh_1/bin/internal/extended_cmd.py", line 1009, in onecmd
    return self.onecmd_orig(line, **opt)
  File "/tmp/zmarshal/run5/PROCNLO_TopFCNC-onlyh_1/bin/internal/extended_cmd.py", line 964, in onecmd_orig
    return func(arg, **opt)
  File "/tmp/zmarshal/run5/PROCNLO_TopFCNC-onlyh_1/bin/internal/amcatnlo_run_interface.py", line 1215, in do_launch
    self.exec_cmd('decay_events -from_cards', postcmd=False)
  File "/tmp/zmarshal/run5/PROCNLO_TopFCNC-onlyh_1/bin/internal/extended_cmd.py", line 1035, in exec_cmd
    stop = Cmd.onecmd_orig(current_interface, line, **opt)
  File "/tmp/zmarshal/run5/PROCNLO_TopFCNC-onlyh_1/bin/internal/extended_cmd.py", line 964, in onecmd_orig
    return func(arg, **opt)
  File "/tmp/zmarshal/run5/PROCNLO_TopFCNC-onlyh_1/bin/internal/common_run_interface.py", line 2006, in do_decay_events
    madspin_cmd.import_command_file(path)
  File "/tmp/zmarshal/MG5_aMC_v2_4_2/madgraph/interface/extended_cmd.py", line 1154, in import_command_file
    self.exec_cmd(line, precmd=True)
  File "/tmp/zmarshal/MG5_aMC_v2_4_2/madgraph/interface/extended_cmd.py", line 1035, in exec_cmd
    stop = Cmd.onecmd_orig(current_interface, line, **opt)
  File "/tmp/zmarshal/MG5_aMC_v2_4_2/madgraph/interface/extended_cmd.py", line 964, in onecmd_orig
    return func(arg, **opt)
  File "/tmp/zmarshal/MG5_aMC_v2_4_2/MadSpin/interface_madspin.py", line 566, in do_launch
    self.options)
  File "/tmp/zmarshal/MG5_aMC_v2_4_2/MadSpin/decay.py", line 2041, in __init__
    self.generate_all_matrix_element()
  File "/tmp/zmarshal/MG5_aMC_v2_4_2/madgraph/various/misc.py", line 169, in f_with_no_logger
    out = f(self, *args, **opt)
  File "/tmp/zmarshal/MG5_aMC_v2_4_2/tests/parallel_tests/test_aloha.py", line 50, in deco_f_set
    out = f(*args, **opt)
  File "/tmp/zmarshal/MG5_aMC_v2_4_2/MadSpin/decay.py", line 2654, in generate_all_matrix_element
    self.get_branching_ratio()
  File "/tmp/zmarshal/MG5_aMC_v2_4_2/MadSpin/decay.py", line 2932, in get_branching_ratio
    width.extract_br(self.list_branches, self.mgcmd)
  File "/tmp/zmarshal/MG5_aMC_v2_4_2/MadSpin/decay.py", line 1348, in extract_br
    self.launch_width_evaluation(to_decay, mgcmd)
  File "/tmp/zmarshal/MG5_aMC_v2_4_2/MadSpin/decay.py", line 1614, in launch_width_evaluation
    self.compute_widths(model, argument)
  File "/tmp/zmarshal/MG5_aMC_v2_4_2/MadSpin/decay.py", line 1651, in compute_widths
    cmd.exec_cmd(line)
  File "/tmp/zmarshal/MG5_aMC_v2_4_2/madgraph/interface/extended_cmd.py", line 1035, in exec_cmd
    stop = Cmd.onecmd_orig(current_interface, line, **opt)
  File "/tmp/zmarshal/MG5_aMC_v2_4_2/madgraph/interface/extended_cmd.py", line 964, in onecmd_orig
    return func(arg, **opt)
  File "/tmp/zmarshal/MG5_aMC_v2_4_2/madgraph/interface/master_interface.py", line 332, in do_co...

[Olivier Mattelaer (olivier-mattelaer)]

correct,

Looks like no one ever tested to have non standard path for NLO computation.
Thanks to help us to fix that.

Cheers,

Olivier
> On Jul 6, 2016, at 18:06, Zachary Marshall <email address hidden> wrote:
>
> Ahhh, spoke too soon. Looks like exactly the same issue on L771 of
> madgraph/iolibs/export_v4.py , in convert_model_to_mg4. Full stack
> below. With the same patch there it seems to get to the end of the job
> properly.
>
> Thanks again,
> Zach
>
>
> launch --parton --nocompile --only_generation -f --name=mcttqHWbNLO5FSNewMod
> Traceback (most recent call last):
> File "/tmp/zmarshal/run5/PROCNLO_TopFCNC-onlyh_1/bin/internal/extended_cmd.py", line 1009, in onecmd
> return self.onecmd_orig(line, **opt)
> File "/tmp/zmarshal/run5/PROCNLO_TopFCNC-onlyh_1/bin/internal/extended_cmd.py", line 964, in onecmd_orig
> return func(arg, **opt)
> File "/tmp/zmarshal/run5/PROCNLO_TopFCNC-onlyh_1/bin/internal/amcatnlo_run_interface.py", line 1215, in do_launch
> self.exec_cmd('decay_events -from_cards', postcmd=False)
> File "/tmp/zmarshal/run5/PROCNLO_TopFCNC-onlyh_1/bin/internal/extended_cmd.py", line 1035, in exec_cmd
> stop = Cmd.onecmd_orig(current_interface, line, **opt)
> File "/tmp/zmarshal/run5/PROCNLO_TopFCNC-onlyh_1/bin/internal/extended_cmd.py", line 964, in onecmd_orig
> return func(arg, **opt)
> File "/tmp/zmarshal/run5/PROCNLO_TopFCNC-onlyh_1/bin/internal/common_run_interface.py", line 2006, in do_decay_events
> madspin_cmd.import_command_file(path)
> File "/tmp/zmarshal/MG5_aMC_v2_4_2/madgraph/interface/extended_cmd.py", line 1154, in import_command_file
> self.exec_cmd(line, precmd=True)
> File "/tmp/zmarshal/MG5_aMC_v2_4_2/madgraph/interface/extended_cmd.py", line 1035, in exec_cmd
> stop = Cmd.onecmd_orig(current_interface, line, **opt)
> File "/tmp/zmarshal/MG5_aMC_v2_4_2/madgraph/interface/extended_cmd.py", line 964, in onecmd_orig
> return func(arg, **opt)
> File "/tmp/zmarshal/MG5_aMC_v2_4_2/MadSpin/interface_madspin.py", line 566, in do_launch
> self.options)
> File "/tmp/zmarshal/MG5_aMC_v2_4_2/MadSpin/decay.py", line 2041, in __init__
> self.generate_all_matrix_element()
> File "/tmp/zmarshal/MG5_aMC_v2_4_2/madgraph/various/misc.py", line 169, in f_with_no_logger
> out = f(self, *args, **opt)
> File "/tmp/zmarshal/MG5_aMC_v2_4_2/tests/parallel_tests/test_aloha.py", line 50, in deco_f_set
> out = f(*args, **opt)
> File "/tmp/zmarshal/MG5_aMC_v2_4_2/MadSpin/decay.py", line 2654, in generate_all_matrix_element
> self.get_branching_ratio()
> File "/tmp/zmarshal/MG5_aMC_v2_4_2/MadSpin/decay.py", line 2932, in get_branching_ratio
> width.extract_br(self.list_branches, self.mgcmd)
> File "/tmp/zmarshal/MG5_aMC_v2_4_2/MadSpin/decay.py", line 1348, in extract_br
> self.launch_width_evaluation(to_decay, mgcmd)
> File "/tmp/zmarshal/MG5_aMC_v2_4_2/MadSpin/decay.py", line 1614, in launch_width_evaluation
> self.compute_widths(model, argument)
> File "/tmp/zmarshal/MG5_aMC_v2_4_2/MadSpin/decay.py", line 1651, in compute_widths
> cmd.exec_cmd(line)
> File "/tmp/zmarshal/MG5_aMC_v2_4_2/madgraph/interface/extended_cmd.py", line 1035, in exec_cmd
> sto...