Overwritte a directory fails in aMC@NLO

[Olivier Mattelaer]

Hi Marco,

I found a small bug in aMC@NLO.

when doing:
! rm -rf NEW
generate p p > t t~ [QCD]; output NEW
everything is fine but if do
! rm -rf NEW
generate p p > t t~ [QCD]; output NEW; output NEW
Then it crash.

On the same kind of error:
! rm -rf NEW
generate p p > t t~
output NEW
generate p p > t t~ [QCD]
output NEW

It crash as well (but not for the same reason).
To avoid those problem, at LO, we have change the behavior a long time ago,
in order to first fully remove the directory and then recreate it (instead of trying to clean it)

Cheers,

Olivier

[marco zaro (marco-zaro)]

Hi Olivier,
thanks for spotting this.
I fixed it and pushed it directly to beta4 (since the fix was trivial).
Here you have the diff (it is the same as for LO MG5 now)
Cheers!
Marco

@@ -445,11 +446,13 @@
                and self._export_format in ['NLO']:
             # Don't ask if user already specified force or noclean
             logger.info('INFO: directory %s already exists.' % self._export_dir)
- logger.info('If you continue this directory will be cleaned')
+ logger.info('If you continue this directory will be deleted and replaced.')
             answer = self.ask('Do you want to continue?', 'y', ['y','n'],
                                                 timeout=self.options['timeout'])
             if answer != 'y':
                 raise self.InvalidCmd('Stopped by user request')
+ else:
+ shutil.rmtree(self._export_dir)

         # Make a Template Copy
         if self._export_format in ['NLO']:

On 22 Mar 2013, at 00:46, Olivier Mattelaer wrote:

> ** Changed in: madgraph5
> Importance: Undecided => Low
>
> --
> You received this bug notification because you are a member of aMC@NLO,
> which is subscribed to MadGraph5.
> Matching subscriptions: aMCatNLO_bug_reports
> https://bugs.launchpad.net/bugs/1158526
>
> Title:
> Overwritte a directory fails in aMC@NLO
>
> Status in The MadGraph Matrix Element Generator version 5:
> New
>
> Bug description:
> Hi Marco,
>
> I found a small bug in aMC@NLO.
>
> when doing:
> ! rm -rf NEW
> generate p p > t t~ [QCD]; output NEW
> everything is fine but if do
> ! rm -rf NEW
> generate p p > t t~ [QCD]; output NEW; output NEW
> Then it crash.
>
> On the same kind of error:
> ! rm -rf NEW
> generate p p > t t~
> output NEW
> generate p p > t t~ [QCD]
> output NEW
>
> It crash as well (but not for the same reason).
> To avoid those problem, at LO, we have change the behavior a long time ago,
> in order to first fully remove the directory and then recreate it (instead of trying to clean it)
>
>
> Cheers,
>
> Olivier
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/madgraph5/+bug/1158526/+subscriptions

[Olivier Mattelaer (olivier-mattelaer)]

Thanks, perfect!!!

I'll link this bug to the branch and put the bug has commited then.

Cheers,

Olivier

[Olivier Mattelaer (olivier-mattelaer)]

Hi Marco,
I face the problem yesterday in 2.0.0.beta4.
So I reopen this bug report

Cheers,

Olivier

[marco zaro (marco-zaro)]

Ciao Olivier,
with the latest 2.0.0beta4 version, i can't reproduce this bug.
Everything is fine, and I am asked if I want to remove the directory after the 2nd output command....
Let me know,
Cheers,

Marco

On 23 Aug 2013, at 17:38, Olivier Mattelaer wrote:

> Hi Marco,
> I face the problem yesterday in 2.0.0.beta4.
> So I reopen this bug report
>
> Cheers,
>
> Olivier
>
> ** Changed in: madgraph5
> Status: Fix Committed => Confirmed
>
> --
> You received this bug notification because you are a member of aMC@NLO,
> which is subscribed to MadGraph5.
> Matching subscriptions: aMCatNLO_bug_reports
> https://bugs.launchpad.net/bugs/1158526
>
> Title:
> Overwritte a directory fails in aMC@NLO
>
> Status in The MadGraph Matrix Element Generator version 5:
> Confirmed
>
> Bug description:
> Hi Marco,
>
> I found a small bug in aMC@NLO.
>
> when doing:
> ! rm -rf NEW
> generate p p > t t~ [QCD]; output NEW
> everything is fine but if do
> ! rm -rf NEW
> generate p p > t t~ [QCD]; output NEW; output NEW
> Then it crash.
>
> On the same kind of error:
> ! rm -rf NEW
> generate p p > t t~
> output NEW
> generate p p > t t~ [QCD]
> output NEW
>
> It crash as well (but not for the same reason).
> To avoid those problem, at LO, we have change the behavior a long time ago,
> in order to first fully remove the directory and then recreate it (instead of trying to clean it)
>
>
> Cheers,
>
> Olivier
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/madgraph5/+bug/1158526/+subscriptions

[Olivier Mattelaer (olivier-mattelaer)]

You right,

I think that my problem was created since I was running via the nohup command...
(I should look at allowing that...)

Thanks,

Olivier

[Olivier Mattelaer (olivier-mattelaer)]

Hi Marco,

I face the problem again:

  File "/nfs/home/fynu/omatt/2.0.0beta4/madgraph/interface/amcatnlo_interface.py", line 471, in do_output
    self.finalize(nojpeg)
  File "/nfs/home/fynu/omatt/2.0.0beta4/madgraph/interface/madgraph_interface.py", line 5190, in finalize
    files.cp(pjoin(MG5DIR, 'MCatNLO-utilities', 'MCatNLO'), self._export_dir)
  File "/nfs/home/fynu/omatt/2.0.0beta4/madgraph/iolibs/files.py", line 144, in cp
    shutil.copytree(path1, path2)
  File "/nfs/soft/python/python-2.7.3-sl5_amd64_gcc41/lib/python2.7/shutil.py", line 175, in copytree
    os.makedirs(dst)
  File "/nfs/soft/python/python-2.7.3-sl5_amd64_gcc41/lib/python2.7/os.py", line 157, in makedirs
    mkdir(name, mode)

Looks like it occur only if MC-utilities is present.
Is this really require such cp?
It make more sense to have only one version of that code and not duplicate it.

Cheers,

Olivier

[marco zaro (marco-zaro)]

Ciao Olivier,
even with the MCatNLO utilities installed, i can't reproduce your bug.
Anyway, i have added few lines in madgraph_interface.py before
> files.cp(pjoin(MG5DIR, 'MCatNLO-utilities', 'MCatNLO'), self._export_dir)
to remove the MCatNLO directory if it is present in the folder.
I have pushed it to 2.0.0beta4, rev 252.
For what concerns the fact of copying the MCatNLO directory, indeed this may not be the most optimal solution, however it is working now, and can also be justified by the fact that one may want to change e.g. the pythia internal parameters by modifying the pythia driver for a given process, but he/she does not want that this modification affects all processes.
Let me know if my modification works.
Cheers,

Marco

@@ -5217,6 +5217,11 @@

             # copy the MCatNLO directory from mcatnlo-utils inside the exported dir
             if os.path.isdir(pjoin(MG5DIR, 'MCatNLO-utilities')):
+ # first remove the MCatNLO folder if present (e.g. if overwriting)
+ try:
+ files.rm(pjoin(self._export_dir, 'MCatNLO'))
+ except OSError:
+ pass
                 files.cp(pjoin(MG5DIR, 'MCatNLO-utilities', 'MCatNLO'), self._export_dir)
             else:
                 logger.warning('MCatNLO-utilities is not installed.')

On 06 Sep 2013, at 06:49, Olivier Mattelaer wrote:

> Hi Marco,
>
> I face the problem again:
>
> File "/nfs/home/fynu/omatt/2.0.0beta4/madgraph/interface/amcatnlo_interface.py", line 471, in do_output
> self.finalize(nojpeg)
> File "/nfs/home/fynu/omatt/2.0.0beta4/madgraph/interface/madgraph_interface.py", line 5190, in finalize
> files.cp(pjoin(MG5DIR, 'MCatNLO-utilities', 'MCatNLO'), self._export_dir)
> File "/nfs/home/fynu/omatt/2.0.0beta4/madgraph/iolibs/files.py", line 144, in cp
> shutil.copytree(path1, path2)
> File "/nfs/soft/python/python-2.7.3-sl5_amd64_gcc41/lib/python2.7/shutil.py", line 175, in copytree
> os.makedirs(dst)
> File "/nfs/soft/python/python-2.7.3-sl5_amd64_gcc41/lib/python2.7/os.py", line 157, in makedirs
> mkdir(name, mode)
>
> Looks like it occur only if MC-utilities is present.
> Is this really require such cp?
> It make more sense to have only one version of that code and not duplicate it.
>
> Cheers,
>
> Olivier
>
> --
> You received this bug notification because you are a bug assignee.
> https://bugs.launchpad.net/bugs/1158526
>
> Title:
> Overwritte a directory fails in aMC@NLO
>
> Status in The MadGraph Matrix Element Generator version 5:
> Fix Committed
>
> Bug description:
> Hi Marco,
>
> I found a small bug in aMC@NLO.
>
> when doing:
> ! rm -rf NEW
> generate p p > t t~ [QCD]; output NEW
> everything is fine but if do
> ! rm -rf NEW
> generate p p > t t~ [QCD]; output NEW; output NEW
> Then it crash.
>
> On the same kind of error:
> ! rm -rf NEW
> generate p p > t t~
> output NEW
> generate p p > t t~ [QCD]
> output NEW
>
> It crash as well (but not for the same reason).
> To avoid those problem, at LO, we have change the behavior a long time ago,
> in order to first fully remove the directory and then recreate it (instead of trying ...