cannot generate events.lhe.hep larger than 4 GB with Herwig 6

[Federico Demartin]

Hi MadGuys,

I'm trying to shower a large events.lhe file with Herwig 6, but the size of the resulting events.lhe.hep file doesn't increase anymore after hitting 4 GB.
This is probably due to the same problem reported here https://bugs.launchpad.net/mg5amcnlo/+bug/1218842 .

I recompiled the StdHEP folder with the -D_FILE_OFFSET_BITS=64 flag, as Paolo suggests in that thread, but without success.
Reading that thread it seems that a fix was released for Pythia; is there a known way to solve this problem for Herwig 6 too?

Thanks,
cheers,

Federico

[Paolo Torrielli (paolo-torrielli)]

Ciao Federico,
actually Pavel Demin was the one who solved the problem in that thread…
It seems strange that what you see depends on the shower.
Have you tried to do the same with Pythia? If it does not work even in that
case, this should mean that the compilation flag has somehow not been
properly understood. If it does, I don't have any clue at the moment, and I
would just recommend to split the event file in parts (for example using
the Utilities/split_events.f script) and showering separately each sub-sample,
if possible.
Cheers.
Paolo

[Paolo Torrielli (paolo-torrielli)]

Hi again,
just a further comment: in order to do the Pythia test,
you don't need to generate another sample. You just
copy the event file, change "HERWIG6" to "PYTHIA6Q"
in the header, and change the gluon MC mass at the end
of the header (in particular, HERWIG6 uses 0.75, while
PYTHIA6 uses 0).
Of course, the stdHEP file you get this way has no physical
meaning if you are at the NLO, but this is just a technical
test.
Cheers.
Paolo

[Federico Demartin (federico-demartin)]

Ciao Paolo,
unfortunately I'm still having problems.
Btw, fyi I'm using 2.0.1 with an NLO process.

1) Hacking the event file in order to use Pythia6Q as you suggested doesn't change the result: once the output .hep file reaches 4GB, its size doesn't increase anymore (I'm monitoring this with ls -l).

2) If I compile the Utilities folder, run split_events to split the .lhe file and ask to split it into 10 files, then I get 10 lines on the screen saying " writing event file /home/fynu/fdemartin/madgraph/MG5_aMC_v2", so it seems that 10 files are generated.
But then, if I go into the Events/run_01 folder, I just find 2 files instead of 10
events.lhe.00 and .01
each with 1/10 the size of the original events.lhe file. So it seems that 8 files are missing.

Cheers,

Federico

[Paolo Torrielli (paolo-torrielli)]

Ciao Federico,

> 1) Hacking the event file in order to use Pythia6Q as you suggested
> doesn't change the result: once the output .hep file reaches 4GB, its
> size doesn't increase anymore (I'm monitoring this with ls -l).
This seems to point towards the compilation flag not being understood
properly. I don't know how much Pavel's solution is machine-independent,
probably asking him directly is the easiest way.
Maybe on your OS you have to put a different flag, I don't know
unfortunately.

> 2) If I compile the Utilities folder, run split_events to split the .lhe file and ask to split it into 10 files, then I get 10 lines on the screen saying " writing event file /home/fynu/fdemartin/madgraph/MG5_aMC_v2", so it seems that 10 files are generated.
> But then, if I go into the Events/run_01 folder, I just find 2 files instead of 10
> events.lhe.00 and .01
> each with 1/10 the size of the original events.lhe file. So it seems that 8 files are missing.
Just a few questions.
Are you running split_events from Events/run_01 (I mean executing ../../Utilities/split_events in run_01)?
Is your event file sane (I mean, it ends with the correct </LesHouchesEvents>
tag and it contains the supposed number of events (you can check
this with grep -rn "</events>" | wc -l))?

Cheers.
Paolo

>
> Cheers,
>
> Federico
>
> --
> You received this bug notification because you are a bug assignee.
> https://bugs.launchpad.net/bugs/1278308
>
> Title:
> cannot generate events.lhe.hep larger than 4 GB with Herwig 6
>
> Status in MadGraph5_aMC@NLO Generator:
> New
>
> Bug description:
> Hi MadGuys,
>
> I'm trying to shower a large events.lhe file with Herwig 6, but the size of the resulting events.lhe.hep file doesn't increase anymore after hitting 4 GB.
> This is probably due to the same problem reported here https://bugs.launchpad.net/mg5amcnlo/+bug/1218842 .
>
> I recompiled the StdHEP folder with the -D_FILE_OFFSET_BITS=64 flag, as Paolo suggests in that thread, but without success.
> Reading that thread it seems that a fix was released for Pythia; is there a known way to solve this problem for Herwig 6 too?
>
> Thanks,
> cheers,
>
> Federico
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/mg5amcnlo/+bug/1278308/+subscriptions

[Pavel Demin (pavel-demin)]

Hi,

Looks like the fix that I did for MG5/pythia-pgs was not applied to MG5_aMC/vendor/StdHEP.

I've just checked the following file

/home/fynu/fdemartin/madgraph/MG5_aMC_v2_0_1/vendor/StdHEP/mcfio/src/mcfio_Block.c

and it still uses xdr_getpos and xdr_setpos functions.

Here is a list of files that should be modified in vendor/StdHEP/mcfio/src

mcfio_Block.c
mcfio_Direct.c
mcfio_Util1.c
mcf_NTuIOFiles.c
mcf_xdr.h

You can install the modified files with the following commands:

cd vendor/StdHEP/mcfio/src
wget http://cp3.irmp.ucl.ac.be/downloads/pythia-pgs-mcfio-files.tgz
tar -vzxf pythia-pgs-mcfio-files.tgz

Then you should rebuild the StdHEP library.

Could somebody, please commit these modifications to the mg5amcnlo repository?

Cheers,

Pavel

[Pavel Demin (pavel-demin)]

BTW. The _FILE_OFFSET_BITS=64 flag is only needed on 32-bit systems. It's not the case for ingri-ui3.

[Federico Demartin (federico-demartin)]

Many thanks Pavel, now the shower seems to work correctly for output files larger than 4 GB.

Paolo, I also managed to have the split_events routine working correctly (I wasn't running it in the correct folder).

In conclusion, the only thing that needs to be done to solve the problem is to commit Pavel's modifications to amcatnlo, as he had already said.

Thanks again guys,
cheers,

Federico

[Paolo Torrielli (paolo-torrielli)]

Hi Pavel,
thanks.
As soon as I know from Olivier what is the new
branch where to push the modifications, I'll do.
Thanks again.
Cheers.
Paolo

On Feb 10, 2014, at 2:36 PM, Federico Demartin <email address hidden> wrote:

> Many thanks Pavel, now the shower seems to work correctly for output
> files larger than 4 GB.
>
> Paolo, I also managed to have the split_events routine working correctly
> (I wasn't running it in the correct folder).
>
> In conclusion, the only thing that needs to be done to solve the problem
> is to commit Pavel's modifications to amcatnlo, as he had already said.
>
> Thanks again guys,
> cheers,
>
> Federico
>
> --
> You received this bug notification because you are a bug assignee.
> https://bugs.launchpad.net/bugs/1278308
>
> Title:
> cannot generate events.lhe.hep larger than 4 GB with Herwig 6
>
> Status in MadGraph5_aMC@NLO Generator:
> New
>
> Bug description:
> Hi MadGuys,
>
> I'm trying to shower a large events.lhe file with Herwig 6, but the size of the resulting events.lhe.hep file doesn't increase anymore after hitting 4 GB.
> This is probably due to the same problem reported here https://bugs.launchpad.net/mg5amcnlo/+bug/1218842 .
>
> I recompiled the StdHEP folder with the -D_FILE_OFFSET_BITS=64 flag, as Paolo suggests in that thread, but without success.
> Reading that thread it seems that a fix was released for Pythia; is there a known way to solve this problem for Herwig 6 too?
>
> Thanks,
> cheers,
>
> Federico
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/mg5amcnlo/+bug/1278308/+subscriptions

[Olivier Mattelaer (olivier-mattelaer)]

Hi Paolo,

You can push those modification in
lp:~maddevelopers/mg5amcnlo/2.1.0

Cheers,

Olivier
On Feb 10, 2014, at 1:50 PM, Paolo Torrielli <email address hidden> wrote:

> Hi Pavel,
> thanks.
> As soon as I know from Olivier what is the new
> branch where to push the modifications, I'll do.
> Thanks again.
> Cheers.
> Paolo
>
>
> On Feb 10, 2014, at 2:36 PM, Federico Demartin <email address hidden> wrote:
>
>> Many thanks Pavel, now the shower seems to work correctly for output
>> files larger than 4 GB.
>>
>> Paolo, I also managed to have the split_events routine working correctly
>> (I wasn't running it in the correct folder).
>>
>> In conclusion, the only thing that needs to be done to solve the problem
>> is to commit Pavel's modifications to amcatnlo, as he had already said.
>>
>> Thanks again guys,
>> cheers,
>>
>> Federico
>>
>> --
>> You received this bug notification because you are a bug assignee.
>> https://bugs.launchpad.net/bugs/1278308
>>
>> Title:
>> cannot generate events.lhe.hep larger than 4 GB with Herwig 6
>>
>> Status in MadGraph5_aMC@NLO Generator:
>> New
>>
>> Bug description:
>> Hi MadGuys,
>>
>> I'm trying to shower a large events.lhe file with Herwig 6, but the size of the resulting events.lhe.hep file doesn't increase anymore after hitting 4 GB.
>> This is probably due to the same problem reported here https://bugs.launchpad.net/mg5amcnlo/+bug/1218842 .
>>
>> I recompiled the StdHEP folder with the -D_FILE_OFFSET_BITS=64 flag, as Paolo suggests in that thread, but without success.
>> Reading that thread it seems that a fix was released for Pythia; is there a known way to solve this problem for Herwig 6 too?
>>
>> Thanks,
>> cheers,
>>
>> Federico
>>
>> To manage notifications about this bug go to:
>> https://bugs.launchpad.net/mg5amcnlo/+bug/1278308/+subscriptions
>
> --
> You received this bug notification because you are subscribed to
> MadGraph5_aMC@NLO.
> https://bugs.launchpad.net/bugs/1278308
>
> Title:
> cannot generate events.lhe.hep larger than 4 GB with Herwig 6
>
> Status in MadGraph5_aMC@NLO Generator:
> New
>
> Bug description:
> Hi MadGuys,
>
> I'm trying to shower a large events.lhe file with Herwig 6, but the size of the resulting events.lhe.hep file doesn't increase anymore after hitting 4 GB.
> This is probably due to the same problem reported here https://bugs.launchpad.net/mg5amcnlo/+bug/1218842 .
>
> I recompiled the StdHEP folder with the -D_FILE_OFFSET_BITS=64 flag, as Paolo suggests in that thread, but without success.
> Reading that thread it seems that a fix was released for Pythia; is there a known way to solve this problem for Herwig 6 too?
>
> Thanks,
> cheers,
>
> Federico
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/mg5amcnlo/+bug/1278308/+subscriptions

[Paolo Torrielli (paolo-torrielli)]

Ok Thanks.
Cheers.
Paolo

On Feb 10, 2014, at 3:02 PM, Olivier Mattelaer <email address hidden> wrote:

> Hi Paolo,
>
> You can push those modification in
> lp:~maddevelopers/mg5amcnlo/2.1.0
>
> Cheers,
>
> Olivier
> On Feb 10, 2014, at 1:50 PM, Paolo Torrielli <email address hidden> wrote:
>
>> Hi Pavel,
>> thanks.
>> As soon as I know from Olivier what is the new
>> branch where to push the modifications, I'll do.
>> Thanks again.
>> Cheers.
>> Paolo
>>
>>
>> On Feb 10, 2014, at 2:36 PM, Federico Demartin <email address hidden> wrote:
>>
>>> Many thanks Pavel, now the shower seems to work correctly for output
>>> files larger than 4 GB.
>>>
>>> Paolo, I also managed to have the split_events routine working correctly
>>> (I wasn't running it in the correct folder).
>>>
>>> In conclusion, the only thing that needs to be done to solve the problem
>>> is to commit Pavel's modifications to amcatnlo, as he had already said.
>>>
>>> Thanks again guys,
>>> cheers,
>>>
>>> Federico
>>>
>>> --
>>> You received this bug notification because you are a bug assignee.
>>> https://bugs.launchpad.net/bugs/1278308
>>>
>>> Title:
>>> cannot generate events.lhe.hep larger than 4 GB with Herwig 6
>>>
>>> Status in MadGraph5_aMC@NLO Generator:
>>> New
>>>
>>> Bug description:
>>> Hi MadGuys,
>>>
>>> I'm trying to shower a large events.lhe file with Herwig 6, but the size of the resulting events.lhe.hep file doesn't increase anymore after hitting 4 GB.
>>> This is probably due to the same problem reported here https://bugs.launchpad.net/mg5amcnlo/+bug/1218842 .
>>>
>>> I recompiled the StdHEP folder with the -D_FILE_OFFSET_BITS=64 flag, as Paolo suggests in that thread, but without success.
>>> Reading that thread it seems that a fix was released for Pythia; is there a known way to solve this problem for Herwig 6 too?
>>>
>>> Thanks,
>>> cheers,
>>>
>>> Federico
>>>
>>> To manage notifications about this bug go to:
>>> https://bugs.launchpad.net/mg5amcnlo/+bug/1278308/+subscriptions
>>
>> --
>> You received this bug notification because you are subscribed to
>> MadGraph5_aMC@NLO.
>> https://bugs.launchpad.net/bugs/1278308
>>
>> Title:
>> cannot generate events.lhe.hep larger than 4 GB with Herwig 6
>>
>> Status in MadGraph5_aMC@NLO Generator:
>> New
>>
>> Bug description:
>> Hi MadGuys,
>>
>> I'm trying to shower a large events.lhe file with Herwig 6, but the size of the resulting events.lhe.hep file doesn't increase anymore after hitting 4 GB.
>> This is probably due to the same problem reported here https://bugs.launchpad.net/mg5amcnlo/+bug/1218842 .
>>
>> I recompiled the StdHEP folder with the -D_FILE_OFFSET_BITS=64 flag, as Paolo suggests in that thread, but without success.
>> Reading that thread it seems that a fix was released for Pythia; is there a known way to solve this problem for Herwig 6 too?
>>
>> Thanks,
>> cheers,
>>
>> Federico
>>
>> To manage notifications about this bug go to:
>> https://bugs.launchpad.net/mg5amcnlo/+bug/1278308/+subscriptions
>
> --
> You received this bug notification because you are a bug assignee.
> https://bugs.launchpad.net/bugs/1278308
>
> Title:
> ...

[Olivier Mattelaer (olivier-mattelaer)]

Hi Paolo,

I guess that the correct status for this bug is "fix commited" isn't it?

Cheers,

Olivier