Comment 9 for bug 1218842

Revision history for this message
Paolo Torrielli (paolo-torrielli) wrote : Re: [Bug 1218842] STDHEP files larger than 4GB are corrupted

Yes, thanks very much.
Cheers.
Paolo

On Nov 28, 2013, at 12:20 AM, Olivier Mattelaer <email address hidden> wrote:

> Thanks a lot for the investigation.
>
> Cheers,
>
> Olivier
> On Nov 27, 2013, at 10:29 PM, Pavel Demin <email address hidden> wrote:
>
>> I've just looked at the mcfio/stdhep code includedd with the pythia-pgs
>> package in MG5.
>>
>> The code in pythia-pgs/src/pythia.f calls the following chain of
>> functions to write an event:
>>
>> 1. stdxwrt defined in pythia-pgs/libraries/PGS4/src/stdhep-dir/src/stdhep/stdxwrt.F
>> 2. stdxwevt defined in pythia-pgs/libraries/PGS4/src/stdhep-dir/src/stdhep/stdxwevt.F
>> 3. mcfio_block defined in pythia-pgs/libraries/PGS4/src/stdhep-dir/mcfio/src/mcfio_Block.c
>>
>> Then he mcfio_block function uses the xdr_getpos and xdr_setpos
>> functions from the XDR library. These functions are used to get and set
>> postion in the output file.
>>
>> The problem is that these functions operate on 32-bit values and return
>> 32-bit values. When the position in file reaches the 4GB limit, the
>> 32-bit position overflows and the mcfio/sdthep librarry starts to write
>> events in the very beginning of the file and thus corrupts the file
>> header.
>>
>> So, my conclusion is that the mcfio/stdhep library used by the pythia-
>> pgs/src/pythia.f program can't work with files larger than 4GB.
>>
>> --
>> You received this bug notification because you are a member of MadTeam,
>> which is subscribed to MadGraph5.
>> https://bugs.launchpad.net/bugs/1218842
>>
>> Title:
>> STDHEP files larger than 4GB are corrupted
>>
>> Status in The MadGraph Matrix Element Generator version 5:
>> Confirmed
>>
>> Bug description:
>> I've just tested several STDHEP files produced by MG+HERWIG6.
>>
>> I've tested 50MB, 3.3GB and 4.0GB files.
>>
>> The 50MB and 3.3GB files can be read without any problem.
>>
>> The 4.0GB file can't be read.
>>
>> In the 4.0GB file, already the 2nd block has a block ID number equal
>> 2. Normally, it should be 4 (Event Header). The following blocks have
>> completely wrong block ID numbers. So, I'd say that it's corrupted
>> from the very beginning.
>>
>> In some cases, adding the following compilation flag helps:
>>
>> -D_FILE_OFFSET_BITS=64
>>
>> Normally, the STDHEP library and maybe the PYTHIA and HERWIG
>> interfaces should be compiled with this flag.
>>
>> If you can tell me where to put this flag, I can redo the test.
>>
>> To manage notifications about this bug go to:
>> https://bugs.launchpad.net/madgraph5/+bug/1218842/+subscriptions
>
> --
> You received this bug notification because you are a bug assignee.
> https://bugs.launchpad.net/bugs/1218842
>
> Title:
> STDHEP files larger than 4GB are corrupted
>
> Status in The MadGraph Matrix Element Generator version 5:
> Confirmed
>
> Bug description:
> I've just tested several STDHEP files produced by MG+HERWIG6.
>
> I've tested 50MB, 3.3GB and 4.0GB files.
>
> The 50MB and 3.3GB files can be read without any problem.
>
> The 4.0GB file can't be read.
>
> In the 4.0GB file, already the 2nd block has a block ID number equal
> 2. Normally, it should be 4 (Event Header). The following blocks have
> completely wrong block ID numbers. So, I'd say that it's corrupted
> from the very beginning.
>
> In some cases, adding the following compilation flag helps:
>
> -D_FILE_OFFSET_BITS=64
>
> Normally, the STDHEP library and maybe the PYTHIA and HERWIG
> interfaces should be compiled with this flag.
>
> If you can tell me where to put this flag, I can redo the test.
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/madgraph5/+bug/1218842/+subscriptions