amd64: Corrupted memory when using Regexp::Grammars

[Olivier Mengué]

Binary package hint: perl

Core dump attached.

This is Perl 5.10.0 on amd64.

I'm trying to use Regexp::Grammars 1.002 from CPAN. This module uses hidden corners of Perl 5's Regexp engine.
Regexp::Grammars is a pure Perl module that has only dependencies on the Perl core modules.
The demo files of the Regexp::Grammars distribution work fine.

However the attached program and its datafile show a case where the memory of Perl seems to be corrupted: when using the R::G debugger, with <debug:on>, R::G says it has matched some strings which are not in my input string but instead elsewhere in the perl memory ("$Regexp::G").
Perl finally crashes (segmentation fault).

./make-Changes-rss.pl < /dev/null

Here are the first lines of the log output:

===================> Trying <grammar> from position 0
0.94 2010-02-19T2 | |...Trying <Version>
    2010-02-19T21:45 | | \_____<Version> matched '0.94'
Use of uninitialized value in substr at /home/dolmen/perl/lib/perl5/Regexp/Grammars.pm line 174, <> line 1.
Use of uninitialized value in subtraction (-) at /home/dolmen/perl/lib/perl5/Regexp/Grammars.pm line 331, <> line 1.
Use of uninitialized value in substr at /home/dolmen/perl/lib/perl5/Regexp/Grammars.pm line 174, <> line 1.
 <~~~~~~~~~~~~~~~~~~ | |...Backtracking 49 chars and trying new match
Revision history for | |...Trying <Date>
Use of uninitialized value in substr at /home/dolmen/perl/lib/perl5/Regexp/Grammars.pm line 174, <> line 1.
Use of uninitialized value in subtraction (-) at /home/dolmen/perl/lib/perl5/Regexp/Grammars.pm line 380, <> line 1.
Use of uninitialized value $Regexp::Grammars::last_try_pos in subtraction (-) at /home/dolmen/perl/lib/perl5/Regexp/Grammars.pm line 380, <> line 1.
                     | | \_____<Date> matched '$Regexp::G'
Use of uninitialized value in substr at /home/dolmen/perl/lib/perl5/Regexp/Grammars.pm line 174, <> line 2.
Use of uninitialized value in subtraction (-) at /home/dolmen/perl/lib/perl5/Regexp/Grammars.pm line 331, <> line 2.
Use of uninitialized value $Regexp::Grammars::last_try_pos in subtraction (-) at /home/dolmen/perl/lib/perl5/Regexp/Grammars.pm line 331, <> line 2.
                     | |...Trying <Time>
Use of uninitialized value in substr at /home/dolmen/perl/lib/perl5/Regexp/Grammars.pm line 174, <> line 2.
                     | | \FAIL <Time>
Use of uninitialized value in substr at /home/dolmen/perl/lib/perl5/Regexp/Grammars.pm line 174, <> line 2.
Use of uninitialized value in subtraction (-) at /home/dolmen/perl/lib/perl5/Regexp/Grammars.pm line 331, <> line 2.
Use of uninitialized value $Regexp::Grammars::last_try_pos in subtraction (-) at /home/dolmen/perl/lib/perl5/Regexp/Grammars.pm line 331, <> line 2.
                     | |...Trying <Date>
Use of uninitialized value in substr at /home/dolmen/perl/lib/perl5/Regexp/Grammars.pm line 174, <> line 2.
Use of uninitialized value in subtraction (-) at /home/dolmen/perl/lib/perl5/Regexp/Grammars.pm line 380, <> line 2.
Use of uninitialized value $Regexp::Grammars::last_try_pos in subtraction (-) at /home/dolmen/perl/lib/perl5/Regexp/Grammars.pm line 380, <> line 2.
                     | | \_____<Date> matched '$Regexp::G'
Use of uninitialized value in substr at /home/dolmen/perl/lib/perl5/Regexp/Grammars.pm line 174, <> line 3.
Use of uninitialized value in subtraction (-) at /home/dolmen/perl/lib/perl5/Regexp/Grammars.pm line 331, <> line 3.
Use of uninitialized value $Regexp::Grammars::last_try_pos in subtraction (-) at /home/dolmen/perl/lib/perl5/Regexp/Grammars.pm line 331, <> line 3.
                     | |...Trying <Time>
Use of uninitialized value in substr at /home/dolmen/perl/lib/perl5/Regexp/Grammars.pm line 174, <> line 3.
                     | | \FAIL <Time>
Use of uninitialized value in substr at /home/dolmen/perl/lib/perl5/Regexp/Grammars.pm line 174, <> line 3.
Use of uninitialized value in subtraction (-) at /home/dolmen/perl/lib/perl5/Regexp/Grammars.pm line 331, <> line 3.
Use of uninitialized value $Regexp::Grammars::last_try_pos in subtraction (-) at /home/dolmen/perl/lib/perl5/Regexp/Grammars.pm line 331, <> line 3.
                     | |...Trying <Date>
Use of uninitialized value in substr at /home/dolmen/perl/lib/perl5/Regexp/Grammars.pm line 174, <> line 3.
Use of uninitialized value in subtraction (-) at /home/dolmen/perl/lib/perl5/Regexp/Grammars.pm line 380, <> line 3.
Use of uninitialized value $Regexp::Grammars::last_try_pos in subtraction (-) at /home/dolmen/perl/lib/perl5/Regexp/Grammars.pm line 380, <> line 3.
                     | | \_____<Date> matched '$Regexp::G'
Use of uninitialized value in substr at /home/dolmen/perl/lib/perl5/Regexp/Grammars.pm line 174, <> line 4.
Use of uninitialized value in subtraction (-) at /home/dolmen/perl/lib/perl5/Regexp/Grammars.pm line 331, <> line 4.
Use of uninitialized value $Regexp::Grammars::last_try_pos in subtraction (-) at /home/dolmen/perl/lib/perl5/Regexp/Grammars.pm line 331, <> line 4.
                     | |...Trying <Time>
Use of uninitialized value in substr at /home/dolmen/perl/lib/perl5/Regexp/Grammars.pm line 174, <> line 4.
                     | | \FAIL <Time>
Use of uninitialized value in substr at /home/dolmen/perl/lib/perl5/Regexp/Grammars.pm line 174, <> line 4.
Use of uninitialized value in subtraction (-) at /home/dolmen/perl/lib/perl5/Regexp/Grammars.pm line 331, <> line 4.
Use of uninitialized value $Regexp::Grammars::last_try_pos in subtraction (-) at /home/dolmen/perl/lib/perl5/Regexp/Grammars.pm line 331, <> line 4.
                     | |...Trying <Date>
Use of uninitialized value in substr at /home/dolmen/perl/lib/perl5/Regexp/Grammars.pm line 174, <> line 4.
Use of uninitialized value in subtraction (-) at /home/dolmen/perl/lib/perl5/Regexp/Grammars.pm line 380, <> line 4.
Use of uninitialized value $Regexp::Grammars::last_try_pos in subtraction (-) at /home/dolmen/perl/lib/perl5/Regexp/Grammars.pm line 380, <> line 4.
                     | | \_____<Date> matched '$Regexp::G'
Use of uninitialized value in substr at /home/dolmen/perl/lib/perl5/Regexp/Grammars.pm line 174, <> line 5.
Use of uninitialized value in subtraction (-) at /home/dolmen/perl/lib/perl5/Regexp/Grammars.pm line 331, <> line 5.
Use of uninitialized value $Regexp::Grammars::last_try_pos in subtraction (-) at /home/dolmen/perl/lib/perl5/Regexp/Grammars.pm line 331, <> line 5.
                     | |...Trying <Time>
Use of uninitialized value in substr at /home/dolmen/perl/lib/perl5/Regexp/Grammars.pm line 174, <> line 5.
                     | | \FAIL <Time>
Use of uninitialized value in substr at /home/dolmen/perl/lib/perl5/Regexp/Grammars.pm line 174, <> line 5.
Use of uninitialized value in subtraction (-) at /home/dolmen/perl/lib/perl5/Regexp/Grammars.pm line 380, <> line 5.
Use of uninitialized value $Regexp::Grammars::last_try_pos in subtraction (-) at /home/dolmen/perl/lib/perl5/Regexp/Grammars.pm line 380, <> line 5.
                     | | \_____<Version> matched 'K; '
Use of uninitialized value in substr at /home/dolmen/perl/lib/perl5/Regexp/Grammars.pm line 174, <> line 6.
Use of uninitialized value in subtraction (-) at /home/dolmen/perl/lib/perl5/Regexp/Grammars.pm line 331, <> line 6.
Use of uninitialized value $Regexp::Grammars::last_try_pos in subtraction (-) at /home/dolmen/perl/lib/perl5/Regexp/Grammars.pm line 331, <> line 6.
                     | |...Trying <Date>
Use of uninitialized value in substr at /home/dolmen/perl/lib/perl5/Regexp/Grammars.pm line 174, <> line 6.
Erreur de segmentation

This is a security problem as it show a memory corruption in the regexp engine: Some program may allow regular expression to come from external source.

ProblemType: Bug
Architecture: amd64
Date: Sat Feb 20 09:46:52 2010
DistroRelease: Ubuntu 9.10
Package: perl 5.10.0-24ubuntu4 [modified: usr/bin/perldoc]
ProcEnviron:
 LANGUAGE=fr_FR.UTF-8
 PATH=(custom, user)
 LANG=fr_FR.UTF-8
 SHELL=/bin/bash
ProcVersionSignature: Ubuntu 2.6.31-19.56-generic
SourcePackage: perl
Uname: Linux 2.6.31-19-generic x86_64

[Olivier Mengué (dolmen)]

Coredump attached.

[Olivier Mengué (dolmen)]

I've also reported the problem to the Regexp::Grammars developer here:
http://rt.cpan.org/Public/Bug/Display.html?id=54819

[Olivier Mengué (dolmen)]

According to Damian Conway, the problem is in Perl's Unicode support.

http://rt.cpan.org/Ticket/Display.html?id=54819#txn-736810

[Olivier Mengué (dolmen)]

Here is the bug I reported upstream on the Perl 5 bug tracker:
http://rt.perl.org/rt3//Public/Bug/Display.html?id=72996

[Olivier Mengué (dolmen)]

Making public, as nobody cares.