Activity log for bug #1915959

Date Who What changed Old value New value Message
2021-02-17 20:01:25 lordaro bug added bug
2021-02-17 20:01:25 lordaro attachment added Proposed patch fixing the issue https://bugs.launchpad.net/bugs/1915959/+attachment/5464650/+files/mod_perl-argv-null-terminator.diff
2021-02-17 20:32:27 Ubuntu Foundations Team Bug Bot tags patch
2021-02-17 20:32:34 Ubuntu Foundations Team Bug Bot bug added subscriber Ubuntu Review Team
2021-02-18 12:16:41 Paride Legovini libapache2-mod-perl2 (Ubuntu): status New Incomplete
2021-02-18 13:02:11 Paride Legovini bug added subscriber Ubuntu Server
2021-02-24 09:05:48 Launchpad Janitor libapache2-mod-perl2 (Ubuntu): status Incomplete Fix Released
2021-03-01 10:08:23 Robie Basak nominated for series Ubuntu Focal
2021-03-01 10:08:23 Robie Basak bug task added libapache2-mod-perl2 (Ubuntu Focal)
2021-03-01 10:08:28 Robie Basak libapache2-mod-perl2 (Ubuntu Focal): status New Triaged
2021-03-01 10:08:33 Robie Basak libapache2-mod-perl2 (Ubuntu Focal): importance Undecided Medium
2021-03-01 10:08:41 Robie Basak tags patch bitesize patch server-next
2021-03-08 09:49:36 lordaro description While setting up a perl web application with mod_perl & apache, apache kept segfaulting. Broke out gdb, and found that it was segfaulting within perl itself Program received signal SIGSEGV, Segmentation fault. 0x00007ffff7358ff5 in perl_parse () from /lib/x86_64-linux-gnu/libperl.so.5.30 (gdb) bt #0 0x00007ffff7358ff5 in perl_parse () from /lib/x86_64-linux-gnu/libperl.so.5.30 #1 0x00007ffff764cd0c in modperl_startup () from /usr/lib/apache2/modules/mod_perl.so #2 0x00007ffff764cc97 in modperl_startup () from /usr/lib/apache2/modules/mod_perl.so #3 0x00007ffff764d0fa in modperl_init () from /usr/lib/apache2/modules/mod_perl.so #4 0x00007ffff764d27b in modperl_hook_init () from /usr/lib/apache2/modules/mod_perl.so #5 0x00005555555b23d4 in ap_run_open_logs () #6 0x000055555558c440 in main () # valgrind apache2 -k start -X ==22529== Memcheck, a memory error detector ==22529== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al. ==22529== Using Valgrind-3.15.0 and LibVEX; rerun with -h for copyright info ==22529== Command: apache2 -k start -X ==22529== ==22529== Invalid read of size 8 ==22529== at 0x564AFF5: perl_parse (in /usr/lib/x86_64-linux-gnu/libperl.so.5.30.0) ==22529== by 0x55A8D0B: modperl_startup (in /usr/lib/apache2/modules/mod_perl.so) ==22529== by 0x55A8C96: modperl_startup (in /usr/lib/apache2/modules/mod_perl.so) ==22529== by 0x55A90F9: modperl_init (in /usr/lib/apache2/modules/mod_perl.so) ==22529== by 0x55A927A: modperl_hook_init (in /usr/lib/apache2/modules/mod_perl.so) ==22529== by 0x1663D3: ap_run_open_logs (in /usr/sbin/apache2) ==22529== by 0x14043F: main (in /usr/sbin/apache2) ==22529== Address 0x5a44000 is not stack'd, malloc'd or (recently) free'd ==22529== ==22529== ==22529== Process terminating with default action of signal 11 (SIGSEGV) ==22529== Access not within mapped region at address 0x5A44000 ==22529== at 0x564AFF5: perl_parse (in /usr/lib/x86_64-linux-gnu/libperl.so.5.30.0) ==22529== by 0x55A8D0B: modperl_startup (in /usr/lib/apache2/modules/mod_perl.so) ==22529== by 0x55A8C96: modperl_startup (in /usr/lib/apache2/modules/mod_perl.so) ==22529== by 0x55A90F9: modperl_init (in /usr/lib/apache2/modules/mod_perl.so) ==22529== by 0x55A927A: modperl_hook_init (in /usr/lib/apache2/modules/mod_perl.so) ==22529== by 0x1663D3: ap_run_open_logs (in /usr/sbin/apache2) ==22529== by 0x14043F: main (in /usr/sbin/apache2) gdb indicated that it was erroring in very early in perl's runtime, before it had got to any perl code. When using debug symbols, the exact line it was failing on was `scriptname = argv[0];` (perl.c:2365) It wasn't possible to reason beyond that as stepping through optimised code even with debug symbols is next to impossible to make any sense of. I did find that building an unoptimised perl made the error go away. I found the following closed issue: https://github.com/Perl/perl5/issues/15806 which describes the same issue I was having. Looking at the source for mod_perl, I found that the argv array passed to perl_parse() is not NULL terminated as is required by perl - ( documentation: https://perldoc.perl.org/perlembed#Adding-a-Perl-interpreter-to-your-C-program ) After patching this, the problem went away and didn't come back. Patch is attached. However, for some reason or another, the problem is very inconsistent. Sometimes it will work fine, other times it will fail repeatedly. I suspect it has something to do with my environment (ubuntu-minimal:focal LXD container). It almost feels like the act of installing all the build dependencies for mod_perl made the crash disappear. However, with a clean container and copying the patched deb across, it does appear to be repeatable: # ls libapache2-mod-perl2_2.0.11-2_amd64.clean.deb libapache2-mod-perl2_2.0.11-2_amd64.patched.deb # dpkg -i libapache2-mod-perl2_2.0.11-2_amd64.clean.deb (Reading database ... 33224 files and directories currently installed.) Preparing to unpack libapache2-mod-perl2_2.0.11-2_amd64.clean.deb ... Unpacking libapache2-mod-perl2 (2.0.11-2) over (2.0.11-2) ... Setting up libapache2-mod-perl2 (2.0.11-2) ... apache2_invoke perl: already enabled # source /etc/apache2/envvars # apache2 -k start -X Segmentation fault (core dumped) # dpkg -i libapache2-mod-perl2_2.0.11-2_amd64.patched.deb (Reading database ... 33224 files and directories currently installed.) Preparing to unpack libapache2-mod-perl2_2.0.11-2_amd64.patched.deb ... Unpacking libapache2-mod-perl2 (2.0.11-2) over (2.0.11-2) ... Setting up libapache2-mod-perl2 (2.0.11-2) ... apache2_invoke perl: already enabled # apache2 -k start -X <success>^C # dpkg -i libapache2-mod-perl2_2.0.11-2_amd64.clean.deb (Reading database ... 33224 files and directories currently installed.) Preparing to unpack libapache2-mod-perl2_2.0.11-2_amd64.clean.deb ... Unpacking libapache2-mod-perl2 (2.0.11-2) over (2.0.11-2) ... Setting up libapache2-mod-perl2 (2.0.11-2) ... apache2_invoke perl: already enabled # apache2 -k start -X Segmentation fault (core dumped) While setting up a perl web application with mod_perl & apache, apache kept segfaulting. Broke out gdb, and found that it was segfaulting within perl itself Program received signal SIGSEGV, Segmentation fault. 0x00007ffff7358ff5 in perl_parse () from /lib/x86_64-linux-gnu/libperl.so.5.30 (gdb) bt #0 0x00007ffff7358ff5 in perl_parse () from /lib/x86_64-linux-gnu/libperl.so.5.30 #1 0x00007ffff764cd0c in modperl_startup () from /usr/lib/apache2/modules/mod_perl.so #2 0x00007ffff764cc97 in modperl_startup () from /usr/lib/apache2/modules/mod_perl.so #3 0x00007ffff764d0fa in modperl_init () from /usr/lib/apache2/modules/mod_perl.so #4 0x00007ffff764d27b in modperl_hook_init () from /usr/lib/apache2/modules/mod_perl.so #5 0x00005555555b23d4 in ap_run_open_logs () #6 0x000055555558c440 in main ()  # valgrind apache2 -k start -X ==22529== Memcheck, a memory error detector ==22529== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al. ==22529== Using Valgrind-3.15.0 and LibVEX; rerun with -h for copyright info ==22529== Command: apache2 -k start -X ==22529== ==22529== Invalid read of size 8 ==22529== at 0x564AFF5: perl_parse (in /usr/lib/x86_64-linux-gnu/libperl.so.5.30.0) ==22529== by 0x55A8D0B: modperl_startup (in /usr/lib/apache2/modules/mod_perl.so) ==22529== by 0x55A8C96: modperl_startup (in /usr/lib/apache2/modules/mod_perl.so) ==22529== by 0x55A90F9: modperl_init (in /usr/lib/apache2/modules/mod_perl.so) ==22529== by 0x55A927A: modperl_hook_init (in /usr/lib/apache2/modules/mod_perl.so) ==22529== by 0x1663D3: ap_run_open_logs (in /usr/sbin/apache2) ==22529== by 0x14043F: main (in /usr/sbin/apache2) ==22529== Address 0x5a44000 is not stack'd, malloc'd or (recently) free'd ==22529== ==22529== ==22529== Process terminating with default action of signal 11 (SIGSEGV) ==22529== Access not within mapped region at address 0x5A44000 ==22529== at 0x564AFF5: perl_parse (in /usr/lib/x86_64-linux-gnu/libperl.so.5.30.0) ==22529== by 0x55A8D0B: modperl_startup (in /usr/lib/apache2/modules/mod_perl.so) ==22529== by 0x55A8C96: modperl_startup (in /usr/lib/apache2/modules/mod_perl.so) ==22529== by 0x55A90F9: modperl_init (in /usr/lib/apache2/modules/mod_perl.so) ==22529== by 0x55A927A: modperl_hook_init (in /usr/lib/apache2/modules/mod_perl.so) ==22529== by 0x1663D3: ap_run_open_logs (in /usr/sbin/apache2) ==22529== by 0x14043F: main (in /usr/sbin/apache2) gdb indicated that it was erroring in very early in perl's runtime, before it had got to any perl code. When using debug symbols, the exact line it was failing on was `scriptname = argv[0];` (perl.c:2365) It wasn't possible to reason beyond that as stepping through optimised code even with debug symbols is next to impossible to make any sense of. I did find that building an unoptimised perl made the error go away. I found the following closed issue: https://github.com/Perl/perl5/issues/15806 which describes the same issue I was having. Looking at the source for mod_perl, I found that the argv array passed to perl_parse() is not NULL terminated as is required by perl - ( documentation: https://perldoc.perl.org/perlembed#Adding-a-Perl-interpreter-to-your-C-program ) After patching this, the problem went away and didn't come back. Patch is attached. However, for some reason or another, the problem is very inconsistent. Sometimes it will work fine, other times it will fail repeatedly. I suspect it has something to do with my environment (ubuntu-minimal:focal LXD container). It almost feels like the act of installing all the build dependencies for mod_perl made the crash disappear. However, with a clean container and copying the patched deb across, it does appear to be repeatable:  # ls libapache2-mod-perl2_2.0.11-2_amd64.clean.deb libapache2-mod-perl2_2.0.11-2_amd64.patched.deb  # dpkg -i libapache2-mod-perl2_2.0.11-2_amd64.clean.deb (Reading database ... 33224 files and directories currently installed.) Preparing to unpack libapache2-mod-perl2_2.0.11-2_amd64.clean.deb ... Unpacking libapache2-mod-perl2 (2.0.11-2) over (2.0.11-2) ... Setting up libapache2-mod-perl2 (2.0.11-2) ... apache2_invoke perl: already enabled  # source /etc/apache2/envvars  # apache2 -k start -X Segmentation fault (core dumped)  # dpkg -i libapache2-mod-perl2_2.0.11-2_amd64.patched.deb (Reading database ... 33224 files and directories currently installed.) Preparing to unpack libapache2-mod-perl2_2.0.11-2_amd64.patched.deb ... Unpacking libapache2-mod-perl2 (2.0.11-2) over (2.0.11-2) ... Setting up libapache2-mod-perl2 (2.0.11-2) ... apache2_invoke perl: already enabled  # apache2 -k start -X <success>^C  # dpkg -i libapache2-mod-perl2_2.0.11-2_amd64.clean.deb (Reading database ... 33224 files and directories currently installed.) Preparing to unpack libapache2-mod-perl2_2.0.11-2_amd64.clean.deb ... Unpacking libapache2-mod-perl2 (2.0.11-2) over (2.0.11-2) ... Setting up libapache2-mod-perl2 (2.0.11-2) ... apache2_invoke perl: already enabled  # apache2 -k start -X Segmentation fault (core dumped)
2021-03-19 13:21:46 Utkarsh Gupta libapache2-mod-perl2 (Ubuntu Focal): assignee Utkarsh Gupta (utkarsh)
2021-03-19 13:41:16 Launchpad Janitor merge proposal linked https://code.launchpad.net/~utkarsh/ubuntu/+source/libapache2-mod-perl2/+git/libapache2-mod-perl2/+merge/399921
2021-03-19 14:10:59 Utkarsh Gupta description While setting up a perl web application with mod_perl & apache, apache kept segfaulting. Broke out gdb, and found that it was segfaulting within perl itself Program received signal SIGSEGV, Segmentation fault. 0x00007ffff7358ff5 in perl_parse () from /lib/x86_64-linux-gnu/libperl.so.5.30 (gdb) bt #0 0x00007ffff7358ff5 in perl_parse () from /lib/x86_64-linux-gnu/libperl.so.5.30 #1 0x00007ffff764cd0c in modperl_startup () from /usr/lib/apache2/modules/mod_perl.so #2 0x00007ffff764cc97 in modperl_startup () from /usr/lib/apache2/modules/mod_perl.so #3 0x00007ffff764d0fa in modperl_init () from /usr/lib/apache2/modules/mod_perl.so #4 0x00007ffff764d27b in modperl_hook_init () from /usr/lib/apache2/modules/mod_perl.so #5 0x00005555555b23d4 in ap_run_open_logs () #6 0x000055555558c440 in main ()  # valgrind apache2 -k start -X ==22529== Memcheck, a memory error detector ==22529== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al. ==22529== Using Valgrind-3.15.0 and LibVEX; rerun with -h for copyright info ==22529== Command: apache2 -k start -X ==22529== ==22529== Invalid read of size 8 ==22529== at 0x564AFF5: perl_parse (in /usr/lib/x86_64-linux-gnu/libperl.so.5.30.0) ==22529== by 0x55A8D0B: modperl_startup (in /usr/lib/apache2/modules/mod_perl.so) ==22529== by 0x55A8C96: modperl_startup (in /usr/lib/apache2/modules/mod_perl.so) ==22529== by 0x55A90F9: modperl_init (in /usr/lib/apache2/modules/mod_perl.so) ==22529== by 0x55A927A: modperl_hook_init (in /usr/lib/apache2/modules/mod_perl.so) ==22529== by 0x1663D3: ap_run_open_logs (in /usr/sbin/apache2) ==22529== by 0x14043F: main (in /usr/sbin/apache2) ==22529== Address 0x5a44000 is not stack'd, malloc'd or (recently) free'd ==22529== ==22529== ==22529== Process terminating with default action of signal 11 (SIGSEGV) ==22529== Access not within mapped region at address 0x5A44000 ==22529== at 0x564AFF5: perl_parse (in /usr/lib/x86_64-linux-gnu/libperl.so.5.30.0) ==22529== by 0x55A8D0B: modperl_startup (in /usr/lib/apache2/modules/mod_perl.so) ==22529== by 0x55A8C96: modperl_startup (in /usr/lib/apache2/modules/mod_perl.so) ==22529== by 0x55A90F9: modperl_init (in /usr/lib/apache2/modules/mod_perl.so) ==22529== by 0x55A927A: modperl_hook_init (in /usr/lib/apache2/modules/mod_perl.so) ==22529== by 0x1663D3: ap_run_open_logs (in /usr/sbin/apache2) ==22529== by 0x14043F: main (in /usr/sbin/apache2) gdb indicated that it was erroring in very early in perl's runtime, before it had got to any perl code. When using debug symbols, the exact line it was failing on was `scriptname = argv[0];` (perl.c:2365) It wasn't possible to reason beyond that as stepping through optimised code even with debug symbols is next to impossible to make any sense of. I did find that building an unoptimised perl made the error go away. I found the following closed issue: https://github.com/Perl/perl5/issues/15806 which describes the same issue I was having. Looking at the source for mod_perl, I found that the argv array passed to perl_parse() is not NULL terminated as is required by perl - ( documentation: https://perldoc.perl.org/perlembed#Adding-a-Perl-interpreter-to-your-C-program ) After patching this, the problem went away and didn't come back. Patch is attached. However, for some reason or another, the problem is very inconsistent. Sometimes it will work fine, other times it will fail repeatedly. I suspect it has something to do with my environment (ubuntu-minimal:focal LXD container). It almost feels like the act of installing all the build dependencies for mod_perl made the crash disappear. However, with a clean container and copying the patched deb across, it does appear to be repeatable:  # ls libapache2-mod-perl2_2.0.11-2_amd64.clean.deb libapache2-mod-perl2_2.0.11-2_amd64.patched.deb  # dpkg -i libapache2-mod-perl2_2.0.11-2_amd64.clean.deb (Reading database ... 33224 files and directories currently installed.) Preparing to unpack libapache2-mod-perl2_2.0.11-2_amd64.clean.deb ... Unpacking libapache2-mod-perl2 (2.0.11-2) over (2.0.11-2) ... Setting up libapache2-mod-perl2 (2.0.11-2) ... apache2_invoke perl: already enabled  # source /etc/apache2/envvars  # apache2 -k start -X Segmentation fault (core dumped)  # dpkg -i libapache2-mod-perl2_2.0.11-2_amd64.patched.deb (Reading database ... 33224 files and directories currently installed.) Preparing to unpack libapache2-mod-perl2_2.0.11-2_amd64.patched.deb ... Unpacking libapache2-mod-perl2 (2.0.11-2) over (2.0.11-2) ... Setting up libapache2-mod-perl2 (2.0.11-2) ... apache2_invoke perl: already enabled  # apache2 -k start -X <success>^C  # dpkg -i libapache2-mod-perl2_2.0.11-2_amd64.clean.deb (Reading database ... 33224 files and directories currently installed.) Preparing to unpack libapache2-mod-perl2_2.0.11-2_amd64.clean.deb ... Unpacking libapache2-mod-perl2 (2.0.11-2) over (2.0.11-2) ... Setting up libapache2-mod-perl2 (2.0.11-2) ... apache2_invoke perl: already enabled  # apache2 -k start -X Segmentation fault (core dumped) [Impact] ======== While setting up a perl web application with mod_perl & apache, apache keeps segfaulting. Broke out gdb, and found that it was segfaulting within perl itself Program received signal SIGSEGV, Segmentation fault. 0x00007ffff7358ff5 in perl_parse () from /lib/x86_64-linux-gnu/libperl.so.5.30 (gdb) bt #0 0x00007ffff7358ff5 in perl_parse () from /lib/x86_64-linux-gnu/libperl.so.5.30 #1 0x00007ffff764cd0c in modperl_startup () from /usr/lib/apache2/modules/mod_perl.so #2 0x00007ffff764cc97 in modperl_startup () from /usr/lib/apache2/modules/mod_perl.so #3 0x00007ffff764d0fa in modperl_init () from /usr/lib/apache2/modules/mod_perl.so #4 0x00007ffff764d27b in modperl_hook_init () from /usr/lib/apache2/modules/mod_perl.so #5 0x00005555555b23d4 in ap_run_open_logs () #6 0x000055555558c440 in main ()  # valgrind apache2 -k start -X ==22529== Memcheck, a memory error detector ==22529== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al. ==22529== Using Valgrind-3.15.0 and LibVEX; rerun with -h for copyright info ==22529== Command: apache2 -k start -X ==22529== ==22529== Invalid read of size 8 ==22529== at 0x564AFF5: perl_parse (in /usr/lib/x86_64-linux-gnu/libperl.so.5.30.0) ==22529== by 0x55A8D0B: modperl_startup (in /usr/lib/apache2/modules/mod_perl.so) ==22529== by 0x55A8C96: modperl_startup (in /usr/lib/apache2/modules/mod_perl.so) ==22529== by 0x55A90F9: modperl_init (in /usr/lib/apache2/modules/mod_perl.so) ==22529== by 0x55A927A: modperl_hook_init (in /usr/lib/apache2/modules/mod_perl.so) ==22529== by 0x1663D3: ap_run_open_logs (in /usr/sbin/apache2) ==22529== by 0x14043F: main (in /usr/sbin/apache2) ==22529== Address 0x5a44000 is not stack'd, malloc'd or (recently) free'd ==22529== ==22529== ==22529== Process terminating with default action of signal 11 (SIGSEGV) ==22529== Access not within mapped region at address 0x5A44000 ==22529== at 0x564AFF5: perl_parse (in /usr/lib/x86_64-linux-gnu/libperl.so.5.30.0) ==22529== by 0x55A8D0B: modperl_startup (in /usr/lib/apache2/modules/mod_perl.so) ==22529== by 0x55A8C96: modperl_startup (in /usr/lib/apache2/modules/mod_perl.so) ==22529== by 0x55A90F9: modperl_init (in /usr/lib/apache2/modules/mod_perl.so) ==22529== by 0x55A927A: modperl_hook_init (in /usr/lib/apache2/modules/mod_perl.so) ==22529== by 0x1663D3: ap_run_open_logs (in /usr/sbin/apache2) ==22529== by 0x14043F: main (in /usr/sbin/apache2) gdb indicated that it was erroring in very early in perl's runtime, before it had got to any perl code. When using debug symbols, the exact line it was failing on was `scriptname = argv[0];` (perl.c:2365) It wasn't possible to reason beyond that as stepping through optimised code even with debug symbols is next to impossible to make any sense of. I did find that building an unoptimised perl made the error go away. I found the following closed issue: https://github.com/Perl/perl5/issues/15806 which describes the same issue I was having. Looking at the source for mod_perl, I found that the argv array passed to perl_parse() is not NULL terminated as is required by perl - ( documentation: https://perldoc.perl.org/perlembed#Adding-a-Perl-interpreter-to-your-C-program ) After patching this, the problem went away and didn't come back. Patch is attached. [Test Plan] ===========  # ls libapache2-mod-perl2_2.0.11-2_amd64.clean.deb libapache2-mod-perl2_2.0.11-2_amd64.patched.deb  # dpkg -i libapache2-mod-perl2_2.0.11-2_amd64.clean.deb (Reading database ... 33224 files and directories currently installed.) Preparing to unpack libapache2-mod-perl2_2.0.11-2_amd64.clean.deb ... Unpacking libapache2-mod-perl2 (2.0.11-2) over (2.0.11-2) ... Setting up libapache2-mod-perl2 (2.0.11-2) ... apache2_invoke perl: already enabled  # source /etc/apache2/envvars  # apache2 -k start -X Segmentation fault (core dumped)  # dpkg -i libapache2-mod-perl2_2.0.11-2_amd64.patched.deb (Reading database ... 33224 files and directories currently installed.) Preparing to unpack libapache2-mod-perl2_2.0.11-2_amd64.patched.deb ... Unpacking libapache2-mod-perl2 (2.0.11-2) over (2.0.11-2) ... Setting up libapache2-mod-perl2 (2.0.11-2) ... apache2_invoke perl: already enabled  # apache2 -k start -X <success>^C  # dpkg -i libapache2-mod-perl2_2.0.11-2_amd64.clean.deb (Reading database ... 33224 files and directories currently installed.) Preparing to unpack libapache2-mod-perl2_2.0.11-2_amd64.clean.deb ... Unpacking libapache2-mod-perl2 (2.0.11-2) over (2.0.11-2) ... Setting up libapache2-mod-perl2 (2.0.11-2) ... apache2_invoke perl: already enabled  # apache2 -k start -X Segmentation fault (core dumped) So after the SRU is performed, apache should no longer segfault. [Where problems could occur] ============================ The problem could occur if the user has manually set some different workaround for this bug and so the usual upgrade could break some of their old configuration(s) or settings.
2021-03-19 14:53:20 Utkarsh Gupta description [Impact] ======== While setting up a perl web application with mod_perl & apache, apache keeps segfaulting. Broke out gdb, and found that it was segfaulting within perl itself Program received signal SIGSEGV, Segmentation fault. 0x00007ffff7358ff5 in perl_parse () from /lib/x86_64-linux-gnu/libperl.so.5.30 (gdb) bt #0 0x00007ffff7358ff5 in perl_parse () from /lib/x86_64-linux-gnu/libperl.so.5.30 #1 0x00007ffff764cd0c in modperl_startup () from /usr/lib/apache2/modules/mod_perl.so #2 0x00007ffff764cc97 in modperl_startup () from /usr/lib/apache2/modules/mod_perl.so #3 0x00007ffff764d0fa in modperl_init () from /usr/lib/apache2/modules/mod_perl.so #4 0x00007ffff764d27b in modperl_hook_init () from /usr/lib/apache2/modules/mod_perl.so #5 0x00005555555b23d4 in ap_run_open_logs () #6 0x000055555558c440 in main ()  # valgrind apache2 -k start -X ==22529== Memcheck, a memory error detector ==22529== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al. ==22529== Using Valgrind-3.15.0 and LibVEX; rerun with -h for copyright info ==22529== Command: apache2 -k start -X ==22529== ==22529== Invalid read of size 8 ==22529== at 0x564AFF5: perl_parse (in /usr/lib/x86_64-linux-gnu/libperl.so.5.30.0) ==22529== by 0x55A8D0B: modperl_startup (in /usr/lib/apache2/modules/mod_perl.so) ==22529== by 0x55A8C96: modperl_startup (in /usr/lib/apache2/modules/mod_perl.so) ==22529== by 0x55A90F9: modperl_init (in /usr/lib/apache2/modules/mod_perl.so) ==22529== by 0x55A927A: modperl_hook_init (in /usr/lib/apache2/modules/mod_perl.so) ==22529== by 0x1663D3: ap_run_open_logs (in /usr/sbin/apache2) ==22529== by 0x14043F: main (in /usr/sbin/apache2) ==22529== Address 0x5a44000 is not stack'd, malloc'd or (recently) free'd ==22529== ==22529== ==22529== Process terminating with default action of signal 11 (SIGSEGV) ==22529== Access not within mapped region at address 0x5A44000 ==22529== at 0x564AFF5: perl_parse (in /usr/lib/x86_64-linux-gnu/libperl.so.5.30.0) ==22529== by 0x55A8D0B: modperl_startup (in /usr/lib/apache2/modules/mod_perl.so) ==22529== by 0x55A8C96: modperl_startup (in /usr/lib/apache2/modules/mod_perl.so) ==22529== by 0x55A90F9: modperl_init (in /usr/lib/apache2/modules/mod_perl.so) ==22529== by 0x55A927A: modperl_hook_init (in /usr/lib/apache2/modules/mod_perl.so) ==22529== by 0x1663D3: ap_run_open_logs (in /usr/sbin/apache2) ==22529== by 0x14043F: main (in /usr/sbin/apache2) gdb indicated that it was erroring in very early in perl's runtime, before it had got to any perl code. When using debug symbols, the exact line it was failing on was `scriptname = argv[0];` (perl.c:2365) It wasn't possible to reason beyond that as stepping through optimised code even with debug symbols is next to impossible to make any sense of. I did find that building an unoptimised perl made the error go away. I found the following closed issue: https://github.com/Perl/perl5/issues/15806 which describes the same issue I was having. Looking at the source for mod_perl, I found that the argv array passed to perl_parse() is not NULL terminated as is required by perl - ( documentation: https://perldoc.perl.org/perlembed#Adding-a-Perl-interpreter-to-your-C-program ) After patching this, the problem went away and didn't come back. Patch is attached. [Test Plan] ===========  # ls libapache2-mod-perl2_2.0.11-2_amd64.clean.deb libapache2-mod-perl2_2.0.11-2_amd64.patched.deb  # dpkg -i libapache2-mod-perl2_2.0.11-2_amd64.clean.deb (Reading database ... 33224 files and directories currently installed.) Preparing to unpack libapache2-mod-perl2_2.0.11-2_amd64.clean.deb ... Unpacking libapache2-mod-perl2 (2.0.11-2) over (2.0.11-2) ... Setting up libapache2-mod-perl2 (2.0.11-2) ... apache2_invoke perl: already enabled  # source /etc/apache2/envvars  # apache2 -k start -X Segmentation fault (core dumped)  # dpkg -i libapache2-mod-perl2_2.0.11-2_amd64.patched.deb (Reading database ... 33224 files and directories currently installed.) Preparing to unpack libapache2-mod-perl2_2.0.11-2_amd64.patched.deb ... Unpacking libapache2-mod-perl2 (2.0.11-2) over (2.0.11-2) ... Setting up libapache2-mod-perl2 (2.0.11-2) ... apache2_invoke perl: already enabled  # apache2 -k start -X <success>^C  # dpkg -i libapache2-mod-perl2_2.0.11-2_amd64.clean.deb (Reading database ... 33224 files and directories currently installed.) Preparing to unpack libapache2-mod-perl2_2.0.11-2_amd64.clean.deb ... Unpacking libapache2-mod-perl2 (2.0.11-2) over (2.0.11-2) ... Setting up libapache2-mod-perl2 (2.0.11-2) ... apache2_invoke perl: already enabled  # apache2 -k start -X Segmentation fault (core dumped) So after the SRU is performed, apache should no longer segfault. [Where problems could occur] ============================ The problem could occur if the user has manually set some different workaround for this bug and so the usual upgrade could break some of their old configuration(s) or settings. [Impact] ======== While setting up a perl web application with mod_perl & apache, apache keeps segfaulting. Broke out gdb, and found that it was segfaulting within perl itself Program received signal SIGSEGV, Segmentation fault. 0x00007ffff7358ff5 in perl_parse () from /lib/x86_64-linux-gnu/libperl.so.5.30 (gdb) bt #0 0x00007ffff7358ff5 in perl_parse () from /lib/x86_64-linux-gnu/libperl.so.5.30 #1 0x00007ffff764cd0c in modperl_startup () from /usr/lib/apache2/modules/mod_perl.so #2 0x00007ffff764cc97 in modperl_startup () from /usr/lib/apache2/modules/mod_perl.so #3 0x00007ffff764d0fa in modperl_init () from /usr/lib/apache2/modules/mod_perl.so #4 0x00007ffff764d27b in modperl_hook_init () from /usr/lib/apache2/modules/mod_perl.so #5 0x00005555555b23d4 in ap_run_open_logs () #6 0x000055555558c440 in main ()  # valgrind apache2 -k start -X ==22529== Memcheck, a memory error detector ==22529== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al. ==22529== Using Valgrind-3.15.0 and LibVEX; rerun with -h for copyright info ==22529== Command: apache2 -k start -X ==22529== ==22529== Invalid read of size 8 ==22529== at 0x564AFF5: perl_parse (in /usr/lib/x86_64-linux-gnu/libperl.so.5.30.0) ==22529== by 0x55A8D0B: modperl_startup (in /usr/lib/apache2/modules/mod_perl.so) ==22529== by 0x55A8C96: modperl_startup (in /usr/lib/apache2/modules/mod_perl.so) ==22529== by 0x55A90F9: modperl_init (in /usr/lib/apache2/modules/mod_perl.so) ==22529== by 0x55A927A: modperl_hook_init (in /usr/lib/apache2/modules/mod_perl.so) ==22529== by 0x1663D3: ap_run_open_logs (in /usr/sbin/apache2) ==22529== by 0x14043F: main (in /usr/sbin/apache2) ==22529== Address 0x5a44000 is not stack'd, malloc'd or (recently) free'd ==22529== ==22529== ==22529== Process terminating with default action of signal 11 (SIGSEGV) ==22529== Access not within mapped region at address 0x5A44000 ==22529== at 0x564AFF5: perl_parse (in /usr/lib/x86_64-linux-gnu/libperl.so.5.30.0) ==22529== by 0x55A8D0B: modperl_startup (in /usr/lib/apache2/modules/mod_perl.so) ==22529== by 0x55A8C96: modperl_startup (in /usr/lib/apache2/modules/mod_perl.so) ==22529== by 0x55A90F9: modperl_init (in /usr/lib/apache2/modules/mod_perl.so) ==22529== by 0x55A927A: modperl_hook_init (in /usr/lib/apache2/modules/mod_perl.so) ==22529== by 0x1663D3: ap_run_open_logs (in /usr/sbin/apache2) ==22529== by 0x14043F: main (in /usr/sbin/apache2) gdb indicated that it was erroring in very early in perl's runtime, before it had got to any perl code. When using debug symbols, the exact line it was failing on was `scriptname = argv[0];` (perl.c:2365) It wasn't possible to reason beyond that as stepping through optimised code even with debug symbols is next to impossible to make any sense of. I did find that building an unoptimised perl made the error go away. I found the following closed issue: https://github.com/Perl/perl5/issues/15806 which describes the same issue I was having. Looking at the source for mod_perl, I found that the argv array passed to perl_parse() is not NULL terminated as is required by perl - ( documentation: https://perldoc.perl.org/perlembed#Adding-a-Perl-interpreter-to-your-C-program ) After patching this, the problem went away and didn't come back. Patch is attached. [Test Plan] ===========  # ls libapache2-mod-perl2_2.0.11-2_amd64.clean.deb libapache2-mod-perl2_2.0.11-2_amd64.patched.deb  # dpkg -i libapache2-mod-perl2_2.0.11-2_amd64.clean.deb (Reading database ... 33224 files and directories currently installed.) Preparing to unpack libapache2-mod-perl2_2.0.11-2_amd64.clean.deb ... Unpacking libapache2-mod-perl2 (2.0.11-2) over (2.0.11-2) ... Setting up libapache2-mod-perl2 (2.0.11-2) ... apache2_invoke perl: already enabled  # source /etc/apache2/envvars  # apache2 -k start -X Segmentation fault (core dumped)  # dpkg -i libapache2-mod-perl2_2.0.11-2_amd64.patched.deb (Reading database ... 33224 files and directories currently installed.) Preparing to unpack libapache2-mod-perl2_2.0.11-2_amd64.patched.deb ... Unpacking libapache2-mod-perl2 (2.0.11-2) over (2.0.11-2) ... Setting up libapache2-mod-perl2 (2.0.11-2) ... apache2_invoke perl: already enabled  # apache2 -k start -X <success>^C  # dpkg -i libapache2-mod-perl2_2.0.11-2_amd64.clean.deb (Reading database ... 33224 files and directories currently installed.) Preparing to unpack libapache2-mod-perl2_2.0.11-2_amd64.clean.deb ... Unpacking libapache2-mod-perl2 (2.0.11-2) over (2.0.11-2) ... Setting up libapache2-mod-perl2 (2.0.11-2) ... apache2_invoke perl: already enabled  # apache2 -k start -X Segmentation fault (core dumped) So after the SRU is performed, apache should no longer segfault. [Where problems could occur] ============================ The argument parsing code is being changed (taking in NULL terminator now), so edge case failures are likely to be in that area. Should be trivial to handle, though.
2021-03-22 10:24:34 Utkarsh Gupta libapache2-mod-perl2 (Ubuntu Focal): status Triaged Fix Committed
2021-03-25 02:31:35 Robie Basak bug added subscriber Ubuntu Stable Release Updates Team
2021-03-25 02:31:36 Robie Basak bug added subscriber SRU Verification
2021-03-25 02:31:39 Robie Basak tags bitesize patch server-next bitesize patch server-next verification-needed verification-needed-focal
2021-03-25 12:12:14 lordaro tags bitesize patch server-next verification-needed verification-needed-focal bitesize patch server-next verification-done-focal verification-needed
2021-04-06 19:58:09 Launchpad Janitor libapache2-mod-perl2 (Ubuntu Focal): status Fix Committed Fix Released
2021-04-06 19:58:14 Brian Murray removed subscriber Ubuntu Stable Release Updates Team