Activity log for bug #1349288

Date Who What changed Old value New value Message
2014-07-28 08:01:51 Marti bug added bug
2014-07-28 08:02:15 Marti attachment removed error.log.txt https://bugs.launchpad.net/ubuntu/+source/apache2/+bug/1349288/+attachment/4164175/+files/error.log.txt
2014-07-28 08:33:38 Marti description On 2014-07-23, Ubuntu released a security update for Apache. Most of our systems use unattended-upgrades and installed this update automatically. On 2014-07-27, logrotate did its weekly rotation and issued the "reload" command to Apache. Since the new mod_status.so was no longer ABI-compatible with the running Apache, it died with an "undefined symbol" error. This happened on 3 of our systems. I guess the majority of Apache users are using logrotate too, so it's too late to fix anything for them. But for some users, this may still be a ticking time bomb. I hope the people responsible for the patching are made aware of this mistake and will avoid similar ABI changes in the future. /var/log/apache2/error.log.1 [Sun Jul 27 06:32:34.453547 2014] [mpm_worker:notice] [pid 1014:tid 139742164682624] AH00297: SIGUSR1 received. Doing graceful restart apache2: Syntax error on line 140 of /etc/apache2/apache2.conf: Syntax error on line 1 of /etc/apache2/mods-enabled/status.load: Cannot load /usr/lib/apache2/modules/mod_status.so into server: /usr/lib/apache2/modules/mod_status.so: undefined symbol: ap_copy_scoreboard_worker /var/log/unattended-upgrades/unattended-upgrades.log 2014-07-24 06:46:22,214 INFO Initial blacklisted packages: 2014-07-24 06:46:22,215 INFO Starting unattended upgrades script 2014-07-24 06:46:22,215 INFO Allowed origins are: ['o=Ubuntu,a=trusty-security'] 2014-07-24 06:46:30,273 INFO Packages that will be upgraded: apache2 apache2-bin apache2-data apache2-utils 2014-07-24 06:46:30,273 INFO Writing dpkg log to '/var/log/unattended-upgrades/unattended-upgrades-dpkg_2014-07-24_06:46:30.273613.log' 2014-07-24 06:46:32,956 INFO All upgrades installed ProblemType: Bug DistroRelease: Ubuntu 14.04 Package: apache2 2.4.7-1ubuntu4.1 ProcVersionSignature: Ubuntu 3.13.0-32.57-generic 3.13.11.4 Uname: Linux 3.13.0-32-generic x86_64 Apache2ConfdDirListing: False ApportVersion: 2.14.1-0ubuntu3.2 Architecture: amd64 Date: Mon Jul 28 10:38:22 2014 InstallationDate: Installed on 2011-02-08 (1265 days ago) InstallationMedia: Ubuntu-Server 10.10 "Maverick Meerkat" - Release amd64 (20101007) ProcEnviron: TERM=xterm PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/usr/bin/zsh SourcePackage: apache2 UpgradeStatus: Upgraded to trusty on 2014-06-16 (41 days ago) On 2014-07-23, Ubuntu released a security update for Apache for the CVE-2014-0226 vulnerability. Most of our systems use unattended-upgrades and installed this update automatically. On 2014-07-27, logrotate did its weekly rotation and issued the "reload" command to Apache. Since the new mod_status.so was no longer ABI-compatible with the running Apache, it died with an "undefined symbol" error. This happened on 4 of our systems. I guess the majority of Apache users are using logrotate with the default settings, so it's too late to fix anything for them. But for some users, this may still be a ticking time bomb. I hope the people responsible for the patching are made aware of this mistake and will avoid applying security updates with ABI changes in the future. /var/log/apache2/error.log.1 [Sun Jul 27 06:32:34.453547 2014] [mpm_worker:notice] [pid 1014:tid 139742164682624] AH00297: SIGUSR1 received. Doing graceful restart apache2: Syntax error on line 140 of /etc/apache2/apache2.conf: Syntax error on line 1 of /etc/apache2/mods-enabled/status.load: Cannot load /usr/lib/apache2/modules/mod_status.so into server: /usr/lib/apache2/modules/mod_status.so: undefined symbol: ap_copy_scoreboard_worker /var/log/unattended-upgrades/unattended-upgrades.log 2014-07-24 06:46:22,214 INFO Initial blacklisted packages: 2014-07-24 06:46:22,215 INFO Starting unattended upgrades script 2014-07-24 06:46:22,215 INFO Allowed origins are: ['o=Ubuntu,a=trusty-security'] 2014-07-24 06:46:30,273 INFO Packages that will be upgraded: apache2 apache2-bin apache2-data apache2-utils 2014-07-24 06:46:30,273 INFO Writing dpkg log to '/var/log/unattended-upgrades/unattended-upgrades-dpkg_2014-07-24_06:46:30.273613.log' 2014-07-24 06:46:32,956 INFO All upgrades installed ProblemType: Bug DistroRelease: Ubuntu 14.04 Package: apache2 2.4.7-1ubuntu4.1 ProcVersionSignature: Ubuntu 3.13.0-32.57-generic 3.13.11.4 Uname: Linux 3.13.0-32-generic x86_64 Apache2ConfdDirListing: False ApportVersion: 2.14.1-0ubuntu3.2 Architecture: amd64 Date: Mon Jul 28 10:38:22 2014 InstallationDate: Installed on 2011-02-08 (1265 days ago) InstallationMedia: Ubuntu-Server 10.10 "Maverick Meerkat" - Release amd64 (20101007) ProcEnviron:  TERM=xterm  PATH=(custom, no user)  LANG=en_US.UTF-8  SHELL=/usr/bin/zsh SourcePackage: apache2 UpgradeStatus: Upgraded to trusty on 2014-06-16 (41 days ago)
2014-07-28 09:00:12 Launchpad Janitor apache2 (Ubuntu): status New Confirmed