Activity log for bug #1430403

Date Who What changed Old value New value Message
2015-03-10 15:42:43 Colin Watson bug added bug
2015-03-10 15:44:24 Launchpad Janitor upstart (Ubuntu): status New Confirmed
2015-03-10 15:48:42 James Hunt upstart (Ubuntu): importance Undecided High
2015-03-10 15:52:44 James Hunt upstart (Ubuntu): assignee James Hunt (jamesodhunt)
2015-03-10 17:40:34 James Hunt attachment added upstart-bug-1430403.diff https://bugs.launchpad.net/ubuntu/+source/upstart/+bug/1430403/+attachment/4340271/+files/upstart-bug-1430403.diff
2015-03-10 18:33:57 Launchpad Janitor branch linked lp:~jamesodhunt/ubuntu/precise/upstart/bug-1430403
2015-03-10 20:19:35 Ubuntu Foundations Team Bug Bot tags patch
2015-03-11 09:27:30 James Hunt nominated for series Ubuntu Precise
2015-03-11 09:27:30 James Hunt bug task added upstart (Ubuntu Precise)
2015-03-11 13:50:57 James Hunt description ubuntu-touch livefs builds have started killing upstart in the host system (in this case, precise, although a similar bug appears to be present in current versions). The livefs build completes, but the host dies shortly after launchpad-buildd starts trying to remove the build chroot. The kernel log looks like this: Mar 10 13:46:55 allspice kernel: [3743880.621603] init: /home/buildd/build-LIVEFSBUILD-22254/chroot-autobuild/build/chroot/etc/init/tty1.conf: Unable to reload configuration after override deletion Mar 10 13:46:55 allspice kernel: [3743880.642455] init: file.c:110: Unhandled error from nih_file_read: No such file or directory Mar 10 13:46:55 allspice kernel: [3743880.754281] init: Caught abort, core dumped Mar 10 13:46:55 allspice kernel: [3743880.754375] init: file.c:110: Unhandled error from nih_file_read: No such file or directory Mar 10 13:46:55 allspice kernel: [3743880.757830] init: Caught abort, core dumped This appears to be because a couple of functions call conf_reload_path, which may leave an nih_error in place if nih_file_read fails, but then do not dispose of the nih_error. The pattern near the end of conf_file_visitor (in precise) is probably appropriate. We're working around this to some extent in livecd-rootfs by removing the .override files first, but it should never be possible for a chroot to crash the host's init. = Summary = The version of Upstart in precise is affected by a bug in the way that ".override" [1] file handling is performed. If a job has an override file ("/etc/init/*.override") and that override file is deleted before the corresponding job configuration file ("/etc/init/*.conf"), there is a possibility of a crash. == Explanation == When a "/etc/init/*.override" file is deleted, Upstart will automatically detect this and reload the corresponding "/etc/init/*.conf" file. However, if the ".conf" file (which the ".override" file corresponded to) is deleted at the same time Upstart attempts to read the ".conf" file, an assertion failure could result. = Code Specifics = The erroneous function is "conf_delete_handler()" which is called whenever a file Upstart is watching gets deleted. The end of this function calls conf_reload_path() but although it logs an error message, it does not consume the error object that gets raised when conf_reload_path() fails. = Affected Releases = This bug is only present in precise: - Upstart override handling was introduced in Upstart v1.3: - Precise currently uses Upstart 1.5-0ubuntu7.2 (and hence is affected). - Lucid currently uses Upstart 0.6.5-8 (hence, not affected). - Trusty and Vivid use much newer versions of the Upstart which no longer contain the problematic code. = Fix = The fix is simply to have conf_delete_handler() consume the error object (by freeing it) when conf_reload_path() fails. = Test Case = A reliable test case is unfortunately not possible to create, since the problem comes down to Upstart racing with the deletion of the ".conf" file. However, the patch is small and it can be seen that every other failing call to conf_reload_path() free's the resulting error object. = Regression Potential = None expected. The problem is difficult to trigger anwyay and the patch can be seen to correct (what is now) an obvious coding error. [1] - http://upstart.ubuntu.com/cookbook/#override-file = Original Description = ubuntu-touch livefs builds have started killing upstart in the host system (in this case, precise, although a similar bug appears to be present in current versions). The livefs build completes, but the host dies shortly after launchpad-buildd starts trying to remove the build chroot. The kernel log looks like this: Mar 10 13:46:55 allspice kernel: [3743880.621603] init: /home/buildd/build-LIVEFSBUILD-22254/chroot-autobuild/build/chroot/etc/init/tty1.conf: Unable to reload configuration after override deletion Mar 10 13:46:55 allspice kernel: [3743880.642455] init: file.c:110: Unhandled error from nih_file_read: No such file or directory Mar 10 13:46:55 allspice kernel: [3743880.754281] init: Caught abort, core dumped Mar 10 13:46:55 allspice kernel: [3743880.754375] init: file.c:110: Unhandled error from nih_file_read: No such file or directory Mar 10 13:46:55 allspice kernel: [3743880.757830] init: Caught abort, core dumped This appears to be because a couple of functions call conf_reload_path, which may leave an nih_error in place if nih_file_read fails, but then do not dispose of the nih_error. The pattern near the end of conf_file_visitor (in precise) is probably appropriate. We're working around this to some extent in livecd-rootfs by removing the .override files first, but it should never be possible for a chroot to crash the host's init.
2015-03-11 14:08:00 James Hunt summary ubuntu-touch livefs builds kill upstart in host [SRU] ubuntu-touch livefs builds kill upstart in host
2015-03-11 14:08:22 James Hunt upstart (Ubuntu): status Confirmed In Progress
2015-03-11 14:14:17 James Hunt bug added subscriber James Hunt
2015-03-11 14:34:10 James Hunt description = Summary = The version of Upstart in precise is affected by a bug in the way that ".override" [1] file handling is performed. If a job has an override file ("/etc/init/*.override") and that override file is deleted before the corresponding job configuration file ("/etc/init/*.conf"), there is a possibility of a crash. == Explanation == When a "/etc/init/*.override" file is deleted, Upstart will automatically detect this and reload the corresponding "/etc/init/*.conf" file. However, if the ".conf" file (which the ".override" file corresponded to) is deleted at the same time Upstart attempts to read the ".conf" file, an assertion failure could result. = Code Specifics = The erroneous function is "conf_delete_handler()" which is called whenever a file Upstart is watching gets deleted. The end of this function calls conf_reload_path() but although it logs an error message, it does not consume the error object that gets raised when conf_reload_path() fails. = Affected Releases = This bug is only present in precise: - Upstart override handling was introduced in Upstart v1.3: - Precise currently uses Upstart 1.5-0ubuntu7.2 (and hence is affected). - Lucid currently uses Upstart 0.6.5-8 (hence, not affected). - Trusty and Vivid use much newer versions of the Upstart which no longer contain the problematic code. = Fix = The fix is simply to have conf_delete_handler() consume the error object (by freeing it) when conf_reload_path() fails. = Test Case = A reliable test case is unfortunately not possible to create, since the problem comes down to Upstart racing with the deletion of the ".conf" file. However, the patch is small and it can be seen that every other failing call to conf_reload_path() free's the resulting error object. = Regression Potential = None expected. The problem is difficult to trigger anwyay and the patch can be seen to correct (what is now) an obvious coding error. [1] - http://upstart.ubuntu.com/cookbook/#override-file = Original Description = ubuntu-touch livefs builds have started killing upstart in the host system (in this case, precise, although a similar bug appears to be present in current versions). The livefs build completes, but the host dies shortly after launchpad-buildd starts trying to remove the build chroot. The kernel log looks like this: Mar 10 13:46:55 allspice kernel: [3743880.621603] init: /home/buildd/build-LIVEFSBUILD-22254/chroot-autobuild/build/chroot/etc/init/tty1.conf: Unable to reload configuration after override deletion Mar 10 13:46:55 allspice kernel: [3743880.642455] init: file.c:110: Unhandled error from nih_file_read: No such file or directory Mar 10 13:46:55 allspice kernel: [3743880.754281] init: Caught abort, core dumped Mar 10 13:46:55 allspice kernel: [3743880.754375] init: file.c:110: Unhandled error from nih_file_read: No such file or directory Mar 10 13:46:55 allspice kernel: [3743880.757830] init: Caught abort, core dumped This appears to be because a couple of functions call conf_reload_path, which may leave an nih_error in place if nih_file_read fails, but then do not dispose of the nih_error. The pattern near the end of conf_file_visitor (in precise) is probably appropriate. We're working around this to some extent in livecd-rootfs by removing the .override files first, but it should never be possible for a chroot to crash the host's init. = Summary = The version of Upstart in precise is affected by a bug in the way that ".override" [1] file handling is performed. If a job has an override file ("/etc/init/*.override") and that override file is deleted before the corresponding job configuration file ("/etc/init/*.conf"), there is a possibility of a crash. == Explanation == When a "/etc/init/*.override" file is deleted, Upstart will automatically detect this and reload the corresponding "/etc/init/*.conf" file. However, if the ".conf" file (which the ".override" file corresponded to) is deleted at the same time Upstart attempts to read the ".conf" file, an assertion failure could result. = Code Specifics = The erroneous function is "conf_delete_handler()" which is called whenever a file Upstart is watching gets deleted. The end of this function calls conf_reload_path() but although it logs an error message, it does not consume the error object that gets raised when conf_reload_path() fails. = Affected Releases = This bug is only present in precise: - Upstart override handling was introduced in Upstart v1.3: - Precise currently uses Upstart 1.5-0ubuntu7.2 (and hence is affected). - Lucid currently uses Upstart 0.6.5-8 (hence, not affected). - Trusty and Vivid use much newer versions of the Upstart which no   longer contain the problematic code. = Fix = The fix is simply to have conf_delete_handler() consume the error object (by freeing it) when conf_reload_path() fails. = Test Case = A reliable test case is unfortunately not possible to create, since the problem comes down to Upstart racing with the deletion of the ".conf" file. However, the patch is small and it can be seen that every other failing call to conf_reload_path() free's the resulting error object. = Workarounds = The problem is only manifested if ".conf" and the ".override" file get deleted one after another, with the ".override" file being deleted first. This implies the following work-arounds to avoid the problem if you wish to delete both files "at the same time": 1) Ensure the ".conf" file is deleted first. 2) Delete the ".override" file first, and then wait for a small period of time before deleting the corresponding ".conf" file. 3) Delete the ".override" file first, then call "sudo initctl reload-configuration" and then delete the corresponding ".conf" file. = Regression Potential = None expected. The problem is difficult to trigger anwyay and the patch can be seen to correct (what is now) an obvious coding error. [1] - http://upstart.ubuntu.com/cookbook/#override-file = Original Description = ubuntu-touch livefs builds have started killing upstart in the host system (in this case, precise, although a similar bug appears to be present in current versions). The livefs build completes, but the host dies shortly after launchpad-buildd starts trying to remove the build chroot. The kernel log looks like this: Mar 10 13:46:55 allspice kernel: [3743880.621603] init: /home/buildd/build-LIVEFSBUILD-22254/chroot-autobuild/build/chroot/etc/init/tty1.conf: Unable to reload configuration after override deletion Mar 10 13:46:55 allspice kernel: [3743880.642455] init: file.c:110: Unhandled error from nih_file_read: No such file or directory Mar 10 13:46:55 allspice kernel: [3743880.754281] init: Caught abort, core dumped Mar 10 13:46:55 allspice kernel: [3743880.754375] init: file.c:110: Unhandled error from nih_file_read: No such file or directory Mar 10 13:46:55 allspice kernel: [3743880.757830] init: Caught abort, core dumped This appears to be because a couple of functions call conf_reload_path, which may leave an nih_error in place if nih_file_read fails, but then do not dispose of the nih_error. The pattern near the end of conf_file_visitor (in precise) is probably appropriate. We're working around this to some extent in livecd-rootfs by removing the .override files first, but it should never be possible for a chroot to crash the host's init.
2015-03-11 14:34:31 James Hunt description = Summary = The version of Upstart in precise is affected by a bug in the way that ".override" [1] file handling is performed. If a job has an override file ("/etc/init/*.override") and that override file is deleted before the corresponding job configuration file ("/etc/init/*.conf"), there is a possibility of a crash. == Explanation == When a "/etc/init/*.override" file is deleted, Upstart will automatically detect this and reload the corresponding "/etc/init/*.conf" file. However, if the ".conf" file (which the ".override" file corresponded to) is deleted at the same time Upstart attempts to read the ".conf" file, an assertion failure could result. = Code Specifics = The erroneous function is "conf_delete_handler()" which is called whenever a file Upstart is watching gets deleted. The end of this function calls conf_reload_path() but although it logs an error message, it does not consume the error object that gets raised when conf_reload_path() fails. = Affected Releases = This bug is only present in precise: - Upstart override handling was introduced in Upstart v1.3: - Precise currently uses Upstart 1.5-0ubuntu7.2 (and hence is affected). - Lucid currently uses Upstart 0.6.5-8 (hence, not affected). - Trusty and Vivid use much newer versions of the Upstart which no   longer contain the problematic code. = Fix = The fix is simply to have conf_delete_handler() consume the error object (by freeing it) when conf_reload_path() fails. = Test Case = A reliable test case is unfortunately not possible to create, since the problem comes down to Upstart racing with the deletion of the ".conf" file. However, the patch is small and it can be seen that every other failing call to conf_reload_path() free's the resulting error object. = Workarounds = The problem is only manifested if ".conf" and the ".override" file get deleted one after another, with the ".override" file being deleted first. This implies the following work-arounds to avoid the problem if you wish to delete both files "at the same time": 1) Ensure the ".conf" file is deleted first. 2) Delete the ".override" file first, and then wait for a small period of time before deleting the corresponding ".conf" file. 3) Delete the ".override" file first, then call "sudo initctl reload-configuration" and then delete the corresponding ".conf" file. = Regression Potential = None expected. The problem is difficult to trigger anwyay and the patch can be seen to correct (what is now) an obvious coding error. [1] - http://upstart.ubuntu.com/cookbook/#override-file = Original Description = ubuntu-touch livefs builds have started killing upstart in the host system (in this case, precise, although a similar bug appears to be present in current versions). The livefs build completes, but the host dies shortly after launchpad-buildd starts trying to remove the build chroot. The kernel log looks like this: Mar 10 13:46:55 allspice kernel: [3743880.621603] init: /home/buildd/build-LIVEFSBUILD-22254/chroot-autobuild/build/chroot/etc/init/tty1.conf: Unable to reload configuration after override deletion Mar 10 13:46:55 allspice kernel: [3743880.642455] init: file.c:110: Unhandled error from nih_file_read: No such file or directory Mar 10 13:46:55 allspice kernel: [3743880.754281] init: Caught abort, core dumped Mar 10 13:46:55 allspice kernel: [3743880.754375] init: file.c:110: Unhandled error from nih_file_read: No such file or directory Mar 10 13:46:55 allspice kernel: [3743880.757830] init: Caught abort, core dumped This appears to be because a couple of functions call conf_reload_path, which may leave an nih_error in place if nih_file_read fails, but then do not dispose of the nih_error. The pattern near the end of conf_file_visitor (in precise) is probably appropriate. We're working around this to some extent in livecd-rootfs by removing the .override files first, but it should never be possible for a chroot to crash the host's init. = Summary = The version of Upstart in precise is affected by a bug in the way that ".override" [1] file handling is performed. If a job has an override file ("/etc/init/*.override") and that override file is deleted before the corresponding job configuration file ("/etc/init/*.conf"), there is a possibility of a crash. == Explanation == When a "/etc/init/*.override" file is deleted, Upstart will automatically detect this and reload the corresponding "/etc/init/*.conf" file. However, if the ".conf" file (which the ".override" file corresponded to) is deleted at the same time Upstart attempts to read the ".conf" file, an assertion failure could result. = Code Specifics = The erroneous function is "conf_delete_handler()" which is called whenever a file Upstart is watching gets deleted. The end of this function calls conf_reload_path() but although it logs an error message, it does not consume the error object that gets raised when conf_reload_path() fails. = Affected Releases = This bug is only present in precise: - Upstart override handling was introduced in Upstart v1.3: - Precise currently uses Upstart 1.5-0ubuntu7.2 (and hence is affected). - Lucid currently uses Upstart 0.6.5-8 (hence, not affected). - Trusty and Vivid use much newer versions of the Upstart which no   longer contain the problematic code. = Fix = The fix is simply to have conf_delete_handler() consume the error object (by freeing it) when conf_reload_path() fails. = Test Case = A reliable test case is unfortunately not possible to create, since the problem comes down to Upstart racing with the deletion of the ".conf" file. However, the patch is small and it can be seen that every other failing call to conf_reload_path() free's the resulting error object. = Workarounds = The problem is only manifested if the ".conf" and the ".override" file get deleted one after another, with the ".override" file being deleted first. This implies the following work-arounds to avoid the problem if you wish to delete both files "at the same time": 1) Ensure the ".conf" file is deleted first. 2) Delete the ".override" file first, and then wait for a small period of time before deleting the corresponding ".conf" file. 3) Delete the ".override" file first, then call "sudo initctl reload-configuration" and then delete the corresponding ".conf" file. = Regression Potential = None expected. The problem is difficult to trigger anwyay and the patch can be seen to correct (what is now) an obvious coding error. [1] - http://upstart.ubuntu.com/cookbook/#override-file = Original Description = ubuntu-touch livefs builds have started killing upstart in the host system (in this case, precise, although a similar bug appears to be present in current versions). The livefs build completes, but the host dies shortly after launchpad-buildd starts trying to remove the build chroot. The kernel log looks like this: Mar 10 13:46:55 allspice kernel: [3743880.621603] init: /home/buildd/build-LIVEFSBUILD-22254/chroot-autobuild/build/chroot/etc/init/tty1.conf: Unable to reload configuration after override deletion Mar 10 13:46:55 allspice kernel: [3743880.642455] init: file.c:110: Unhandled error from nih_file_read: No such file or directory Mar 10 13:46:55 allspice kernel: [3743880.754281] init: Caught abort, core dumped Mar 10 13:46:55 allspice kernel: [3743880.754375] init: file.c:110: Unhandled error from nih_file_read: No such file or directory Mar 10 13:46:55 allspice kernel: [3743880.757830] init: Caught abort, core dumped This appears to be because a couple of functions call conf_reload_path, which may leave an nih_error in place if nih_file_read fails, but then do not dispose of the nih_error. The pattern near the end of conf_file_visitor (in precise) is probably appropriate. We're working around this to some extent in livecd-rootfs by removing the .override files first, but it should never be possible for a chroot to crash the host's init.
2015-03-11 22:51:23 Timo Aaltonen upstart (Ubuntu Precise): status New Fix Committed
2015-03-11 22:51:25 Timo Aaltonen bug added subscriber Ubuntu Stable Release Updates Team
2015-03-11 22:51:26 Timo Aaltonen bug added subscriber SRU Verification
2015-03-11 22:51:32 Timo Aaltonen tags patch patch verification-needed
2015-05-26 09:55:52 Adam Conrad tags patch verification-needed patch verification-done
2015-05-26 09:56:27 Launchpad Janitor upstart (Ubuntu Precise): status Fix Committed Fix Released
2015-05-26 09:56:32 Adam Conrad removed subscriber Ubuntu Stable Release Updates Team