[ubuntu-touch] system not recovering automatically when a critical service reaches the upstart respawn limit

[Ricardo Salveti]

current build number: 26
device name: mako
channel: ubuntu-touch/vivid-proposed
last update: 2014-11-19 16:43:47
version version: 26
version ubuntu: 20141119
version device: 20141119
version custom: 20141119

Useful for ubuntu-touch mainly.

Currently there is no way for the system to recover itself after a critical service reaches the upstart respawn limit (e.g. ofono crashing in a reboot loop). This is bad for the user as he will never really know that core services are not running anymore, and might not be able to receive calls or messages.

[Ricardo Salveti (rsalveti)]

You can easily reproduce that by hand by killing ofono in loop (e.g. 'pkill -9 ofono' a few times in a row). Upstart will give up and it'll never be started again.

[Ricardo Salveti (rsalveti)]

We discussed possible ways to fix this issue during our last device sprint, and the suggestion for the short term was to have a watchdog job that could watch critical system and session services, and automatically reboot the phone when a process reaches the upstart respawn limit.

[Ricardo Salveti (rsalveti)]

Fix available at https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/landing-003, test plan at https://wiki.ubuntu.com/Process/Merges/TestPlan/upstart-watchdog