missing dependencies leaves the container running blocking further jobs

[Michał Sawicz]

An example job:
http://s-jenkins.ubuntu-ci:8080/job/autopilot-testrunner-otto-trusty/809/console

Apart from the fact that the error message is not helpful:
/var/local/autopilot/setup.log: unity8 : Depends: libunity-mir1 but it is not going to be installed

The job is stuck until it times out:
Build timed out (after 60 minutes). Marking the build as failed.

That, along with the fact that we're down to just one otto runner, means that turnaround time is, unnecessarily, really high.

[Vincent Ladeuil (vila)]

http://s-jenkins.ubuntu-ci:8080/job/autopilot-testrunner-otto-saucy/1336/console is another example:

/var/local/autopilot/setup.log: The following packages have unmet dependencies:
/var/local/autopilot/setup.log: ubuntu-ui-toolkit-autopilot : Depends: python-autopilot (>= 1.4) but 1.3.1+13.10.20131023.1-0ubuntu1 is to be installed

[Jean-Baptiste Lallement (jibel)]

This is not a bug in Otto but in a custom version of the script that calls autopilot for medium tests of the upstream merger. I don't know where it comes from but it is not from Otto. Moving to ubuntu-ci-services-itself

[Jean-Baptiste Lallement (jibel)]

In the referenced job the failure is not detected because in usr/local/bin/run-autopilot.sh (from container ps-radeon-hd8350:/var/lib/lxc/trusty-amd64-20131023-0529/) in install_debs() apt is called with:

66 echo "Package list: ${package_list}" >> $AP_SETUP_LOGFILE
67 sudo apt-get -y --force-yes install $package_list >> $AP_SETUP_LOGFILE

and there is no error handling

You should do something like this instead
rc=0
sudo apt-get -y --force-yes install $package_list||rc=$? >> $AP_SETUP_LOGFILE
if [ $rc -gt 0 ]; then
    # Terminate gracefully
   ...