Cannot run tests on device

[Alan Pope 🍺🐧🐱 🦄]

Flashed device with:-

$ ubuntu-device-flash --channel=ubuntu-touch/vivid --wipe

Which succeeded:-

$ adb shell system-image-cli --info
current build number: 1
device name: flo
channel: ubuntu-touch/vivid
last update: 2015-02-23 18:38:35
version version: 1
version ubuntu: 20141201
version device: 20141119
version custom: 20141201

Tried running tests:-

adt-run --click com.ubuntu.clock --- ssh -s adb

Failed with loads of update errors / 404s.

http://paste.ubuntu.com/10376178/

I followed the instructions buried at the bottom of https://wiki.ubuntu.com/Touch/Testing

Two issues here:-

1) The instructions listed are buried at the bottom of the page
2) The instructions provided don't actually work.

[Nicholas Skaggs (nskaggs)]

Note the following bugs which are related:

https://bugs.launchpad.net/ubuntu/+source/autopkgtest/+bug/1342838

https://bugs.launchpad.net/ubuntu/+source/autopkgtest/+bug/1408350

I thought autopkgtest was supposed to automagically fallback and update if apt-get failed? Also how do you manually force the update?

[Nicholas Skaggs (nskaggs)]

Seems --setup-commands might help:

adt-run --setup-commands "apt-get update" --click com.ubuntu.clock --- ssh -s adb

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Martin Pitt (pitti)]

Nicholas: no, adt-run does not automatically update apt. This requires making the device r/w (at least for now); right now you need to specify "--setup-commands ro-apt-update" if you test an old image whose apt indexes have test dependencies which don't exist any more.

[Martin Pitt (pitti)]

Also, we really shouldn't run apt-get update without explicitly asking to. It would meant that you run an old image with test dependencies which changed since the image was built, which is always prone to failure. Fixing this properly requires re-thinking how we build our archives, and we need to keep the older debs around for much much longer than a day. But I realize this isn't going to happen anytime soon, so the next best thing is to use ro-apt-update and live with the fact that we sometimes can't test old images any more as the archive moved ahead.

Marking as a duplicate of bug 1408350 as there isn't much more that we can sensibly do here.