snappy-selftest: fake newer version for upgrade test

[Martin Pitt]

Once we made snappy-selftest capable of reboot tests (bug 1424583) and added reboot tests (bug 1424584), figure out how to fake a newer image to upgrade to, so that we can test the upgrader of the latest image. This does not necessarily need to involve some actual changes, just a new image revision and triggering the machinery to do a "null" upgrade.

See https://docs.google.com/a/canonical.com/document/d/1R_Tw0N0QbEpjFeYf9XnVV8Gp8ldT2Ig0PO6MfR-kuSM for some further information.

[James Hunt (jamesodhunt)]

Hi Martin,

Is this now wishlist simply due to other priorities, or a change in strategy for upgrade testing?

[Martin Pitt (pitti)]

Hi James,

I mostly set this to wishlist because it's not a malfunction but an improvement. But I don't care much either way, if you want to interpret this as urgency and not as bug severity.

Am 16. März 2015 17:51:17 OEZ, schrieb James Hunt <email address hidden>:
>Hi Martin,
>
>Is this now wishlist simply due to other priorities, or a change in
>strategy for upgrade testing?
>
>--
>You received this bug notification because you are subscribed to the
>bug
>report.
>https://bugs.launchpad.net/bugs/1424586
>
>Title:
> snappy-selftest: fake newer version for upgrade test
>
>To manage notifications about this bug go to:
>https://bugs.launchpad.net/snappy-ubuntu/+bug/1424586/+subscriptions

--
Diese Nachricht wurde von meinem Android-Mobiltelefon mit K-9 Mail gesendet.

[Martin Pitt (pitti)]

In the google doc, could you please describe how to modify the revision numbers for this to make sense and work? Thanks!

[Steve Langasek (vorlon)]

We don't want to fake the revision numbers here. Instead, we want to import the existing images onto a channel on a testing instance of the ubuntu-system-image server; synthesize a test upgrade of some kind; and upgrade from this, to ensure end-to-end testing of the upgrade support.

The google doc describes the manual tests we're doing today in the absence of support for this; it's not the way we should automate these tests going forward.

[Steve Langasek (vorlon)]

BTW, I have a list of test cases for system image updates which was written for the phone, and which have never been automated. Automatic generation of synthetic channels to use for testing is a very important part of this. I understand that Barry has some code in the system-image client test suite which should be reusable for this purpose.

https://docs.google.com/a/canonical.com/document/d/1gyzIlPBAnDYchU6vVNzdL0Qnct4s8fwfZtPyzDuou9A/edit?usp=docs_home

[Michael Vogt (mvo)]

See lp:~mvo/snappy/selftest-upgrade-rollback for a implementation of this (not quite ready, needs some cleanup as its duplicating quite a bit of code right now).

[Ricardo Salveti (rsalveti)]

Adding fix commited for 15.04.1 since we're not merging the selftests yet for 15.04 (but instead using the common branch for both rolling and 15.04). We should merge it as part of 15.04.2.