Silo tracking old package version and bailing out on unhandled exception

[Łukasz Zemczak]

This error seems to be 100% reproducible on one silo - not sure if that's just a singular case, but I think Timo once mentioned having similar problems. Consider this silo and log:

https://ci-train.ubuntu.com/job/ubuntu-rtm-landing-008-1-build/105/console

The silo had previously in its contents oxide-qt 1.5.5~rtm-0ubuntu1 (and errored out when trying to build it - it was a silo sync). This got then overriden by a direct upload of 1.5.5-0ubuntu1~rtm1. As we can see it correctly finds 1.5.5-0ubuntu1~rtm1 in the PPA but then bails out with an exception while trying to do something with the old version. I tried different combinations of the build job (watch_only, normal build, listing oxide-qt, force rebuilt etc.) and noting seems to clear the state.

As mentioned, it doesn't seem to happen frequently, but it's basically blocking this selected silo for moving forward.

[Robert Bruce Park (robru)]

Yeah, so there's a few things going on here. Ultimately, the silo is in an inconsistent state, due to what happened here:

https://ci-train.ubuntu.com/job/ubuntu-rtm-landing-008-1-build/100/console

2015-03-25 23:26:15,247 ERROR Uncaught exception: UnicodeDecodeError: 'utf8' codec can't decode byte 0xa0 in position 138394060: invalid start byte

Which was a result of debdiff outputting invalid utf8 characters, which is a result of oxide source tree containing non-utf8 files.

So I can fix the clean_source() function to handle that unexpected directory, but it's not really going to help much as there isn't really a way to generate a diff. Perhaps I'll look into saving the diff to disk rather than decoding it into memory, if that's possible then we can at least get the full content diff saved in jenkins, but even still the code that trims that down to the packaging diff will still fail because the data just isn't utf8 (and we have no way to detect what encoding it is).

[Robert Bruce Park (robru)]

So I've landed a branch that is able to correctly clear out that unexpected directory, and we're right back to the original traceback with the UnicodeDecodeError:

https://ci-train.ubuntu.com/job/ubuntu-rtm-landing-008-1-build/109/console

[Robert Bruce Park (robru)]

I've also landed a branch that makes the watch phase only look at the newest available version in the silo, so that stuff about the previous version shouldn't come up again.

Ultimately, the thing about not being able to generate a diff is due to oxide source tree containing non-utf8 files, so you'll have to just proceed with publication without the diff.