armhf/ppc64el tests might use earlier binary packages than britney sees
| Affects | Status | Importance | Assigned to | Milestone | |
|---|---|---|---|---|---|
| | Auto Package Testing |
High
|
Martin Pitt | ||
Bug Description
Steve and Matthias reported that in some cases autopkgtests got triggered too early:
- armhf was triggered although the armhf build was not in the archive yet
- ppc64el build was triggered after it was built/published, but the corresponding amd64 (arch: all) was still missing
| Changed in auto-package-testing: | |
| status: | New → Incomplete |
| importance: | Undecided → High |
| Martin Pitt (pitti) wrote : | #1 |
| Martin Pitt (pitti) wrote : | #2 |
The first time cantor/
I: [Mon Aug 31 22:39:06 2015] - Requesting cantor/
I: [Mon Aug 31 22:39:06 2015] - Requesting cantor/
I: [Mon Aug 31 22:39:06 2015] - Requesting cantor/
I: [Mon Aug 31 22:39:06 2015] - Requesting cantor/
cantor (4:15.04.2-0ubuntu4 to 4:15.08.0-0ubuntu1)
Maintainer: Debian/Kubuntu Qt/KDE Maintainers
Section: universe/math
0 days old
autopkgtest for cantor 4:15.08.0-0ubuntu1: amd64: Test in progress, armhf: Test in progress, i386: Test in progress, ppc64el: Test in progress
Not considered
i. e. it was allegedly built at that time already. The armhf build log has "Finished at 20150831-2212", which leaves 27 minutes for the publisher in between, which is at least plausible. This then triggered the already linked test log above.
So I'm afraid I don't yet have information about what went wrong. I'll create some tests for this.
| Martin Pitt (pitti) wrote : | #3 |
And I regularly see that britney does wait for the last build to finish, and packages to become installable before it triggers tests. E. g. right now
systemd (224-2ubuntu2 to 225-1ubuntu1)
Maintainer: Ubuntu Developers
0 days old
missing build on armhf: libnss-myhostname, libnss-mymachines, libpam-systemd, libsystemd-
Not considered
nvidia-
Maintainer: Debian NVIDIA Maintainers
Section: multiverse/misc
311 days old
nvidia-
nvidia-
nvidia-
nvidia-
Not considered
So, Matthias/Steve, can you please point to some specific examples where this went wrong? Thanks!
| Steve Langasek (vorlon) wrote : Re: [Bug 1490899] Re: britney might trigger tests before builds are complete | #4 |
On Tue, Sep 01, 2015 at 02:56:44PM -0000, Martin Pitt wrote:
> So, Matthias/Steve, can you please point to some specific examples where
> this went wrong? Thanks!
This is the failing cantor test run:
That's with cantor 4:15.07.
We're speculating as to the cause of the failure, because the error message
is opaque: "Test dependencies are unsatisfiable" doesn't tell us what is
unsatisfiable. Logging could be improved.
Ah, thanks. That run was indeed triggerd by the previous version https:/
> "Test dependencies are unsatisfiable" doesn't tell us what is unsatisfiable.
It actually does, further up; that's apt-get's Debug::
Investigating (0) cantor-
Broken cantor-
Considering octave:armhf 1 as a solution to cantor-
Holding Back cantor-
So I think this was mostly just confusion due to uploading cantor twice within a few hours, but I think this mostly worked out as it should have.
he main thing that's left here as a potential trap AFAICS is the mirror delay between ftpmaster.internal (what britney uses) and ports.ubuntu.com (or whichever other mirror we are using). I'll fix the armhf/ppc64 test boxes to use ftpmaster.internal.
| summary: |
- britney might trigger tests before builds are complete + armhf/ppc64el tests might use earlier binary packages than britney sees |
| Changed in auto-package-testing: | |
| status: | Incomplete → In Progress |
| assignee: | nobody → Martin Pitt (pitti) |
| Martin Pitt (pitti) wrote : | #6 |
The armhf and ppc64el runners now use ftpmaster.internal. The ppc64el nodes don't have direct access to it, so I have to use squid.internal as a proxy, but that's (hopefully) still be better than the mirror delay.
| Changed in auto-package-testing: | |
| status: | In Progress → Fix Released |
I think the matching britney run is http://
I: [Mon Aug 31 23:06:20 2015] - test cantor/
I: [Mon Aug 31 23:06:20 2015] - test cantor/
I: [Mon Aug 31 23:06:20 2015] - test cantor/
I: [Mon Aug 31 23:06:20 2015] - test cantor/
[...]
I: [Mon Aug 31 23:06:23 2015] - Fetched test result for cantor/
I: [Mon Aug 31 23:06:23 2015] - -> matches pending request for triggers {('octave', '4.0.0-3ubuntu4'), ('cantor', '4:15.08.
I: [Mon Aug 31 23:06:23 2015] - -> matches pending request for triggers {('octave', '4.0.0-3ubuntu4')}
I: [Mon Aug 31 23:06:23 2015] - Fetched test result for cantor/
E: [Mon Aug 31 23:06:23 2015] - -> does not match any pending request!
E: [Mon Aug 31 23:06:23 2015] - -> does not match any pending request!
I: [Mon Aug 31 23:06:23 2015] - Fetched test result for cantor/
I: [Mon Aug 31 23:06:23 2015] - -> matches pending request for triggers {('octave', '4.0.0-3ubuntu4'), ('cantor', '4:15.08.
I: [Mon Aug 31 23:06:23 2015] - -> matches pending request for triggers {('octave', '4.0.0-3ubuntu4')}
I: [Mon Aug 31 23:06:23 2015] - Fetched test result for cantor/
E: [Mon Aug 31 23:06:23 2015] - -> does not match any pending request!
E: [Mon Aug 31 23:06:23 2015] - -> does not match any pending request!
I: [Mon Aug 31 23:06:23 2015] - Fetched test result for cantor/
I: [Mon Aug 31 23:06:23 2015] - -> matches pending request for triggers {('octave', '4.0.0-3ubuntu4'), ('cantor', '4:15.08.
I: [Mon Aug 31 23:06:23 2015] - -> matches pending request for triggers {('octave', '4.0.0-3ubuntu4')}
I: [Mon Aug 31 23:06:23 2015] - Fetched test result for cantor/
E: [Mon Aug 31 23:06:23 2015] - -> does not match any pending request!
E: [Mon Aug 31 23:06:23 2015] - -> does not match any pending request!
I: [Mon Aug 31 23:06:23 2015] - Fetched test result for cantor/
I: [Mon Aug 31 23:06:23 2015] - -> matches pending request for triggers {('octave', '4.0.0-3ubuntu4'), ('cantor', '4:15.08.
E: [Mon Aug 31 23:06:23 2015] - -> does not match any pending request!
[...]
trying: cantor
accepted: cantor
ori: 259+0: a-72:a-
pre: 259+0: a-72:a-
now: 257+0: a-72:a-
all: cantor
and in the corresponding http://
cantor (4:15.04.2-0ubuntu4 to 4:...