Attempts to fetch wrong version for package

[Andreas Hasenack]

This is still being investigated.

Looking at the test failures of the noble software-properties[1], we see a pattern:

1604s autopkgtest [19:21:53]: testbed running kernel: Linux 6.8.0-31-generic #31-Ubuntu SMP PREEMPT_DYNAMIC Sat Apr 20 00:40:06 UTC 2024
1605s autopkgtest [19:21:54]: @@@@@@@@@@@@@@@@@@@@ apt-source software-properties
1606s blame: software-properties
1606s badpkg: rules extract failed with exit code 1
1606s autopkgtest [19:21:55]: ERROR: erroneous package: rules extract failed with exit code 1

With a debug run, we can see it trying to download a version of software-properties that is not published:

autopkgtest: DBG: testbed command ['sh', '-ec', 'exec 3>&1 >&2; set -x; cd /; builddir=$(mktemp -d /tmp/autopkgtest.XIJbve/build.XXX); cd $builddir; \nOUT=$(apt-get source -d -q --only-source software-properties=0.99.48.1 2>&1) || RC=$?;\nif [ -n "$RC" ]; then\n if echo "$OUT" | grep -q "Unable to find a source package"; then\n exit 1;\n else\n exit $RC;\n fi;\nfi;\necho "$OUT" | grep ^Get: || true;\ndpkg-source -x software-properties_*.dsc src >/dev/null; chmod -R a+rX .; cd [a-z0-9]*/.; pwd >&3; sed -n "1 {s/).*//; s/ (/\\n/; p}" debian/changelog >&3'], kind build, sout pipe, serr raw, env []
+ cd /
+ mktemp -d /tmp/autopkgtest.XIJbve/build.XXX
+ builddir=/tmp/autopkgtest.XIJbve/build.AlK
+ cd /tmp/autopkgtest.XIJbve/build.AlK
+ apt-get source -d -q --only-source software-properties=0.99.48.1
+ OUT=Reading package lists...
E: Can not find version '0.99.48.1' of package 'software-properties'
E: Unable to find a source package for software-properties
+ RC=100
+ [ -n 100 ]
+ echo Reading package lists...
E: Can not find version '0.99.48.1' of package 'software-properties'
E: Unable to find a source package for software-properties
+ grep -q Unable to find a source package
+ exit 1
blame: software-properties
badpkg: rules extract failed with exit code 1
autopkgtest: DBG: testbed command exited with code 1
autopkgtest [13:27:58]: ERROR: erroneous package: rules extract failed with exit code 1

$ rmadison -a source software-properties | grep noble
$ rmadison -asource software-properties | grep noble
 software-properties | 0.99.48 | noble | source
 software-properties | 0.99.49 | noble-proposed | source

But there is a src:software-properties-qt package at that version:

$ rmadison -a source software-properties-qt | grep noble

$ rmadison -asource software-properties-qt | grep noble
 software-properties-qt | 0.99.48.1 | noble/universe | source

IRC logs of a conversation about this: [2]

1. https://autopkgtest.ubuntu.com/packages/s/software-properties/noble/amd64
2. https://irclogs.ubuntu.com/2024/05/21/%23ubuntu-devel.html#t15:53

[Andreas Hasenack (ahasenack)]

Right now in noble we have two distinct source packages both producing bin:software-properties-qt:

src:software-properties 0.99.48
src:software-properties-qt 0.99.48.1

[Paride Legovini (paride)]

This may fix it:

https://salsa.debian.org/paride/autopkgtest/-/commits/lp2066290

I want to see CI pass before creating a MR, but unfortunately the images used by CI are a bit broken at the moment.

Also: I'll add a test for this scenario.

[Andreas Hasenack (ahasenack)]

I retried the failing tests with software-properties from proposed, which is at a higher version, that seems to workaround this issue.

[Paride Legovini (paride)]

Thanks. I'm running some autopkgtests locally (tests/autopkgtest LxcRunner) so see if we can deliver a fix faster, without waiting for things to happen on salsa.

[Paride Legovini (paride)]

$ AUTOPKGTEST_TEST_UNINSTALLED=yes AUTOPKGTEST_TEST_LXC=autopkgtest-testing-amd64 \
   tests/autopkgtest LxcRunner

[...]
----------------------------------------------------------------------
Ran 84 tests in 7189.824s

OK (skipped=2)

[Paride Legovini (paride)]

Merged: https://salsa.debian.org/ci-team/autopkgtest/-/merge_requests/355

[Launchpad Janitor (janitor)]

This bug was fixed in the package autopkgtest - 5.37ubuntu2

---------------
autopkgtest (5.37ubuntu2) oracular; urgency=medium

  * New delta, all upstream cherry-picks:
    - t/autopkgtest-build-docker: use ports.ubuntu.com on non-amd64 Ubuntu
    - t/autopkgtest: test_git_source_build: skip if remote repo is unreachable
    - d/t/lxd: restrict architectures via Architecture and enable arm64
    - d/rules: don't run pre-commit at package build time

 -- Paride Legovini <email address hidden> Wed, 26 Jun 2024 13:16:53 +0200

[Robie Basak (racb)]

This is still affecting Ubuntu infrastructure - at least it was on 23 July. See for example: https://autopkgtest.ubuntu.com/packages/s/software-properties/noble/amd64

[Andreas Hasenack (ahasenack)]

Is the fix deployed? We are still seeing failures because of the situation in noble (see comment #1), and still have to use the workaround from comment #3.

[Paride Legovini (paride)]

The fix is not deployed: we still need to rebase the production branch on top of autopkgtest 5.37. We normally don't rush this: we want to see how new autopkgtest versions behave on the Debian infra, and to let Ubuntu developers use the package from the archive for a while. This is especially true when nontrivial changes land in autopkgtest, and we have plenty of those in 5.35 and 5.36.

This said, we normally cherry-pick individual important fixes. This specifically was considered non-urgent because an AA dropped src:software-properties-qt from the archive, which should have removed this corner case. However: I see it only happened on Oracular, so we got bitten again in Noble.

Given that this is has been worked around via ad-hoc triggering, we should also drop the -qt package from Noble after software-properties 0.99.49 migrates I believe.