Failed to fetch kernel source from ckt ppa for ubuntu_kernel_selftests on nodes of Intel MAAS

[Po-Hsu Lin]

Issue found with:
  * 5.4.0-122-generic with node spitfire / wildcat
  * 5.14.0-1043.48 F-oem with node vought

Note that this issue does not exist with J / J-oem-5.17 with node wildcat and vought

deb-src fetch with command:
  apt-get source --download-only linux-modules-extra-5.4.0-122-generic

is failing with:
 E: Failed to fetch http://ppa.launchpad.net/canonical-kernel-team/ppa/ubuntu/pool/main/l/linux/linux_5.4.0.orig.tar.gz Undetermined Error [IP: 185.125.190.52 80]

Test log:
 Running 'apt-get source --download-only linux-modules-extra-5.4.0-122-generic'
 Reading package lists...
 Picking 'linux' as source package instead of 'linux-modules-extra-5.4.0-122-generic'
 NOTICE: 'linux' packaging is maintained in the 'Git' version control system at:
 git://git.launchpad.net/~ubuntu-kernel/ubuntu/+source/linux/+git/focal
 Please use:
 git clone git://git.launchpad.net/~ubuntu-kernel/ubuntu/+source/linux/+git/focal
 to retrieve the latest (possibly unreleased) updates to the package.
 Need to get 180 MB of source archives.
 Get:1 http://ppa.launchpad.net/canonical-kernel-team/ppa/ubuntu focal/main linux 5.4.0-122.138 (tar) [170 MB]
 Err:1 http://ppa.launchpad.net/canonical-kernel-team/ppa/ubuntu focal/main linux 5.4.0-122.138 (tar)
   Undetermined Error [IP: 185.125.190.52 80]
 Get:2 http://ppa.launchpad.net/canonical-kernel-team/ppa/ubuntu focal/main linux 5.4.0-122.138 (diff) [9603 kB]
 Get:3 http://ppa.launchpad.net/canonical-kernel-team/ppa/ubuntu focal/main linux 5.4.0-122.138 (dsc) [7335 B]
 Fetched 9611 kB in 48s (199 kB/s)
 W: Download is performed unsandboxed as root as file 'linux_5.4.0.orig.tar.gz' couldn't be accessed by user '_apt'. - pkgAcquire::Run (13: Permission denied)
 E: Failed to fetch http://ppa.launchpad.net/canonical-kernel-team/ppa/ubuntu/pool/main/l/linux/linux_5.4.0.orig.tar.gz Undetermined Error [IP: 185.125.190.52 80]
 E: Failed to fetch some archives.

We don't have this issue in previous cycles.

[Po-Hsu Lin (cypressyew)]

The interesting thing is that if you run the same command twice, the first attempt will fail around 71% of 170 MB in about 45 seconds. Downloaded file size is 116M.

The second attempt will continue the download and this time it will complete successfully.

This is reproducible: 4 out of 4 attempts.

[Colin Watson (cjwatson)]

This seems to be a network reliability issue, not a "does connectivity work at all" issue. Therefore, you should raise this with IS (via RT), not with Launchpad - we can't help with this sort of thing.

[Po-Hsu Lin (cypressyew)]

OK thanks for the feedback.

This is also affecting node vought with bionic 4.15

[Po-Hsu Lin (cypressyew)]

This issue has been solved by setting the following parameter in /etc/apt/apt.conf.d/80-retries:
  Acquire::Retries "3" into

Patch landed in CKCT.