apt(-get) source fails to use credentials from /etc/apt/auth.conf(.d)

[Alex Murray]

I have configured apt-src access to the private ESM PPAs via entries in /etc/apt/sources.list.d/ubuntu-security.list as follows:

deb-src https://private-ppa.launchpad.net/ubuntu-esm/esm-infra-security/ubuntu trusty main

and then added credentials as follows to /etc/apt/auth.conf.d/ubuntu-security.conf:

machine private-ppa.launchpad.net/ubuntu-esm/esm-infra-security/ubuntu login alexmurray password XXXXXXXX

Running apt-get update then succeeds - but if I then try and run `apt-get source` to download from the PPA it fails:

$ apt-get source --only-source intel-microcode/trusty
Reading package lists... Done
Selected version '3.20201110.0ubuntu0.14.04.2' (trusty) for intel-microcode
NOTICE: 'intel-microcode' packaging is maintained in the 'Git' version control system at:
https://salsa.debian.org/hmh/intel-microcode.git
Please use:
git clone https://salsa.debian.org/hmh/intel-microcode.git
to retrieve the latest (possibly unreleased) updates to the package.
Need to get 3,447 kB of source archives.
Err:1 https://private-ppa.launchpad.net/ubuntu-esm/esm-infra-security/ubuntu trusty/main intel-microcode 3.20201110.0ubuntu0.14.04.2 (tar)
  401 Unauthorized [IP: 2001:67c:1560:8008::15 443]
Err:2 https://private-ppa.launchpad.net/ubuntu-esm/esm-infra-security/ubuntu trusty/main intel-microcode 3.20201110.0ubuntu0.14.04.2 (dsc)
  401 Unauthorized [IP: 2001:67c:1560:8008::15 443]
E: Failed to fetch https://private-ppa.launchpad.net/ubuntu-esm/esm-infra-security/ubuntu/pool/main/i/intel-microcode/intel-microcode_3.20201110.0ubuntu0.14.04.2.tar.xz 401 Unauthorized [IP: 2001:67c:1560:8008::15 443]
E: Failed to fetch https://private-ppa.launchpad.net/ubuntu-esm/esm-infra-security/ubuntu/pool/main/i/intel-microcode/intel-microcode_3.20201110.0ubuntu0.14.04.2.dsc 401 Unauthorized [IP: 2001:67c:1560:8008::15 443]
E: Failed to fetch some archives.

However if I edit /etc/apt/sources.list.d/ubuntu-security.list above to specify the credentials in-line then it succeeds:

deb-src https://alexmurray:<email address hidden>/ubuntu-esm/esm-infra-security/ubuntu trusty main

$ apt-get source --only-source intel-microcode/trusty
Reading package lists... Done
Selected version '3.20201110.0ubuntu0.14.04.2' (trusty) for intel-microcode
NOTICE: 'intel-microcode' packaging is maintained in the 'Git' version control system at:
https://salsa.debian.org/hmh/intel-microcode.git
Please use:
git clone https://salsa.debian.org/hmh/intel-microcode.git
to retrieve the latest (possibly unreleased) updates to the package.
Need to get 3,447 kB of source archives.
Get:1 https://private-ppa.launchpad.net/ubuntu-esm/esm-infra-security/ubuntu trusty/main intel-microcode 3.20201110.0ubuntu0.14.04.2 (tar) [3,446 kB]
Get:2 https://private-ppa.launchpad.net/ubuntu-esm/esm-infra-security/ubuntu trusty/main intel-microcode 3.20201110.0ubuntu0.14.04.2 (dsc) [1,604 B]
Fetched 3,447 kB in 5s (657 kB/s)
dpkg-source: info: extracting intel-microcode in intel-microcode-3.20201110.0ubuntu0.14.04.2
dpkg-source: info: unpacking intel-microcode_3.20201110.0ubuntu0.14.04.2.tar.xz

However now apt(-get) update complains about having credentials manually listed in the apt sources:

$ sudo apt update
...
N: Usage of apt_auth.conf(5) should be preferred over embedding login information directly in the sources.list(5) entry for 'https://private-ppa.launchpad.net/ubuntu-esm/esm-infra-security/ubuntu'

ProblemType: Bug
DistroRelease: Ubuntu 20.10
Package: apt 2.1.10
ProcVersionSignature: Ubuntu 5.8.0-28.30-generic 5.8.14
Uname: Linux 5.8.0-28-generic x86_64
NonfreeKernelModules: zfs zunicode zavl icp zcommon znvpair
ApportVersion: 2.20.11-0ubuntu50
Architecture: amd64
CasperMD5CheckResult: skip
CurrentDesktop: ubuntu:GNOME
Date: Fri Nov 13 09:09:54 2020
InstallationDate: Installed on 2020-10-11 (32 days ago)
InstallationMedia: Ubuntu 20.10 "Groovy Gorilla" - Beta amd64 (20200930)
SourcePackage: apt
UpgradeStatus: No upgrade log present (probably fresh install)

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Gabriel Kaufmann (info-typoworx)]

Also affects me. But instead of "only" source-packages all updates fail with Unauthorized 401. ESM is unusable in fact working like that! In my case Ubuntu 16.04 is affected which still should work for ESM Updates and there are about 300 updates proposed for me, but I'm not able to download them.

[Nils Büchner (n.buechner)]

This seems to be a simple permission problem.
The user is not allowed to read /etc/apt/auth.conf.d/90ubuntu-advantage

So instead of:
apt source libopenexr25
use:
sudo apt source libopenexr25

If you need to run it as a user fix the permissions of
/etc/apt/auth.conf.d/90ubuntu-advantage

[Aaron Rainbolt (arraybolt3)]

Could you try changing the permissions back to something more strict (maybe 0600), and then changing ownership to the `apt` user/group?

[Julian Andres Klode (juliank)]

I noticed that too yesterday but it's hard to make this right I think. Maybe the default permissions should give read access to the auth.conf snippet for users in the admin group or whatever it's called.

Because the problem to guard against are shared systems where credentials for Pro shouldn't leak to unprivileged users, so there's always gonna be users unable to use apt download.

[Renan Rodrigo (renanrodrigo)]

Gabriel Kaufmann (info-typoworx) wrote on 2022-08-24 (last edit on 2022-08-24):

> Also affects me. But instead of "only" source-packages all updates fail with Unauthorized 401. ESM is unusable in fact working like that! In my case Ubuntu 16.04 is affected which still should work for ESM Updates and there are about 300 updates proposed for me, but I'm not able to download them.

Hello, Gabriel
To have access to the esm repositories you need an Ubuntu Pro subscription (https://ubuntu.com/pro). If you are attached to a subscription, `pro enable esm-infra` will give you access.

If you still get 401s when the service is enabled, please open a bug against `ubuntu-advantage-tools` (https://bugs.launchpad.net/ubuntu/+source/ubuntu-advantage-tools/+filebug) so we can help you.