Missing apt keys causes issues on deployed nodes

[Jeff Lane ]

Not sure exactly how this happened but the Tokyo office set up a MAAS server and a Fujitsu server to test some fixes on.

The system appears to have been deployed using our curtin_userdata as the /etc/installed-by-maas file is present and shows MAAS 2.1.2 was used for install.

However, I see this when trying to do an apt update:
ubuntu@cx1640-1:~$ sudo apt update
sudo: unable to resolve host cx1640-1
Hit:1 http://us.archive.ubuntu.com/ubuntu xenial InRelease
Hit:3 http://us.archive.ubuntu.com/ubuntu xenial-updates InRelease
Hit:5 http://us.archive.ubuntu.com/ubuntu xenial-security InRelease
Get:2 http://ppa.launchpad.net/hardware-certification/public/ubuntu xenial InRelease [17.6 kB]
Get:4 http://ppa.launchpad.net/firmware-testing-team/ppa-fwts-stable/ubuntu xenial InRelease [17.6 kB]
Err:2 http://ppa.launchpad.net/hardware-certification/public/ubuntu xenial InRelease
  The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 101B58111FEADE10
Err:4 http://ppa.launchpad.net/firmware-testing-team/ppa-fwts-stable/ubuntu xenial InRelease
  The following signatures couldn't be verified because the public key is not available: NO_PUBKEY A1E2B0920C32E7C6
Reading package lists... Done
W: GPG error: http://ppa.launchpad.net/hardware-certification/public/ubuntu xenial InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 101B58111FEADE10
E: The repository 'http://ppa.launchpad.net/hardware-certification/public/ubuntu xenial InRelease' is not signed.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.
W: GPG error: http://ppa.launchpad.net/firmware-testing-team/ppa-fwts-stable/ubuntu xenial InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY A1E2B0920C32E7C6
E: The repository 'http://ppa.launchpad.net/firmware-testing-team/ppa-fwts-stable/ubuntu xenial InRelease' is not signed.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.
ubuntu@cx1640-1:~$ cat /etc/apt/sources.list
deb http://us.archive.ubuntu.com/ubuntu/ xenial main restricted universe multiverse
deb http://us.archive.ubuntu.com/ubuntu/ xenial-updates main restricted universe multiverse
deb http://us.archive.ubuntu.com/ubuntu/ xenial-security main restricted universe multiverse
ubuntu@cx1640-1:~$ ls /etc/apt/sources.list.d/
certification.list
ubuntu@cx1640-1:~$ cat /etc/apt/sources.list.d/certification.list
deb http://ppa.launchpad.net/hardware-certification/public/ubuntu xenial main
deb http://ppa.launchpad.net/firmware-testing-team/ppa-fwts-stable/ubuntu xenial main

Once I manually run add-apt-repository to re-create the repo and import the key, I am able to update and now apt shows correct data:

BEFORE:
ubuntu@cx1640-1:~$ apt-cache policy stress-ng
stress-ng:
  Installed: 0.07.12-0ppa1
  Candidate: 0.07.12-0ppa1
  Version table:
 *** 0.07.12-0ppa1 500
        500 http://ppa.launchpad.net/hardware-certification/public/ubuntu xenial/main amd64 Packages
        100 /var/lib/dpkg/status
     0.05.23-1ubuntu2 500
        500 http://us.archive.ubuntu.com/ubuntu xenial-updates/universe amd64 Packages
     0.05.23-1 500
        500 http://us.archive.ubuntu.com/ubuntu xenial/universe amd64 Packages

AFTER manually re-running apt-add-repository:
ubuntu@cx1640-1:~$ apt-cache policy stress-ng
stress-ng:
  Installed: 0.07.12-0ppa1
  Candidate: 0.07.16-1ppa1
  Version table:
     0.07.16-1ppa1 500
        500 http://ppa.launchpad.net/hardware-certification/public/ubuntu xenial/main amd64 Packages
 *** 0.07.12-0ppa1 100
        100 /var/lib/dpkg/status
     0.05.23-1ubuntu2 500
        500 http://us.archive.ubuntu.com/ubuntu xenial-updates/universe amd64 Packages
     0.05.23-1 500
        500 http://us.archive.ubuntu.com/ubuntu xenial/universe amd64 Packages

[Rod Smith (rodsmith)]

Might apt and apt-get respond differently? I've never run into this, but I almost always use apt-get. Of course, this needs to be fixed either way; I'm just hoping for a quick workaround in the meantime.

[Rod Smith (rodsmith)]

I've played around a bit with this, and I think that using "apt-get" rather than "apt" is a reasonable workaround in the short term. The "apt-get" program seems to be less picky about the imported keys and allows the use of our PPAs, whereas "apt" seems to be much more finicky. That said, as this was a fresh install (with canonical-certification-server already available), I wasn't able to test with many packages that actually needed updating.

[Jeff Lane  (bladernr)]

So another possible avenue to fix this (and possibly other issues) is
to use MAAS 2.x's ability to add arbitrary rrepos and PPAs. I don't
know if this works with local mirrors yet, but we can start with
trying to add the PPAS via MAAS's webUI and then see if we can deploy
the tools without the code in curtin_userdata. I'm already working on
this much... though I haven't moved the card on the board yet.

On Mon, Feb 6, 2017 at 4:36 PM, Rod Smith <email address hidden> wrote:
> I've played around a bit with this, and I think that using "apt-get"
> rather than "apt" is a reasonable workaround in the short term. The
> "apt-get" program seems to be less picky about the imported keys and
> allows the use of our PPAs, whereas "apt" seems to be much more finicky.
> That said, as this was a fresh install (with canonical-certification-
> server already available), I wasn't able to test with many packages that
> actually needed updating.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1662231
>
> Title:
> Missing apt keys causes issues on deployed nodes
>
> Status in maas-cert-server:
> New
>
> Bug description:
> Not sure exactly how this happened but the Tokyo office set up a MAAS
> server and a Fujitsu server to test some fixes on.
>
> The system appears to have been deployed using our curtin_userdata as
> the /etc/installed-by-maas file is present and shows MAAS 2.1.2 was
> used for install.
>
> However, I see this when trying to do an apt update:
> ubuntu@cx1640-1:~$ sudo apt update
> sudo: unable to resolve host cx1640-1
> Hit:1 http://us.archive.ubuntu.com/ubuntu xenial InRelease
> Hit:3 http://us.archive.ubuntu.com/ubuntu xenial-updates InRelease
> Hit:5 http://us.archive.ubuntu.com/ubuntu xenial-security InRelease
> Get:2 http://ppa.launchpad.net/hardware-certification/public/ubuntu xenial InRelease [17.6 kB]
> Get:4 http://ppa.launchpad.net/firmware-testing-team/ppa-fwts-stable/ubuntu xenial InRelease [17.6 kB]
> Err:2 http://ppa.launchpad.net/hardware-certification/public/ubuntu xenial InRelease
> The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 101B58111FEADE10
> Err:4 http://ppa.launchpad.net/firmware-testing-team/ppa-fwts-stable/ubuntu xenial InRelease
> The following signatures couldn't be verified because the public key is not available: NO_PUBKEY A1E2B0920C32E7C6
> Reading package lists... Done
> W: GPG error: http://ppa.launchpad.net/hardware-certification/public/ubuntu xenial InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 101B58111FEADE10
> E: The repository 'http://ppa.launchpad.net/hardware-certification/public/ubuntu xenial InRelease' is not signed.
> N: Updating from such a repository can't be done securely, and is therefore disabled by default.
> N: See apt-secure(8) manpage for repository creation and user configuration details.
> W: GPG error: http://ppa.launchpad.net/firmware-testing-team/ppa-fwts-stable/ubuntu xenial InRelease: The following signatures couldn't be verified because the...