GPG keys are not shown in Software and Updates

[MadhuSoodanan]

[Impact]

software-properties-gtk does not show the registered apt gpg keys in the "Authentication" tab on my Ubuntu 21.10 and 22.04 systems.

[Test Plan]

1. Launch software-properties-gtk
2. Click on the "Authentication" tab
3. At least one key should be listed

[Where problems could occur]

I fail to come up with possible problems. The failures was a regression and the fixed version has a test case.

[Other Infos]

ProblemType: Bug
DistroRelease: Ubuntu 22.04
Package: ubuntu-release-upgrader-core 1:22.04.10
ProcVersionSignature: Ubuntu 5.15.0-27.28-generic 5.15.30
Uname: Linux 5.15.0-27-generic x86_64
ApportVersion: 2.20.11-0ubuntu82
Architecture: amd64
CasperMD5CheckResult: pass
CrashDB: ubuntu
CurrentDesktop: ubuntu:GNOME
Date: Tue Apr 26 20:48:04 2022
InstallationDate: Installed on 2022-04-25 (0 days ago)
InstallationMedia: Ubuntu 22.04 LTS "Jammy Jellyfish" - Release amd64 (20220419)
PackageArchitecture: all
SourcePackage: ubuntu-release-upgrader
Symptom: ubuntu-release-upgrader
UpgradeStatus: No upgrade log present (probably fresh install)

[Sebastien Bacher (seb128)]

Thank you for taking the time to report this bug and helping to make Ubuntu better. Unfortunately, we cannot work on this bug because your description didn't include enough information. You may find it helpful to read "How to report bugs effectively" http://www.chiark.greenend.org.uk/~sgtatham/bugs.html. We'd be grateful if you would then provide a more complete description of the problem.

We have instructions on debugging some types of problems at http://wiki.ubuntu.com/DebuggingProcedures.

At a minimum, we need:

1. The specific steps or actions you took that caused you to encounter the problem.
2. The behavior you expected.
3. The behavior you actually encountered (in as much detail as possible).

Please also ensure that you include the release and flavour of Ubuntu that you are using.

Thank you!

[MadhuSoodanan (mt-madhu)]

sb_release -crid
apt-key list
xxxx-V15-ADA 5.15.0-27-generic #28-Ubuntu SMP Thu Apr 14 04:55:28 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Distributor ID: Ubuntu
Description: Ubuntu 22.04 LTS
Release: 22.04
Codename: jammy
Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)).
/etc/apt/trusted.gpg.d/ubuntu-advantage-esm-infra-trusty.gpg
------------------------------------------------------------
pub rsa4096 2019-04-17 [SC]
      56F7 650A 24C9 E9EC F87C 4D8D 4067 E403 13CB 4B13
uid [ unknown] Ubuntu Extended Security Maintenance Automatic Signing Key v2 <email address hidden>
sub rsa4096 2019-04-17 [E]

/etc/apt/trusted.gpg.d/ubuntu-keyring-2012-cdimage.gpg
------------------------------------------------------
pub rsa4096 2012-05-11 [SC]
      8439 38DF 228D 22F7 B374 2BC0 D94A A3F0 EFE2 1092
uid [ unknown] Ubuntu CD Image Automatic Signing Key (2012) <email address hidden>

/etc/apt/trusted.gpg.d/ubuntu-keyring-2018-archive.gpg
------------------------------------------------------
pub rsa4096 2018-09-17 [SC]
      F6EC B376 2474 EDA9 D21B 7022 8719 20D1 991B C93C
uid [ unknown] Ubuntu Archive Automatic Signing Key (2018) <email address hidden>

[MadhuSoodanan (mt-madhu)]

dmesg | grep 'X.509'
dmesg: read kernel buffer failed: Operation not permitted

[Manfred Hampl (m-hampl)]

Due to the fact that the system in question showed two different issues, there is a mix of information related to the two problems.

I have tried to clean that up a bit.

[Manfred Hampl (m-hampl)]

Fact is: software-properties-gtk does not show the registered apt gpg keys in the "Authentication" tab on my Ubuntu 21.10 and 22.04 systems.
My Ubuntu 20.04 shows the registered keys in the window.

[Sebastien Bacher (seb128)]

updating the description to be about the key issue, please report a new bug about the livepatch problem including a screenshot

[Dan Streetman (ddstreet)]

I believe the problem is that 'apt-key' uses only the /etc/apt/trusted.gpg file, while software-properties now (correctly) creates individual /etc/apt/trusted.gpg.d/*.gpg files for each key added.

The software-properties AptAuth.py module should probably be changed to stop using apt-key, and/or apt-key should be fixed to correctly use all the 'trusted.gpg' files (main file and subdir/'drop-in' files).

This evaluation is just from a quick scan of apt-key source, so someone might want to verify that apt-key is actually the problem.

[H. Hösch (hhoesch-deactivatedaccount)]

apt-key list works. It also shows the keys in /etc/apt/trusted.gpg.d/

[H. Hösch (hhoesch-deactivatedaccount)]

subprocess.Popen was substituted with subprocess.run in the AptAuth.py. That seems
to be the reason. They behave differently here. The other operations seem to work,
but it does not read the keys with subprocess.run. It only reads the keys with subprocess.Popen.
I do not exactly know why. I wrote a small testfile and they seem to behave differently.
It seems as if they had a different return value. I'm not sure. The behavior of these two seems
to be different here. From the python documentation that should not be the case, I would say.
In the python documentation they seem to be nearly equal, but that does not seem to be the case.

[H. Hösch (hhoesch-deactivatedaccount)]

I would say that the application window gets no value when you use subprocess.run.

When you change subprocess.run to subprocess.Popen it works. I can try to post a screenshot.

I do not know if that works. That is the actual version.

[H. Hösch (hhoesch-deactivatedaccount)]

Now I change the AptAuth.py. I changed subprocess.run to subprocess.Popen.

(In the passage where it reads the keys)
I did not change anything else.

(The version in Ubuntu 20.04 had a p.close() or something additionally.)

Here it works. It shows the keys

[H. Hösch (hhoesch-deactivatedaccount)]

The version from Ubuntu 20.04 has a p.close() at the end to close the subprocess.
I have it here in my download folder. Perhaps it would be better to include that.
The problem is that I can't make a package. I could upload a AptAuth.py that works.

I can't understand why the importance was set to low. Ubuntu is a bit like a
commercial operating system where some people even pay for support. Then something
like that should work, I would say. But that is my opinion.

[H. Hösch (hhoesch-deactivatedaccount)]

Here is a working version of the AptAuth.py for Ubuntu 22.04.

The testing version of Ubuntu might have another software-properties-version.

I just changed the subprocess.run to subprocess.Popen and added a p.close to end the subprocess
where it reads the keys. I left the rest as it is. I did not want to change more than necessary.

Other methods of this class use subprocess.run, there you don't need to end the subprocess, I would
say. The subprocess.run ends the subprocess automatically, I would say, but the subprocess.Popen
seemingly needs a manual closing of the subprocess.

[Benjamin Drung (bdrung)]

This regression was introduce in https://git.launchpad.net/software-properties/commit/?id=d70fcb4445a912e1bc88e56dd8474467a288c640 and was included in software-properties 0.99.2. Therefore jammy and kinetic carry that regression. focal is not affected despite having version 0.99.9.8 (but this version number should be 0.98.9.8 instead).

[Benjamin Drung (bdrung)]

subprocess.run is nicer that subprocess.Popen since you do not have to close the opened process. The underlying problem is that subprocess.Popen gave you the output line by line, but stdout from subprocess.run is the complete output. To iterate over the lines you have to call splitlines() first. Otherwise only the first line is taken into account and all following lines will be ignored.

I added a test case and fixed the code (besides some packaging clean-up). Thanks H. Hösch for the analysis and for finding the culprit.

[Łukasz Zemczak (sil2100)]

Hello MadhuSoodanan, or anyone else affected,

Accepted software-properties into jammy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/software-properties/0.99.22.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-jammy to verification-done-jammy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-jammy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Launchpad Janitor (janitor)]

This bug was fixed in the package software-properties - 0.99.25

---------------
software-properties (0.99.25) kinetic; urgency=medium

  * Fix GPG keys are not shown in Software and Updates (LP: #1970449)
  * Remove trailing spaces
  * Drop obsolete imports from __future__
  * Remove duplicate keys in desktop files
  * d/copyright: Use https URLs instead of http
  * d/copyright: Correct path to add-apt-repository.1
  * Drop obsolete Conflicts & Replaces
  * Fix spelling mistake of repository
  * Set Rules-Requires-Root: no
  * Bump Standards-Version to 4.6.1
  * Add Vcs-Browser

 -- Benjamin Drung <email address hidden> Tue, 26 Jul 2022 13:03:28 +0200

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted software-properties (0.99.22.3) for jammy have finished running.
The following regressions have been reported in tests triggered by the package:

crash/8.0.0-1ubuntu1 (arm64)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/jammy/update_excuses.html#software-properties

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Manfred Hampl (m-hampl)]

I tested the updated package on my system and I can confirm that the new version now correctly shows the apt keys. Other actions in software-properties-gtk like changing the repository server and disabling -proposed still work as before.

For the autopkgtest regression that has been reported in comment #21:
This seems to be caused by a transient network hickup, see relevant output from the log below:

Ign:1 http://ddebs.ubuntu.com jammy-updates/main arm64 linux-image-unsigned-5.15.0-41-generic-dbgsym arm64 5.15.0-41.44
Err:1 http://ddebs.ubuntu.com jammy-updates/main arm64 linux-image-unsigned-5.15.0-41-generic-dbgsym arm64 5.15.0-41.44
  Error reading from server. Remote end closed connection [IP: 185.125.190.18 80]
Fetched 17.3 kB in 25min 53s (11 B/s)
E: Failed to fetch http://ddebs.ubuntu.com/pool/main/l/linux/linux-image-unsigned-5.15.0-41-generic-dbgsym_5.15.0-41.44_arm64.ddeb Error reading from server. Remote end closed connection [IP: 185.125.190.18 80]
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
autopkgtest [14:56:44]: test live: -----------------------]
live FAIL non-zero exit status 100

I assume that re-running the autopkgtest will show that this is no regression in the package.
In my opinion verification has succeeded, but due to the autopkgtest failure report I do not dare to change the tags accordingly.

[Benjamin Drung (bdrung)]

Thanks for the testing and looking into the crash/8.0.0-1ubuntu1 (arm64) failure. I retried the test. If that test passes, you can tag this SRU as successful.

[Manfred Hampl (m-hampl)]

Re-running the autopkgtest has succeeded, https://autopkgtest.ubuntu.com/packages/d/crash/jammy/arm64
Confirming the SRU.

[Launchpad Janitor (janitor)]

This bug was fixed in the package software-properties - 0.99.22.3

---------------
software-properties (0.99.22.3) jammy; urgency=medium

  * Fix GPG keys are not shown in Software and Updates (LP: #1970449)
  * Fix spelling mistake of repository

 -- Benjamin Drung <email address hidden> Tue, 26 Jul 2022 13:37:33 +0200

[Łukasz Zemczak (sil2100)]

The verification of the Stable Release Update for software-properties has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.