bash-completion incorrectly shows source package names for APT

[Norbert]

[Impact]

Source packages have been included in apt-cache pkgnames, and virtual packages have not been included in apt-cache pkgnames --all-names. The former leads to completions autocompleting to source package names where they only should complete to binaries.

[Test case]
An automated test case is included in the test suite

test-ubuntu-bug-1876495-pkgnames-virtual

It verifies that pkgnames does not return source package names, and that pkgnames --all-names does return source package names and virtual package names.

[Where problems could occur]
In the pkgnames command only. If there's a bug, it could exclude or include packages it should not.

[Original bug report]
Steps to reproduce:
1. Have Ubuntu 20.04 LTS installed
2. Open terminal, enter one of the commands below

2a. apt install brisk<Tab>
2b. apt source brisk-menu<Tab>
2c. apt-get install brisk<Tab>
2d. apt-get source brisk-menu<Tab>
2e. apt-cache policy brisk<Tab>

Expected results:
* The bash-completion should not lead to package name as there are no binary packages named with starting part "brisk" (see https://packages.ubuntu.com/search?suite=all&searchon=names&keywords=brisk )

Actual results:
* all commands below get completed to the name of source package - in this example named "brisk-menu"
(see https://packages.ubuntu.com/search?suite=all&section=all&arch=any&keywords=brisk&searchon=sourcenames). This is absolutely unexpected, as user can not really install this package in binary form.

ProblemType: Bug
DistroRelease: Ubuntu Kylin 20.04
Package: bash-completion 1:2.10-1ubuntu1 [origin: Ubuntu]
ProcVersionSignature: Ubuntu 5.4.0-28.32-lowlatency 5.4.30
Uname: Linux 5.4.0-28-lowlatency x86_64
NonfreeKernelModules: zfs zunicode zavl icp zcommon znvpair
ApportVersion: 2.20.11-0ubuntu27
Architecture: amd64
CasperMD5CheckResult: skip
CurrentDesktop: MATE
Date: Sat May 2 20:35:48 2020
Dependencies:

InstallationDate: Installed on 2020-04-22 (9 days ago)
InstallationMedia: Ubuntu-MATE 20.04 LTS "Focal Fossa" - Release amd64 (20200422)
PackageArchitecture: all
SourcePackage: bash-completion
UpgradeStatus: No upgrade log present (probably fresh install)

[Julian Andres Klode (juliank)]

source is certainly a bad example, as it does take source names. But yes, this is a minor inconvenience sometimes.

This needs quite a bit of work to fix, in a coordinated fashion, across the two packages, and I'm not sure it's worth it. Probably needs some investigation whether we already return some other invalid names.

[Julian Andres Klode (juliank)]

Though maybe it only needs work in apt, we'll see.

[Norbert (nrbrtx)]

Still not fixed as per https://askubuntu.com/q/1286600/66509 .

[Norbert (nrbrtx)]

The aforementioned example was happened for `sudo apt install pyqt5<Tab>` and `sudo apt-get install pyqt5<Tab>` .

Please fix this serious bug in essentially critical system component as fast as it possible. It confuses users.

20.10 is also affected.

[Julian Andres Klode (juliank)]

There are two hilarious bugs in here:

(1) We call _config->FindB("APT::Cache::AllNames","false"), trying to set the default to false, but we pass a char* instead, which gets cast to true (as it's != nullptr). So, pkgnames always runs with --all-names

(2) We accidentally exclude virtual packages, hence why we've not seen that before by excluding any package that does not have a version, by checking I.FindPkg("any")->VersionList == 0.

Now the trick here is that the check in (2) fails for source packages, as I.FindPkg("any") does not find a valid package (it returns a pointer to the start of the cache), but that generates a != 0 VersionList.

Anyway, I've got the fix.

[Launchpad Janitor (janitor)]

This bug was fixed in the package apt - 2.1.12

---------------
apt (2.1.12) unstable; urgency=medium

  [ Julian Andres Klode ]
  * pkgnames: Correctly set the default for AllNames to false (LP: #1876495)
  * pkgnames: Do not exclude virtual packages with --all-names
  * Remove expired domain that became nsfw from debian/changelog
  * Do not immediately configure m-a: same packages in lockstep (LP: #1871268)

  [ Américo Monteiro ]
  * Portuguese manpages translation update (Closes: #968414)

  [ David Kalnischkies ]
  * Rename CMake find_package helpers to avoid developer warnings
  * Install translated apt-patterns(7) man pages
  * Remove ancient versions support from apts postinst
  * Update libapt-pkg6.0 symbols file
  * Refresh lintian-overrides of apt and libapt-pkg-doc

 -- Julian Andres Klode <email address hidden> Mon, 23 Nov 2020 17:52:37 +0100

[Łukasz Zemczak (sil2100)]

Hello Norbert, or anyone else affected,

Accepted apt into groovy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/apt/2.1.10ubuntu0.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-groovy to verification-done-groovy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-groovy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Łukasz Zemczak (sil2100)]

Hello Norbert, or anyone else affected,

Accepted apt into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/apt/2.0.4 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted apt (2.1.10ubuntu0.2) for groovy have finished running.
The following regressions have been reported in tests triggered by the package:

reprotest/0.7.15 (amd64, arm64, s390x)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/groovy/update_excuses.html#apt

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted apt (2.0.4) for focal have finished running.
The following regressions have been reported in tests triggered by the package:

gcc-10/10.2.0-5ubuntu1~20.04 (amd64)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/focal/update_excuses.html#apt

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Julian Andres Klode (juliank)]

autopkgtest has passed for 2.0.4 and 2.1.10ubuntu0.2, GCC failure was unrelated cloud problem on amd64, reprotest seems unrelated. both retried too.

[Launchpad Janitor (janitor)]

This bug was fixed in the package apt - 2.1.10ubuntu0.2

---------------
apt (2.1.10ubuntu0.2) groovy; urgency=medium

  [ JCGoran ]
  * Fix "extended_states" typo in apt-mark(8) (Closes: #969086)

  [ Jordi Mallach ]
  * Fix typo in Catalan translation.

  [ Julian Andres Klode ]
  * doc: Bump Ubuntu release from focal to groovy
  * pkgnames: Correctly set the default for AllNames to false, and do not
    exclude virtual packages if --all-names is specified (LP: #1876495)
  * Remove expired domain that became nsfw from debian/changelog
  * patterns: Terminate short pattern by ~ and ! (LP: #1911676)
  * Improve immediate configuration handling (LP: #1871268)
    - Do not immediately configure m-a: same packages in lockstep
    - Ignore failures from immediate configuration. This does not change the
      actual installation ordering - we never passed the return code to the
      caller and installation went underway anyway if it could be ordered at a
      later stage, this just removes spurious after-the-fact errors.

  [ Frans Spiesschaert ]
  * Dutch manpages translation update (Closes: #970037)

  [ Américo Monteiro ]
  * Portuguese manpages translation update (Closes: #968414)

  [ Helge Kreutzmann ]
  * German program translation update (Closes: #977938)

  [ Vangelis Skarmoutsos ]
  * Greek program translation update

 -- Julian Andres Klode <email address hidden> Thu, 14 Jan 2021 12:29:25 +0100

[Łukasz Zemczak (sil2100)]

The verification of the Stable Release Update for apt has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package apt - 2.0.4

---------------
apt (2.0.4) focal; urgency=medium

  [ Julian Andres Klode ]
  * Merge 2.0.2ubuntu0.1 and 2.0.2ubuntu0.2 security updates with 2.0.3
    release.
  * pkgnames: Correctly set the default for AllNames to false, and do not
    exclude virtual packages if --all-names is specified (LP: #1876495)
  * Remove expired domain that became nsfw from debian/changelog
  * patterns: Terminate short pattern by ~ and ! (LP: #1911676)
  * Improve immediate configuration handling (LP: #1871268)
    - Do not immediately configure m-a: same packages in lockstep
    - Ignore failures from immediate configuration. This does not change the
      actual installation ordering - we never passed the return code to the
      caller and installation went underway anyway if it could be ordered at a
      later stage, this just removes spurious after-the-fact errors.

  [ JCGoran ]
  * Fix "extended_states" typo in apt-mark(8) (Closes: #969086)

apt (2.0.3) focal; urgency=medium

  [ Frans Spiesschaert ]
  * Dutch manpages translation update (Closes: #956313)

  [ David Kalnischkies ]
  * Add correct std namespace to nullptr_t

  [ Julian Andres Klode ]
  * Reinstate * wildcards (Closes: #953531) (LP: #1872200)
  * apt list: Fix behavior of regex vs fnmatch vs wildcards
  * debian/gbp.conf: We're in the 2.0.y branch
  * CI: Test on ubuntu:focal, not debian:unstable

 -- Julian Andres Klode <email address hidden> Thu, 14 Jan 2021 12:49:00 +0100