Ubuntu links missing or broken in /usr/share/debootstrap/scripts

[Lastique]

[Justification]
Fix for an SRU regression. The previous SRU dropped the 'scripts' symlinks for all recent Ubuntu releases because the links were no longer needed for debootstrap's internal use. However, this broke compatibility with third-party code which (unnecessarily) hard-coded the script argument to debootstrap.

[Test case]
1. apt install ubuntu-dev-tools pbuilder arch-test
2. pbuilder-dist jammy create

[Where things could go wrong]
Provided the diff contents are correct, this is a straight revert of a change from the preceding SRU and should not regress behavior. The revert means that when bootstrapping the releases that were manually added before, debootstrap will follow the symlink instead of using the new code, which in inconsequential.

[Original description]
debootstrap package does not have links for many Ubuntu versions in /usr/share/debootstrap/scripts. In particular, jammy is missing.

See the file list:

https://packages.ubuntu.com/jammy-updates/all/debootstrap/filelist

Also, /usr/share/debootstrap/scripts/hirsute link points to groovy, which is also missing (i.e. hirsute is a broken symlink).

This breaks pbuilder, which no longer recognizes distro names and fails with errors like this:

Warning: Unknown distribution "jammy". Do you want to continue [y|N]?
Aborting as requested.

ProblemType: Bug
DistroRelease: Ubuntu 22.04
Package: debootstrap 1.0.126+nmu1ubuntu0.4
ProcVersionSignature: Ubuntu 5.19.0-1027.28~22.04.1-lowlatency 5.19.17
Uname: Linux 5.19.0-1027-lowlatency x86_64
NonfreeKernelModules: vtsspp sep5 socperf3 pax nvidia_modeset nvidia
ApportVersion: 2.20.11-0ubuntu82.5
Architecture: amd64
CasperMD5CheckResult: unknown
CurrentDesktop: KDE
Date: Tue Jun 20 19:34:53 2023
InstallationDate: Installed on 2015-05-01 (2971 days ago)
InstallationMedia: Kubuntu 15.04 "Vivid Vervet" - Release amd64 (20150422)
PackageArchitecture: all
SourcePackage: debootstrap
UpgradeStatus: Upgraded to jammy on 2022-05-14 (402 days ago)

[Lastique (andysem)]

I'll note that this is a regression introduced by one of the recent updates (not sure which exact version). Previously, pbuilder for jammy was working on the same system.

[Lastique (andysem)]

The problem is not present in debootstrap version 1.0.126+nmu1 available in jammy.

[Keith Maxwell (maxwell-k)]

I think this bug is related to https://bugs.launchpad.net/ubuntu/+source/debootstrap/+bug/2024200

Perhaps a duplicate

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Keith Maxwell (maxwell-k)]

I think the relevant item in the changelog is:

> * Update debootstrap to automatically use the gutsy script by default
> if the target is an Ubuntu release known to distro-info, eliminating
> the need to upload debootstrap for each new Ubuntu release.
> * Add dependency on distro-info, which is part of ubuntu-minimal, for the
> above
> * Remove all the old symlinks that are no longer needed.

http://changelogs.ubuntu.com/changelogs/pool/main/d/debootstrap/debootstrap_1.0.128+nmu2ubuntu4/changelog

I think that the relevant commit is
https://git.launchpad.net/ubuntu/+source/debootstrap/commit/?id=2e6949c9017d7231bc9d927109f2c068d631e3eb

Somehow "automatically" isn't working for me with pbuilder on Jammy. The exact steps for me are to start an new VM for Jammy 22.04 and run:

```
sudo apt-get update
sudo apt-get install --yes ubuntu-dev-tools
pbuilder-dist jammy create

```

The error message is:

```
Warning: Unknown distribution "jammy". Do you want to continue [y|N]?
```

[Steve Langasek (vorlon)]

The symlinks were dropped in the update because they were no longer necessary for debootstrap's internal use. I didn't count on the fact that the various callers of debootstrap would hard-code the use of the script argument.

We should fix this by restoring the symlinks that were dropped in the SRU.

Going forward, we will NOT be doing SRUs to add NEW symlinks to debootstrap. So to use tools on existing stable series to bootstrap newer series, those tools will need to be fixed to not hard-code the script argument. We have updated mk-sbuild for this. I have marked the pbuilder tasks on this bug "Won't fix" because I do not expect anyone to put in the effort to do an SRU of this universe package. But if someone does, it would be acceptable under SRU policy.

[Steve Langasek (vorlon)]

Marking 2024200 as a duplicate of this bug rather than the other way around, since 2024200 describes an issue that's specific to focal and this bug applies to all existing stable series.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Steve Langasek (vorlon)]

For mantic and later, the symlinks are gone and not coming back.

[Lastique (andysem)]

Are you saying that unless someone (other than Canonical) fixes pbuilder, it will permanently stop working starting with Ubuntu 23.10 (Mantic Minotaur)? Meaning it will be broken in the next LTS (24.04) as well?

[Lastique (andysem)]

Also note that the hirsute link is broken, which should probably be fixed in Mantic.

[Christopher Marshall (slackwarerefugee)]

I just ran into this bug today while working through this tutorial: https://packaging.ubuntu.com/html/fixing-a-bug.html#testing-the-fix

Specifically this command line:
pbuilder-dist <release> build ../<package>_<version>.dsc

I am trying to learn how to compile binary packages from source packages.

Is there a work around for this problem? How would I create the required symlinks to fix this myself?

[Steve Langasek (vorlon)]

I've just noticed from https://bugs.launchpad.net/ubuntu/+source/pbuilder/+bug/2024483/comments/6 that this isn't pbuilder itself which is hard-coding the script argument and failing, but pbuilder-dist. Reassigning to ubuntu-dev-tools.

[Alexis Wilke (alexis-m2osw)]

I noticed this this morning and can confirm that the "jammy" file is missing on all of my systems and as a result I can't build packages anymore. Re-adding those files to debootstrap seems to be me like the simplest at the moment, until the other systems (a.k.a. pbuilder-dist) are fixed.

[Leo Lin (0xff07)]

Copying files from older version of debootstrap seems to work for me as well. I copied files from version 1.0.95ubuntu0.12 in particular:

$ wget "https://launchpad.net/ubuntu/+archive/primary/+sourcefiles/debootstrap/1.0.95ubuntu0.12/debootstrap_1.0.95ubuntu0.12.tar.gz"
$ tar -xf debootstrap_1.0.95ubuntu0.12.tar.gz
$ cp -r scripts/* /usr/share/debootstrap/scripts/

After that pbuilder-dist could recognize jammy.

[Andreas Hasenack (ahasenack)]

Hello Lastique, or anyone else affected,

Accepted debootstrap into lunar-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/debootstrap/1.0.128+nmu2ubuntu1.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-lunar to verification-done-lunar. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-lunar. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Andreas Hasenack (ahasenack)]

Hello Lastique, or anyone else affected,

Accepted debootstrap into kinetic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/debootstrap/1.0.127ubuntu1.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-kinetic to verification-done-kinetic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-kinetic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Andreas Hasenack (ahasenack)]

Hello Lastique, or anyone else affected,

Accepted debootstrap into jammy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/debootstrap/1.0.126+nmu1ubuntu0.5 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-jammy to verification-done-jammy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-jammy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Lastique (andysem)]

I've tested 1.0.126+nmu1ubuntu0.5 by running `pbuilder-dist jammy create` and the command successfully created a new .tgz filesystem for Jammy. The error about unknown distribution didn't appear.

[Andreas Hasenack (ahasenack)]

Hello Lastique, or anyone else affected,

Accepted debootstrap into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/debootstrap/1.0.118ubuntu1.11 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted debootstrap (1.0.126+nmu1ubuntu0.5) for jammy have finished running.
The following regressions have been reported in tests triggered by the package:

debuerreotype/0.14-1 (ppc64el)
livecd-rootfs/2.765.20 (amd64)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/jammy/update_excuses.html#debootstrap

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Alexis Wilke (alexis-m2osw)]

I tested with https://launchpad.net/ubuntu/+source/debootstrap/1.0.128+nmu2ubuntu1.2 and it worked for me. My build system built many of my packages (It stopped in the middle, but that's probably an issue on my end). Thank you for the quick fix.

[Steve Langasek (vorlon)]

lunar:

$ sudo apt install ubuntu-dev-tools pbuilder arch-test
$ pbuilder-dist jammy create
Warning: Unknown distribution "jammy". Do you want to continue [y|N]? y
Cannot determine if "create" is a valid architecture. Please install the arch-test package and retry.
$

debootstrap from lunar-proposed:

$ pbuilder-dist jammy create
W: /home/vorlon/.pbuilderrc does not exist
I: Logging to /home/vorlon/pbuilder/jammy_result/last_operation.log
W: cgroups are not available on the host, not using them.
I: Distribution is jammy.
[...]
^C
$

[Steve Langasek (vorlon)]

Actually, the above test without lunar-proposed also didn't have arch-test installed, I didn't notice the error message there was unrelated to debootstrap.

So in fact, it appears that pbuilder-dist does NOT fail with the debootstrap symlinks removed, it only throws a prompt which the user can bypass.

[Steve Langasek (vorlon)]

On kinetic:

$ sudo apt install ubuntu-dev-tools pbuilder arch-test
$ pbuilder-dist jammy create
Warning: Unknown distribution "jammy". Do you want to continue [y|N]?
$

With kinetic-proposed:

$ pbuilder-dist jammy create
W: /home/vorlon/.pbuilderrc does not exist
I: Logging to /home/vorlon/pbuilder/jammy_result/last_operation.log
W: cgroups are not available on the host, not using them.
I: Distribution is jammy.
I: Current time: Fri Jul 7 21:42:44 UTC 2023
I: pbuilder-time-stamp: 1688766164
I: Building the build environment
I: running debootstrap
/usr/sbin/debootstrap
I: Target architecture can be executed
I: Retrieving InRelease
[...]
^C
$

[Steve Langasek (vorlon)]

Because focal was rolled back (due to the d-i impact), the focal-updates version does not show the problem:

$ sudo apt install ubuntu-dev-tools pbuilder arch-test
$ pbuilder-dist jammy create
W: /home/vorlon/.pbuilderrc does not exist
I: Logging to /home/vorlon/pbuilder/jammy_result/last_operation.log
W: cgroups are not available on the host, not using them.
I: Distribution is jammy.
I: Current time: Fri Jul 7 22:33:46 UTC 2023
I: pbuilder-time-stamp: 1688769226
I: Building the build environment
I: running debootstrap
/usr/sbin/debootstrap
I: Target architecture can be executed
[...]
^C
$

debootstrap from focal-proposed:

$ pbuilder-dist jammy create
W: /home/vorlon/.pbuilderrc does not exist
I: Logging to /home/vorlon/pbuilder/jammy_result/last_operation.log
W: cgroups are not available on the host, not using them.
I: Distribution is jammy.
I: Current time: Fri Jul 7 22:37:13 UTC 2023
I: pbuilder-time-stamp: 1688769433
I: Building the build environment
I: running debootstrap
/usr/sbin/debootstrap
I: Target architecture can be executed
I: Retrieving InRelease
^CManually aborted.
$

So this can be considered verified for focal as well.

[Steve Langasek (vorlon)]

For one last double-check, I downloaded

https://launchpad.net/ubuntu/+source/debootstrap/1.0.118ubuntu1.9/+build/26228958/+files/debootstrap_1.0.118ubuntu1.9_all.deb

to confirm that pbuilder-dist in focal does show the issue when the symlink is missing, and:

$ pbuilder-dist jammy create
Warning: Unknown distribution "jammy". Do you want to continue [y|N]?
$

[Launchpad Janitor (janitor)]

This bug was fixed in the package debootstrap - 1.0.128+nmu2ubuntu1.2

---------------
debootstrap (1.0.128+nmu2ubuntu1.2) lunar; urgency=medium

  * Restore the old symlinks removed in the previous SRU. While not
    needed by debootstrap itself, other tools may call debootstrap with
    the optional 'script' argument (which was always unnecessary when
    its value was the same as the 'suite' argument), and this change
    broke compatibility with them. LP: #2024483.

 -- Steve Langasek <email address hidden> Sat, 24 Jun 2023 00:07:30 -0700

[Łukasz Zemczak (sil2100)]

The verification of the Stable Release Update for debootstrap has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package debootstrap - 1.0.127ubuntu1.3

---------------
debootstrap (1.0.127ubuntu1.3) kinetic; urgency=medium

  * Restore the old symlinks removed in the previous SRU. While not
    needed by debootstrap itself, other tools may call debootstrap with
    the optional 'script' argument (which was always unnecessary when
    its value was the same as the 'suite' argument), and this change
    broke compatibility with them. LP: #2024483.

 -- Steve Langasek <email address hidden> Fri, 23 Jun 2023 23:56:20 -0700

[Łukasz Zemczak (sil2100)]

Steve, I see that the jammy debootstrap upload has a repeating autopkgtest failure in livecd-rootfs, but that one seems to be unrelated (I see "1632s gzip: stdout: No space left on device" happening there). This is certainly something that needs a separate investigation, but I won't block the release of this update on that. Would be great if we had a follow up investiagtion/bug for this.

[Launchpad Janitor (janitor)]

This bug was fixed in the package debootstrap - 1.0.126+nmu1ubuntu0.5

---------------
debootstrap (1.0.126+nmu1ubuntu0.5) jammy; urgency=medium

  * Restore the old symlinks removed in the previous SRU. While not
    needed by debootstrap itself, other tools may call debootstrap with
    the optional 'script' argument (which was always unnecessary when
    its value was the same as the 'suite' argument), and this change
    broke compatibility with them. LP: #2024483.

 -- Steve Langasek <email address hidden> Sat, 24 Jun 2023 00:12:13 -0700

[Launchpad Janitor (janitor)]

This bug was fixed in the package debootstrap - 1.0.118ubuntu1.11

---------------
debootstrap (1.0.118ubuntu1.11) focal; urgency=medium

  * Restore the old symlinks removed in the previous SRU. While not
    needed by debootstrap itself, other tools may call debootstrap with
    the optional 'script' argument (which was always unnecessary when
    its value was the same as the 'suite' argument), and this change
    broke compatibility with them. LP: #2024483.

debootstrap (1.0.118ubuntu1.10) focal; urgency=medium

  * Add Breaks: on ubuntu-dev-tools (<< 0.190~) to ensure compatibility with
    mk-sbuild.

debootstrap (1.0.118ubuntu1.9) focal; urgency=medium

  * Update debootstrap to automatically use the gutsy script by default
    if the target is an Ubuntu release known to distro-info, eliminating
    the need to upload debootstrap for each new Ubuntu release.
  * Add dependency on distro-info, which is part of ubuntu-minimal, for the
    above
  * Remove all the old symlinks that are no longer needed.
  * LP: #2020530.

 -- Steve Langasek <email address hidden> Sat, 24 Jun 2023 00:14:37 -0700