The unminimize command fails to reinstall packages with missing files in the `share` and `locale` directories
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
cloud-images |
Fix Released
|
Undecided
|
Unassigned | ||
livecd-rootfs (Ubuntu) |
Fix Released
|
Undecided
|
Utkarsh Gupta | ||
Jammy |
Fix Released
|
Undecided
|
Utkarsh Gupta | ||
Lunar |
Fix Released
|
Undecided
|
Utkarsh Gupta |
Bug Description
[Impact]
========
The `unminimize` command in livecd-rootfs doesn't work out of the box. When `unminimize` is run, it results into the following errors:
#9 47.54 Reinstalling packages with system documentation in /usr/share/doc/ ..
#9 48.76 dpkg-query: error: --search needs at least one file name pattern argument
#9 48.76
#9 48.76 Use --help for help about querying packages.
#9 48.80 Reading package lists...
#9 49.49 Building dependency tree...
#9 49.62 Reading state information...
#9 49.76 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
#9 49.76 Restoring system translations...
#9 50.97 dpkg-query: error: --search needs at least one file name pattern argument
#9 50.97
#9 50.97 Use --help for help about querying packages.
#9 51.01 Reading package lists...
#9 51.71 Building dependency tree...
#9 51.85 Reading state information...
#9 52.00 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
#9 53.18 Documentation has been restored successfully.
#9 DONE 54.3s
Specifically, the error is:
dpkg-query: error: --search needs at least one file name pattern argument
The problem is that prior to dpkg/1.21.0, there was a bug where dpkg -V/--verify couldn't list all the correct packages correctly but with that being fix and in archive since Jammy, this works perfectly but the syntax to report the missing files have changed. It just prints 'missing' now. With that new format, we can now fix the regex to simply list the packages.
[Test Plan]
===========
1. Create an image with the changes:
$ ubuntu-bartender --build-provider gce --hook-extras-dir "$(pwd)
2. Boot into that image. Ensure it's a minimized image.
3. Run update and unminimize.
# apt update
# unminimize
This will result in the above error.
# echo $?
1
Also notes that the above command failed to run successfully.
OTOH, with the fixed package, unminimize will run just fine and "echo $?" would say 0.
[Where Problems Could Occur]
=======
unminimize is already broken so not a lot of things can go wrong at this point. However, if a user has already made amendments to their unminimize script and adapted it to work differently for them, this update could break their existing set up and functionality. But I don't envision that happening a lot, really.
[OG Bug Report]
===============
I am building a custom Docker image based on:
https:/
I added a line that runs `yes | unminmize':
Here's from build log :
https:/
And here are the errors from the `uminmize` command:
#9 47.54 Reinstalling packages with system documentation in /usr/share/doc/ ..
#9 48.76 dpkg-query: error: --search needs at least one file name pattern argument
#9 48.76
#9 48.76 Use --help for help about querying packages.
#9 48.80 Reading package lists...
#9 49.49 Building dependency tree...
#9 49.62 Reading state information...
#9 49.76 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
#9 49.76 Restoring system translations...
#9 50.97 dpkg-query: error: --search needs at least one file name pattern argument
#9 50.97
#9 50.97 Use --help for help about querying packages.
#9 51.01 Reading package lists...
#9 51.71 Building dependency tree...
#9 51.85 Reading state information...
#9 52.00 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
#9 53.18 Documentation has been restored successfully.
#9 DONE 54.3s
Specifically, the error is:
dpkg-query: error: --search needs at least one file name pattern argument
I copped `/usr/local/
When looking for packages with missing files in the `/usr/bin/man` directory, the script does this:
dpkg -S /usr/share/man/ |sed 's|, |\n|g;s|: [^:]*$||' | DEBIAN_
However, for the `/usr/share/docs/` the script does this:
dpkg --verify --verify-format rpm | awk '/..5...... \/usr\/share\/doc/ {print $2}' | sed 's|/[^/]*$||' | sort |uniq \
| xargs dpkg -S | sed 's|, |\n|g;s|: [^:]*$||' | uniq | DEBIAN_
The script runs the same command for the `/usr/share/
In both cases, no packages ever make it to the `xargs dpkg -S` command, so the script throws an error. However, because the error occurs in a pipe, it is lost, and the script continues despite the `set -e` at the start (which is a bug in itself).
However, with a little experimentation, I found that the original command for `/usr/local/bin` works perfectly for the `/usr/share/docs/` and `/usr/share/
Why not run the same command for all three directories? You could even wrap it in a function.
However, you could improve performance by combining all three into a single command.
The `unminimize` script notes:
# Reinstallation takes place in two steps because a single dpkg --verified
# command generates very long parameter list for "xargs dpkg -S" and may go
# over ARG_MAX. Since many packages have man pages the second download
# handles a much smaller amount of packages.
I understand the concern about hitting `ARG_MAX`, but I think a better solution to that problem would be to use `xargs -n` to set a limit on the maximum number of arguments. This change would allow xargs to run `dpkg -S` as many times as need (to avoid hitting ARG_MAX).
Doing it this way would also speed up the script.
There are many packages with missing files in multiple minmized directories, meaning that as it stands, even if the bug was fixed, some packages would be re-installed twice, maybe even three times.
(I have seen this for myself because I essentially wrote a shorter version of the script for my own use.)
Related branches
- Simon Déziel (community): Needs Fixing
- Philip Roche (community): Approve
- Canonical Foundations Team: Pending requested
-
Diff: 54 lines (+14/-6)2 files modifieddebian/changelog (+7/-0)
live-build/auto/build (+7/-6)
- Andrew Cloke (community): Approve
- Steve Langasek: Abstain
- Philip Roche (community): Approve
-
Diff: 54 lines (+14/-6)2 files modifieddebian/changelog (+7/-0)
live-build/auto/build (+7/-6)
- John Chittum (community): Approve
- Ubuntu Core Development Team: Pending requested
-
Diff: 27 lines (+4/-4)1 file modifiedlive-build/auto/build (+4/-4)
Changed in cloud-images: | |
status: | New → Fix Released |
Changed in livecd-rootfs (Ubuntu): | |
assignee: | nobody → Utkarsh Gupta (utkarsh) |
Changed in livecd-rootfs (Ubuntu Jammy): | |
assignee: | nobody → Utkarsh Gupta (utkarsh) |
Changed in livecd-rootfs (Ubuntu Lunar): | |
assignee: | nobody → Utkarsh Gupta (utkarsh) |
description: | updated |
Changed in livecd-rootfs (Ubuntu Jammy): | |
status: | Incomplete → New |
Changed in livecd-rootfs (Ubuntu Lunar): | |
status: | Incomplete → New |
summary: |
- The unminimize command fails to reinstall packags with missing files in + The unminimize command fails to reinstall packages with missing files in the `share` and `locale` directories |
tags: |
added: verification-done removed: verification-needed |
tags: | removed: block-proposed-jammy |
Thank you for reporting the issue and giving suggestions in great detail! Also very timely as myself and another script maintainer were just having conversations about the minimize/unminimize flow.
I've added `livecd-rootfs` package for tracking, as the script lives within those build scripts. We'll take some time to dig into the suggestions and issues and come back with more info.