Discuss further entries in essential which might instead be minimal

[Christian Ehrhardt ]

I was comparing and analyzing a recent docker image for more things we could maybe drop to be more minimal.

I found a few which are in platform.git/required that we might be able to move to minimal.

Not sure what the best practice is here to minimize the container images - put it to minimal as recommends and then remove it in livecd-rootfs? Anyway that would be up to you, the discussion first ...

#1 e2fsprogs
For a cloud image and anything bootable with disks - absolutely.
But for a container which by definition already comes on its FS without disk access.
I think this could be dropped.

#2 diffutils
There is no call to diff other than the references from its own docu and bzdiff/zdiff, ...
Now while we need bzip2/gunzip and such, we don't need bzdiff/zdiff.
Should that be split and removed along with diffutils?

#3 dash
The comment says "Much faster default shell" whicih is right, but bash+dash in essential.
Again probably ok for cloud-images and everything bigger, but for minimal container images one should do - which of them I don't know.

I just was curious on those three after checking the image if they might be removable.
(Triggered by danwest asking about images sizes)

[Julian Andres Klode (juliank)]

#1 The problem with e2fsprogs - it does not include only tools for managing ext* filesystems, it also includes chattr and lsattr.

#3 We can't switch back from dash to bash, as we clearly have some dashims in various scripts that don't work in bash; and to get rid of bash, you'd have to remove scripts explicitly using it.

[Steve Langasek (vorlon)]

These are interesting suggestions, but note that two of these packages are Essential: yes (and the third is Important: yes). We will not produce Ubuntu images that are missing Essential packages, as such an image is unreliable with respect to correct behavior of packages installed later (since Essential packages do not need to be declared as dependencies).

So these are possible improvements to future versions of Ubuntu, but this improvement must be handled as a migration which removes the Essential flag.

[Steve Langasek (vorlon)]

(also, we should drop the Essential flag from bash, NOT from dash)

[Christian Ehrhardt  (paelzer)]

For #1 thanks for the hint Julian.
Would it make sense to break the really essential bits of e2fsprogs into a new binary "fsprogs" then. And make e2fsprogs depend on it to not break anyone.
After that the essential seed could be changed from e2fsprogs -> fsprogs

And yes - this would also be a transition from "Essential: yes" as well.
I did not expect this to be resolved super fast anyway, but to start it or it would be done never.

[Christian Ehrhardt  (paelzer)]

So it seems we have two optional tasks identified that could be done in this context:
- e2fsprogs
  - break fsprogs from e2fsprogs
  - then make the main e2fsprogs non essential
  - remove e2fsprogs from e.g. container images
- drop essential flag from bash
  - is dash considered compatible enough? could we drop bash in containers then?

What about #2 (diffutils):
- I assume it might be needed in the apt/dpkg context to diff show old/new config?
- If that is true could it detect if diff is missing and then fall back to not show the user the
  delta of conffiles?
- Then this could be dropped as well from most minimal images

[Steve Langasek (vorlon)]

On Fri, May 17, 2019 at 06:41:12AM -0000, Christian Ehrhardt  wrote:
> So it seems we have two optional tasks identified that could be done in
> this context:
> - e2fsprogs
> - break fsprogs from e2fsprogs
> - then make the main e2fsprogs non essential
> - remove e2fsprogs from e.g. container images

If fsprogs is to be split out, I think that's a change that should be agreed
in Debian first.

And this will only save ~1MB on the unpacked system. So while it's worth
pushing this along, it's not something we should treat as urgent.

Also, you briefly confused me by saying "make the main e2fsprogs non
essential". e2fsprogs is not Essential, it's Important; so packages are not
allowed to rely on it without a dependency and it is valid to remove it from
images.

> - drop essential flag from bash
> - is dash considered compatible enough? could we drop bash in
> containers then?

"compatible" with what?

The problem is that there's no definition of *what* within an Essential
package you're allowed to assume to be installed on the system. So
historically, a large number of packages assume that the path /bin/bash will
be available as an interpreter, without any dependencies being declared.

So this is a long migration. I want to make sure it's understood that
removing the Essential flag from packages is a change that must be
coordinated with Debian, not something we should do downstream in Ubuntu.

> What about #2 (diffutils):
> - I assume it might be needed in the apt/dpkg context to diff show old/new
> config?
> - If that is true could it detect if diff is missing and then fall back to
> not show the user the delta of conffiles?
> - Then this could be dropped as well from most minimal images

I think it would be a huge usability issue to not be able to use the 'diff'
option in dpkg conffile prompts, whenever these prompts appear on upgrade.
And dpkg does need diffutils in order to display the diffs (I've just
confirmed this in the dpkg source). OTOH this is only used when
interactively invoking dpkg, so probably fits the "not required on a minimal
system" criteria.

On the gripping hand, this package is also Essential.

[Christian Ehrhardt  (paelzer)]

> If fsprogs is to be split out, I think that's a change that should be agreed
> in Debian first.

Debian would need to agree to split e2fsprogs - ack

> And this will only save ~1MB on the unpacked system. So while it's worth
> pushing this along, it's not something we should treat as urgent.

This is just for a few MB => Not urgent - ack

> Also, you briefly confused me by saying "make the main e2fsprogs non
> essential". e2fsprogs is not Essential, it's Important; so packages are not
> allowed to rely on it without a dependency and it is valid to remove it from
> images.

Sorry I have checked e2fsprogs on Bionic for Essential:
$ apt-cache show e2fsprogs | grep Essential
Essential: yes

But I agree, in later releases it is already non-essential.
Sorry for the confusion.

> > - drop essential flag from bash
> > - is dash considered compatible enough? could we drop bash in
> > containers then?
>
> "compatible" with what?
>
> The problem is that there's no definition of *what* within an Essential
> package you're allowed to assume to be installed on the system. So
> historically, a large number of packages assume that the path /bin/bash will
> be available as an interpreter, without any dependencies being declared.
>
> So this is a long migration. I want to make sure it's understood that
> removing the Essential flag from packages is a change that must be
> coordinated with Debian, not something we should do downstream in Ubuntu.

Yeah long+complex+with-debian, I had no other expectations on this one.

> > What about #2 (diffutils):
> > - I assume it might be needed in the apt/dpkg context to diff show old/new
> > config?
> > - If that is true could it detect if diff is missing and then fall back to
> > not show the user the delta of conffiles?
> > - Then this could be dropped as well from most minimal images
>
> I think it would be a huge usability issue to not be able to use the 'diff'
> option in dpkg conffile prompts, whenever these prompts appear on upgrade.
> And dpkg does need diffutils in order to display the diffs (I've just
> confirmed this in the dpkg source). OTOH this is only used when
> interactively invoking dpkg, so probably fits the "not required on a minimal
> system" criteria.
>
> On the gripping hand, this package is also Essential.

Ok, so another rather painful transition for not too much gain.

It seems overall this discussion has identified a few potential
changes, but for all of them the effort/gain ratio is rather bad.
I'm personally fine keeping this on a low prio for now. One might get
to it for loving the case in general or one day prio might be bumped
because carving off the last few MBs becomes more important.