Bionic: debian-installer FTBFS because udev-udeb depends on libkmod2 not libkmod2-udeb

[Mauricio Faria de Oliveira]

[Impact]

 * debian-installer currently FTBFS on Bionic because
   udev-udeb depends on libkmod2 not on libkmod2-udeb.

 * This regression has been introduced in kmod 24-1ubuntu3.3,
   and it propagated into udev-udeb in systemd 237-3ubuntu10.40.

 * See the '[Details]' section for the details. :)

[Fix]

 * That kmod change fixes a FTBFS with newer debhelper on Eoan and later,
   but it is not required with older debhelper on Bionic. Just revert it.

 * There is no need to "fix" this in Eoan and later (debhelper >= 12.3)

 * No-change rebuild of systemd (for udev-udeb to pick up libkmod2-udeb.)

[Test Case]

 * Try to build debian-installer on Bionic; it FTBFS:

 The following packages have unmet dependencies:
  udev-udeb : Depends: libkmod2 but it is not installable
 E: Unable to correct problems, you have held broken packages.

 * Check if libkmod2's shlibs file contains an udeb line:

   $ dpkg-deb --control libkmod2_<version>_<arch>.deb dir
   $ cat dir/shlibs
   libkmod 2 libkmod2
   udeb: libkmod 2 libkmod2-udeb # this is good.

[Regression Potential]

 * The fix only changes the libkmod2's shlibs file to include
   an udeb line (so udebs that depend on that library package
   should depend on the -udeb package.)

   Thus the regression potential is limited to the installer,
   and the udeb packages which depend on libkmod2 (udev-udeb
   only, probably.)

   Regressions could be seen as failures to load libkmod2.so
   and/or not finding its dynamic symbols.

 * There is a no-change rebuild of systemd involved/required,
   so some dependency updates may potentially impact systemd.

[Details]

debian-installer currently FTBFS on Bionic because
udev-udeb depends on libkmod2 not on libkmod2-udeb:

 The following packages have unmet dependencies:
  udev-udeb : Depends: libkmod2 but it is not installable
 E: Unable to correct problems, you have held broken packages.

This regression has been introduced in kmod 24-1ubuntu3.3,
and it propagated into udev-udeb in systemd 237-3ubuntu10.40.

 kmod (24-1ubuntu3.3) bionic; urgency=medium
 <...>
   * Removed --add-udeb from dh_makeshlibs, since libkmod2-udeb does not
     actually contain a library. (Closes: #939779)
 <...>

    From dh_makeshlibs(1):

 --add-udeb=udeb
    Create an additional line for udebs in the shlibs file
    and use udeb as the package name for udebs to depend on
    instead of the regular library package.

    Before:

 $ dpkg-deb --control libkmod2_24-1ubuntu3.2_amd64.deb libkmod2-u32-control

 $ cat libkmod2-u32-control/shlibs
 libkmod 2 libkmod2
 udeb: libkmod 2 libkmod2-udeb

 $ dpkg-deb --info udev-udeb_237-3ubuntu10.39_amd64.udeb | grep Depends:
  Depends: ..., libkmod2-udeb, ...

    After:

 $ dpkg-deb --control libkmod2_24-1ubuntu3.3_amd64.deb libkmod2-u33-control

 $ cat libkmod2-u33-control/shlibs
 libkmod 2 libkmod2

 $ dpkg-deb --info udev-udeb_237-3ubuntu10.40_amd64.udeb | grep Depends:
  Depends: ..., libkmod2, ...

That kmod change fixes a FTBFS with newer debhelper on Eoan and later,
but it is not required with older debhelper on Bionic:

    Eoan:

 $ dpkg -s debhelper | grep Version:
 Version: 12.6.1ubuntu2

 $ dpkg-buildpackpage
 <...>
 dh_makeshlibs --add-udeb=libkmod2-udeb -- -c4
 dh_makeshlibs: The udeb libkmod2-udeb does not contain any shared libraries but --add-udeb=libkmod2-udeb was passed!?
 make: *** [debian/rules:120: .stamp-binary] Error 255
 dpkg-buildpackage: error: debian/rules binary subprocess returned exit status 2

    Bionic:

 $ dpkg -s debhelper | grep Version:
 Version: 11.1.6ubuntu2

 $ dpkg-buildpackpage
 <...>
 dh_makeshlibs --add-udeb=libkmod2-udeb -- -c4
 dh_shlibdeps -- --warnings=7
 <...>
 dpkg-buildpackage: info: binary and diff upload (original source NOT included)

There is no need to "fix" this in Eoan and later, as debhelper >= 12.3
has support for udeb auto-detection in dh_makeshlibs(1) -- and the udeb
line is added to shlibs by default (without --add-udeb):

    From dh_makeshlibs(1):

 Since debhelper 12.3, dh_makeshlibs will by default add an additional udeb line for udebs
 in the shlibs file, when the udeb has the same name as the deb followed by a "-udeb" suffix
 <...>
 Please use the --add-udeb and --no-add-udeb options below when this auto-detection is insufficient.

 $ dpkg-deb --control libkmod2_26-1ubuntu1.1_amd64.deb libkmod2-eoan-control
 $ cat libkmod2-eoan-control/shlibs
 libkmod 2 libkmod2
 udeb: libkmod 2 libkmod2-udeb

 $ dpkg-deb --control libkmod2_27-1ubuntu2_amd64.deb libkmod2-focal-control
 $ cat libkmod2-focal-control/shlibs
 libkmod 2 libkmod2 (>= 27)
 udeb: libkmod 2 libkmod2-udeb (>= 27)

 $ dpkg-deb --control libkmod2_27+20200310-2ubuntu1_amd64.deb libkmod2-groovy-control
 $ cat libkmod2-groovy-control/shlibs
 libkmod 2 libkmod2 (>= 27+20200310)
 udeb: libkmod 2 libkmod2-udeb (>= 27+20200310)

This also needs a no-change rebuild on systemd, after kmod is built/available in archive.

[Mauricio Faria de Oliveira (mfo)]

There's a systemd upload for Bionic with this version number already.

I guess that is a larger change that may be intended for post-release,
while this change is needed for release, so d-i builds w/ new kernels.

So, keeping the same number for now, in case this needs to trump that.

[Mauricio Faria de Oliveira (mfo)]

Currently building the packages on PPA to verify all three
parts (kmod, systemd, d-i) work correctly with the changes.

[Rafael David Tinoco (rafaeldtinoco)]

Very nice catch @mfo,

I'm sorry to have introduced this issue and I'm glad you found it.

Thanks for sharing. I'm +1 on both uploads to fix the issue.

[Mauricio Faria de Oliveira (mfo)]

Apparenlty the systemd package in the upload queue will be reviewed.

For my test packages, @ddstreet told me that a pure no-change rebuild
of systemd would FTBFS on arm64 due to bug #1886197.

So I added that patch to this systemd "almost no-change rebuild" patch.

[Mauricio Faria de Oliveira (mfo)]

With the test packages for kmod and systemd in this PPA [1]
I can confirm that udev-udeb now Depends: on libkmod2-udeb.

From the build log:

 Package: udev-udeb
 ...
 Depends: libblkid1-udeb (>= 2.31), libc6-udeb (>= 2.27), libkmod2-udeb, util-linux-udeb

Just uploaded kmod to Bionic.

[1] https://launchpad.net/~mfo/+archive/ubuntu/lp1889297/

[Mauricio Faria de Oliveira (mfo)]

Finally, with the kmod change, plus the systemd rebuild,
now d-i builds successfully on all archs in the PPA [1].

[1] https://launchpad.net/~mfo/+archive/ubuntu/lp1889297/

[Brian Murray (brian-murray)]

Hello Mauricio, or anyone else affected,

Accepted kmod into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/kmod/24-1ubuntu3.5 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Mauricio Faria de Oliveira (mfo)]

Verification done on bionic-proposed.

The shlibs file of libkmod2 contains the udeb line:

    $ pull-lp-debs libkmod2 bionic
    $ dpkg-deb --control libkmod2_24-1ubuntu3.5_amd64.deb dir
    $ cat dir/shlibs
    libkmod 2 libkmod2
    udeb: libkmod 2 libkmod2-udeb

Then building systemd in a PPA with proposed enabled
does result in udev-udeb depending on libkmod2-udeb:

    Build log:
    ...
    Get:51 http://ftpmaster.internal/ubuntu bionic-proposed/main amd64 libkmod2 amd64 24-1ubuntu3.5 [40.2 kB]
    ...
     Package: udev-udeb
     ...
     Depends: libblkid1-udeb (>= 2.31), libc6-udeb (>= 2.27), libkmod2-udeb, util-linux-udeb
    ...

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted kmod (24-1ubuntu3.5) for bionic have finished running.
The following regressions have been reported in tests triggered by the package:

systemd/237-3ubuntu10.41 (i386)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/bionic/update_excuses.html#kmod

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Mauricio Faria de Oliveira (mfo)]

The failing autopkgtest is in the 'upstream' tests
(search for 'TEST RUN: Job-related tests' twice.)

It's very probably unrelated: apparently the testsuite.service
ended unexpectedly, due to a SIGCHLD because the sleep.service
(the child iiuic) got a SIGTERM unexpectedly too, and finished.
According to the systemd journal in the artifacts.

Just trigerred a re-run since that has no relation with kmod.

[Mauricio Faria de Oliveira (mfo)]

That cleared up the autopkgtest regression.

systemd [bionic/i386]
Version Triggers Date Duration Requester Result
237-3ubuntu10.41 kmod/24-1ubuntu3.5 2020-07-29 23:55:56 UTC 1h 03m 06s mfo pass ...

[Mauricio Faria de Oliveira (mfo)]

The systemd upload has been accepted into bionic-proposed,
and udev-udeb depends on likmod2-udeb. All good.

 Package: udev-udeb
 Source: systemd
 Version: 237-3ubuntu10.42
 ...
 Depends: libblkid1-udeb (>= 2.31), libc6-udeb (>= 2.27), libkmod2-udeb, util-linux-udeb

The existing d-i upload to bump the master/HWE kernels (version 20101020ubuntu543.16)
should pick it up, and now build just fine for 18.04.5.

cheers!

[Mauricio Faria de Oliveira (mfo)]

Hi @mhodson,

Could you please clarify your change 'no longer affects: kmod'?

This bug is actually/fixed in the kmod source package. Thanks!

[Launchpad Janitor (janitor)]

This bug was fixed in the package kmod - 24-1ubuntu3.5

---------------
kmod (24-1ubuntu3.5) bionic; urgency=medium

  * Re-introduce --add-udeb=libkmod2-udeb to dh_makeshlibs.
    Fixes d-i FTBFS due to udev-udeb depending on libkmod2
    instead of libkmod2-udeb. (LP: #1889297)

 -- Mauricio Faria de Oliveira <email address hidden> Tue, 28 Jul 2020 11:46:39 -0300

[Łukasz Zemczak (sil2100)]

The verification of the Stable Release Update for kmod has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Mauricio Faria de Oliveira (mfo)]

Marking systemd as Fix Released as a new version/build that incorporates this kmod fix has been released (version 237-3ubuntu10.42)