Needs obsoletion tag in dkms.conf for improperly versioned drivers

[Martin Pitt]

I DKMSified the dvb-v4l drivers from hg head to provide support for newer DVB-T USB sticks for Ubuntu 8.04. It mostly works well now, except for one thing: the autoinstallation fails when I install a new kernel ABI.

The reason is that those drivers need to be dkms install'ed with --force; the modules constantly get support for new hardware, but their versions do not get bumped (at least not with every hg commit that changes coverage; maybe with the next merge into linux git head). I can add the --force into the debian/postinst template for mkdeb, but there does not seem to be a way to use install --force in /etc/init.d/dkms_autoinstaller.

Maybe this should become an option in dkms.conf? AUTOINSTALL=force, or keep AUTOINSTALL=yes and add AUTOINSTALL_FORCE=yes ?

If you fine with the general approach, I can try to cook up a patch.

[Matt Domsch (matt-domsch)]

Philosophically, I don't like the autoinstaller forcing anything. It isn't smart enough to know the right thing to do in the absence of a version number bump.

I don't understand how after an ABI bump the autoinstaller wouldn't work. /lib/modules/2.6.24-14-generic becomes 2.6.24-15-generic, so the autoinstaller (with AUTOINSTALL=yes) should build and install any old modules, assuming they're not present with the same or greater version already in linux-2.6.24-25-generic.

[Matt Domsch (matt-domsch)]

Martin, can you post your package code?

[Martin Pitt (pitti)]

But if I need install --force for the initial install, then I need it for the next kernel ABI for the very same reason. Which is that the modules are not versioned properly, so I need to install the modules from the v4l-dvb tree even if they have the same version as in linux-image. I. e. it's exactly the "assuming they're not present with the same or greater version already" assumption that breaks.

This is primarily a concern when going from 2.6.24-17 to 2.6.24-18, where I know that I need the new modules. For going to 2.6.24-25 I would actually need the OBSOLETE-BY: field we discussed about, so that autoinstall won't build it for a new upstream kernel.

I put my current DKMSified package to http://people.ubuntu.com/~pitti/tmp/.

I also attach the dkms.conf for this.

[Martin Pitt (pitti)]

FYI, this is my shell script which generates dkms.conf. Since v4l-dvb builds 210 modules, figuring out the module names and location by hand is not a fun job, so I automated it. This also cares about the add , mkdeb, remove procedure (thus it should only be done if v4l-dvb is not yet registered to dkms; I'll clean that up later, it's just the first shot at this).

[Matt Domsch (matt-domsch)]

Here's the thing... MODULE_VERSION() tags aren't used by anything in the kernel at all. The only reason they exist is to help tools like DKMS (and in fact, I know of no other tools that use them.) So if we're building DKMS packages, I'd really like the MODULE_VERSION tags to mean something. For all the other drivers Dell has had to update since we got MODULE_VERSION() added, those maintainers keep their MODULE_VERSION tags meaningful. (yes, I know, with 220 modules, that may be more difficult - yours is an extreme case of lots of individual drivers in a single deb).

If the MODULE_VERSION tags aren't going to be meaningful, then they should be removed from the drivers completely.

On the same grounds, if MODULE_VERSION tags can't be used, then yes, we'll have to allow DKMS to handle version comparisons / upgrades / etc in their absence. I hadn't wanted to go this route, preferring them be done in MODULE_VERSION tags. :-(

your make-dkms.sh is cool.

[Martin Pitt (pitti)]

Hi Matt,

Matt Domsch [2008-06-05 14:09 -0000]:
> Here's the thing... MODULE_VERSION() tags aren't used by anything in
> the kernel at all. The only reason they exist is to help tools like
> DKMS (and in fact, I know of no other tools that use them.) So if we're
> building DKMS packages, I'd really like the MODULE_VERSION tags to mean
> something. For all the other drivers Dell has had to update since we
> got MODULE_VERSION() added, those maintainers keep their MODULE_VERSION
> tags meaningful.

I absolutely understand. I can only guess that the upstream devs just
want to do it once per "release" (merge to upstream linux), and not
for each commit.

> If the MODULE_VERSION tags aren't going to be meaningful, then they
> should be removed from the drivers completely.

Patching them out completely shouldn't be too hard, if that helps and
will quiesce the version sanity check. If you think that's better than
using --force, I can go that route.

> your make-dkms.sh is cool.

Well, a blunt hack born out of necessity.. :-)

Thanks,

Martin

[Matt Domsch (matt-domsch)]

If either the module in the kernel, or the module coming from DKMS, are missing the MODULE_VERSION tag, then the version comparison can't be done by DKMS (and isn't), so it won't exit if you're not using --force.

Removing the MODULE_VERSION tags from the modules going in via DKMS would seem the sensible thing to do, if the authors aren't keeping them at least somewhat meaningful. With version numbers like "1.0" for most, I expect they're not being kept meaningful.

There's a related problem, though. If you were going to count on using the module's 'srcversion' checksum field to match, to know when DKMS should stop replacing the ones in the kernel, then we know they _won't_ match, even if they're otherwise identical, as soon as you've removed the MODULE_VERSION tags from the ones in the DKMS package, but not the ones in the kernel. Doh. So you still need the "Obsoletes By" logic, which I'd been hoping to avoid.

I'm on vacation today, but will look at Mario's Obsoletes By code again next week. My biggest problem with it is that it forces any package built using it to be distro-specific, while for nearly all Dell DKMSified packages we produce, it's a single package for multiple distributions and distro versions. I really want that field to be distribution/version specific somehow. DKMS does have some inkling of which OS version is running, though it's little-used. see function distro_version(). Note too - because it calls 'rpm' to get this information, it's problematic. RPM isn't recursive, thus calling RPM query functions to get the OS version, if DKMS happens to be invoked from an rpm %post script, has the potential to deadlock. In practice we've gotten lucky here, but it scares me.

[Mario Limonciello (superm1)]

Matt:

Perhaps a little off topic for this bug, why don't you just use lsb_release instead of recursive RPM calls to determine the OS version? lsb_release should be more responsive at least too?

Mario Limonciello
Dell | Linux Engineering
<email address hidden>

-----Original Message-----
From: <email address hidden> on behalf of Matt Domsch
Sent: Fri 6/6/2008 8:29 AM
To: <email address hidden>
Subject: [Bug 237358] Re: dkms_autoinstaller does not support install with--force

If either the module in the kernel, or the module coming from DKMS, are
missing the MODULE_VERSION tag, then the version comparison can't be
done by DKMS (and isn't), so it won't exit if you're not using --force.

Removing the MODULE_VERSION tags from the modules going in via DKMS
would seem the sensible thing to do, if the authors aren't keeping them
at least somewhat meaningful. With version numbers like "1.0" for most,
I expect they're not being kept meaningful.

There's a related problem, though. If you were going to count on using
the module's 'srcversion' checksum field to match, to know when DKMS
should stop replacing the ones in the kernel, then we know they _won't_
match, even if they're otherwise identical, as soon as you've removed
the MODULE_VERSION tags from the ones in the DKMS package, but not the
ones in the kernel. Doh. So you still need the "Obsoletes By" logic,
which I'd been hoping to avoid.

I'm on vacation today, but will look at Mario's Obsoletes By code again
next week. My biggest problem with it is that it forces any package
built using it to be distro-specific, while for nearly all Dell
DKMSified packages we produce, it's a single package for multiple
distributions and distro versions. I really want that field to be
distribution/version specific somehow. DKMS does have some inkling of
which OS version is running, though it's little-used. see function
distro_version(). Note too - because it calls 'rpm' to get this
information, it's problematic. RPM isn't recursive, thus calling RPM
query functions to get the OS version, if DKMS happens to be invoked
from an rpm %post script, has the potential to deadlock. In practice
we've gotten lucky here, but it scares me.

--
dkms_autoinstaller does not support install with --force
https://bugs.launchpad.net/bugs/237358
You received this bug notification because you are a member of The Dell
Team, which is subscribed to Dell Ubuntu Project.

[Martin Pitt (pitti)]

Mario Limonciello [2008-06-06 17:19 -0000]:
> Perhaps a little off topic for this bug, why don't you just use
> lsb_release instead of recursive RPM calls to determine the OS version?
> lsb_release should be more responsive at least too?

That's what I was just going to propose as well. In Apport and Jockey
I use "lsb_release -sir".

Matt, thanks for the detailled explanations. It indeed seems to be the
very use case I had in mind when suggesting an obsoletion field. Here
I'd just set it to 2.6.25 (and I can always upload a followup package
which bumps it further), so an upstream oriented field would work as a
start.

[Matt Domsch (matt-domsch)]

Here's a thought then. dkms.conf is really just eval'd as a shell script. You could do something like:

if [ -x /usr/bin/lsb_release ]; then
  if [ "$(/usr/bin/lsb_release -sir)" == "Ubuntu 8.04" ]; then
  OBSOLETES_BY="2.6.25"
  fi
fi

My RHEL and SLES boxes are offline atm, but if everyone is providing /usr/bin/lsb_release now, I'd be more than happy to make use of that. I'd seen it before, but long ago forgotten about it. Thanks for the reminder!

[Matt Domsch (matt-domsch)]

Mario, if you'll send me a docs patch to show the proper usage of OBSOLETES_BY, I'll apply it, then that whole branch you've been working on.

[Mario Limonciello (superm1)]

Matt:

My existing patch should be touching the man page too.

Mario Limonciello
Dell | Linux Engineering
<email address hidden>

-----Original Message-----
From: <email address hidden> on behalf of Matt Domsch
Sent: Sat 6/7/2008 8:12 AM
To: <email address hidden>
Subject: [Bug 237358] Re: dkms_autoinstaller does not support install with--force

Mario, if you'll send me a docs patch to show the proper usage of
OBSOLETES_BY, I'll apply it, then that whole branch you've been working
on.

--
dkms_autoinstaller does not support install with --force
https://bugs.launchpad.net/bugs/237358
You received this bug notification because you are a member of The Dell
Team, which is subscribed to Dell Ubuntu Project.

[Matt Domsch (matt-domsch)]

Mario's branch has been added to the master branch now. See http://linux.dell.com/git dkms.git tree for OBSOLETE_BY. I also added an admonition against using OBSOLETE_BY to the manpage.

[Martin Pitt (pitti)]

Thanks for getting OBSOLETE-BY in! Looking forward to a new upstream version.

I agree to your reasoning why --force in the autoinstall init script is bad. Removing improper MODULE_VERSION() tags indeed sounds like the right solution here, I'll change my v4l-dvb DKMSification accordingly. I adjusted the bug title to reflect the current solution now.

[Martin Pitt (pitti)]

I dropped --force and replaced it with some sed'ing, and now everything Just Works (TM). Thanks a lot! I attach my updated make-dkms.sh script, just in case anyone is interested.

I put the package into my PPA now (https://launchpad.net/~pitti/+archive) and blogged about it (http://martinpitt.wordpress.com/2008/06/10/packaged-dvb-t-drivers-for-ubuntu-804/, syndicated on planet.ubuntu.com) to get more feedback. Thanks a lot for all your help!

[mirak (mirak-mirak)]

hi, I also tried to do dkms package of v4l-dvb and I have the same issue with --force needed for the install

however I don't understand the OBSOLETE_BY solution despite the man page documentation in ubuntu 8.10

[Timothy R. Chavez (timrchavez)]

The bug task for the somerville project has been removed by an automated script. This bug has been cloned on that project and is available here: https://bugs.launchpad.net/bugs/1305812