openafs kernel modules don't build with edgy kernel 2.6.17

[Alex Mauer]

The kernel modules won't build with kernel 2.6.17-4 using module-assistant:
  CC [M] /usr/src/modules/openafs/src/libafs/MODLOAD-2.6.17-4-686-MP/afs_analyze.o
In file included from /usr/src/modules/openafs/src/afs/afs_osi.h:443,
                 from /usr/src/modules/openafs/src/rx/rx_clock.h:88,
                 from /usr/src/modules/openafs/src/rx/rx.h:35,
                 from /usr/src/modules/openafs/src/afs/afsincludes.h:26,
                 from /usr/src/modules/openafs/src/libafs/MODLOAD-2.6.17-4-686-MP/afs_analyze.c:36:
/usr/src/modules/openafs/src/afs/LINUX/osi_machdep.h:55:2: error: #error Not sure what to do about rlim (should be in the Linux task struct somewhere....)

[Björn Torkelsson (torkel)]

It is fixed in 1.4.2. Unfortunately it missed the release of 6.10.

Don't know if it is possible to get 1.4.2 in -updates?

See https://wiki.ubuntu.com/StableReleaseUpdates?highlight=%28Updates%29 for more information.

I have been running 1.4.2 (client) on Edgy since it was released and haven't noticed any regression yet...

[Ted Anderson (ota-surfvi)]

I also had this problem when trying to install openafs-modules-source into a Ubuntu 6.10. Following the advice here I installed openafs-modules-source_1.4.2-3_all.deb from Ubuntu Feisty. This compiled and installed fine and AFS now works. However, I do not yet have a lot of experience with it.

This seems like an important bug to fix, since without it OpenAFS is broken in Ubuntu Edgy. The above mentioned link describes the qualifications for fixing a stable release, one of which is "Bugs which represent severe regressions from the previous release of Ubuntu". Since OpenAFS is now broken, this can be considered a serious regression from earlier releases where OpenAFS did work (right? I haven't used earlier version of Ubuntu).

Since this isn't a bug fix, per se, there isn't any backporting needed. I don't know what is needed to retarget the openafs-modules-source package at a buildable tarball. The Debian bug listed above (#358203) describes a problem with 2.6.16 and fixes it with 1.4.1-1, but Ubuntu Edgy includes[1] openafs 1.4.1-4 which does not work with kernel 2.6.17. Is a more current description of the bug needed to raise the severity/importance of this defect?

[1] http://packages.ubuntu.com/cgi-bin/search_packages.pl?searchon=names&version=all&exact=1&keywords=openafs-modules-source

[Russ Allbery (rra-debian)]

(I'm the Debian OpenAFS maintainer and don't use Ubuntu, but I'm happy to help out if I can.)

I'm curious exactly what is failing. To know what's failing, I'd need to see the config.log output around the test for rlim. Something is causing compilation against the kernel headers to fail, and I don't know off-hand of anything prior to 2.6.18 that causes those failures. There are changes in 2.6.18 that can cause this, which were fixed in the OpenAFS 1.4.2 release.

Note that there are other types of brokenness besides compilation failures that can affect AMD64 in particular, but I think they were all introduced in 2.6.18.

The compilation failure that you're seeing is very generic and only means that the Linux header files didn't compile properly with the expectations that OpenAFS uses. I strongly suspect that you're not actually seeing Bug#358203, which as you mention was already fixed in the version that's in Ubuntu, but instead are seeing some completely different problem that just has the same very generic symptoms.

[alphamerik (alphamerik)]

Thanks for looking into this Russ, I don't understand why it is so hard to get this fixed. Is Sam distracted by other things? Are there political reasons? What is wrong with moving to 1.4.2, why track down bugs in 1.4.1? So many questions... so few answers.

[Russ Allbery (rra-debian)]

Well, Sam doesn't have anything to do with fixing this bug. Sam, like myself, is one of the maintainers of the Debian package. I don't use Ubuntu, have no upload access to Ubuntu, and know nothing about its release cycle. So far as I know, the same is true of Sam. Insofar as Launchpad implies that Sam has anything to do with what ends up in Ubuntu, it's not being particularly accurate.

I could only speculate on why the fix wasn't backported to Edgy, and since I'm not involved in Ubuntu (only in OpenAFS), I don't want to do that.

Looking at the config.log that you attached (thank you!), here's the actual problem:

/lib/modules/2.6.17-10-generic/build/include/asm/processor.h:76: error: 'CONFIG_X86_L1_CACHE_SHIFT' undeclared here (not in a function)
/lib/modules/2.6.17-10-generic/build/include/asm/processor.h:76: error: requested alignment is not a constant

So basically, the kernel headers as included by the OpenAFS configure script don't compile. This is very typical of the error message that you're seeing. Notice in the configure log that every test related to the Linux kernel returns "no", and then the resulting source won't build.

Almost every time we see an error like this, what it means is that the Linux kernel maintainers have changed the headers and now certain other headers have to be included before the ones that OpenAFS is including. The kernel maintainers do this all the time and without warning and believe that it's the responsibility of modules to try to keep up, and OpenAFS then tries to adapt in the next version. I expect that in this case this is one of the many Linux kernel build issues that was fixed in 1.4.2, which is why the current Debian packages work.

It's probably possible to come up with a backport fix for just this problem by inspecting the configure script in 1.4.2 and seeing what additional headers it includes when doing kernel probes, although that may be made more complicated by the fact that I think 1.4.2 does kernel probes in general in a better (but different) way. Coming up with that sort of backport is fairly tedious, though.

[Helge Stenström (h-stenstrom)]

According to the Synaptic package handling program on Ubuntu Edgy 6.10, the package maintainer for openafs-client, openafs-doch and openafs-module-source is Sam Hartman <email address hidden>. The packages are from "universe", whatever that means. Is it a mistake from Ubuntu's side to name Sam as the package maintainer. The latest openafs package for Ubuntu 6.10 is still 1..4.1-4.

[Andrew Bassett (awbassett)]

I'm guessing Sam's name remains in there from the sync to the debian tree since there isn't an Ubuntu maintainer.

[Nick Fishman (bsdlogical)]

Like Helge said, the latest package is still 1.4.1-4. If Sam is not the maintainer for the Ubuntu package (even though the Ubuntu repository lists him as such), we should find a new maintainer and get this problem solved.

I built all the OpenAFS packages from the Debian repository, as suggested by JDahl on http://ubuntuforums.org/showthread.php?p=1701049#post1701049. Using that method, everything works on Ubuntu 6.10 2.6.17-10-generic. Here are details from his post - hopefully they'll help until the right people (whoever they may be) can get their act together:

1) Make a directory "openafs" and download the packages to it from:
$ mkdir openafs; cd openafs
$ wget http://ftp.debian.org/debian/pool/ma....2.orig.tar.gz
$ wget http://ftp.debian.org/debian/pool/ma....4.2-2.diff.gz

2) Unpack and patch
$ tar xzvf openafs_1.4.2.orig.tar.gz
$ zcat openafs_1.4.2-2.diff.gz | patch -p0

3) Install necessary dependencies
$ sudo apt-get build-dep openafs-client
$ sudo apt-get install build-essential fakeroot

4) Build the Debian packages (takes a long time)
$ cd openafs-1.4.2
$ chmod u+x debian/rules
$ dpkg-buildpackage -rfakeroot
Compilation may fail if you don't have all dependencies installed. Just install the missing packages and rebuild.

5) Install the packages
$ sudo dpkg -i ../openafs-modules-source_1.4.2-2_all.deb openafs-client_1.4.2-2_i386.deb

6) Build the kernel module
$ sudo apt-get install module-assistant
$ sudo module-assistant prepare openafs-modules
$ sudo module-assistant auto-build openafs-modules

7) Install the packages (and say [Y]es to overwriting /etc config scripts; there's a bug in the old ones)
$ sudo dpkg -i /usr/src/openafs-modules-2.6.17-10-386_1.4.2-2+2.6.17-10.33_i386.deb

Step 6 is slightly modified from his original post, as I believe he mistakenly included it. An excerpt from my package list:

root@krypton:~# dpkg -l | grep afs
ii openafs-client 1.4.2-4 AFS distributed filesystem client support
ii openafs-modules-2.6.17-10-generic 1.4.2-4+2.6.17.1-10.34 AFS distributed filesystem kernel module
ii openafs-modules-source 1.4.2-4 AFS distributed filesystem kernel module sou

I hope this issue can be resolved soon.

[Johan Christiansen (johandc)]

Bump.

Are there happening anything with regard to the 1.4.2 SRU?
OpenAFS is completely broken in edgy, and nobody is doing anything about it.
I understand we are missing an OpenAFS maintainer on Ubuntu, correct me if i'm wrong, but aren't there anything we can do?

[Achim Bohnet (allee)]

Hi Johan,

well I've rebuild 1.4.2-3 from feisty on edgy and use it without problems (-4* did FTBFS).
I tried right now an edgy rebuild of 1.4.4-1 from feisty and it works too (only quick tests
with an afs client).

IMHO we should skip the patched 1.4.2 and go to plain '1.4.4-1' ;) So what 'we' can do is to
try 1.4.4 and report success/failure here.

I use a little bit different method that uses dch, so the selfbuilt deb version is smaller than a
backported one (~ach0edgy1 < ~edgy1 ). In my sources.list in edgy I have:

deb-src http://apt-proxy:9999/ubuntu/ feisty main restricted universe multiverse

to make building backports easier.

   mkdir temp
   cd temp
   apt-get source openafs # <-needs the feisty deb-src like entry above
   cd openafs-1.4.4/
   dch -v 1.4.4-1~ach0edgy1 -D edgy "rebuild on edgy"
   debuild -us -uc
   sudo dpkg -i ../openafs-modules-source_1.4.4-1~ach0edgy1_all.deb

  # module assistant can be use as normal user too, but for whatever reason still do it on root (*flush*)
   (cd /usr/src/ && sudo module-assistant update)
   (cd /usr/src/ && sudo module-assistant build openafs)

    sudo /etc/init.d/openafs-client stop
    sudo dpkg -i /usr/src/openafs-modules-2.6.17-11-generic_1.4.4-1~ach0edgy1+2.6.17.1-11.35_i386.deb
    dpkg -l openafs-* | grep ^i
    ls ../*1.4.4-1~ach0edgy1*.deb
   # for a client, e.g.
    sudo dpkg -i ../openafs-client_1.4.4-1~ach0edgy1_i386.deb ../openafs-kpasswd_1.4.4-1~ach0edgy1_i386.deb
    sudo /etc/init.d/openafs-client start

I can provide the debs and srcs in an apt-repo if there an need.

[Björn Torkelsson (torkel)]

Unless somebody follows the instructions in https://wiki.ubuntu.com/MOTU/SRU I don't think there will be an update of OpenAFS.

In my opinion an SRU of OpenAFS should happen as the current version in Edgy does not build and also has some security issues (see #94787).

I have very limited time at the moment and I don't have access to any edgy machine, so I hope someone else can step up and do the needed work to get a SRU of OpenAFS.

At work we have rebuilt 1.4.4-1 for Dapper and deployed it on all our clusters and afs servers and the still works :-)

[Johan Christiansen (johandc)]

After a somewhat lenghty discussion on #ubuntu-motu, it has been established, that it's probably not a good idea to try a SRU, since the shift from 1.4.1 to a newer version will probably be rejected.

Instead, it was decided to try and get 1.4.4 into backports. I'm currently looking into this, but am not sure entirely what to do. I guess allee has already proven that the 1.4.4 modules source from feisty can compile with the 2.6.17 kernel included in edgy. As far as i understand, this is enough to post a backport request.

I will therefore try to do the same as allee, and build the feisty modules on edgy and test them. If i succeed i will post a backport request.

[Nick Fishman (bsdlogical)]

Johan, allee, and Björn,

I followed the steps that allee posted, and I successfully built the feisty modules on edgy. I didn't run into any problems, except that I had to install the "devscripts" package.

Everything works over here. I installed libpam-openafs-session and openafs-krb5, so the feisty modules work great with kerberized OpenAFS.

Here's some output:
bsdlogical@krypton:~$ dpkg -l | grep afs | grep ^i
ii libpam-openafs-session 1.0-5 PAM Module to get AFS tokens and set up PAG
ii openafs-client 1.4.4-1~ach0edgy1 AFS distributed filesystem client support
ii openafs-krb5 1.4.4-1~ach0edgy1 AFS distributed filesystem Kerberos 5 integr
ii openafs-modules-2.6.17-11-generic 1.4.4-1~ach0edgy1+2.6.17.1-11.35 AFS distributed filesystem kernel module
ii openafs-modules-source 1.4.4-1~ach0edgy1 AFS distributed filesystem kernel module sou
bsdlogical@krypton:~$ uname -a
Linux krypton.bsdlogical.com 2.6.17-11-generic #2 SMP Thu Feb 1 19:52:28 UTC 2007 i686 GNU/Linux

Good luck with the backport request!

[Johan Christiansen (johandc)]

Thank you Nick.

I also successfully built the feisty modules and tested the Client. I browsed an image library constantly for about 1 hour, and everything seemed to work. I haven't tested the server components on edgy, but I am running 1.4.4-1 on a feisty server and they work great there as well...

I did however run into a problem with unloading the openafs kernel module. This is a known bug (debian #417917) - and not relevant to the backporting issue since it also apply for 1.4.2. The problem appeared only on my QEMU test machine, and not on my friends edgy-laptop where i tried the same.

All in all, 1.4.4-1 seems to fix _A LOT_ of issues with openafs posted over the last couple of months, and the initial tests done by Nick, Allee and Björn suggests that 1.4.4-1 is working in edgy. I only feel that somebody should test the server packages before we submit the backport request? Do you agree?

[Johan Christiansen (johandc)]

It looks like I had already created a backport request over a month ago, i have updated it, with reference to this discussion. It does not look like it has been given much attention though, so how do we push the edgy-backports team to do something about it?

The request is:
https://bugs.launchpad.net/edgy-backports/+bug/90711

[Johan Christiansen (johandc)]

Changing to confirmed, since 1.4.1 is reported broken in edgy.

[Nick Fishman (bsdlogical)]

Johan, I will test the server packages (openafs-fileserver and openafs-dbserver) on Edgy and report what I find.

Otherwise, since we have (almost) all the information we need, we just need to reiterate our request individually - whether it be on IRC or by email. There's a large enough OpenAFS community that's affected by this. I'm sure the request will be granted.

[Johan Christiansen (johandc)]

Nick, that sounds great! But what do you mean by reiterate our requests individually, and what information are you suggesting that we are missing at the present time?

Another question, about the maintainer. Sam Hartman is listed here, but he hasn't been really active on the bug-reports. I think we need somebody who can change priority of the bugs. Anyone who knows Sam that can answer to this? Russ has already given an answer, but how be it that Sam is listed as maintainer, when he's in fact not... If we can't find a maintainer, maybe we should assign the source package to the MOTU group or something insted, so we at least have a possibility to have some control...

[Björn Torkelsson (torkel)]

Sam is listed as maintainer as he is one of the the Debian maintainers. He (and) Ross have nothing to do with Ubuntu, except that they are doing a really great job in Debian and are very responsive to bugs in Ubuntu too, which means that we don't have to divert the package from the Debian version.

If you look at the latest version, MOTU is listed as maintainer and Sam is listed as Original-Maintainer. I guess that LP hasn't catched up that yet and that you can just ignore the fact that Sam is listed as Maintainer. MOTU already have full control over OpenAFS and have always had.

[Nick Fishman (bsdlogical)]

Johan, sorry if I wasn't clear. I meant that we still had not tested the server packages to make sure they work. I ran some tests a while ago with the backported openafs-fileserver and openafs-dbserver packages - they work great on Edgy with the backported modules. Here's some information on the server I used:

bsdlogical@knuth:~$ dpkg -l | grep afs | grep ^i
ii libpam-openafs-session 1.0-7 PAM Module to get AFS tokens and set up PAG
ii openafs-client 1.4.4-1~ach0edgy1 AFS distributed filesystem client support
ii openafs-dbserver 1.4.4-1~ach0edgy1 AFS distributed filesystem database server
ii openafs-doc 1.4.4-1~ach0edgy1 AFS distributed filesystem documentation
ii openafs-fileserver 1.4.4-1~ach0edgy1 AFS distributed filesystem file server
ii openafs-krb5 1.4.4-1~ach0edgy1 AFS distributed filesystem Kerberos 5 integr
ii openafs-modules-2.6.17-11-generic 1.4.4-1~ach0edgy1+2.6.17.1-11.35 AFS distributed filesystem kernel module
bsdlogical@knuth:~$ uname -a
Linux knuth.dupontmanual.org 2.6.17-11-generic #2 SMP Tue Mar 13 23:32:38 UTC 2007 i686 GNU/Linux

By reiterating our requests individually, I meant that each one of us should contact the maintainer on our own - that way, we will get noticed.

Like Björn mentioned, we probably should send emails to MOTU to ensure that he/she is aware of our findings.

[Johan Christiansen (johandc)]

Nick, The problem is that there are no maintainer of the openafs source package in ubuntu! So we have to apply somebody from MOTU to do the update. Please submit your testing results to the backport request here:
https://bugs.launchpad.net/edgy-backports/+bug/90711

John Dong has already tested the server modules, and with three of us having tested the client modules, i hope we can get this backport approved.

[William Grant (wgrant)]

Fixed in Gutsy.

[Anders Kaseorg (andersk)]

It was already fixed in Feisty, too…this bug is about Edgy.

[William Grant (wgrant)]

Even so, the Ubuntu task doesn't need to remain open.

[Johan Christiansen (johandc)]

Changing to Fix Released since 1.4.4-1 has been backported and no SRU will ever go into edgy anyways.