libsane upgrade does not install /etc/hotplug/usb/libusbscanner

[Emmanuel Touzery]

my system is dist-upgraded from hoary. in hoary scanning worked perfectly. I
have a USB scanner mustek bearpaw supported directly by xsane (i must just copy
the firmware).
running 'xsane' says that there are no devices available.
running 'sudo xsane' successfully detects my scanner and lets me scan (but first
warns me that running xsane as root is unsafe and unsupported).

so i think there's a permission problem somewhere. I'm not too sure where but i
assume /dev?
in system->administration->users that user has ticked the right "use scanner
devices" (I didn't touch those settings).

I join two straces: one of running xsane as normal user (which doesn't detect
the scanner) and one of running xsane as root (which works).

[Emmanuel Touzery (emmanuel-touzery)]

Created an attachment (id=4050)
gzip of strace of xsane when ran as normal user, doesn't detect the scanner

[Emmanuel Touzery (emmanuel-touzery)]

Created an attachment (id=4051)
gzip of strace of xsane when ran as root, detects the scanner

both attachments are gzips, NOT .tar.gz (decompress with gunzip, not tar)

[Emmanuel Touzery (emmanuel-touzery)]

this is breezy as of today (24-09-2005) btw.

[Emmanuel Touzery (emmanuel-touzery)]

Oh, I maybe found the reason. It seems it's access /proc not /dev after all.
Then I don't know which package is responsible. Maybe the kernel...

from the strace as normal user:
open("/proc/bus/usb/002/001", O_RDWR) = -1 EACCES (Permission denied)

from the strace as root:
open("/proc/bus/usb/001/003", O_RDWR) = 8

It's a difference, but I don't know if it's the reason of the failure.

[Crispin Flowerday (crispin-flowerday-deactivatedaccount)]

I have similar problems; when my system first starts up, the usb device (under
/proc/bus/usb/) is owned by root, and group root, however if I unplug my scanner
(it is a canon n670u) and plug it back in again, the relevant device is now
correctly owned by group scanner:

-rw-rw---- 1 root scanner 57 2005-09-25 14:07 /proc/bus/usb/002/004

In short, on startup hotplug (or something in the stack) doesn't chgrp the
device to the correct owner.

[Emmanuel Touzery (emmanuel-touzery)]

I confirm.. I didn't realise that, but it's true that it's enough to
unconnect/reconnect the scanner after boot and the scanner works without having
to be root. if the scanner is connected at boot, it won't work, as Crispin
Flowerday described.

[Martin Pitt (pitti)]

I can confirm that with my camera. After boot, /proc/bus/usb/003/* are owned by
root:root. After replugging it or after "sudo /etc/init.d/hotplug restart" the
device permissions are correct.

The permissions are changed by /etc/hotplug.d/usb/*.hotplug. However, at the
time hotplug runs, /proc and /sys should be already mounted. Scott, do you have
an idea about why that fails at boot?

[Matt Zimmerman (mdz)]

*** Bug 22567 has been marked as a duplicate of this bug. ***

[indypende (indypende-deactivatedaccount)]

I confirm! unplug/replug of the scanner or sudo chmod a+rw /proc/bus/usb/***/***
resolve temporary the problem. But when i reboot, everything return as the
beginning.
I've similar problem with my usb webcam! maybe we can find a solution for
everithing. We must solve before Breezy's official release...

[Richard Kleeman (kleeman)]

I believe that #12496 is a duplicate of this (grumpy initial reporter ;-))

[Matt Zimmerman (mdz)]

*** Bug 18760 has been marked as a duplicate of this bug. ***

[Matt Zimmerman (mdz)]

(In reply to comment #10)
> I believe that #12496 is a duplicate of this (grumpy initial reporter ;-))

Thanks. This is a good example of the difference that a good bug report makes.
 In bug #18760, you described what you thought was the cause of the problem
(which was incorrect on its face: these files aren't missing, but merely in a
different location), rather than describing the problem itself. This and other
useful tips can be found in more detail this article:
http://www.chiark.greenend.org.uk/~sgtatham/bugs.html

[Richard Kleeman (kleeman)]

Just to set the record straight on this. I actually inserted these files into
their hoary positions and the permissions issue was solved.

[Scott James Remnant (Canonical) (canonical-scott)]

What modules are actually associated with these scanners?

If that module is loaded in the initramfs, then the /etc/hotplug.d directory
won't be run because it's not in the initramfs!

Note that there are fairly large bugs in /etc/hotplug.d/usb right now -- for
example that libgphoto2.hotplug script that chowns every USB device to
root:plugdev ... :)

[Richard Kleeman (kleeman)]

As I understand it from reading the sane docs, in 2.6 kernels libusb is the
module doing the work. Here is a discussion from their webpage:

Information about USB scanners:
================================

With Linux 2.4.* you could either use the kernel scanner module or libusb to
access USB scanners. In Linux 2.6.4 the kernel scanner module was removed.
Therefore with this and later kernels libusb must be used.

While SANE automatically uses libusb when the library and its header file were
present during the build of sane-backends, setting permissions will require some
attention. So if scanimage -L lists your scanner as root but not as normal user
read on this text.

The device files used by libusb are located in /proc/bus/usb/
(e.g. /proc/bus/usb/001/003). The exact file name can be found out by running
sane-find-scanner which would print "libusb:001:003" in this case. While
setting permissions with e.g. "chmod a+rw /proc/bus/usb/001/003" works,
this change is not permanent. The permissions will be reset when the scanner is
replugged or Linux is rebooted.

One solution to set permissions on-the-fly are the Linux hot-plug tools that
should come with any current distribution. Your distribution should have set up
the scripts to automatically change permissions correctly. Look for
"libsane.usermap" and "libusbscanner" in /etc/hotplug/usb. Usually you must just
add the users that are allowed to access the scanner to group "scanner". To make
that change active, the user must login again. For more details, see your
distribution's documentation e.g. for Debian: README.debian.gz.

If you build SANE from source you can use the hotplug script that comes with
SANE. See the tools/hotplug/ directory in the source distribution. Please refer
to the README in that directory for the details.

Gentoo users: If your USB scanner is not detected at all check that USE=usb is
set when emerging.

[Richard Kleeman (kleeman)]

Correction: libusb is a userspace library not a kernel module.

[Scott James Remnant (Canonical) (canonical-scott)]

Sadly those documents are out of date,

/etc/hotplug (and even /etc/hotplug.d) are heavily deprecated.

These scripts should instead be written as a udev rules file.

There is backwards-compatibility-shimmy in there at the moment, but it does look
like it's not exactly working properly.

[Richard Kleeman (kleeman)]

Oh like this you mean?:
http://gentoo-wiki.com/HOWTO_Installing_USB_Scanner#UDEV

[Martin Pitt (pitti)]

(In reply to comment #14)
> What modules are actually associated with these scanners?
>
> If that module is loaded in the initramfs, then the /etc/hotplug.d directory
> won't be run because it's not in the initramfs!

Please note that I have the same problem with my camera; gphoto directly uses
libusb for communicating with it, no special module. I originally changed the
bug title to be more generic, but it was changed back now.

> Note that there are fairly large bugs in /etc/hotplug.d/usb right now -- for
> example that libgphoto2.hotplug script that chowns every USB device to
> root:plugdev ... :)

Ouch - Back in the time when this script was in /etc/hotplug/usb/libgphoto, it
was only called if there was a matching vendor/product ID in
/etc/hotplug/usb/libgphoto.usermap. Recently the script had to be moved to
/etc/hotplug.d/usb since the old location did not work any more. So are usermaps
completely broken now? How is that supposed to work now?

[Martin Pitt (pitti)]

(In reply to comment #17)
> Sadly those documents are out of date,
>
> /etc/hotplug (and even /etc/hotplug.d) are heavily deprecated.
>
> These scripts should instead be written as a udev rules file.

BTW, I don't think that udev rules file can cover permissions in /proc/bus/usb,
or can they? That's why libgphoto and libsane use hotplug scripts in the first
place.

[Emmanuel Touzery (emmanuel-touzery)]

(In reply to comment #20)
> BTW, I don't think that udev rules file can cover permissions in /proc/bus/usb,
> or can they? That's why libgphoto and libsane use hotplug scripts in the first
> place.

I don't know anything about all of this, but <email address hidden> seems to have
posted an URL describing exactly that in this thread.
http://gentoo-wiki.com/HOWTO_Installing_USB_Scanner#UDEV

it seems udev can set permissions, check the link.

[Martin Pitt (pitti)]

(In reply to comment #21)
> (In reply to comment #20)
> > BTW, I don't think that udev rules file can cover permissions in /proc/bus/usb,
> > or can they? That's why libgphoto and libsane use hotplug scripts in the first
> > place.
>
> I don't know anything about all of this, but <email address hidden> seems to have
> posted an URL describing exactly that in this thread.
> http://gentoo-wiki.com/HOWTO_Installing_USB_Scanner#UDEV
>
> it seems udev can set permissions, check the link.

I doubt it; when I switch on my camera, neither udev nor /sysfs show *any*
reaction. Devices which do not have sysfs support can't be covered by udev,
which is the reason why we use hotplug scripts. If neither /etc/hotplug/ nor
/etc/hotplug.d/ are called at bootup for some reason, then we are screwed and we
cannot make cameras and scanners work properly.

[Martin Pitt (pitti)]

(In reply to comment #22)

> I doubt it; when I switch on my camera, neither udev nor /sysfs show *any*
> reaction. Devices which do not have sysfs support can't be covered by udev,
> which is the reason why we use hotplug scripts. If neither /etc/hotplug/ nor
> /etc/hotplug.d/ are called at bootup for some reason, then we are screwed and we
> cannot make cameras and scanners work properly.

Correction: sometimes there is a sysfs file, sometimes not. I don't know when it
appears, and we should not make big architectural changes for Breezy anyway.

I talked with Scott about this, and we will radically clean up this stuff in
Dapper. For Breezy I will revert from the hotplug-ng style /etc/hotplug.d/usb to
the old and proven /etc/hotplug/usb with usermap, which works better.

[Martin Pitt (pitti)]

 sane-backends (1.0.15-9ubuntu6) breezy; urgency=low
 .
   * Switch back to /etc/hotplug/usb/ script since the handling of
     /etc/hotplug.d/usb is broken:
     - Deactivate 02_tools_hotplug-ng.dpatch
     - debian/rules: Install files from tools/hotplug/ again.
     - debian/libsane.postinst: Delete the transitional removal of hotplug
       scripts, remove the hotplug-ng script instead.
     - Ubuntu #16232

Since I don't have a scanner here, I would appreciate if you can test this
thorougly. Having the scanner attached at boot should work fine now, and
hotplugging it should work, too.

Thanks!

[Scott James Remnant (Canonical) (canonical-scott)]

The long term solution (dapper) for this is:

http://www.kernel.org/git/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=fbf82fd2e1f4e679c60516d772d1862c941ca845

libusb also gets a patch to look under /dev/bus/usb/devices/NNN/NNN instead

Then USB_DEVICEFS goes away.

[Emmanuel Touzery (emmanuel-touzery)]

Well it didn't work for me. Actually it's worse than before...
I'll detail:

1. when upgrading it said there were changes in the file usermap (IIRC). I asked
for diff and the diff window was empty. I said "replace" my file. Right now
/etc/hotplug/usb/libsane.usermap is 49605 bytes on my computer.

2. scanner connected, after boot, permissions are still root-root

3. deconnecting and reconnecting the scanner without rebooting, permissions are
still root-root (regression)

3. running xsane as normal user _freezes the computer_ !!! Running as root
offers me the choice between the scanner or my webcam (spca5xx). before breezy
didn't offer me that choice it ignored the webcam. I don't know if that update
or another one that I did today caused that. running xsane as root offers me the
choice webcam or scanner. running as normal user after typing 'sudo rmmod
spca5xx' doesn't crash but doesn't detect the scanner

Maybe someone can tell me how to downgrade my libsane to the previous version to
check if the spca5xx mess is connected or if another of my upgrades caused this.
I don't know how to do that...

in any case it seems the upgrade didn't fix it.

[Crispin Flowerday (crispin-flowerday-deactivatedaccount)]

This fix worked for me: I upgraded sane-utils, libsane, libgphoto2 and
libphoto2-port0 and after a reboot the permissions root/scanner. I can't hotplug
at the moment, as I am not at home at the moment.

[Emmanuel Touzery (emmanuel-touzery)]

about my spca problem, from http://bugzilla.ubuntu.com/show_bug.cgi?id=15809
http://bugzilla.ubuntu.com/show_bug.cgi?id=14566 i guess this patch is not
responsible. if anything the patch caused xsane to detect the webcam which is a
plus. the freeze is due to the kernel apparently.
in hoary i compiled spca myself and xsane did show it. so it's ok to show it.

so i guess from my report you can just remember:
1. spca is shown and it's correct, it seems before it wasn't by bug of hotplug-ng
2. i still don't get the proper permissions but only root-root

i'm available if i can help for debugging info.

[Emmanuel Touzery (emmanuel-touzery)]

maybe one difference between me for whom it didn't work and Crispin Flowerday
for whom it worked could be that i upgraded from Hoary? I got this strange
message that my usermap file changed at upgrade... I don't know what kind of
installation does Crispin Flowerday has.

[Richard Kleeman (kleeman)]

Works perfectly now for me. I had to manually remove the .sane subdirectory in
my home directory as previous running of xsane as root had created a whole lot
of root owned config files. Once these were removed xsane works perfectly and
the scanner scans very nicely. Hotplugging also works as expected. Thanks guys.

[indypende (indypende-deactivatedaccount)]

i've also upgraded from hoary, downgraded the version but this continue to be
unresolved. I hope thet in a new breezy installation the problem disappear. Now
i cant acces the scanner with xsane. Not as root and as normal user.
sane-find-scanner show me that the device as been foundeed.
???

[Martin Pitt (pitti)]

After the update, which files are in /etc/hotplug/usb?

I will try an upgrade from Hoary to reproduce the upgrade problems. But an
upgrade from the previous version worked fine for me...

[Martin Pitt (pitti)]

(In reply to comment #26)
> Well it didn't work for me. Actually it's worse than before...
> I'll detail:
>
> 1. when upgrading it said there were changes in the file usermap (IIRC). I asked
> for diff and the diff window was empty. I said "replace" my file.

I just did a Hoary->Breezy upgrade and I did not get this question. Technically,
this file is a conffile in both Hoary and Breezy, so you should only get the
question if you manually modified the file.

> Right now
> /etc/hotplug/usb/libsane.usermap is 49605 bytes on my computer.

That's the current one, right.

> 2. scanner connected, after boot, permissions are still root-root
>
> 3. deconnecting and reconnecting the scanner without rebooting, permissions are
> still root-root (regression)

Scott, this seems to be one of the "hotplug scripts are not called" bugs we
talked about yesterday. Could this be a fallout from #15240?

> 3. running xsane as normal user _freezes the computer_ !!! Running as root
> offers me the choice between the scanner or my webcam (spca5xx). before breezy
> didn't offer me that choice it ignored the webcam. I don't know if that update
> or another one that I did today caused that. running xsane as root offers me the
> choice webcam or scanner. running as normal user after typing 'sudo rmmod
> spca5xx' doesn't crash but doesn't detect the scanner

This is a completely different bug (in the spca5xx kernel driver), see #15809.

> Maybe someone can tell me how to downgrade my libsane to the previous version to
> check if the spca5xx mess is connected or if another of my upgrades caused this.

libsane has got nothing to do with the spca5xx driver; however, you can
downgrade libsane by downloading

http://archive.ubuntu.com/ubuntu/pool/main/s/sane-backends/libsane_1.0.15-9ubuntu5_i386.deb

and installing it with "sudo dpkg -i libsane_1.0.15-9ubuntu5_i386.deb. However,
I hope that we can fix the hotplug script calling soon.

[Martin Pitt (pitti)]

(In reply to comment #28)
> about my spca problem, from http://bugzilla.ubuntu.com/show_bug.cgi?id=15809
> http://bugzilla.ubuntu.com/show_bug.cgi?id=14566 i guess this patch is not
> responsible. if anything the patch caused xsane to detect the webcam which is a
> plus. the freeze is due to the kernel apparently.
> in hoary i compiled spca myself and xsane did show it. so it's ok to show it.

Sorry, when typing comment #33, I did not yet see this comment. Does the spca5x
driver work now with the latest kernel? Otherwise you should reopen #14566.

[Emmanuel Touzery (emmanuel-touzery)]

here are the files in my /etc/hotplug/usb, as requested:

/etc/hotplug/usb:
total 176
-rwxr-xr-x 1 root root 493 2004-11-28 05:00 kino-jogshuttle
-rw-r--r-- 1 root root 457 2004-11-28 05:00 kino.usermap
-rwxr-xr-x 1 root root 220 2005-09-27 14:43 libgphoto2
-rw-r--r-- 1 root root 96304 2005-09-27 17:33 libgphoto2.usermap
-rw-r--r-- 1 root root 49605 2005-09-27 16:14 libsane.usermap
-rwxr-xr-x 1 root root 881 2005-09-16 03:13 logitechmouse
-rw-r--r-- 1 root root 3695 2005-09-16 03:13 logitechmouse.usermap

[Martin Pitt (pitti)]

(In reply to comment #35)
> here are the files in my /etc/hotplug/usb, as requested:
>
> /etc/hotplug/usb:
> total 176
> -rwxr-xr-x 1 root root 493 2004-11-28 05:00 kino-jogshuttle
> -rw-r--r-- 1 root root 457 2004-11-28 05:00 kino.usermap
> -rwxr-xr-x 1 root root 220 2005-09-27 14:43 libgphoto2
> -rw-r--r-- 1 root root 96304 2005-09-27 17:33 libgphoto2.usermap
> -rw-r--r-- 1 root root 49605 2005-09-27 16:14 libsane.usermap
> -rwxr-xr-x 1 root root 881 2005-09-16 03:13 logitechmouse
> -rw-r--r-- 1 root root 3695 2005-09-16 03:13 logitechmouse.usermap

Thanks, that indeed showed the problem: libusbscanner is missing. It is shipped
in the package, and it is installed both on a fresh install and on a
hoary->breezy upgrade. Can you please do

  sudo dpkg -i --force-confmiss
/var/cache/apt/archives/libsane_1.0.15-9ubuntu6_amd64.deb

and check that /etc/hotplug/usb/libusbscanner exists after that? Then your
scanner should work.

I'm not sure why that file is missing. Did you do some up/downgrading experiments?

[Emmanuel Touzery (emmanuel-touzery)]

(In reply to comment #36)
> Thanks, that indeed showed the problem: libusbscanner is missing. It is shipped
> in the package, and it is installed both on a fresh install and on a
> hoary->breezy upgrade. Can you please do
>
> sudo dpkg -i --force-confmiss
> /var/cache/apt/archives/libsane_1.0.15-9ubuntu6_amd64.deb
>
> and check that /etc/hotplug/usb/libusbscanner exists after that? Then your
> scanner should work.
>
> I'm not sure why that file is missing. Did you do some up/downgrading experiments?
>

Thank you! I'll try that after work. I didn't try downgrading, nor did anything
special with libsane. I did try 2-3 times to "reinstall" libsane in synaptic
after the package upgrade didn't help.
but my best bet: spca5xx froze my computer several times when testing this, and
maybe this caused file loss (even though i'm on ext3). strange though because i
tried reinstalling libsane even after the freezes. i'll report if this dpkg
command helps me. Thank you again for the support. (about spca I didn't upgrade
my kernel yet, I assume it'll fix it)

[Martin Pitt (pitti)]

(In reply to comment #37)

> but my best bet: spca5xx froze my computer several times when testing this, and
> maybe this caused file loss (even though i'm on ext3). strange though because i
> tried reinstalling libsane even after the freezes. i'll report if this dpkg
> command helps me.

This is a perculiarity of dpkg: once a conffile is removed, dpkg will never
reinstall it without being forced to (--force-confmiss) since deleting a
configuration file is a way of manual configuration which must be respected.
However, the file should not go away automatically in the first place, between
hoary and the latest breezy version it has always been a proper conffile, so if
at all, this is only an intra-breezy upgrade bug.

I leave this open until you tested the new package.

[Emmanuel Touzery (emmanuel-touzery)]

ok, i confirm that your dpkg suggestion fixed the problem (and dpkg actually
wrote that the file is missing and that it's adding it back. now all works, you
can close the bug, thank you very much!
(however spca still freezes my computer i'll reopen the bug)

[Scott James Remnant (Canonical) (canonical-scott)]

Martin, how did you move the conffile from /etc/hotplug to /etc/hotplug.d in the
first place? Could that be what causes this?

[Martin Pitt (pitti)]

(In reply to comment #40)
> Martin, how did you move the conffile from /etc/hotplug to /etc/hotplug.d in the
> first place? Could that be what causes this?

I didn't at all, since the scripts are completely different. ubuntu6 just ships
the new conffile in /etc/hotplug/usb and its postinst removes the old one in
/etc/hotplug.d/usb.

However, I have an idea: A previous postinst (which changed from /etc/hotplug/
to /etc/hotplug.d, as done by Debian some months ago) also just removed the
"old" conffile in /etc/hotplug/usb, probably at a time when the package was
still installed. This was probably regarded as manual removal. Is there a way to
re-register a conffile, to get the --force-confmiss effect? In any case this is
just an intra-Breezy effect, but since this is pretty common, it would be nice
to get a general solution.

Since the original bug is now fixed, I change the bug specs.

[Scott James Remnant (Canonical) (canonical-scott)]

(In reply to comment #41)
> (In reply to comment #40)
> > Martin, how did you move the conffile from /etc/hotplug to /etc/hotplug.d in the
> > first place? Could that be what causes this?
>
> However, I have an idea: A previous postinst (which changed from /etc/hotplug/
> to /etc/hotplug.d, as done by Debian some months ago) also just removed the
> "old" conffile in /etc/hotplug/usb, probably at a time when the package was
> still installed. This was probably regarded as manual removal.
>
That's exactly what I meant :p

> Is there a way to re-register a conffile, to get the --force-confmiss effect?
> In any case this is just an intra-Breezy effect, but since this is pretty common,
> it would be nice to get a general solution.
>
Sadly it is registered as removed; the only way would be to manually copy it
into place if the destination doesn't exist in the maintainer scripts. Perhaps
in preinst upgrade copy it from /etc/hotplug.d/usb to /etc/hotplug again.

[Martin Pitt (pitti)]

The conffile juggling is obsolete in Dapper anyway, since we are going to change
it to udev rules and remove all hotplug files. Setting to pending to get it out
of my way without closing it.

[Martin Pitt (pitti)]

current dapper version does not use hotplug any more, so this is obsolete.