USB devices are not shown in /procu/bus/usb/ causing VMWare to fail to see them

[Mark Shuttleworth]

I am running VMWare 5.5.0 on Linux flash 2.6.15-18-686 #1 SMP PREEMPT Thu Mar 9 15:29:22 UTC 2006 i686 GNU/Linux from linux-image-686 version 2.6.15.17.

Using Windows XP as the guest OS under Ubuntu, I'm unable to detect new USB devices being connected to the system.

According to the documentation, VMWare looks in /proc/bus/usb/ for devices. On my system, this directory exists but is always empty no matter which devices I plug into the system. We do have a /sys/bus/usb/devices/ which appears to be populated but I don't think that is in the same format as the old /proc/bus/usb/

Either VMWare needs to be updated to support the /sys/bus/usb/ interface, or we need to figure out how to populate /proc/bus/usb/

Ben, please figure out who to subscribe from VMWare it would be good to get independent verification of the issue. I will in the mean time download VMWare 5.5.1 and see if this fixes the problem.

Note: the original reporter indicated the bug was in package 'linux-image'; however, that package was not published in Ubuntu.

[Ben Collins (ben-collins)]

I'll check into this.

[Ben Collins (ben-collins)]

Ok, there really isn't a problem. You just need to run this command:

mount -t usbfs usbfs /proc/bus/usb

So this is a matter of some other thing needing to setup this filesystem. Maybe usbutils.

[Mark Shuttleworth (sabdfl)]

There is a problem, because VMWare and others don't do that mount
automatically, and even experienced users might miss the trick here. Can
we mount usbfs automatically? Or can we make it policy that application
packages which depend on it ensure that it is automatically mounted when
they themselves are installed, check it is available when they run, and
give a useful GUI warning when they fail?

[Ben Collins (ben-collins)]

On Thu, 2006-03-23 at 16:03 +0000, Mark Shuttleworth wrote:
> Public bug report changed:
> https://launchpad.net/malone/bugs/35004
>
> Comment:
>
> There is a problem, because VMWare and others don't do that mount
> automatically, and even experienced users might miss the trick here. Can
> we mount usbfs automatically? Or can we make it policy that application
> packages which depend on it ensure that it is automatically mounted when
> they themselves are installed, check it is available when they run, and
> give a useful GUI warning when they fail?

That's the reason I pushed it to usbutils. It's about the only package
that I think could be responsible for this.

IMO, VMWare's init script should ensure it is mounted. The reason it
isn't is likely because it is a legacy interface. VMWare should actually
be using sysfs.

--
Ubuntu - http://www.ubuntu.com/
Debian - http://www.debian.org/
Linux 1394 - http://www.linux1394.org/
SwissDisk - http://www.swissdisk.com/

[Mark Shuttleworth (sabdfl)]

Ben Collins wrote:
> IMO, VMWare's init script should ensure it is mounted. The reason it
> isn't is likely because it is a legacy interface. VMWare should actually
> be using sysfs.
>
But 5.5.1 is not, and unless the version that is current when Dapper is
released has fixed this, we need to figure out how to address it. Please
could you ping your contacts at VMWare?

[Jeffrey Baker (jwbaker)]

This problem still exists in VMWare Workstation 5.5.2. VMWare expects usbfs to be mounted somewhere, although the mountpoint is configurable. And really, why shouldn't they expect it to be mounted? I spent a good hour trying to figure this one out and when I found that usbfs wasn't mounted I was pretty surprised. It's like not having /proc.

[Ben Collins (ben-collins)]

initscripts already mounts some other psuedo filesystems, so I'll add this there.

[Scott James Remnant (Canonical) (canonical-scott)]

Ben, please don't do this just yet

Note that it's very deliberately commented out -- we'll reenable it for the release candidate.

I've reassigned this to me.

And could someone please fix vmware?

[Scott James Remnant (Canonical) (canonical-scott)]

Further details:

/proc/bus/usb is a pseudo-filesystem that contains special files that magically map to individual usb devices

It's been deprecated and unsupported for a while now, and is on gregkh's hit list of things to be removed from the kernel entirely.

It's been replaced by /dev/bus/usb, a udev-maintained directory in the same format (but using character devices instead of files).

Applications can use this without knowing the difference, with just a path change.

Note that there's no /dev/bus/usb/devices (equivalent to the old /proc/bus/usb/devices file). An application that wants to know that kind of detail would need to iterate /sys/bus/usb/devices or use HAL).

We disable /proc/bus/usb in development releases to prevent any software from using it, and generally encourage people away from it. We re-enable it in the release candidate, but only for the root user.

[Scott James Remnant (Canonical) (canonical-scott)]

We came up with an evil hack for the release, uploaded

[Mark Shuttleworth (sabdfl)]

Thanks guys, it's important vmware *just work*. Be nice to have a
non-evil fix in place for Edgy+1 :-)

Mark

[Scott James Remnant (Canonical) (canonical-scott)]

The non-evil fix is to get vmware to fix their broken software.

Note that the "fix" will be removed as soon as edgy+1 opens, as usual

[Mark Shuttleworth (sabdfl)]

Scott James Remnant wrote:
> The non-evil fix is to get vmware to fix their broken software.
>
Ever the master of tact :-). Can you write that up as something clear
but diplomatic that I can send to them for consideration? I.e. "in order
to achieve the same effect using current kernel tools we would like you
to consider foo bar baz..."

[Petr Vandrovec (petr-vmware)]

Scott, can you point in-kernel documentation to me which says that usbfs is deprecated? I cannot find any document saying that, and for VMware purposes /dev/bus/usb does not replace /proc/bus/usb - /proc/bus/usb/devices on which we rely on for providing plugin/removal notifications is not present in /dev/bus/usb, and all alternative approaches we've tried (hacky replacing system-wide hotplug script, trying inotify/dnotify on sysfs, registering for netlink hardware updates) either do not work at all, or do work only on small subset of kernels we (VMware) support.

So I would appreciate if you can recommend another interface which allows us to get notified when USB device is plugged in or removed - preferrably one which will work with older distros as well, and definitely one you are keeping maintained for period of time simillar to /proc/bus/usb (that is, 4 years at minimum).

Thanks, Petr Vandrovec

[Mark Shuttleworth (sabdfl)]

Well said!

[Jeffrey Baker (jwbaker)]

Petr: the "right way" these days would be to use HAL and DBUS to receive notification of device additions and removals. HAL underlies functionality in GNOME and is likely to remain for a long time.

On the other hand, I think it is the responsibility of the distribution to insulate the users from the whims of the upstream developers. Just because Mr. Kroah-Hartman decides that usbfs is "deprecated" doesn't mean Ubuntu should drop it without careful consideration.

[Scott James Remnant (Canonical) (canonical-scott)]

Just gathering the information for you now, Petr. There's no in-kernel documentation about this, but that's not really unusual because the kernel guys aren't exactly the best documenters in the world. The authorative stance on this is from Greg's book and just general e-mail.

Note that this is not a problem unique to Ubuntu, SuSE Linux 10.1 does not mount /proc/bus/usb either; and I do not believe the current Fedora development branch does. Debian are talking about dropping it in etch+1.

  http://www.helios.de/support/ti.phtml?100
  http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=360165

/dev/bus/usb has replaced /proc/bus/usb for the NNN/XXX devices, simply because the latter does not allow any kind of permission control other than at a filesystem level. Because /dev/bus/usb is generated by udev, we have that kind of control.

  http://wiki.linuxfromscratch.org/blfs/ticket/1680

For most software, this is simply a path change from /proc/bus/usb to /dev/bus/usb; the canon way seems to be to check /dev/bus/usb first, and fall back to /proc/bus/usb if the /dev/bus/usb directory does not exist. That way you're compatible with the udev-based distributions and the hotplug/monolothic-based distributions.

Some distributions (like Ubuntu releases) actually include both, however when they do /dev/bus/usb is preferable because the /proc/bus/usb devices can only be written to by the root user. The /dev/bus/usb devices can be written to if the user has permission based on the security settings and their group membership.

Obviously this isn't a complete solution, as /dev/bus/usb does not contain a "devices" file with summary information. If using /dev/bus/usb, you should use an alternate method of detecting USB devices.

To obtain a list of the usb devices, read the /sys/bus/usb/devices directory; which contains a symlink for each known device. The name of the symlink provides the bus id, the target directory contains various files wiht the device information (much easier to parse than the old file).

E.g. to obtain the descriptive name, vendor id and product id of every USB device (in shell):

  for DEVICE in /sys/bus/usb/devices/*; do
    [ -f $DEVICE/product ] && cat $DEVICE/product
    [ -f $DEVICE/idProduct ] && cat $DEVICE/idProduct
    [ -f $DEVICE/idVendor ] && cat $DEVICE/idVendor
  done

Insert and remove notification can also now be done *reliably*, without resorting to polling files or directories and comparing the difference.

Depending how dirty you'd like to get with the system, you can obtain these events using HAL, udev or a kernel netlink socket. HAL is the nice object-oriented, dbus-based way; but that'd introduce a dependency for you. The kernel netlink socket is a bit too low-level perhaps. Given that you know /dev/bus/usb exists, you know that they have udev, so that's probably the right way.

 - Have vmware read from a UNIX domain datagram socket in the abstract namespace, e.g. @/com/vmware/udev/event

 - Install a file named /etc/udev/rules.d/95-vmware.rules that contains the following:

  SUBSYSTEM=="usb_device", RUN+="socket:/com/vmware/udev/event"

Every time a device is added or removed, you will receive a strin...

[Petr Vandrovec (petr-vmware)]

Thanks Scott! VMware Workstation/Player are already crashing on recent Ubuntu builds due to hal-dbus interdependencies (we end up with both libdbus 0.6 and 0.9 loaded in one process, and this does not work quite well: http://www.vmware.com/community/thread.jspa?threadID=53507&tstart=0), so I think we'll try to avoid depending on hal more than necessary.

[Micah Dowty (micah-vmware)]

Thanks, Scott.

These are basically the same potential solutions I discovered when first analyzing this bug. I'd like to add my perspective to this discussion, since I'll probably be implementing VMware's support for /dev/bus/usb.

The three separate issues in /dev/bus/usb migration deserve separate consideration:

1. Opening and interacting with individual USB devices

This should be as simple as using /dev/bus/usb/<bus>/<device> instead of /proc/bus/usb. Permissions issues aren't a huge concern at the moment, since VMware uses its suid-root privileges to open devices- but that's a separate discussion. Besides compatibility with Ubuntu/SuSE/etc, this would be a step towards being a better citizen with regard to permissions on desktop machines.

2. Enumerating attached devices

We used /proc/bus/usb/devices for this previously. The new solution is sysfs, as you mentioned. Sysfs has all the information our products need, but it has some disadvantages. For any program which needs multiple pieces of information about multiple devices, it's complicated and racy. With /proc/bus/usb/devices, I could atomically read all information about all devices. With sysfs, devices could appear/disappear/change at any time. This problem is tractable, but adds complexity.

3. Notifications of connected/removed devices

With /proc/bus/usb/devices, one could just poll() to get notified when something changed. Without that file, there are several other options to consider.

We could have a helper program executed via udev, as you mentioned. That would work fine for our purposes, but I'd rather avoid the headache of installing and maintaining udev rules for all Linux distros VMware supports.

Then there's HAL. This is often touted as the "correct" solution- and I'm sure it is for your average gnome or kde app. Our USB code, however, needs to run in environments where I doubt we can rely on HAL to exist. VMware already does use HAL, but only from the GUI- not from the actual virtual machine process. As Petr mentioned, we've already had problems with HAL/DBus compatibility.

The solution I'm leaning toward right now is to listen directly from the netlink socket. It does require root privileges, but we can use our suid-root bit. The only real downsides I see are:

 - Theoretically, the format of the messages on the netlink socket may change.

 - The uevent netlink socket is new in Linux 2.6.10. There are distros in the wild (SLES9 SP3) with older kernels which don't mount /proc/bus/usb.

[Crispin Flowerday (crispin-flowerday-deactivatedaccount)]

It is worth pointing out, that this 'hack' doesn't actually help vmware - my copy of workstation doesn't pick up usb devices being inserted, I have to suspend / resume the image for them to be picked up.