Ubuntu 8.04.3 amd64 on Dell PE R610 : Front USB ports do not detect newly inserted USB device

[DenverSerrao]

I'm running Ubuntu 8.04.3 amd64 on the Dell PowerEdge R610 and seeing an issue where the OS does not recognize a USB device that is plugged into the front USB ports unless it was plugged in during boot. After boot if I insert the device into the back USB port, the OS detects it immediately. However, if I plug it into the front USB port, it does not detect it.

If I run a lsusb command, then at that point the OS does detect the USB device though. I'm attaching dmesg outputs with a USB drive attached during boot and with no USB drive attached during boot. I see the same behavior even with a USB CD-ROM as well.

[DenverSerrao (denver-serrao)]

Output of dmesg when USB drive is attached during boot

Here are the highlights
[127] Server recognizes USB drive connected to front USB port during boot (sda)
[224] Removed USB drive from front usb port
Re-Inserted USB drive into front USB port, does not get automatically detected.
[278 – 283] Run lsusb command, and then the USB drive is detected
[457] Removed USB drive from front usb port
[463-468] Re-Inserted USB drive into back usb port. USB drive now is automatically detected.
[585] Removed USB drive from back usb port
Re-Inserted USB drive into front USB port, does not get automatically detected.
[633-639] Run lsusb command, and then the USB drive is detected

[DenverSerrao (denver-serrao)]

Output of dmesg when a USB drive is not attached during boot.

Server booted up. Insert USB drive into front USB port. Does not get detected
[748-754] Run lsusb command, and USB drive is detected
[773] Removed USB drive from front USB port
[782-787] Re-Inserted USB drive into back usb port. USB drive now is automatically detected.
[808] Removed USB drive from back usb port
Re-Inserted USB drive into front USB port, does not get automatically detected.
[830-836] Run lsusb command, and then the USB drive is detected
[848] Remove USB drive
[884-895] Insert USB DVD-ROM into back USB port. Automatically detected
[904] Removed USB DVD-ROM
Re-Inserted USB DVD-ROM into front USB port, does not get automatically detected.
[939-945] Run lsusb command, and USB DVD-ROM is detected

[DenverSerrao (denver-serrao)]

lspci with usb-dvd and usb-drive attached and detected

[DenverSerrao (denver-serrao)]

dmidecode

[DenverSerrao (denver-serrao)]

lspci -vvv

[DenverSerrao (denver-serrao)]

lsusb

[stuart hayes (stuart-hayes)]

This appears to be similar to issue #263139.

[DenverSerrao (denver-serrao)]

The front USB ports are internally connected through a USB hub while the back ports are directly connected to the USB host controllers. So I tried connecting a USB drive through a USB hub connected to the back ports, and then I saw the same behavior as the front ports where the OS does not detect a newly connected USB drive.

If we remove the ehci_hcd module, then the OS does detect the USB drive after removing and plugging it back in. We are limited to the lower speed uhci_hcd module then though.

[DenverSerrao (denver-serrao)]

Booting to the 8.10 LiveCD, I see that that kernel (2.6.27-7-generic) does allow the OS to recognize a newly inserted USB device. So looks like the issue was fixed in the ehci_hcd module in that kernel.

[DenverSerrao (denver-serrao)]

Looks similar to this issue
https://bugs.launchpad.net/ubuntu/+source/linux-source-2.6.20/+bug/88746

Is there already a fix for 8.04 for the ehci_hcd module?

[Jerone Young (jerone)]

I was able to reproduce this issue on my laptop. Very easy to reproduce.

1) Take a USB port
2) Connect USB hub to it
3) Try any USB device connected to the USB hub and notice that it never see it seen by the kernel.

So this isn't a machine specific issue. It's a problem for any USB hub.

Though it does appear to relate to LP#88746 as pointed out above. If you unload then reload the ehci_hcd module then it works.

[Jerone Young (jerone)]

So looks like my findings where actually wrong. I just needed to wait a few seconds. Once I did things worked fine. Will have to try it on the actual Dell machine the problem is being reported on.

[DenverSerrao (denver-serrao)]

So this problem is seen on the Dell PE R610 and R710. It is NOT seen on the 2950-III

[DenverSerrao (denver-serrao)]

Usb hub info from the R610

[Jerone Young (jerone)]

This problem can be seen on any machine actually. I was seeing it earlier. The key is to NOT run "lsusb". This kicks off a fresh scan of the USB bus and then it's seen.

So to reproduce:

1) Connect USB hub to usb port
2) Connect any device to it .. in this case try a USB stick .. though it's for any
* DO NOT RUN "lsusb"
3) Check dmesg and notice the device is not seen
4) Run "lsusb" and notice now your device is seen.

Appears a problem only with this kernel. As reported it works fine with 8.10. Will try and narrow down the change that may fix this issue.

[Jerone Young (jerone)]

Narrowing the fix down. This problem does not appear on upstream kernels 2.24.4 and up. Have to try lower point releases. But I was able to get a binary for 2.24.4 & up here:
http://kernel.ubuntu.com/~kernel-ppa/mainline/

Now to try lower 2.24 point releases.

[Jerone Young (jerone)]

Trying upstream 2.24.0 , 2.24.1, & 2.24.2 with the server kernel config all work without issue. I also tried 2.6.25 before and that worked without issue also.

So their appears to be a patch in our kernel that has caused a regression.

[Jerone Young (jerone)]

This is a regression in the hardy server kernel starting with version 2.6.24-21.42 in the Ubuntu archive. Here are results of me trying various kernels from the archive:

kernel Result
--------- ------------
2.6.24-16.30 works!
2.6.24-20.38 works!
2.6.24-21.42 Broken!
2.6.24-21.43 Broken!
2.6.24-23.52 Broken!

[Jerone Young (jerone)]

This patch is potentially what is causing the issue:
http://kernel.ubuntu.com/git?p=ubuntu/ubuntu-hardy.git;a=commit;h=6ca9208c22280928597bcc738bf4248de0ab359d

[Jerone Young (jerone)]

Trying the patch above with vanilla 2.6.24.0 source you see the problem. The 2.6.24 source worked fine without this patch (see past comments).

So the patch pointed above is the source of the problem here.

[Andy Whitcroft (apw)]

As there is a worth around (running lsusb) and this affects a limited number of machines, marking this Medium.

[stuart hayes (stuart-hayes)]

I bisected it... this was fixed with commit d1f114d12bb4db3147e1b1342ae31083c5a79c84 (USB: EHCI: fix remote-wakeup regression), which makes sense.

http://kernel.ubuntu.com/git?p=ubuntu/ubuntu-hardy.git;a=commit;h=d1f114d12bb4db3147e1b1342ae31083c5a79c84

[Jerone Young (jerone)]

Confirmed that the patch pointed to by Stuart does fix the issue.

[Andy Whitcroft (apw)]

Ok I've pulled the patch into our kernel tree and build some official test kernels. If you could confirm that these packages also fix the issue and report back here. Thanks! Kernels are at the url below:

    http://people.canonical.com/~apw/lp406419-hardy/

[DenverSerrao (denver-serrao)]

Thanks Andy

I confirmed that installing the the linux-headers-2.6.24-24_2.6.24-24.58~lp406419apw1_all.deb does resolve the issue on the 2.6.25-25-server flavor that the customer is running.

Looks like all is good. Can we get an official pkg that we can release to the customer?

Thanks
Denver

[DenverSerrao (denver-serrao)]

Sorry, I had a few typo's in the above post.

I meant to say that the installing the linux-headers-2.6.24-24_2.6.24-24.58~lp406419apw1_all.deb package works on the 2.6.24-24-server flavor of the kernel that the customer is running.

Thanks
Denver

[Jerone Young (jerone)]

@Andy
         Any reason you would mark the progress as incomplete?

[Jerone Young (jerone)]

@Andy
         I've confirmed that the kernel you put together with the patch does fix the issue.

[DenverSerrao (denver-serrao)]

Thanks folks. So just to clarify, will you be getting an official package to release to the customer? Functionally, this does resolve the issue. The only thing I was not sure about were the references to "apw" in the deb names and dependencies.

[Jerone Young (jerone)]

@DenverSerrao
     It's best to wait till the next SRU release kernel for 8.04.3 is released with this fix in it. Andy do know a date on that ?

[DenverSerrao (denver-serrao)]

Hi Jerone

If the next release is in a few weeks, then that might be acceptable to the customer. If it's much later than that, we'd probably need to give them a beta package to tide them over till the next release.

Best
Denver

[Stefan Bader (smb)]

8.04.3 has been done already (July 2009) so the next point release which can include the fix would be .4 (Jan 2010) but the fix would be available in the update/proposed kernels before.

[DenverSerrao (denver-serrao)]

Discussed further with the customer, and they are fine with living with this issue, until it is fixed in the 8.04.4 release.

Thanks very much for your help on this folks.

[DenverSerrao (denver-serrao)]

Just to clarify, can we get a confirmation that this is tabled to be included in the 8.04.4 release?

Thanks
Denver

[Stefan Bader (smb)]

I am just now working to include this change into the next Hardy proposed
upload. So, if things go all well, this might be available in the next update
of the kernel and then automatically in 8.04.4

[Steve Langasek (vorlon)]

Andy, is this still an issue as karmic?

[Steve Langasek (vorlon)]

Accepted into hardy-proposed, the package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

[Jerone Young (jerone)]

@Steve
         This issue was introduced into on of the 8.04 SRU kernels. The patch here is actually upstream fix to the patch that was included in SRU that caused the issue. It is not an issue in any other version past 8.04 since both patch was upstream.

[Steve Langasek (vorlon)]

ok, marking as resolved for karmic.

[Jerone Young (jerone)]

@Andy
         When do we expect to have an SRU kernel in proposed with this fix?

[Stefan Bader (smb)]

Since last week: https://launchpad.net/ubuntu/hardy/+source/linux/2.6.24-24.60

[Stefan Bader (smb)]

Can someone who was previously having this problem, try the proposed kernel and report here whether this solves the issues. Thanks.

[DenverSerrao (denver-serrao)]

I pointed apt to archive.ubuntu.com/ubuntu/ hardy-proposed restricted main multiverse universe

After the apt-get update and upgrade, and a reboot, looks like the new kernel works. It does recognize the newly inserted USB device. Great work all :)

Thanks
Denver

[Launchpad Janitor (janitor)]

This bug was fixed in the package linux - 2.6.24-24.60

---------------
linux (2.6.24-24.60) hardy-proposed; urgency=low

  [Stefan Bader]

  * SAUCE: init: Add extra mark_rodata_ro quirk for Acer Aspire One
    - LP: #322867

  [Upstream Kernel Changes]

  * USB: EHCI: fix remote-wakeup regression
    - LP: #406419

 -- Stefan Bader <email address hidden> Tue, 18 Aug 2009 18:25:47 +0200