Mike - checking if you have found a workaround or any temporary fix for this issue ? I have faced the same problem on an R410 with iDRAC running trusty kernel.

Following are the logs from my server:

[3903494.645100] usb 5-2: reset full-speed USB device number 2 using uhci_hcd
[3903499.760216] usb 5-2: device descriptor read/64, error -110
[3903499.984358] usb 5-2: device descriptor read/64, error -71
[3903500.200497] usb 5-2: reset full-speed USB device number 2 using uhci_hcd
[3903500.320567] usb 5-2: device descriptor read/64, error -71
[3903500.544709] usb 5-2: device descriptor read/64, error -71
[3903500.760848] usb 5-2: reset full-speed USB device number 2 using uhci_hcd
[3903501.169123] usb 5-2: device not accepting address 2, error -71
[3903501.281153] usb 5-2: reset full-speed USB device number 2 using uhci_hcd
[3903501.689411] usb 5-2: device not accepting address 2, error -71
[3903501.689842] usb 5-2: USB disconnect, device number 2
[3903717.219659] INFO: task khubd:119 blocked for more than 120 seconds.
[3903717.220045]       Not tainted 3.13.0-100-generic #147-Ubuntu
[3903717.220441] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[3903717.220975] khubd           D ffff88012b293180     0   119      2 0x00000000
[3903717.220980]  ffff880128f85c80 0000000000000046 ffff880128ece000 0000000000013180
[3903717.220984]  ffff880128f85fd8 0000000000013180 ffff880128ece000 ffff8801253da0e8
[3903717.220988]  ffff8801253da0ec ffff880128ece000 00000000ffffffff ffff8801253da0f0
[3903717.220992] Call Trace:
[3903717.221002]  [<ffffffff8172ee59>] schedule_preempt_disabled+0x29/0x70
[3903717.221006]  [<ffffffff81730c95>] __mutex_lock_slowpath+0x135/0x1b0
[3903717.221010]  [<ffffffff81730d2f>] mutex_lock+0x1f/0x2f
[3903717.221016]  [<ffffffff81544894>] usb_disconnect+0x64/0x200
[3903717.221019]  [<ffffffff81547483>] hub_port_connect_change+0xd3/0xb30
[3903717.221023]  [<ffffffff81541fcd>] ? hub_port_status+0xdd/0x120
[3903717.221026]  [<ffffffff815483b4>] hub_events+0x4d4/0xa20
[3903717.221030]  [<ffffffff81548935>] hub_thread+0x35/0x150
[3903717.221034]  [<ffffffff810adad0>] ? prepare_to_wait_event+0x100/0x100
[3903717.221037]  [<ffffffff81548900>] ? hub_events+0xa20/0xa20
[3903717.221041]  [<ffffffff8108dc79>] kthread+0xc9/0xe0
[3903717.221045]  [<ffffffff8108dbb0>] ? kthread_create_on_node+0x1c0/0x1c0
[3903717.221050]  [<ffffffff8173afa8>] ret_from_fork+0x58/0x90
[3903717.221053]  [<ffffffff8108dbb0>] ? kthread_create_on_node+0x1c0/0x1c0
[3903717.221064] INFO: task kworker/2:1:12543 blocked for more than 120 seconds.
[3903717.221495]       Not tainted 3.13.0-100-generic #147-Ubuntu
[3903717.221828] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[3903717.222285] kworker/2:1     D ffff88012b253180     0 12543      2 0x00000000
[3903717.222299] Workqueue: events hid_reset [usbhid]
[3903717.222301]  ffff880036091850 0000000000000046 ffff8800c4c40000 0000000000013180
[3903717.222304]  ffff880036091fd8 0000000000013180 ffff8800c4c40000 ffff880036091998
[3903717.222308]  ffff8800360919a0 7fffffffffffffff ffff8800c4c40000 ffff8800c4c40000
[3903717.222312] Call Trace:
[3903717.222315]  [<ffffffff8172e949>] schedule+0x29/0x70
[3903717.222318]  [<ffffffff8172db99>] schedule_timeout+0x279/0x310
[3903717.222323]  [<ffffffff810a16f8>] ? __enqueue_entity+0x78/0x80
[3903717.222327]  [<ffffffff810a7b2d>] ? enqueue_entity+0x2ad/0xc00
[3903717.222331]  [<ffffffff8172f456>] wait_for_completion+0xa6/0x150
[3903717.222336]  [<ffffffff8109d2f0>] ? wake_up_state+0x20/0x20
...
...