Same problem on ASUS Z97-WS after upgrading Ubuntu 14.10 to 15.04:

uname:
         Linux 3.19.0-22-generic #22-Ubuntu SMP Tue Jun 16 17:15:15 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

systemd-analyze:
         Startup finished in 2.077s (kernel) + 45.086s (userspace) = 47.164s

systemd-analyze blame:
         39.863s systemd-udev-settle.service

lsusb:
Bus 006 Device 002: ID 8087:8001 Intel Corp. 
Bus 006 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 005 Device 002: ID 8087:8009 Intel Corp. 
Bus 005 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 002 Device 004: ID 174c:3074 ASMedia Technology Inc. 
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 008: ID 0557:2213 ATEN International Co., Ltd CS682 2-Port USB 2.0 DVI KVM Switch
Bus 001 Device 007: ID 0557:8021 ATEN International Co., Ltd 
Bus 001 Device 006: ID 0409:005a NEC Corp. HighSpeed Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

lsusb -s 1
Bus 006 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 005 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

dmesg log (usb filtered):
[    0.283460] ACPI: bus type USB registered
[    0.283470] usbcore: registered new interface driver usbfs
[    0.283476] usbcore: registered new interface driver hub
[    0.283489] usbcore: registered new device driver usb
[    0.547961] xhci_hcd 0000:00:14.0: new USB bus registered, assigned bus number 1
[    0.548096] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002
[    0.548111] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    0.548128] usb usb1: Product: xHCI Host Controller
[    0.548139] usb usb1: Manufacturer: Linux 3.19.0-22-generic xhci-hcd
[    0.548154] usb usb1: SerialNumber: 0000:00:14.0
[    0.548227] hub 1-0:1.0: USB hub found
[    0.549410] xhci_hcd 0000:00:14.0: new USB bus registered, assigned bus number 2
[    0.549450] usb usb2: New USB device found, idVendor=1d6b, idProduct=0003
[    0.549465] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    0.549482] usb usb2: Product: xHCI Host Controller
[    0.549493] usb usb2: Manufacturer: Linux 3.19.0-22-generic xhci-hcd
[    0.549508] usb usb2: SerialNumber: 0000:00:14.0
[    0.549572] hub 2-0:1.0: USB hub found
[    0.550080] xhci_hcd 0000:10:00.0: new USB bus registered, assigned bus number 3
[    0.610642] usb usb3: New USB device found, idVendor=1d6b, idProduct=0002
[    0.610659] usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    0.610675] usb usb3: Product: xHCI Host Controller
[    0.610687] usb usb3: Manufacturer: Linux 3.19.0-22-generic xhci-hcd
[    0.610701] usb usb3: SerialNumber: 0000:10:00.0
[    0.610780] hub 3-0:1.0: USB hub found
[    0.610861] xhci_hcd 0000:10:00.0: new USB bus registered, assigned bus number 4
[    0.610918] usb usb4: New USB device found, idVendor=1d6b, idProduct=0003
[    0.610933] usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    0.610950] usb usb4: Product: xHCI Host Controller
[    0.610961] usb usb4: Manufacturer: Linux 3.19.0-22-generic xhci-hcd
[    0.610975] usb usb4: SerialNumber: 0000:10:00.0
[    0.611048] hub 4-0:1.0: USB hub found
[    0.611117] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    0.611218] ehci-pci 0000:00:1a.0: new USB bus registered, assigned bus number 5
[    0.626629] ehci-pci 0000:00:1a.0: USB 2.0 started, EHCI 1.00
[    0.626658] usb usb5: New USB device found, idVendor=1d6b, idProduct=0002
[    0.626673] usb usb5: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    0.626690] usb usb5: Product: EHCI Host Controller
[    0.627496] usb usb5: Manufacturer: Linux 3.19.0-22-generic ehci_hcd
[    0.628304] usb usb5: SerialNumber: 0000:00:1a.0
[    0.629180] hub 5-0:1.0: USB hub found
[    0.631710] ehci-pci 0000:00:1d.0: new USB bus registered, assigned bus number 6
[    0.646659] ehci-pci 0000:00:1d.0: USB 2.0 started, EHCI 1.00
[    0.647503] usb usb6: New USB device found, idVendor=1d6b, idProduct=0002
[    0.648314] usb usb6: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    0.649121] usb usb6: Product: EHCI Host Controller
[    0.649922] usb usb6: Manufacturer: Linux 3.19.0-22-generic ehci_hcd
[    0.650727] usb usb6: SerialNumber: 0000:00:1d.0
[    0.651604] hub 6-0:1.0: USB hub found
[    0.654091] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[    0.656508] uhci_hcd: USB Universal Host Controller Interface driver
[    0.858871] usb 1-2: new high-speed USB device number 2 using xhci_hcd
[    0.942915] usb 5-1: new high-speed USB device number 2 using ehci-pci
[    0.962960] usb 6-1: new high-speed USB device number 2 using ehci-pci
[    1.075399] usb 5-1: New USB device found, idVendor=8087, idProduct=8009
[    1.076309] usb 5-1: New USB device strings: Mfr=0, Product=0, SerialNumber=0
[    1.077452] hub 5-1:1.0: USB hub found
[    1.095436] usb 6-1: New USB device found, idVendor=8087, idProduct=8001
[    1.096370] usb 6-1: New USB device strings: Mfr=0, Product=0, SerialNumber=0
[    1.097473] hub 6-1:1.0: USB hub found
[    1.267212] usb 2-2: device not accepting address 2, error -22
[    2.023751] usb 2-2: new SuperSpeed USB device number 4 using xhci_hcd
[    2.148028] usb 2-2: New USB device found, idVendor=174c, idProduct=3074
[    2.149070] usb 2-2: New USB device strings: Mfr=2, Product=3, SerialNumber=0
[    2.150061] usb 2-2: Product: ASM107x
[    2.151013] usb 2-2: Manufacturer: ASUS Tek.
[    2.164339] hub 2-2:1.0: USB hub found
[   11.007927] usb 1-2: device descriptor read/all, error -110
[   11.119936] usb 1-2: new high-speed USB device number 3 using xhci_hcd
[   21.256938] usb 1-2: device descriptor read/all, error -110
[   21.368996] usb 1-2: new high-speed USB device number 4 using xhci_hcd
[   26.389580] usb 1-2: device descriptor read/8, error -110
[   31.514060] usb 1-2: device descriptor read/8, error -110
[   31.730162] usb 1-2: new high-speed USB device number 5 using xhci_hcd
[   36.750614] usb 1-2: device descriptor read/8, error -110
[   41.875228] usb 1-2: device descriptor read/8, error -110
[   41.979225] usb usb1-port2: unable to enumerate USB device
[   42.091317] usb 1-14: new high-speed USB device number 6 using xhci_hcd
[   42.219603] usb 1-14: New USB device found, idVendor=0409, idProduct=005a
[   42.219605] usb 1-14: New USB device strings: Mfr=0, Product=0, SerialNumber=0
[   42.219999] hub 1-14:1.0: USB hub found
[   42.517861] usb 1-14.4: new full-speed USB device number 7 using xhci_hcd
[   42.620250] usb 1-14.4: New USB device found, idVendor=0557, idProduct=8021
[   42.620251] usb 1-14.4: New USB device strings: Mfr=0, Product=0, SerialNumber=0
[   42.620339] usb 1-14.4: ep 0x81 - rounding interval to 1024 microframes, ep desc says 2040 microframes
[   42.620621] hub 1-14.4:1.0: USB hub found
[   42.892066] usb 1-14.4.1: new full-speed USB device number 8 using xhci_hcd
[   43.002183] usb 1-14.4.1: New USB device found, idVendor=0557, idProduct=2213
[   43.002186] usb 1-14.4.1: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[   43.002187] usb 1-14.4.1: Product: DVI DL KVMP
[   43.002188] usb 1-14.4.1: Manufacturer: ATEN
[   43.002295] usb 1-14.4.1: ep 0x81 - rounding interval to 64 microframes, ep desc says 80 microframes
[   43.002300] usb 1-14.4.1: ep 0x83 - rounding interval to 32 microframes, ep desc says 40 microframes
[   43.028279] usbcore: registered new interface driver usbhid
[   43.028283] usbhid: USB HID core driver
[   43.030230] input: ATEN DVI DL KVMP as /devices/pci0000:00/0000:00:14.0/usb1/1-14/1-14.4/1-14.4.1/1-14.4.1:1.0/0003:0557:2213.0001/input/input18
[   43.088435] hid-generic 0003:0557:2213.0001: input,hidraw0: USB HID v1.00 Keyboard [ATEN DVI DL KVMP] on usb-0000:00:14.0-14.4.1/input0
[   43.088533] input: ATEN DVI DL KVMP as /devices/pci0000:00/0000:00:14.0/usb1/1-14/1-14.4/1-14.4.1/1-14.4.1:1.1/0003:0557:2213.0002/input/input19
[   43.144318] hid-generic 0003:0557:2213.0002: input,hidraw1: USB HID v1.00 Mouse [ATEN DVI DL KVMP] on usb-0000:00:14.0-14.4.1/input1