@Bernhard, @Mathias
I'm not sure anymore if the issue is related to xhci, because of the lates WARNINGs and traces.
I tested a PCIe card
Network controller: Realtek Semiconductor Co., Ltd. RTL8821AE 802.11ac PCIe Wireless Network Adapter
and running into similar issues:
12506.901197] wlp3s0: deauthenticating from 00:24:d4:9e:e8:c4 by local choice (Reason: 3=DEAUTH_LEAVING)
[12506.902535] ------------[ cut here ]------------
[12506.902589] WARNING: CPU: 1 PID: 15941 at net/mac80211/rx.c:804 ieee80211_rx_napi.cold+0xc/0x67 [mac80211]
[12506.902590] Modules linked in: nfnetlink_queue nfnetlink_log nfnetlink mt7601u ccm fuse nls_iso8859_1 nls_cp437 vfat fat nvidia_drm(POE) nvidia_modeset(POE) nvidia(POE) uvcvideo videobuf2_vmalloc videobuf2_memops videobuf2_v4l2 videobuf2_common videodev mc snd_soc_skl snd_soc_hdac_hda snd_hda_ext_core snd_soc_skl_ipc snd_soc_sst_ipc rtsx_usb_ms snd_soc_sst_dsp rtl8821ae snd_soc_acpi_intel_match x86_pkg_temp_thermal memstick snd_soc_acpi intel_powerclamp btcoexist coretemp r8169 kvm_intel snd_soc_core rtl_pci rtlwifi snd_hda_codec_hdmi snd_compress ac97_bus kvm mac80211 snd_hda_codec_realtek ipmi_devintf ipmi_msghandler snd_pcm_dmaengine snd_hda_codec_generic irqbypass ledtrig_audio cfg80211 realtek libphy joydev mousedev iTCO_wdt i915 iTCO_vendor_support crct10dif_pclmul crc32_pclmul libarc4 ghash_clmulni_intel btusb btrtl aesni_intel btbcm btintel bluetooth aes_x86_64 snd_hda_intel snd_hda_codec crypto_simd cryptd i2c_algo_bit i2c_hid glue_helper drm_kms_helper asus_nb_wmi intel_rapl_msr
[12506.902624]  asus_wmi drm sparse_keymap mei_hdcp snd_hda_core intel_cstate mxm_wmi intel_uncore intel_rapl_perf intel_gtt agpgart ecdh_generic snd_hwdep pcspkr rfkill syscopyarea snd_pcm sysfillrect ecc sysimgblt fb_sys_fops tpm_crb input_leds snd_timer elan_i2c tpm_tis tpm_tis_core snd int3403_thermal tpm i2c_i801 evdev rng_core soundcore processor_thermal_device intel_rapl_common mac_hid idma64 intel_xhci_usb_role_switch int340x_thermal_zone roles intel_soc_dts_iosf mei_me int3400_thermal mei acpi_thermal_rel intel_pch_thermal intel_lpss_pci intel_lpss asus_wireless wmi battery ac sg crypto_user ip_tables x_tables ext4 crc32c_generic crc16 mbcache jbd2 rtsx_usb_sdmmc mmc_core rtsx_usb hid_generic usbhid hid sr_mod cdrom sd_mod serio_raw atkbd libps2 ahci libahci libata xhci_pci crc32c_intel i8042 xhci_hcd scsi_mod serio
[12506.902660] CPU: 1 PID: 15941 Comm: Netlink Monitor Tainted: P        W  OE     5.3.8-arch1-1 #1
[12506.902661] Hardware name: ASUSTeK COMPUTER INC. X555UB/X555UB, BIOS X555UB.301 02/20/2017
[12506.902684] RIP: 0010:ieee80211_rx_napi.cold+0xc/0x67 [mac80211]
[12506.902687] Code: 38 48 81 c1 70 04 00 00 48 81 c6 38 01 00 00 e8 0a 10 77 dc b8 01 00 00 00 e9 26 4b fb ff 48 c7 c7 60 ab eb c0 e8 b7 23 25 dc <0f> 0b 48 89 ef e8 7f f8 89 dc e9 d1 5b fb ff 48 c7 c7 60 ab eb c0
[12506.902688] RSP: 0000:ffffb624c0120e10 EFLAGS: 00010246
[12506.902690] RAX: 0000000000000024 RBX: ffff8ee22cae07a0 RCX: 0000000000000000
[12506.902691] RDX: 0000000000000000 RSI: ffff8ee23ba97708 RDI: 00000000ffffffff
[12506.902692] RBP: ffff8ee1ab8b8400 R08: 00000000000014eb R09: 0000000000000001
[12506.902692] R10: 0000000000000000 R11: 0000000000000001 R12: 0000000000000000
[12506.902694] R13: 0000000000000001 R14: 0000000000000006 R15: 0000000000000000
[12506.902695] FS:  00007f27b8866700(0000) GS:ffff8ee23ba80000(0000) knlGS:0000000000000000
[12506.902696] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[12506.902697] CR2: 00005599e0bfe008 CR3: 00000002417b6002 CR4: 00000000003606e0
[12506.902699] Call Trace:
[12506.902702]  <IRQ>
[12506.902707]  ? __mod_zone_page_state+0x66/0xa0
[12506.902712]  ? kmem_cache_free_bulk+0x2e1/0x450
[12506.902730]  ieee80211_tasklet_handler+0xbc/0xd0 [mac80211]
[12506.902734]  tasklet_action_common.isra.0+0x4a/0xb0
[12506.902737]  __do_softirq+0x114/0x332
[12506.902740]  irq_exit+0xd4/0xf0
[12506.902742]  do_IRQ+0xbb/0x130
[12506.902746]  common_interrupt+0xf/0xf
[12506.902747]  </IRQ>
[12506.902749] RIP: 0033:0x55b7f9be923d
[12506.902751] Code: 0f 84 9e 00 00 00 48 8b 10 48 83 e2 fe 48 09 ca 48 89 56 08 48 89 30 48 8b 4e 08 48 83 e1 fe 74 09 48 8b 51 08 f6 c2 01 75 06 <48> 89 4e 08 59 c3 48 83 e2 fe 48 89 51 08 48 8b 4e 08 48 89 ca 48
[12506.902752] RSP: 002b:00007f27b8864a10 EFLAGS: 00000246 ORIG_RAX: ffffffffffffffda
[12506.902754] RAX: 00007f27a6800770 RBX: 00007f27a6201628 RCX: 0000000000000000
[12506.902755] RDX: 0000000000000001 RSI: 00007f27a6201628 RDI: 0000000000000000
[12506.902755] RBP: 0000000000000000 R08: 0000000000000000 R09: 00007f27c3f0a3b6
[12506.902756] R10: 0000000000000000 R11: 0000000000000000 R12: 00007f27c8e00110
[12506.902757] R13: 00007f27a6800f80 R14: 00000000ffffffff R15: 00007f27a5d01100
[12506.902760] ---[ end trace 934096102c096064 ]---
[12506.990611] audit: type=1131 audit(1573296338.199:239): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=wpa_supplicant comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
[12507.259354] ------------[ cut here ]------------
[12507.259405] WARNING: CPU: 1 PID: 16380 at net/mac80211/rx.c:804 ieee80211_rx_napi.cold+0xc/0x67 [mac80211]
[12507.259406] Modules linked in: nfnetlink_queue nfnetlink_log nfnetlink mt7601u ccm fuse nls_iso8859_1 nls_cp437 vfat fat nvidia_drm(POE) nvidia_modeset(POE) nvidia(POE) uvcvideo videobuf2_vmalloc videobuf2_memops videobuf2_v4l2 videobuf2_common videodev mc snd_soc_skl snd_soc_hdac_hda snd_hda_ext_core snd_soc_skl_ipc snd_soc_sst_ipc rtsx_usb_ms snd_soc_sst_dsp rtl8821ae snd_soc_acpi_intel_match x86_pkg_temp_thermal memstick snd_soc_acpi intel_powerclamp btcoexist coretemp r8169 kvm_intel snd_soc_core rtl_pci rtlwifi snd_hda_codec_hdmi snd_compress ac97_bus kvm mac80211 snd_hda_codec_realtek ipmi_devintf ipmi_msghandler snd_pcm_dmaengine snd_hda_codec_generic irqbypass ledtrig_audio cfg80211 realtek libphy joydev mousedev iTCO_wdt i915 iTCO_vendor_support crct10dif_pclmul crc32_pclmul libarc4 ghash_clmulni_intel btusb btrtl aesni_intel btbcm btintel bluetooth aes_x86_64 snd_hda_intel snd_hda_codec crypto_simd cryptd i2c_algo_bit i2c_hid glue_helper drm_kms_helper asus_nb_wmi intel_rapl_msr
[12507.259436]  asus_wmi drm sparse_keymap mei_hdcp snd_hda_core intel_cstate mxm_wmi intel_uncore intel_rapl_perf intel_gtt agpgart ecdh_generic snd_hwdep pcspkr rfkill syscopyarea snd_pcm sysfillrect ecc sysimgblt fb_sys_fops tpm_crb input_leds snd_timer elan_i2c tpm_tis tpm_tis_core snd int3403_thermal tpm i2c_i801 evdev rng_core soundcore processor_thermal_device intel_rapl_common mac_hid idma64 intel_xhci_usb_role_switch int340x_thermal_zone roles intel_soc_dts_iosf mei_me int3400_thermal mei acpi_thermal_rel intel_pch_thermal intel_lpss_pci intel_lpss asus_wireless wmi battery ac sg crypto_user ip_tables x_tables ext4 crc32c_generic crc16 mbcache jbd2 rtsx_usb_sdmmc mmc_core rtsx_usb hid_generic usbhid hid sr_mod cdrom sd_mod serio_raw atkbd libps2 ahci libahci libata xhci_pci crc32c_intel i8042 xhci_hcd scsi_mod serio
[12507.259466] CPU: 1 PID: 16380 Comm: sudo Tainted: P        W  OE     5.3.8-arch1-1 #1
[12507.259466] Hardware name: ASUSTeK COMPUTER INC. X555UB/X555UB, BIOS X555UB.301 02/20/2017
[12507.259483] RIP: 0010:ieee80211_rx_napi.cold+0xc/0x67 [mac80211]
[12507.259485] Code: 38 48 81 c1 70 04 00 00 48 81 c6 38 01 00 00 e8 0a 10 77 dc b8 01 00 00 00 e9 26 4b fb ff 48 c7 c7 60 ab eb c0 e8 b7 23 25 dc <0f> 0b 48 89 ef e8 7f f8 89 dc e9 d1 5b fb ff 48 c7 c7 60 ab eb c0
[12507.259485] RSP: 0000:ffffb624c0120e10 EFLAGS: 00010246
[12507.259486] RAX: 0000000000000024 RBX: ffff8ee22cae07a0 RCX: 0000000000000000
[12507.259487] RDX: 0000000000000000 RSI: ffff8ee23ba97708 RDI: 00000000ffffffff
[12507.259488] RBP: ffff8ee1ab8b8b00 R08: 0000000000001511 R09: 0000000000000001
[12507.259488] R10: 0000000000000000 R11: 0000000000000001 R12: 0000000000000000
[12507.259489] R13: 0000000000000001 R14: 0000000000000006 R15: 0000000000000000
[12507.259490] FS:  00007fe70c78b740(0000) GS:ffff8ee23ba80000(0000) knlGS:0000000000000000
[12507.259491] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[12507.259491] CR2: 00007fe70ca9c1b0 CR3: 00000001f9502001 CR4: 00000000003606e0
[12507.259492] Call Trace:
[12507.259494]  <IRQ>
[12507.259499]  ? __mod_zone_page_state+0x66/0xa0
[12507.259502]  ? kmem_cache_free_bulk+0x2e1/0x450
[12507.259512]  ieee80211_tasklet_handler+0xbc/0xd0 [mac80211]
[12507.259515]  tasklet_action_common.isra.0+0x4a/0xb0
[12507.259517]  __do_softirq+0x114/0x332
[12507.259519]  irq_exit+0xd4/0xf0
[12507.259521]  do_IRQ+0xbb/0x130
[12507.259524]  common_interrupt+0xf/0xf
[12507.259524]  </IRQ>
[12507.259527] RIP: 0010:__do_page_fault+0x12c/0x4f0
[12507.259528] Code: a9 ff ff ff 7f 0f 85 cf 01 00 00 48 85 db 0f 84 c6 01 00 00 41 f6 84 24 88 00 00 00 03 0f 84 4a 01 00 00 fb 66 0f 1f 44 00 00 <41> bf 54 00 00 00 0f 1f 44 00 00 44 89 f8 48 89 e9 83 c8 01 83 e1
[12507.259529] RSP: 0000:ffffb624c4a2fec8 EFLAGS: 00000206 ORIG_RAX: ffffffffffffffda
[12507.259530] RAX: 0000000080000000 RBX: ffff8ee21a601980 RCX: 0000000000000000
[12507.259530] RDX: 00007fe70ca9c1b0 RSI: 0000000000000000 RDI: ffffb624c4a2ff58
[12507.259531] RBP: 0000000000000007 R08: 0000000000000000 R09: 0000000000000000
[12507.259531] R10: 0000000000000000 R11: 0000000000000000 R12: ffffb624c4a2ff58
[12507.259532] R13: 00007fe70ca9c1b0 R14: 00007fe70ca9c000 R15: 0000000000000000
[12507.259535]  do_page_fault+0x31/0x130
[12507.259536]  page_fault+0x3e/0x50
[12507.259538] RIP: 0033:0x7fe70c9a190d
[12507.259539] Code: 64 48 8b 04 25 28 00 00 00 48 89 44 24 38 31 c0 8b 05 b7 a8 0f 00 85 c0 0f 8e af 02 00 00 83 c0 01 83 f8 64 0f 8f 53 01 00 00 <89> 05 9d a8 0f 00 80 3d 16 87 0f 00 00 0f 84 a3 02 00 00 48 8b 05
[12507.259540] RSP: 002b:00007ffc8325d380 EFLAGS: 00010287
[12507.259540] RAX: 000000000000000a RBX: 000055b4c6e1fc80 RCX: 0000000000000400
[12507.259541] RDX: 000055b4c6e1fcb0 RSI: 000055b4c6e1fc80 RDI: 000055b4c6e15740
[12507.259541] RBP: 0000000000000400 R08: 00007ffc8325d418 R09: 0000000000000041
[12507.259542] R10: 0000000000000000 R11: 00007fe70ca96a40 R12: 000055b4c6e1fc80
[12507.259543] R13: 000055b4c6e15740 R14: 000055b4c6e1fcb0 R15: 0000000000000000
[12507.259544] ---[ end trace 934096102c096065 ]---
[12507.907345] IPv6: ADDRCONF(NETDEV_CHANGE): wlp3s0: link becomes ready
[12516.922094] audit: type=1131 audit(1573296348.129:240): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=NetworkManager-dispatcher comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'

And we have a simliar bug report regarding an ATHEROS driver, here:
https://bugzilla.kernel.org/show_bug.cgi?id=198701

So if it is really a driver issue and not related to xhci, we have this issue on rt2800usb, mt76 and ath9k_htc.