Kernel crashes immediately when accessing network

[Marco Wodrich]

After the last kernel update i boot my laptop and then start chromium browser. I was suprised that the browser no starting up. So i look at the output of dmesg and see the panic.

ProblemType: Bug
DistroRelease: Ubuntu 14.04
Package: linux-image-generic-lts-utopic 3.16.0.37.29
ProcVersionSignature: Ubuntu 3.16.0-33.44~14.04.1-generic 3.16.7-ckt7
Uname: Linux 3.16.0-33-generic x86_64
ApportVersion: 2.14.1-0ubuntu3.10
Architecture: amd64
Date: Mon May 18 18:57:19 2015
EcryptfsInUse: Yes
InstallationDate: Installed on 2012-10-28 (932 days ago)
InstallationMedia: Ubuntu GNOME Remix 12.10 "Quantal Quetzal" - Release amd64(20121023)
ProcEnviron:
 TERM=xterm
 PATH=(custom, no user)
 LANG=de_DE.UTF-8
 SHELL=/bin/bash
SourcePackage: linux-meta-lts-utopic
UpgradeStatus: No upgrade log present (probably fresh install)

[Marco Wodrich (mwodrich)]

The kernel panic:

------------[ cut here ]------------
[ 230.528076] kernel BUG at /build/buildd/linux-lts-utopic-3.16.0/net/ipv4/tcp_output.c:878!
[ 230.528155] invalid opcode: 0000 [#1] SMP
[ 230.528203] Modules linked in: rpcsec_gss_krb5 nfsv4 nls_utf8 isofs vhost_net vhost macvtap macvlan autofs4 bridge stp llc arc4 iwldvm mac80211 intel_rapl x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_codec_hdmi kvm snd_hda_codec_conexant snd_hda_codec_generic iwlwifi crct10dif_pclmul btusb crc32_pclmul ghash_clmulni_intel snd_hda_intel snd_hda_controller nfsd snd_hda_codec aesni_intel auth_rpcgss nfs_acl snd_hwdep aes_x86_64 snd_seq_midi snd_seq_midi_event lrw thinkpad_acpi bluetooth cfg80211 snd_pcm gf128mul glue_helper snd_rawmidi rtsx_pci_ms ablk_helper nfs snd_seq cryptd memstick snd_seq_device 6lowpan_iphc snd_timer nvram joydev lockd parport_pc snd sunrpc ppdev mei_me mei shpchp soundcore serio_raw binfmt_misc fscache mac_hid lp parport btrfs xor raid6_pq i915 i2c_algo_bit drm_kms_helper rtsx_pci_sdmmc psmouse r8169 ahci drm mii rtsx_pci libahci wmi video
[ 230.529160] CPU: 1 PID: 2419 Comm: Chrome_IOThread Not tainted 3.16.0-37-generic #51~14.04.1-Ubuntu
[ 230.529223] Hardware name: LENOVO 6885EXG/6885EXG, BIOS HEET47WW (1.28 ) 10/24/2014
[ 230.529247] task: ffff880402cdbd20 ti: ffff8803d3948000 task.ti: ffff8803d3948000
[ 230.529270] RIP: 0010:[<ffffffff816b845f>] [<ffffffff816b845f>] tcp_transmit_skb+0x72f/0x900
[ 230.529302] RSP: 0018:ffff8803d394bae8 EFLAGS: 00010246
[ 230.529319] RAX: 00000000000002c0 RBX: ffff8800c1490700 RCX: 00000000000000d0
[ 230.529352] RDX: 0000000000000001 RSI: ffff880403bdf600 RDI: ffff8803d0d01000
[ 230.529397] RBP: ffff8803d394bb40 R08: 00000000000002c0 R09: 0000000000000500
[ 230.529441] R10: ffff88040dc03500 R11: ffff8800c1490ce4 R12: ffff880403bdf600
[ 230.529486] R13: ffff8803d0ca1700 R14: 00000000000000ae R15: ffff880403bdf600
[ 230.529531] FS: 00007f8314069700(0000) GS:ffff88041e240000(0000) knlGS:0000000000000000
[ 230.529581] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 230.529617] CR2: 00003d742c856008 CR3: 00000003d8e52000 CR4: 00000000001407e0
[ 230.529661] Stack:
[ 230.529676] 00ff8803d0ca1700 ffff8800c1490700 00000000000000b0 ffff8800c1490700
[ 230.529732] 00000000000000ae ffffffff81398a63 ffff8800c1490700 ffff880403bdee00
[ 230.529787] ffff8803d0ca1700 00000000000000ae ffff880403bdf600 ffff8803d394bb98
[ 230.529841] Call Trace:
[ 230.529864] [<ffffffff81398a63>] ? memcpy_fromiovecend+0x83/0xb0
[ 230.529905] [<ffffffff816b9a9e>] tcp_connect+0x47e/0x7a0
[ 230.529943] [<ffffffff816be537>] tcp_v4_connect+0x317/0x470
[ 230.529981] [<ffffffff816d3fd5>] __inet_stream_connect+0xa5/0x320
[ 230.530023] [<ffffffff813863f0>] ? cpumask_next_and+0x30/0x50
[ 230.530065] [<ffffffff811b79c6>] ? kmem_cache_alloc_trace+0x1c6/0x1f0
[ 230.530108] [<ffffffff816aad23>] ? tcp_sendmsg+0x4e3/0xce0
[ 230.530145] [<ffffffff816aad60>] tcp_sendmsg+0x520/0xce0
[ 230.530185] [<ffffffff8133457d>] ? aa_sk_perm.isra.4+0x6d/0x150
[ 230.530225] [<ffffffff816d52d4>] inet_sendmsg+0x64/0xb0
[ 230.530261] [<ffffffff81329c31>] ? apparmor_socket_sendmsg+0x21/0x30
[ ...

[Qishuai Liu (lqs)]

I have encountered this bug too. It seems that this bug is triggered by sending a "tcp fast open" request with sendto().

[ 382.110408] ------------[ cut here ]------------
[ 382.112844] kernel BUG at /build/buildd/linux-lts-utopic-3.16.0/net/ipv4/tcp_output.c:878!
[ 382.114282] invalid opcode: 0000 [#3] SMP
[ 382.114282] Modules linked in: joydev hid_generic ppdev kvm_intel kvm crct10dif_pclmul crc32_pclmul ghash_clmulni_intel aesni_intel aes_x86_64 lrw gf128mul glue_helper ablk_helper cryptd usbhid hid serio_raw pvpanic i2c_piix4 parport_pc mac_hid lp parport nls_utf8 isofs floppy psmouse pata_acpi
[ 382.114282] CPU: 1 PID: 2091 Comm: ss-server Tainted: G D 3.16.0-37-generic #51~14.04.1-Ubuntu
[ 382.114282] Hardware name: Fedora Project OpenStack Nova, BIOS 0.5.1 01/01/2011
[ 382.114282] task: ffff88003af38000 ti: ffff88003d73c000 task.ti: ffff88003d73c000
[ 382.114282] RIP: 0010:[<ffffffff816b845f>] [<ffffffff816b845f>] tcp_transmit_skb+0x72f/0x900
[ 382.114282] RSP: 0018:ffff88003d73fae8 EFLAGS: 00010246
[ 382.114282] RAX: 00000000000002c0 RBX: ffff88003c5dc600 RCX: 00000000000000d0
[ 382.114282] RDX: 0000000000000001 RSI: ffff880036418800 RDI: ffff880036b61000
[ 382.114282] RBP: ffff88003d73fb40 R08: 00000000000002c0 R09: 0000000000000500
[ 382.114282] R10: ffff88003e001500 R11: ffff88003c5dcbe4 R12: ffff880036418800
[ 382.114282] R13: ffff88003c1b8740 R14: 0000000000000000 R15: ffff880036418800
[ 382.114282] FS: 00007fe77c448740(0000) GS:ffff88003fd00000(0000) knlGS:0000000000000000
[ 382.114282] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 382.114282] CR2: 00000000025a37c8 CR3: 000000003b00c000 CR4: 00000000001407e0
[ 382.114282] Stack:
[ 382.114282] 00ff88003c1b8740 ffff88003c5dc600 0000000000000000 ffff88003c5dc600
[ 382.114282] 0000000000000000 ffff88003c5dc600 ffff88003c5dc600 ffff880036418400
[ 382.114282] ffff88003c1b8740 0000000000000000 ffff880036418800 ffff88003d73fb98
[ 382.114282] Call Trace:
[ 382.114282] [<ffffffff816b9a9e>] tcp_connect+0x47e/0x7a0
[ 382.114282] [<ffffffff816be537>] tcp_v4_connect+0x317/0x470
[ 382.114282] [<ffffffff816d3fd5>] __inet_stream_connect+0xa5/0x320
[ 382.114282] [<ffffffff811b79c6>] ? kmem_cache_alloc_trace+0x1c6/0x1f0
[ 382.114282] [<ffffffff816aad23>] ? tcp_sendmsg+0x4e3/0xce0
[ 382.114282] [<ffffffff816aad60>] tcp_sendmsg+0x520/0xce0
[ 382.114282] [<ffffffff8116b6fa>] ? __alloc_pages_nodemask+0x15a/0x2a0
[ 382.114282] [<ffffffff8133457d>] ? aa_sk_perm.isra.4+0x6d/0x150
[ 382.114282] [<ffffffff816d52d4>] inet_sendmsg+0x64/0xb0
[ 382.114282] [<ffffffff81329c31>] ? apparmor_socket_sendmsg+0x21/0x30
[ 382.114282] [<ffffffff81648bbb>] sock_sendmsg+0x8b/0xc0
[ 382.114282] [<ffffffff8105af5c>] ? __do_page_fault+0x20c/0x560
[ 382.114282] [<ffffffff811f1243>] ? __fdget+0x13/0x20
[ 382.114282] [<ffffffff81648d61>] SYSC_sendto+0x121/0x1c0
[ 382.114282] [<ffffffff8105b361>] ? trace_do_page_fault+0x41/0x100
[ 382.114282] [<ffffffff812ebbe6>] ? security_file_fcntl+0x16/0x20
[ 382.114282] [<ffffffff8164983e>] SyS_sendto+0xe/0x10
[ 382.114282] [<ffffffff8176d20d>] system_call_fastpath+0x1a/0x1f
[ 382.114282] Code: 58 ab f9 ff...

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Marco Wodrich (mwodrich)]

It seems like this post, http://www.openwall.com/lists/oss-security/2015/04/18/2, describe the problem exactly. Please correct me if it is wrong.

[Marco Wodrich (mwodrich)]

I do further investigations on the debian bugtracker. The bug is caused by TCP fast open. If you you disable TCP fast open no crash is caused.

To disable TCP fast open do the following:

echo 0 > /proc/sys/net/ipv4/tcp_fastopen.