Comment 7 for bug 1702665

Revision history for this message
Sorah Fukumori (sorah) wrote : Re: 4.4.0-83-generic + Docker + EC2 t2.small frequently crashes at cgroup_rmdir GPF

Additional logs from 4.4.0 machine by running `dmesg -w` until crash.

[ 3031.276097] ------------[ cut here ]------------
[ 3031.276106] WARNING: CPU: 0 PID: 31804 at /build/linux-0uniEn/linux-4.4.0/net/ipv6/addrconf_core.c:159 in6_dev_finish_destroy+0x6b/0xc0()
[ 3031.276108] Modules linked in: veth binfmt_misc xt_nat xt_comment xt_tcpudp ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 xt_addrtype iptable_filter ip_tables xt_conntrack x_tables nf_nat nf_conntrack br_netfilter bridge stp llc aufs isofs ppdev input_leds serio_raw
 parport_pc parport autofs4 btrfs xor raid6_pq crct10dif_pclmul crc32_pclmul ghash_clmulni_intel aesni_intel aes_x86_64 lrw gf128mul glue_helper ablk_helper cryptd psmouse floppy
[ 3031.276131] CPU: 0 PID: 31804 Comm: kworker/u30:3 Not tainted 4.4.0-83-generic #106-Ubuntu
[ 3031.276132] Hardware name: Xen HVM domU, BIOS 4.2.amazon 02/16/2017
[ 3031.276136] Workqueue: netns cleanup_net
[ 3031.276137] 0000000000000286 00000000e9d24b1b ffff880043383ba0 ffffffff813f9513
[ 3031.276140] 0000000000000000 ffffffff81d75940 ffff880043383bd8 ffffffff81081322
[ 3031.276141] ffff880070297800 0000000000000000 0000000000000006 ffff880043383ca8
[ 3031.276143] Call Trace:
[ 3031.276148] [<ffffffff813f9513>] dump_stack+0x63/0x90
[ 3031.276151] [<ffffffff81081322>] warn_slowpath_common+0x82/0xc0
[ 3031.276153] [<ffffffff8108146a>] warn_slowpath_null+0x1a/0x20
[ 3031.276154] [<ffffffff8181ba0b>] in6_dev_finish_destroy+0x6b/0xc0
[ 3031.276157] [<ffffffff817f17e6>] ip6_route_dev_notify+0x116/0x130
[ 3031.276159] [<ffffffff810a1b1a>] notifier_call_chain+0x4a/0x70
[ 3031.276161] [<ffffffff810a1c96>] raw_notifier_call_chain+0x16/0x20
[ 3031.276163] [<ffffffff8172f655>] call_netdevice_notifiers_info+0x35/0x60
[ 3031.276165] [<ffffffff81739e7d>] netdev_run_todo+0x16d/0x320
[ 3031.276168] [<ffffffff817316c9>] ? rollback_registered_many+0x2c9/0x340
[ 3031.276171] [<ffffffff8174540e>] rtnl_unlock+0xe/0x10
[ 3031.276173] [<ffffffff81732a37>] default_device_exit_batch+0x147/0x170
[ 3031.276176] [<ffffffff810c4140>] ? __wake_up_sync+0x20/0x20
[ 3031.276178] [<ffffffff8172a892>] ops_exit_list.isra.4+0x52/0x60
[ 3031.276179] [<ffffffff8172b8e2>] cleanup_net+0x1c2/0x2a0
[ 3031.276182] [<ffffffff8109a585>] process_one_work+0x165/0x480
[ 3031.276184] [<ffffffff8109a8eb>] worker_thread+0x4b/0x4c0
[ 3031.276186] [<ffffffff8109a8a0>] ? process_one_work+0x480/0x480
[ 3031.276187] [<ffffffff810a0c25>] kthread+0xe5/0x100
[ 3031.276189] [<ffffffff810a0b40>] ? kthread_create_on_node+0x1e0/0x1e0
[ 3031.276192] [<ffffffff81840f0f>] ret_from_fork+0x3f/0x70
[ 3031.276193] [<ffffffff810a0b40>] ? kthread_create_on_node+0x1e0/0x1e0
[ 3031.276194] ---[ end trace 5d921dfe814c9c7c ]---
[ 3031.276195] ------------[ cut here ]------------
[ 3031.276197] WARNING: CPU: 0 PID: 31804 at /build/linux-0uniEn/linux-4.4.0/net/ipv6/addrconf_core.c:160 in6_dev_finish_destroy+0x83/0xc0()
[ 3031.276198] Modules linked in: veth binfmt_misc xt_nat xt_comment xt_tcpudp ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 xt_addrtype iptable_filter ip_tables xt_conntrack x_tables nf_nat nf_conntrack br_netfilter bridge stp llc aufs isofs ppdev input_leds serio_raw parport_pc parport autofs4 btrfs xor raid6_pq crct10dif_pclmul crc32_pclmul ghash_clmulni_intel aesni_intel aes_x86_64 lrw gf128mul glue_helper ablk_helper cryptd psmouse floppy
[ 3031.276211] CPU: 0 PID: 31804 Comm: kworker/u30:3 Tainted: G W 4.4.0-83-generic #106-Ubuntu
[ 3031.276212] Hardware name: Xen HVM domU, BIOS 4.2.amazon 02/16/2017
[ 3031.276213] Workqueue: netns cleanup_net
[ 3031.276214] 0000000000000286 00000000e9d24b1b ffff880043383ba0 ffffffff813f9513
[ 3031.276216] 0000000000000000 ffffffff81d75940 ffff880043383bd8 ffffffff81081322
[ 3031.276217] ffff880070297800 0000000000000000 0000000000000006 ffff880043383ca8
[ 3031.276219] Call Trace:
[ 3031.276220] [<ffffffff813f9513>] dump_stack+0x63/0x90
[ 3031.276222] [<ffffffff81081322>] warn_slowpath_common+0x82/0xc0
[ 3031.276224] [<ffffffff8108146a>] warn_slowpath_null+0x1a/0x20
[ 3031.276225] [<ffffffff8181ba23>] in6_dev_finish_destroy+0x83/0xc0
[ 3031.276226] [<ffffffff817f17e6>] ip6_route_dev_notify+0x116/0x130
[ 3031.276228] [<ffffffff810a1b1a>] notifier_call_chain+0x4a/0x70
[ 3031.276229] [<ffffffff810a1c96>] raw_notifier_call_chain+0x16/0x20
[ 3031.276231] [<ffffffff8172f655>] call_netdevice_notifiers_info+0x35/0x60
[ 3031.276233] [<ffffffff81739e7d>] netdev_run_todo+0x16d/0x320
[ 3031.276234] [<ffffffff817316c9>] ? rollback_registered_many+0x2c9/0x340
[ 3031.276237] [<ffffffff8174540e>] rtnl_unlock+0xe/0x10
[ 3031.276239] [<ffffffff81732a37>] default_device_exit_batch+0x147/0x170
[ 3031.276240] [<ffffffff810c4140>] ? __wake_up_sync+0x20/0x20
[ 3031.276242] [<ffffffff8172a892>] ops_exit_list.isra.4+0x52/0x60
[ 3031.276243] [<ffffffff8172b8e2>] cleanup_net+0x1c2/0x2a0
[ 3031.276245] [<ffffffff8109a585>] process_one_work+0x165/0x480
[ 3031.276247] [<ffffffff8109a8eb>] worker_thread+0x4b/0x4c0
[ 3031.276249] [<ffffffff8109a8a0>] ? process_one_work+0x480/0x480
[ 3031.276250] [<ffffffff810a0c25>] kthread+0xe5/0x100
[ 3031.276251] [<ffffffff810a0b40>] ? kthread_create_on_node+0x1e0/0x1e0
[ 3031.276253] [<ffffffff81840f0f>] ret_from_fork+0x3f/0x70
[ 3031.276254] [<ffffffff810a0b40>] ? kthread_create_on_node+0x1e0/0x1e0
[ 3031.276255] ---[ end trace 5d921dfe814c9c7d ]---
[ 3031.276260] BUG: unable to handle kernel NULL pointer dereference at 0000000000000448
[ 3031.280059] IP: [<ffffffff8181b9cd>] in6_dev_finish_destroy+0x2d/0xc0
[ 3031.280059] PGD 7bbf6067 PUD 7ba63067 PMD 0
[ 3031.280059] Oops: 0000 [#1] SMP
[ 3031.280059] Modules linked in: veth binfmt_misc xt_nat xt_comment xt_tcpudp ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 xt_addrtype iptable_filter ip_tables xt_conntrack x_tables nf_nat nf_conntrack br_netfilter bridge stp llc aufs isofs ppdev input_leds serio_raw parport_pc parport autofs4 btrfs xor raid6_pq crct10dif_pclmul crc32_pclmul ghash_clmulni_intel aesni_intel aes_x86_64 lrw gf128mul glue_helper ablk_helper cryptd psmouse floppy
[ 3031.280059] CPU: 0 PID: 31804 Comm: kworker/u30:3 Tainted: G W 4.4.0-83-generic #106-Ubuntu
[ 3031.280059] Hardware name: Xen HVM domU, BIOS 4.2.amazon 02/16/2017
[ 3031.280059] Workqueue: netns cleanup_net
[ 3031.280059] task: ffff88001608c600 ti: ffff880043380000 task.ti: ffff880043380000
[ 3031.280059] RIP: 0010:[<ffffffff8181b9cd>] [<ffffffff8181b9cd>] in6_dev_finish_destroy+0x2d/0xc0
[ 3031.280059] RSP: 0018:ffff880043383bf8 EFLAGS: 00010246
[ 3031.280059] RAX: 0000000000000024 RBX: ffff880070297800 RCX: 0000000000000006
[ 3031.280059] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000009
[ 3031.280059] RBP: ffff880043383c08 R08: 000000000000000a R09: 00000000000002e0
[ 3031.280059] R10: 0000000000000004 R11: 00000000000002e0 R12: 0000000000000000
[ 3031.280059] R13: 0000000000000006 R14: ffff880043383ca8 R15: 0000000000000000
[ 3031.280059] FS: 0000000000000000(0000) GS:ffff88007d200000(0000) knlGS:0000000000000000
[ 3031.280059] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 3031.280059] CR2: 0000000000000448 CR3: 0000000036d4c000 CR4: 00000000001406f0
[ 3031.280059] Stack:
[ 3031.280059] ffff880007803100 0000000000000000 ffff880043383c20 ffffffff817f17e6
[ 3031.280059] 00000000ffffffed ffff880043383c58 ffffffff810a1b1a ffff880043383ca8
[ 3031.280059] 0000000000000006 ffff880005580000 00000001000a6343 00000000000000fa
[ 3031.280059] Call Trace:
[ 3031.280059] [<ffffffff817f17e6>] ip6_route_dev_notify+0x116/0x130
[ 3031.280059] [<ffffffff810a1b1a>] notifier_call_chain+0x4a/0x70
[ 3031.280059] [<ffffffff810a1c96>] raw_notifier_call_chain+0x16/0x20
[ 3031.280059] [<ffffffff8172f655>] call_netdevice_notifiers_info+0x35/0x60
[ 3031.280059] [<ffffffff81739e7d>] netdev_run_todo+0x16d/0x320
[ 3031.280059] [<ffffffff817316c9>] ? rollback_registered_many+0x2c9/0x340
[ 3031.280059] [<ffffffff8174540e>] rtnl_unlock+0xe/0x10
[ 3031.280059] [<ffffffff81732a37>] default_device_exit_batch+0x147/0x170
[ 3031.280059] [<ffffffff810c4140>] ? __wake_up_sync+0x20/0x20
[ 3031.280059] [<ffffffff8172a892>] ops_exit_list.isra.4+0x52/0x60
[ 3031.280059] [<ffffffff8172b8e2>] cleanup_net+0x1c2/0x2a0
[ 3031.280059] [<ffffffff8109a585>] process_one_work+0x165/0x480
[ 3031.280059] [<ffffffff8109a8eb>] worker_thread+0x4b/0x4c0
[ 3031.280059] [<ffffffff8109a8a0>] ? process_one_work+0x480/0x480
[ 3031.280059] [<ffffffff810a0c25>] kthread+0xe5/0x100
[ 3031.280059] [<ffffffff810a0b40>] ? kthread_create_on_node+0x1e0/0x1e0
[ 3031.280059] [<ffffffff81840f0f>] ret_from_fork+0x3f/0x70
[ 3031.280059] [<ffffffff810a0b40>] ? kthread_create_on_node+0x1e0/0x1e0
[ 3031.280059] Code: 44 00 00 55 48 8d 47 08 48 89 e5 41 54 53 48 39 47 08 48 89 fb 4c 8b 27 75 3e 48 83 7b 18 00 75 4f 48 83 bb b8 02 00 00 00 75 60 <49> 8b 84 24 48 04 00 00 65 ff 08 8b 83 58 01 00 00 85 c0 74 5e
[ 3031.280059] RIP [<ffffffff8181b9cd>] in6_dev_finish_destroy+0x2d/0xc0
[ 3031.280059] RSP <ffff880043383bf8>
[ 3031.280059] CR2: 0000000000000448
[ 3031.280059] ---[ end trace 5d921dfe814c9c7e ]---