Hi,
I got this on 4.4.0-116-generic :
[266447.178906] ------------[ cut here ]------------ [266447.178919] WARNING: CPU: 0 PID: 3343 at /build/linux-fQ94TU/linux-4.4.0/net/core/dev.c:2448 skb_warn_bad_offload+0xd1/0x120() [266447.178924] qr-8a4bd2f3-04: caps=(0x00000184075b59e9, 0x00000184075b59e9) len=2058 data_len=582 gso_size=1406 gso_type=1 ip_summed=3 [266447.178926] Modules linked in: kpatch_livepatch_Ubuntu_4_4_0_116_140_generic_36(OE) xt_REDIRECT nf_nat_redirect ip6table_mangle xt_nat xt_mark xt_connmark ip6table_raw iptable_raw ip_gre ip_tunnel gre xt_recent ip6t_REJECT nf_reject_ipv6 nf_log_ipv6 xt_hl ip6t_rt nf_conntrack_ipv6 ipt_REJECT nf_reject_ipv4 nf_log_ipv4 nf_log_common xt_LOG xt_limit xt_addrtype xt_conntrack dccp_diag dccp tcp_diag udp_diag inet_diag unix_diag veth netlink_diag nbd ip6table_filter ip6_tables xt_CHECKSUM iptable_mangle xt_tcpudp ipt_MASQUERADE nf_nat_masquerade_ipv4 xt_comment iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_filter ip_tables x_tables openvswitch nf_defrag_ipv6 nf_conntrack binfmt_misc ipmi_ssif intel_rapl x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel bridge stp kvm llc [266447.178988] irqbypass sb_edac edac_core ipmi_si hpilo ipmi_msghandler serio_raw 8250_fintek lpc_ich shpchp acpi_power_meter ioatdma mac_hid dca ib_iser rdma_cm iw_cm ib_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi autofs4 btrfs raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear crct10dif_pclmul crc32_pclmul ghash_clmulni_intel aesni_intel aes_x86_64 lrw gf128mul glue_helper ablk_helper cryptd psmouse tg3 ptp pps_core hpsa scsi_transport_sas wmi fjes [266447.179045] CPU: 0 PID: 3343 Comm: haproxy Tainted: G W OE K 4.4.0-116-generic #140-Ubuntu [266447.179048] Hardware name: HP ProLiant DL360 Gen9/ProLiant DL360 Gen9, BIOS P89 01/22/2018 [266447.179050] 0000000000000286 6f5737dc927d16dd ffff880a3646b818 ffffffff813ffc13 [266447.179055] ffff880a3646b860 ffffffff81d715e8 ffff880a3646b850 ffffffff810830f2 [266447.179059] ffff880064507ce8 ffff880b4ff12000 0000000000000001 ffff8808723268fc [266447.179064] Call Trace: [266447.179072] [<ffffffff813ffc13>] dump_stack+0x63/0x90 [266447.179079] [<ffffffff810830f2>] warn_slowpath_common+0x82/0xc0 [266447.179084] [<ffffffff8108318c>] warn_slowpath_fmt+0x5c/0x80 [266447.179090] [<ffffffff81405d92>] ? ___ratelimit+0xa2/0xe0 [266447.179094] [<ffffffff8173ff81>] skb_warn_bad_offload+0xd1/0x120 [266447.179100] [<ffffffff81740155>] skb_checksum_help+0x185/0x1a0 [266447.179106] [<ffffffffc0524072>] checksum_tg+0x22/0x29 [xt_CHECKSUM] [266447.179112] [<ffffffffc03a4e21>] ipt_do_table+0x301/0x720 [ip_tables] [266447.179117] [<ffffffffc03a4e69>] ? ipt_do_table+0x349/0x720 [ip_tables] [266447.179122] [<ffffffff810bae37>] ? enqueue_entity+0x3a7/0xd20 [266447.179129] [<ffffffffc04980c9>] iptable_mangle_hook+0x39/0x107 [iptable_mangle] [266447.179134] [<ffffffff81778e48>] nf_iterate+0x68/0x80 [266447.179137] [<ffffffff81778ed3>] nf_hook_slow+0x73/0xd0 [266447.179143] [<ffffffff8178642f>] ip_output+0xcf/0xe0 [266447.179148] [<ffffffff81785880>] ? __ip_flush_pending_frames.isra.42+0x90/0x90 [266447.179153] [<ffffffff81785b8b>] ip_local_out+0x3b/0x50 [266447.179158] [<ffffffff81785eb4>] ip_queue_xmit+0x154/0x380 [266447.179164] [<ffffffff8179e2bb>] tcp_transmit_skb+0x51b/0x980 [266447.179169] [<ffffffff8179e8f6>] tcp_write_xmit+0x1d6/0xf40 [266447.179174] [<ffffffff8179f8f1>] __tcp_push_pending_frames+0x31/0xd0 [266447.179178] [<ffffffff8178d7ec>] tcp_push+0xec/0x110 [266447.179181] [<ffffffff81791969>] tcp_sendmsg+0x739/0xb80 [266447.179186] [<ffffffff817bd91b>] inet_sendmsg+0x6b/0xa0 [266447.179192] [<ffffffff81724b5e>] sock_sendmsg+0x3e/0x50 [266447.179196] [<ffffffff81725151>] SYSC_sendto+0x101/0x190 [266447.179203] [<ffffffff81003938>] ? syscall_trace_enter_phase1+0xc8/0x140 [266447.179208] [<ffffffff81725c7e>] SyS_sendto+0xe/0x10 [266447.179215] [<ffffffff8184efc8>] entry_SYSCALL_64_fastpath+0x1c/0xbb [266447.179218] ---[ end trace 4c054fa8d617289c ]---
Joseph, can we maybe get a new kernel to test ? The one in your last comment is gone.
Note that this isn't appearing a lot (~10 occurrences on a server with 3 days of uptime)
Hi,
I got this on 4.4.0-116-generic :
[266447.178906] ------------[ cut here ]------------ linux-fQ94TU/ linux-4. 4.0/net/ core/dev. c:2448 skb_warn_ bad_offload+ 0xd1/0x120( ) 4075b59e9, 0x00000184075b59e9) len=2058 data_len=582 gso_size=1406 gso_type=1 ip_summed=3 livepatch_ Ubuntu_ 4_4_0_116_ 140_generic_ 36(OE) xt_REDIRECT nf_nat_redirect ip6table_mangle xt_nat xt_mark xt_connmark ip6table_raw iptable_raw ip_gre ip_tunnel gre xt_recent ip6t_REJECT nf_reject_ipv6 nf_log_ipv6 xt_hl ip6t_rt nf_conntrack_ipv6 ipt_REJECT nf_reject_ipv4 nf_log_ipv4 nf_log_common xt_LOG xt_limit xt_addrtype xt_conntrack dccp_diag dccp tcp_diag udp_diag inet_diag unix_diag veth netlink_diag nbd ip6table_filter ip6_tables xt_CHECKSUM iptable_mangle xt_tcpudp ipt_MASQUERADE nf_nat_ masquerade_ ipv4 xt_comment iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_filter ip_tables x_tables openvswitch nf_defrag_ipv6 nf_conntrack binfmt_misc ipmi_ssif intel_rapl x86_pkg_ temp_thermal intel_powerclamp coretemp kvm_intel bridge stp kvm llc iscsi autofs4 btrfs raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear crct10dif_pclmul crc32_pclmul ghash_clmulni_intel aesni_intel aes_x86_64 lrw gf128mul glue_helper ablk_helper cryptd psmouse tg3 ptp pps_core hpsa scsi_transport_sas wmi fjes c13>] dump_stack+ 0x63/0x90 0f2>] warn_slowpath_ common+ 0x82/0xc0 18c>] warn_slowpath_ fmt+0x5c/ 0x80 d92>] ? ___ratelimit+ 0xa2/0xe0 f81>] skb_warn_ bad_offload+ 0xd1/0x120 155>] skb_checksum_ help+0x185/ 0x1a0 072>] checksum_ tg+0x22/ 0x29 [xt_CHECKSUM] e21>] ipt_do_ table+0x301/ 0x720 [ip_tables] e69>] ? ipt_do_ table+0x349/ 0x720 [ip_tables] e37>] ? enqueue_ entity+ 0x3a7/0xd20 0c9>] iptable_ mangle_ hook+0x39/ 0x107 [iptable_mangle] e48>] nf_iterate+ 0x68/0x80 ed3>] nf_hook_ slow+0x73/ 0xd0 42f>] ip_output+0xcf/0xe0 880>] ? __ip_flush_ pending_ frames. isra.42+ 0x90/0x90 b8b>] ip_local_ out+0x3b/ 0x50 eb4>] ip_queue_ xmit+0x154/ 0x380 2bb>] tcp_transmit_ skb+0x51b/ 0x980 8f6>] tcp_write_ xmit+0x1d6/ 0xf40 8f1>] __tcp_push_ pending_ frames+ 0x31/0xd0 7ec>] tcp_push+0xec/0x110 969>] tcp_sendmsg+ 0x739/0xb80 91b>] inet_sendmsg+ 0x6b/0xa0 b5e>] sock_sendmsg+ 0x3e/0x50 151>] SYSC_sendto+ 0x101/0x190 938>] ? syscall_ trace_enter_ phase1+ 0xc8/0x140 c7e>] SyS_sendto+0xe/0x10 fc8>] entry_SYSCALL_ 64_fastpath+ 0x1c/0xbb
[266447.178919] WARNING: CPU: 0 PID: 3343 at /build/
[266447.178924] qr-8a4bd2f3-04: caps=(0x0000018
[266447.178926] Modules linked in: kpatch_
[266447.178988] irqbypass sb_edac edac_core ipmi_si hpilo ipmi_msghandler serio_raw 8250_fintek lpc_ich shpchp acpi_power_meter ioatdma mac_hid dca ib_iser rdma_cm iw_cm ib_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp libiscsi_tcp libiscsi scsi_transport_
[266447.179045] CPU: 0 PID: 3343 Comm: haproxy Tainted: G W OE K 4.4.0-116-generic #140-Ubuntu
[266447.179048] Hardware name: HP ProLiant DL360 Gen9/ProLiant DL360 Gen9, BIOS P89 01/22/2018
[266447.179050] 0000000000000286 6f5737dc927d16dd ffff880a3646b818 ffffffff813ffc13
[266447.179055] ffff880a3646b860 ffffffff81d715e8 ffff880a3646b850 ffffffff810830f2
[266447.179059] ffff880064507ce8 ffff880b4ff12000 0000000000000001 ffff8808723268fc
[266447.179064] Call Trace:
[266447.179072] [<ffffffff813ff
[266447.179079] [<ffffffff81083
[266447.179084] [<ffffffff81083
[266447.179090] [<ffffffff81405
[266447.179094] [<ffffffff8173f
[266447.179100] [<ffffffff81740
[266447.179106] [<ffffffffc0524
[266447.179112] [<ffffffffc03a4
[266447.179117] [<ffffffffc03a4
[266447.179122] [<ffffffff810ba
[266447.179129] [<ffffffffc0498
[266447.179134] [<ffffffff81778
[266447.179137] [<ffffffff81778
[266447.179143] [<ffffffff81786
[266447.179148] [<ffffffff81785
[266447.179153] [<ffffffff81785
[266447.179158] [<ffffffff81785
[266447.179164] [<ffffffff8179e
[266447.179169] [<ffffffff8179e
[266447.179174] [<ffffffff8179f
[266447.179178] [<ffffffff8178d
[266447.179181] [<ffffffff81791
[266447.179186] [<ffffffff817bd
[266447.179192] [<ffffffff81724
[266447.179196] [<ffffffff81725
[266447.179203] [<ffffffff81003
[266447.179208] [<ffffffff81725
[266447.179215] [<ffffffff8184e
[266447.179218] ---[ end trace 4c054fa8d617289c ]---
Joseph, can we maybe get a new kernel to test ? The one in your last comment is gone.
Note that this isn't appearing a lot (~10 occurrences on a server with 3 days of uptime)