network problems after update to kernel 3.2.0-65 - thunderbird/imap/dovecot

[grufo]

Since we installed the latest kernel update on our firewall this weekend (3.2.0-65.98) we got massive problems connecting from thunderbird (windows client) to an external imap-server (dovecot with tls). With our Linux client we got no problems.

Opening or sending mails with attachments which was done in a second before the update, took some minutes after the update to kernel 3.2.0-65!

Installing the kernel version 3.2.0-64 on our firewall system fixed the problem, also the newer kernel linux-generic-lts-trusty did fix the problem.

[Brad Figg (brad-figg)]

This bug is missing log files that will aid in diagnosing the problem. From a terminal window please run:

apport-collect 1337281

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

[grufo (ml-grufo)]

No Logfiles...

Maybe this could help a litte bit...:?
Two times seq 1984 with same chksum - is that normal?

09:11:16.453942 IP (tos 0x0, ttl 127, id 9250, offset 0, flags [DF], proto TCP (6), length 114)
    192.168.98.7.53479 > 192.168.1.21.993: Flags [P.], cksum 0x04c1 (correct), seq 1910:1984, ack 19602, win 16286, length 74
09:11:16.467028 IP (tos 0x0, ttl 127, id 9284, offset 0, flags [DF], proto TCP (6), length 162)
    192.168.98.7.53479 > 192.168.1.21.993: Flags [P.], cksum 0xdabb (correct), seq 1984:2106, ack 19692, win 16263, length 122
09:11:16.773648 IP (tos 0x0, ttl 127, id 9452, offset 0, flags [DF], proto TCP (6), length 162)
    192.168.98.7.53479 > 192.168.1.21.993: Flags [P.], cksum 0xdabb (correct), seq 1984:2106, ack 19692, win 16263, length 122
09:11:16.774403 IP (tos 0x0, ttl 127, id 9453, offset 0, flags [DF], proto TCP (6), length 52)
    192.168.98.7.53479 > 192.168.1.21.993: Flags [.], cksum 0x9c66 (correct), seq 2106, ack 19692, win 16263, options [nop,nop,sack 1 {25532:26980}], length 0
09:11:16.775437 IP (tos 0x0, ttl 127, id 9454, offset 0, flags [DF], proto TCP (6), length 52)
    192.168.98.7.53479 > 192.168.1.21.993: Flags [.], cksum 0x9610 (correct), seq 2106, ack 21152, win 16425, options [nop,nop,sack 1 {25532:26980}], length 0
09:11:16.776425 IP (tos 0x0, ttl 127, id 9455, offset 0, flags [DF], proto TCP (6), length 52)
    192.168.98.7.53479 > 192.168.1.21.993: Flags [.], cksum 0x905c (correct), seq 2106, ack 22612, win 16425, options [nop,nop,sack 1 {25532:26980}], length 0
09:11:16.777020 IP (tos 0x0, ttl 127, id 9456, offset 0, flags [DF], proto TCP (6), length 52)
    192.168.98.7.53479 > 192.168.1.21.993: Flags [.], cksum 0x8e54 (correct), seq 2106, ack 22612, win 16425, options [nop,nop,sack 1 {25532:27500}], length 0
09:11:16.777792 IP (tos 0x0, ttl 127, id 9457, offset 0, flags [DF], proto TCP (6), length 52)
    192.168.98.7.53479 > 192.168.1.21.993: Flags [.], cksum 0x88a0 (correct), seq 2106, ack 24072, win 16425, options [nop,nop,sack 1 {25532:27500}], length 0
09:11:16.778118 IP (tos 0x0, ttl 127, id 9458, offset 0, flags [DF], proto TCP (6), length 40)
    192.168.98.7.53479 > 192.168.1.21.993: Flags [.], cksum 0x0acb (correct), seq 2106, ack 27500, win 16425, length 0
09:11:16.859798 IP (tos 0x0, ttl 127, id 9472, offset 0, flags [DF], proto TCP (6), length 114)
    192.168.98.7.53479 > 192.168.1.21.993: Flags [P.], cksum 0x182a (correct), seq 2106:2180, ack 27500, win 16425, length 74
09:11:17.060652 IP (tos 0x0, ttl 127, id 9684, offset 0, flags [DF], proto TCP (6), length 40)
    192.168.98.7.53479 > 192.168.1.21.993: Flags [.], cksum 0x0a4a (correct), seq 2180, ack 27574, win 16406, length 0

[Charles Jordan (photoman5311)]

I experienced this myself but it seems to be all outbound NAT traffic.
I set grub to boot and older kernel and everything went back to normal.
From a workstation behind the firewall, do a speed test at speakeasy.net and notice that the download speeds are really not affected but when it goes to test your upload it can't even complete the test. My kernel version was 3.2.0-65.
I hope they fix this soon - it has caused a lot of problems for me and my customers.

[grufo (ml-grufo)]

I agree - it seems to be all about outbound NAT traffic!

[Dirk Bonenkamp (dirk-proactive)]

I have similar trouble. We have a Ubuntu NAT box as firewall, after installing 3.2.0-65-generic most of our clients experienced issues. I could not reproduce what this exactly was caused by, but reverting to 3.2.0-64-generic solved the problem.

[Vitas (vitas)]

Sorry of not fluently english...

This bug (or feature?) costs us 24 hours working in our routing center (2 Linux NAT boxes, 2 all-purpose Linux routers, ~ 15 VLANS), beating heads to the wall, changing a half of VLANs structure, checking a half of switches and about 200 angry phone calls from customers :). The last day to holidays :).
We think, the kernel 3.2.0-65 was made by Microsoft :). Reverting to 3.2.0-64 solved this NAT problems

[lmhd (maylein)]

Same problem as descibed here: http://ubuntuforums.org/showthread.php?t=2231953

[Peter Thomassen (mail-peter-thomassen)]

I have the impression that packets larger than the MTU are not treated correctly. It would be worthwhile if someone who knows more about this did some MTU and IMCP "fragmentation needed" functionality tests.

The buggy kernel update seems to contains this patch: http://patchwork.ozlabs.org/patch/345509/
... which is related to my suspicion. It might be useful to double-check that patch for side-effects.

[Peter Thomassen (mail-peter-thomassen)]

http://lists.openwall.net/netdev/2014/06/10/108 confirms and has information about this issue in version 3.4.92. Hope it helps

[lmhd (maylein)]

Is this bug fixed in the 3.2.0-67.101?
This bug should be treated as an urgent security problem. It prevents further kernel updates espacially on firewall machines.

[Tjakko Tjakkes (tjakko-tjakkes)]

Problem also still occurs on 3.2.0.-67.101

[YAMAMOTO Hirotaka (ymmt2005)]

It seems that the patch "net: ipv4: ip_forward: fix inverted local_df" need to be
applied with this netfilter fix:

http://kernel.suse.com/cgit/kernel/commit/?id=895162b1101b3ea5db08ca6822ae9672717efec0

[YAMAMOTO Hirotaka (ymmt2005)]

Update to #12.

We built a kernel with the patch and tried it, but the problem still occurs.

Next, we will revert "net: ipv4: ip_forward: fix inverted local_df test" patch.

[YAMAMOTO Hirotaka (ymmt2005)]

Update to #12 & #13.

Reverting "net: ipv4: ip_forward: fix inverted local_df test" resolves the problem.

For sure, this is it:
http://kernel.ubuntu.com/git?p=ubuntu/ubuntu-precise.git;a=commitdiff;h=5be90996677a55d3c2371debce40c45066587c9b

[YAMAMOTO Hirotaka (ymmt2005)]

Debian BTS:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=754173

They reverted two commits to fix this problem.

[Tim Gardner (timg-tpi)]

Proposed reverts on the kteam list: https://lists.ubuntu.com/archives/kernel-team/2014-July/046494.html

[Drew (drewscogin)]

I can confirm what YAMAMOTO Hirotaka (ymmt2005) said. I upgraded from 3.2.0-64-generic to 3.2.0-67 and had lots of issues with Windows users connecting to network shares on a different subnet among other things. I rolled back to 3.2.0-64 and the problems disappeared. All Linux and Mac workstations saw no issues whatsoever.

Are there any fixes for this that can be accomplished within /etc/sysctl.conf ? Thank you!

[Brad Figg (brad-figg)]

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-precise' to 'verification-done-precise'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

[Brad Figg (brad-figg)]

@YAMAMOTO,

Can you verify that the kernel in -proposed fixes this issue?

Thanks

[YAMAMOTO Hirotaka (ymmt2005)]

I will verify the kernel tomorrow (it is midnight in Japan).

[YAMAMOTO Hirotaka (ymmt2005)]

@Brad

Verified that the proposed kernel works like a charm!

Thank you.

[Launchpad Janitor (janitor)]

This bug was fixed in the package linux - 3.2.0-68.102

---------------
linux (3.2.0-68.102) precise; urgency=low

  [ Joseph Salisbury ]

  * Release Tracking Bug
    - LP: #1355387

  [ Joseph Salisbury ]

  * [Config] updateconfigs after Linux 3.2.62 update

  [ Upstream Kernel Changes ]

  * Revert "net: ipv4: ip_forward: fix inverted local_df test"
    - LP: #1337281
  * Revert "net: ip, ipv6: handle gso skbs in forwarding path"
    - LP: #1337281
  * Yama: handle 32-bit userspace prctl
    - LP: #1338883
  * mm: highmem: don't treat PKMAP_ADDR(LAST_PKMAP) as a highmem address
    - LP: #1348572
  * bluetooth: hci_ldisc: fix deadlock condition
    - LP: #1348572
  * genirq: Sanitize spurious interrupt detection of threaded irqs
    - LP: #1348572
  * UBIFS: fix an mmap and fsync race condition
    - LP: #1348572
  * Input: synaptics - add min/max quirk for the ThinkPad W540
    - LP: #1348572
  * ACPI: Fix conflict between customized DSDT and DSDT local copy
    - LP: #1348572
  * HID: core: fix validation of report id 0
    - LP: #1348572
  * IB/srp: Fix a sporadic crash triggered by cable pulling
    - LP: #1348572
  * reiserfs: drop vmtruncate
    - LP: #1348572
  * reiserfs: call truncate_setsize under tailpack mutex
    - LP: #1348572
  * ARM: imx: fix error handling in ipu device registration
    - LP: #1348572
  * matroxfb: perform a dummy read of M_STATUS
    - LP: #1348572
  * USB: Avoid runtime suspend loops for HCDs that can't handle
    suspend/resume
    - LP: #1348572
  * ARM: 8051/1: put_user: fix possible data corruption in put_user
    - LP: #1348572
  * Input: synaptics - T540p - unify with other LEN0034 models
    - LP: #1348572
  * mac80211: fix IBSS join by initializing last_scan_completed
    - LP: #1348572
  * drm/i915: s/DRM_ERROR/DRM_DEBUG in i915_gem_execbuffer.c
    - LP: #1348572
  * drm/i915: Only copy back the modified fields to userspace from
    execbuffer
    - LP: #1348572
  * ahci: add PCI ID for Marvell 88SE91A0 SATA Controller
    - LP: #1348572
  * ext4: fix zeroing of page during writeback
    - LP: #1348572
  * ext4: fix wrong assert in ext4_mb_normalize_request()
    - LP: #1348572
  * IB/qib: Fix port in pkey change event
    - LP: #1348572
  * IB/ipath: Translate legacy diagpkt into newer extended diagpkt
    - LP: #1348572
  * USB: sierra: fix AA deadlock in open error path
    - LP: #1348572
  * USB: sierra: fix urb and memory leak in resume error path
    - LP: #1348572
  * USB: sierra: fix urb and memory leak on disconnect
    - LP: #1348572
  * USB: sierra: fix remote wakeup
    - LP: #1348572
  * USB: option: fix runtime PM handling
    - LP: #1348572
  * USB: usb_wwan: fix urb leak in write error path
    - LP: #1348572
  * USB: usb_wwan: fix race between write and resume
    - LP: #1348572
  * USB: usb_wwan: fix write and suspend race
    - LP: #1348572
  * USB: usb_wwan: fix urb leak at shutdown
    - LP: #1348572
  * USB: usb_wwan: fix potential blocked I/O after resume
    - LP: #1348572
  * USB: cdc-acm: fix write and suspend race
    - LP: #1348572
  * USB: cdc-acm: fix write and resume race
    - LP: #1348572
  * USB: cdc-acm: fix broken runtime suspend
    - LP: #1348572
  * USB: ...