Kernel-panic with 3.13.0-64.104 generic kernel (BUG at net/core/skbuff:1290)

Bug #1497184 reported by Ioann on 2015-09-18
24
This bug affects 4 people
Affects Status Importance Assigned to Milestone
linux (Ubuntu)
High
Unassigned
Trusty
High
Unassigned
Vivid
High
Unassigned

Bug Description

Hi.
I get randomly kernel panic with 3.13-64 kernel after some hours of work.
This happens on 2 different computers with x86 and x86_kernel.
First(x86_64): Toshiba Tecra A10-181 Intel Core i3-370M 8Gb DDR3 ram
Second(x86): Desktop with AMD Athlon 64 x2 4800+ 4Gb DDR1 ram

In attached photo you can see panic on laptop.
Every time i other kernel panic message.

With kernel 3.13-63-generic kernel i have uptime more than week.

Techa host: uname -a
Linux nb-ubuntu-ioann 3.13.0-64-generic #104-Ubuntu SMP Wed Sep 9 12:36:12 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

Desktop host:
Linux ioann-old-desktop 3.13.0-64-generic #104-Ubuntu SMP Wed Sep 9 12:36:12 UTC 2015 x86 x86 x86 GNU/Linux

Description: Ubuntu 14.04.3 LTS
Release: 14.04

ProblemType: Bug
DistroRelease: Ubuntu 14.04
Package: linux-image-3.13.0-64-generic 3.13.0-64.104
ProcVersionSignature: Ubuntu 3.13.0-64.104-generic 3.13.11-ckt26
Uname: Linux 3.13.0-64-generic x86_64
ApportVersion: 2.14.1-0ubuntu3.13
Architecture: amd64
AudioDevicesInUse:
 USER PID ACCESS COMMAND
 /dev/snd/controlC0: unim95 2004 F.... pulseaudio
CRDA: Error: [Errno 2] No such file or directory: 'iw'
CurrentDesktop: XFCE
CurrentDmesg:
 [ 26.552940] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
 [ 52.301084] audit_printk_skb: 108 callbacks suppressed
 [ 52.301088] type=1400 audit(1442566720.780:48): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="/usr/lib/cups/backend/cups-pdf" pid=2283 comm="apparmor_parser"
 [ 52.301096] type=1400 audit(1442566720.780:49): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="/usr/sbin/cupsd" pid=2283 comm="apparmor_parser"
 [ 52.301679] type=1400 audit(1442566720.780:50): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="/usr/sbin/cupsd" pid=2283 comm="apparmor_parser"
Date: Fri Sep 18 11:02:21 2015
HibernationDevice: RESUME=UUID=7bb2db85-2152-4e84-ba1a-fb4a3c7c82ca
MachineType: TOSHIBA TECRA A11
ProcFB: 0 inteldrmfb
ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-3.13.0-64-generic root=UUID=314c867a-d6f1-4af2-b92b-510042a6940b ro quiet splash vt.handoff=7
RelatedPackageVersions:
 linux-restricted-modules-3.13.0-64-generic N/A
 linux-backports-modules-3.13.0-64-generic N/A
 linux-firmware 1.127.15
SourcePackage: linux
UpgradeStatus: No upgrade log present (probably fresh install)
dmi.bios.date: 06/27/2011
dmi.bios.vendor: TOSHIBA
dmi.bios.version: Version 3.20
dmi.board.asset.tag: 0000000000
dmi.board.name: Portable PC
dmi.board.vendor: TOSHIBA
dmi.board.version: Version A0
dmi.chassis.asset.tag: 0000000000
dmi.chassis.type: 10
dmi.chassis.vendor: TOSHIBA
dmi.chassis.version: Version 1.0
dmi.modalias: dmi:bvnTOSHIBA:bvrVersion3.20:bd06/27/2011:svnTOSHIBA:pnTECRAA11:pvrPTSE0E-0LP046GR:rvnTOSHIBA:rnPortablePC:rvrVersionA0:cvnTOSHIBA:ct10:cvrVersion1.0:
dmi.product.name: TECRA A11
dmi.product.version: PTSE0E-0LP046GR
dmi.sys.vendor: TOSHIBA

Ioann (yanunim95) wrote :
Ioann (yanunim95) wrote :

Here is second part of screen photo of panic.

This change was made by a bot.

Changed in linux (Ubuntu):
status: New → Confirmed
Andy Whitcroft (apw) on 2015-09-18
summary: - Kernel-panic with 3.13-64 generic kernel
+ Kernel-panic with 3.13-64 generic kernel (BUG at net/core/skbuff:1290
summary: - Kernel-panic with 3.13-64 generic kernel (BUG at net/core/skbuff:1290
+ Kernel-panic with 3.13-64 generic kernel (BUG at net/core/skbuff:1290)
summary: - Kernel-panic with 3.13-64 generic kernel (BUG at net/core/skbuff:1290)
+ Kernel-panic with 3.130-64.104 generic kernel (BUG at
+ net/core/skbuff:1290)
Andy Whitcroft (apw) on 2015-09-18
tags: added: regression-proposed
Luis Henriques (henrix) wrote :

I suspect the issue is related with commit

commit 738ac1ebb96d02e0d23bc320302a6ea94c612dec
Author: Herbert Xu <email address hidden>
Date: Mon Jul 13 16:04:13 2015 +0800

    net: Clone skb before setting peeked flag

This commit was included in Trusty 3.13.0-64.104, and introduces a use-after-free bug, fixed with upstream commit

commit a0a2a6602496a45ae838a96db8b8173794b5d398
Author: Herbert Xu <email address hidden>
Date: Tue Aug 4 15:42:47 2015 +0800

    net: Fix skb_set_peeked use-after-free bug

which is not in Trusty yet.

I've built a test kernel that includes this commit here: http://people.canonical.com/~henrix/lp1497184/amd64/
Could you please try to reproduce the issue using this kernel? Thanks

summary: - Kernel-panic with 3.130-64.104 generic kernel (BUG at
+ Kernel-panic with 3.13.0-64.104 generic kernel (BUG at
net/core/skbuff:1290)
Andy Whitcroft (apw) on 2015-09-18
Changed in linux (Ubuntu Trusty):
status: New → Confirmed
Changed in linux (Ubuntu Vivid):
status: New → Confirmed
Changed in linux (Ubuntu):
importance: Undecided → High
Changed in linux (Ubuntu Trusty):
importance: Undecided → High
Changed in linux (Ubuntu Vivid):
importance: Undecided → High
tags: added: bios-outdated-3.50
Ioann (yanunim95) wrote :

Hi. Got panic again. New panic messages are in attach.
It was direct while router updated my IP address.

Arjen Lentz (arjen-lentz) wrote :

Hi henrix - I'm trying your patched kernel, will update soon with results.
I too get the kernel panics in 3.13.0-64, but not in -63.
In -64 they are very frequent, so it shouldn't take long to see one again it re-occurs anyway with your patch in place.

Ioann (yanunim95) wrote :

henrix.

Now I'm using you kerenel. It made 24 hours uptime. So your patch is working.

Arjen Lentz (arjen-lentz) wrote :

I concur with loann, although mine has only been up for 2h20. With the unpatched kernel it would've long panicked.

Andy Whitcroft (apw) wrote :

@Arjen -- this package has not been released into the LTS it is only in -proposed for testing. This is exactly why we build and publish these kernels into -proposed and why they sit there for some time before being released into -updates. In this case the kernel carries a regression, one which has been detected before release to the general population. It would not be recommended to run any kernel from -proposed on a production system, though testing on less critical systems is always appreciated.

Arjen Lentz (arjen-lentz) wrote :

@apw / @henrix - ah, that's good. No worries.
That explains why only my laptop has had the issue, as indeed I do pull in updates from proposed there and that's fine - happy to help catch stuff early. So, all is as it should be! awesome. thanks!

Luis Henriques (henrix) on 2015-09-22
Changed in linux (Ubuntu Vivid):
status: Confirmed → Fix Committed
Changed in linux (Ubuntu Trusty):
status: Confirmed → Fix Committed
Changed in linux (Ubuntu):
status: Confirmed → Invalid
aleandro (aleandrodasilva) wrote :

Hello. I updated with the proposed updates my ununtu 14.04 and installed the 3.13.0-64.104 kernel on the 24 of september. I got a first kernel panic after many hours of work and after reboot even after about an hour. I removed this kernel and installed the 3.13.0-65.-generic which was in the repositories of ubuntu. I hope this new one is not affected by that bug.

Launchpad Janitor (janitor) wrote :
Download full text (5.4 KiB)

This bug was fixed in the package linux - 3.13.0-65.105

---------------
linux (3.13.0-65.105) trusty; urgency=low

  [ Brad Figg ]

  * Release Tracking Bug
    - LP: #1498108

  [ Upstream Kernel Changes ]

  * net: Fix skb_set_peeked use-after-free bug
      - LP: #1497184

linux (3.13.0-64.104) trusty; urgency=low

  [ Luis Henriques ]

  * Release Tracking Bug
    - LP: #1493803

  [ Chris J Arges ]

  * [Config] DEFAULT_IOSCHED="deadline" for ppc64el
    - LP: #1469829

  [ Upstream Kernel Changes ]

  * tcp: fix recv with flags MSG_WAITALL | MSG_PEEK
    - LP: #1486146
  * libceph: abstract out ceph_osd_request enqueue logic
    - LP: #1488035
  * libceph: resend lingering requests with a new tid
    - LP: #1488035
  * n_tty: Refactor input_available_p() by call site
    - LP: #1397976
  * tty: Fix pty master poll() after slave closes v2
    - LP: #1397976
  * md: use kzalloc() when bitmap is disabled
    - LP: #1493305
  * ata: pmp: add quirk for Marvell 4140 SATA PMP
    - LP: #1493305
  * libata: add ATA_HORKAGE_BROKEN_FPDMA_AA quirk for HP 250GB SATA disk
    VB0250EAVER
    - LP: #1493305
  * libata: add ATA_HORKAGE_NOTRIM
    - LP: #1493305
  * libata: force disable trim for SuperSSpeed S238
    - LP: #1493305
  * libata: increase the timeout when setting transfer mode
    - LP: #1493305
  * libata: Do not blacklist M510DC
    - LP: #1493305
  * mac80211: clear subdir_stations when removing debugfs
    - LP: #1493305
  * ALSA: hda - Add new GPU codec ID 0x10de007d to snd-hda
    - LP: #1493305
  * drm: Stop resetting connector state to unknown
    - LP: #1493305
  * usb: dwc3: Reset the transfer resource index on SET_INTERFACE
    - LP: #1493305
  * usb: xhci: Bugfix for NULL pointer deference in xhci_endpoint_init()
    function
    - LP: #1493305
  * xhci: Calculate old endpoints correctly on device reset
    - LP: #1493305
  * xhci: report U3 when link is in resume state
    - LP: #1493305
  * xhci: prevent bus_suspend if SS port resuming in phase 1
    - LP: #1493305
  * xhci: do not report PLC when link is in internal resume state
    - LP: #1493305
  * USB: OHCI: Fix race between ED unlink and URB submission
    - LP: #1493305
  * usb-storage: ignore ZTE MF 823 card reader in mode 0x1225
    - LP: #1493305
  * blkcg: fix gendisk reference leak in blkg_conf_prep()
    - LP: #1493305
  * tile: use free_bootmem_late() for initrd
    - LP: #1493305
  * Input: usbtouchscreen - avoid unresponsive TSC-30 touch screen
    - LP: #1493305
  * md/raid1: fix test for 'was read error from last working device'.
    - LP: #1493305
  * mmc: omap_hsmmc: Fix DTO and DCRC handling
    - LP: #1493305
  * isdn/gigaset: reset tty->receive_room when attaching ser_gigaset
    - LP: #1493305
  * mmc: sdhci-pxav3: fix platform_data is not initialized
    - LP: #1493305
  * mmc: block: Add missing mmc_blk_put() in power_ro_lock_show()
    - LP: #1493305
  * mmc: sdhci-esdhc: Make 8BIT bus work
    - LP: #1493305
  * bonding: correctly handle bonding type change on enslave failure
    - LP: #1493305
  * net: Clone skb before setting peeked flag
    - LP: #1493305
  * bridge: mdb: fix double add notification
    - LP: #1493305
  * usb: gadget: mv_udc_c...

Read more...

Changed in linux (Ubuntu Trusty):
status: Fix Committed → Fix Released
Launchpad Janitor (janitor) wrote :
Download full text (13.2 KiB)

This bug was fixed in the package linux - 3.19.0-30.33

---------------
linux (3.19.0-30.33) vivid; urgency=low

  [ Brad Figg ]

  * Release Tracking Bug
    - LP: #1498065
  * Revert "[Config]
    MFD_INTEL_LPSS/MFD_INTEL_LPSS_ACPI/MFD_INTEL_LPSS_PCI=m"
    - LP: #1498137
  * [Config] Disable the MFD_INTEL_LPSS* driver

linux (3.19.0-30.32) vivid; urgency=low

  [ Brad Figg ]

  * Release Tracking Bug
    - LP: #1498065

  [ Upstream Kernel Changes ]

  * net: Fix skb_set_peeked use-after-free bug
    - LP: #1497184

linux (3.19.0-29.31) vivid; urgency=low

  [ Luis Henriques ]

  * Release Tracking Bug
    - LP: #1493902

  [ Ander Conselvan de Oliveira ]

  * SAUCE: i915_bpo: Set ddi_pll_sel in DP MST path
    - LP: #1483320

  [ Chris J Arges ]

  * [Config] DEFAULT_IOSCHED="deadline" for ppc64el
    - LP: #1469829

  [ Chris Wilson ]

  * SAUCE: i915_bpo: drm/i915: Flag the execlists context object as dirty
    after every use
    - LP: #1489501

  [ Daniel Vetter ]

  * SAUCE: i915_bpo: drm/i915: Only dither on 6bpc panels
    - LP: #1489501

  [ David Henningsson ]

  * SAUCE: drm/i915: Add audio pin sense / ELD callback
    - LP: #1490895
  * SAUCE: drm/i915: Call audio pin/ELD notify function
    - LP: #1490895
  * SAUCE: ubuntu/i915: Call audio pin/ELD notify function
    - LP: #1490895
  * SAUCE: ALSA: hda - Add "hdac_acomp" global variable
    - LP: #1490895
  * SAUCE: ALSA: hda - allow codecs to access the i915 pin/ELD callback
    - LP: #1490895
  * SAUCE: ALSA: hda - Wake the codec up on pin/ELD notify events
    - LP: #1490895

  [ Jani Nikula ]

  * SAUCE: i915_bpo: Revert "drm/i915: Allow parsing of variable size child
    device entries from VBT"
    - LP: #1489501

  [ Maarten Lankhorst ]

  * SAUCE: i915_bpo: drm/i915: calculate primary visibility changes instead
    of calling from set_config
    - LP: #1489501
  * SAUCE: i915_bpo: drm/i915: Commit planes on each crtc separately.
    - LP: #1489501

  [ Thulasimani,Sivakumar ]

  * SAUCE: i915_bpo: Revert "drm/i915: Add eDP intermediate frequencies for
    CHV"
    - LP: #1489501
  * SAUCE: i915_bpo: drm/i915: remove HBR2 from chv supported list
    - LP: #1489501
  * SAUCE: i915_bpo: drm/i915: Avoid TP3 on CHV
    - LP: #1489501

  [ Timo Aaltonen ]

  * Revert "SAUCE: i915_bpo: drm/i915: Allow parsing of variable size child
    device entries from VBT, addendum v2"
    - LP: #1489501
  * SAUCE: Migrate Broadwell to i915_bpo.
    - LP: #1483320

  [ Upstream Kernel Changes ]

  * tcp: fix recv with flags MSG_WAITALL | MSG_PEEK
    - LP: #1486146
  * powerpc/powernv: Fix the overflow of OPAL message notifiers head array
    - LP: #1487085
  * xhci: call BIOS workaround to enable runtime suspend on Intel Braswell
    - LP: #1489292
  * PM / QoS: Make it possible to expose device latency tolerance to
    userspace
    - LP: #1488395
  * ACPI / PM: Attach ACPI power domain only once
    - LP: #1488395
  * Driver core: wakeup the parent device before trying probe
    - LP: #1488395
  * klist: implement klist_prev()
    - LP: #1488395
  * driver core: implement device_for_each_child_reverse()
    - LP: #1488395
  * mfd: make mfd_remove_devices() iterate in reverse order
    ...

Changed in linux (Ubuntu Vivid):
status: Fix Committed → Fix Released
Matt Riedemann (mriedem) wrote :
Download full text (6.5 KiB)

Ever since we started picking up the 3.13.0-65 kernel in the openstack CI system on 9/29 we've been seeing lots of random failures across 3 branches (juno/kilo/master) with ebtables, details are in:

https://bugs.launchpad.net/nova/+bug/1501558

An example failure:

http://logs.openstack.org/62/229962/1/check/gate-tempest-dsvm-full/28bb6b6/logs/screen-n-cpu.txt.gz?level=TRACE#_2015-10-01_17_39_59_1872015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] Traceback (most recent call last):
2015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] File "/opt/stack/new/nova/nova/compute/manager.py", line 2155, in _build_resources
2015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] yield resources
2015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] File "/opt/stack/new/nova/nova/compute/manager.py", line 2009, in _build_and_run_instance
2015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] block_device_info=block_device_info)
2015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 2444, in spawn
2015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] block_device_info=block_device_info)
2015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 4516, in _create_domain_and_network
2015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] xml, pause=pause, power_on=power_on)
2015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 4446, in _create_domain
2015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] guest.launch(pause=pause)
2015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] File "/opt/stack/new/nova/nova/virt/libvirt/guest.py", line 141, in launch
2015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] self._encoded_xml, errors='ignore')
2015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 195, in __exit__
2015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] six.reraise(self.type_, self.value, self.tb)
2015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] File "/opt/stack/new/nova/nova/virt/libvirt/guest.py", line 136, in launch
2015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] return self._domain.cr...

Read more...

Stefan Bader (smb) wrote :

Matt, please open a new bug report. As your problem neither does seem to be unexpected complete hangs or crashes, nor isolated to 3.13.0-64.104 this unlikely is the same issue. Ideally run "ubuntu-bug linux" from an affected machine. Note also that 3.13.0-64.104 never left the proposed stage. So only people that have the proposed archive enabled have ever seen that kernel version. Thanks.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers