Networking Hang - sch_generic.c:261

Bug #554694 reported by IAmACarpetLicker
12
This bug affects 2 people
Affects Status Importance Assigned to Milestone
linux (Ubuntu)
Expired
Undecided
Unassigned

Bug Description

This problem was present in the previous release of Ubuntu, 9.10. But with every update has slowly gotten worse.

It started with the networking just giving out under high load, and now it seems to cut out when moving any file over 250MB.

I think it may be something to do with scheduling of packets when at high speed, this is because it can do slow transfers over 24/48 hours spanning many GB, but when it comes to trying to retrieve files from its disks at high speeds, it dies. With it being a headless box, I have to hard reset, and this is what is contained in the logs:

Apr 1 23:37:14 download-server kernel: [ 718.011280] ------------[ cut here
]------------
Apr 1 23:37:14 download-server kernel: [ 718.011305] WARNING: at
/build/buildd/linux-2.6.32/net/sched/sch_generic.c:261
dev_watchdog+0x262/0x270()
Apr 1 23:37:14 download-server kernel: [ 718.011312] Hardware name:
EasyNote_MX37-U-004
Apr 1 23:37:14 download-server kernel: [ 718.011318] NETDEV WATCHDOG: eth0
(sis190): transmit queue 0 timed out
Apr 1 23:37:14 download-server kernel: [ 718.011323] Modules linked in: vmnet
parport_pc vsock vmci vmmon ppdev snd_hda_codec_realtek nfsd exportfs nfs lockd
nfs_acl auth_rpcgss sunrpc arc4 snd_hda_intel snd_hda_codec snd_hwdep ath5k
snd_pcm snd_timer fbcon tileblit font bitblit softcursor mac80211 ath snd lp
video output soundcore snd_page_alloc vga16fb vgastate sis190 cfg80211 shpchp
mii asus_laptop led_class sis_agp parport sata_sis
Apr 1 23:37:14 download-server kernel: [ 718.011425] Pid: 0, comm: swapper
Not tainted 2.6.32-17-server #26-Ubuntu
Apr 1 23:37:14 download-server kernel: [ 718.011430] Call Trace:
Apr 1 23:37:14 download-server kernel: [ 718.011435] <IRQ>
[<ffffffff81066ceb>] warn_slowpath_common+0x7b/0xc0
Apr 1 23:37:14 download-server kernel: [ 718.011456] [<ffffffff81066d91>]
warn_slowpath_fmt+0x41/0x50
Apr 1 23:37:14 download-server kernel: [ 718.011465] [<ffffffff81489842>]
dev_watchdog+0x262/0x270
Apr 1 23:37:14 download-server kernel: [ 718.011476] [<ffffffff81080767>] ?
insert_work+0x77/0xc0
Apr 1 23:37:14 download-server kernel: [ 718.011488] [<ffffffff810397a9>] ?
default_spin_lock_flags+0x9/0x10
Apr 1 23:37:14 download-server kernel: [ 718.011497] [<ffffffff814895e0>] ?
dev_watchdog+0x0/0x270
Apr 1 23:37:14 download-server kernel: [ 718.011505] [<ffffffff81077417>]
run_timer_softirq+0x197/0x340
Apr 1 23:37:14 download-server kernel: [ 718.011516] [<ffffffff810943a0>] ?
tick_sched_timer+0x0/0xc0
Apr 1 23:37:14 download-server kernel: [ 718.011525] [<ffffffff8108f113>] ?
ktime_get+0x63/0xe0
Apr 1 23:37:14 download-server kernel: [ 718.011534] [<ffffffff8106e227>]
__do_softirq+0xb7/0x1e0
Apr 1 23:37:14 download-server kernel: [ 718.011542] [<ffffffff81093f8a>] ?
tick_program_event+0x2a/0x30
Apr 1 23:37:14 download-server kernel: [ 718.011551] [<ffffffff810142ec>]
call_softirq+0x1c/0x30
Apr 1 23:37:14 download-server kernel: [ 718.011559] [<ffffffff81015cb5>]
do_softirq+0x65/0xa0
Apr 1 23:37:14 download-server kernel: [ 718.011566] [<ffffffff8106e0c5>]
irq_exit+0x85/0x90
Apr 1 23:37:14 download-server kernel: [ 718.011576] [<ffffffff8155c021>]
smp_apic_timer_interrupt+0x71/0x9c
Apr 1 23:37:14 download-server kernel: [ 718.011584] [<ffffffff81013cb3>]
apic_timer_interrupt+0x13/0x20
Apr 1 23:37:14 download-server kernel: [ 718.011589] <EOI>
[<ffffffff8130c7ce>] ? acpi_idle_enter_simple+0x117/0x14b
Apr 1 23:37:14 download-server kernel: [ 718.011606] [<ffffffff8130c7c7>] ?
acpi_idle_enter_simple+0x110/0x14b
Apr 1 23:37:14 download-server kernel: [ 718.011617] [<ffffffff81448c77>] ?
cpuidle_idle_call+0xa7/0x140
Apr 1 23:37:14 download-server kernel: [ 718.011627] [<ffffffff81011e63>] ?
cpu_idle+0xb3/0x110
Apr 1 23:37:14 download-server kernel: [ 718.011636] [<ffffffff8154ee91>] ?
start_secondary+0xa8/0xaa
Apr 1 23:37:14 download-server kernel: [ 718.011643] ---[ end trace
62bcf8b592c12c43 ]---

There is a kernel bugzilla open for this: https://bugzilla.kernel.org/show_bug.cgi?id=15398
There doesn't seem to be any progress though.

The version of Ubuntu I am using is 10.04 BETA 2 x86_64.
It isn't just the 64bit kernel this happens with as it happened to me on the x86 kernel also.

ProblemType: Bug
DistroRelease: Ubuntu 10.04
Package: linux-image-2.6.32-19-server 2.6.32-19.28
Regression: Yes
Reproducible: Yes
ProcVersionSignature: Ubuntu 2.6.32-19.28-server 2.6.32.10+drm33.1
Uname: Linux 2.6.32-19-server x86_64
AlsaVersion: Advanced Linux Sound Architecture Driver Version 1.0.21.
AplayDevices: Error: [Errno 2] No such file or directory
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/by-path', '/dev/snd/controlC0', '/dev/snd/hwC0D0', '/dev/snd/pcmC0D0c', '/dev/snd/pcmC0D0p', '/dev/snd/timer'] failed with exit code 1:
CRDA: Error: [Errno 2] No such file or directory
Card0.Amixer.info: Error: [Errno 2] No such file or directory
Card0.Amixer.values: Error: [Errno 2] No such file or directory
Date: Sat Apr 3 17:38:28 2010
HibernationDevice: RESUME=/dev/sda2
InstallationMedia: Error: [Errno 13] Permission denied: '/var/log/installer/media-info'
Lsusb:
 Bus 003 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
 Bus 002 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
 Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
MachineType: Packard Bell BV EasyNote_MX37-U-004
ProcCmdLine: BOOT_IMAGE=/vmlinuz-2.6.32-19-server root=UUID=837814bc-e909-4696-a34a-9c30fb295b36 ro quiet splash
ProcEnviron:
 LANG=en_GB.UTF-8
 SHELL=/bin/bash
RelatedPackageVersions: linux-firmware 1.33
RfKill:
 0: phy0: Wireless LAN
  Soft blocked: no
  Hard blocked: no
SourcePackage: linux
dmi.bios.date: 10/24/2007
dmi.bios.vendor: American Megatrends Inc.
dmi.bios.version: 208
dmi.board.asset.tag: ATN12345678901234567
dmi.board.name: T12C
dmi.board.vendor: PACKARD BELL BV
dmi.board.version: 1.0
dmi.chassis.type: 10
dmi.chassis.vendor: PACKARD BELL BV
dmi.modalias: dmi:bvnAmericanMegatrendsInc.:bvr208:bd10/24/2007:svnPackardBellBV:pnEasyNote_MX37-U-004:pvrPC10E00301:rvnPACKARDBELLBV:rnT12C:rvr1.0:cvnPACKARDBELLBV:ct10:cvr:
dmi.product.name: EasyNote_MX37-U-004
dmi.product.version: PC10E00301
dmi.sys.vendor: Packard Bell BV

Revision history for this message
IAmACarpetLicker (sam-feel-the-darkness) wrote :
Revision history for this message
IAmACarpetLicker (sam-feel-the-darkness) wrote :

I've been exploring what makes the networking error occur, and I've noticed that it happens more when transferring files via either AFP using netatalk or NFSv4 using the nfs-kernel-server package. The speeds when copying are incredible, even over wireless, which is why I try to use them.
When I copy over Samba, it is less frequent and tends to happen when it is very big files, such as BlueRay images of around 10-15GB. But otherwise it is quite stable. The only difference is, the speeds are a LOT slower, almost half the speed.

Revision history for this message
IAmACarpetLicker (sam-feel-the-darkness) wrote :

Sorry, I forgot to add, I have tried it with the latest built mainline kernel and I'm still getting the same thing, although it does seem to be going longer without cutting out, but still can't copy a full 500MB file without it happening.

Revision history for this message
Jeremy Foshee (jeremyfoshee) wrote :

Hi IAmACarpetLicker,

If you could also please test the latest upstream kernel available that would be great. It will allow additional upstream developers to examine the issue. Refer to https://wiki.ubuntu.com/KernelMainlineBuilds . Once you've tested the upstream kernel, please remove the 'needs-upstream-testing' tag. This can be done by clicking on the yellow pencil icon next to the tag located at the bottom of the bug description and deleting the 'needs-upstream-testing' text. Please let us know your results.

Thanks in advance.

    [This is an automated message. Apologies if it has reached you inappropriately; please just reply to this message indicating so.]

tags: added: kj-triage
Changed in linux (Ubuntu):
status: New → Incomplete
Revision history for this message
IAmACarpetLicker (sam-feel-the-darkness) wrote :

I've been using the upstream kernel for a while to try and test this.
I was previously on 2.6.34rc1 and I have tonight moved to 2.6.34rc6.
I will let you know the results on rc6 as and when the next crash occurs, or as the case may be, doesn't occur.

Here is a full start to finish log from tonight of this happening:
http://notes.sassybox.net/index.php/view/66511304

It has been happening quite frequently tonight, about once every 25mins.

I have just noticed that on the hardware I am using, it supports Gigabit Ethernet. I wonder with it being what looks like a scheduling problem, it has something to do with it running at 100Mbit/sec rather than full capacity. I am hoping to get a gigabit router this month so should be able to check this thought, and I will let you know the results.

tags: removed: needs-upstream-testing
Revision history for this message
IAmACarpetLicker (sam-feel-the-darkness) wrote :

Just got my first crash on the rc6 kernel.

Is is very much still an issue.

Full kernel log from system startup on new kernel:
http://notes.sassybox.net/index.php/view/25960536

Revision history for this message
Maci (gergo-nemeth) wrote :
Download full text (3.8 KiB)

This bug affects me also. This used to take effect, when I have huge traffic.

The log:
Jun 21 13:02:05 picimaci-laptop kernel: [21553.040063] WARNING: at /build/buildd/linux-2.6.32/net/sched/sch_generic.c:261 dev_watchdog+0x262/0x270()
Jun 21 13:02:05 picimaci-laptop kernel: [21553.040068] Hardware name: F5SL
Jun 21 13:02:05 picimaci-laptop kernel: [21553.040073] NETDEV WATCHDOG: eth0 (sis190): transmit queue 0 timed out
Jun 21 13:02:05 picimaci-laptop kernel: [21553.040077] Modules linked in: binfmt_misc ppdev vboxnetadp vboxnetflt vboxdrv snd_hda_codec_si3054 snd_hda_codec_realtek snd_hda_intel snd_hda_codec snd_hwdep snd_pcm_oss snd_mixer_oss snd_pcm arc4 snd_seq_dummy snd_seq_oss snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq snd_timer fbcon ath5k snd_seq_device tileblit mac80211 font joydev sis190 snd bitblit ath fglrx(P) soundcore softcursor mii sis_agp snd_page_alloc cfg80211 vga16fb uvcvideo vgastate videodev psmouse usbhid vhba v4l1_compat serio_raw hid v4l2_compat_ioctl32 video output asus_laptop led_class lp parport tcp_illinois jfs usb_storage sata_sis
Jun 21 13:02:05 picimaci-laptop kernel: [21553.040176] Pid: 0, comm: swapper Tainted: P 2.6.32-22-generic #36-Ubuntu
Jun 21 13:02:05 picimaci-laptop kernel: [21553.040181] Call Trace:
Jun 21 13:02:05 picimaci-laptop kernel: [21553.040185] <IRQ> [<ffffffff81066d0b>] warn_slowpath_common+0x7b/0xc0
Jun 21 13:02:05 picimaci-laptop kernel: [21553.040203] [<ffffffff81066db1>] warn_slowpath_fmt+0x41/0x50
Jun 21 13:02:05 picimaci-laptop kernel: [21553.040210] [<ffffffff81476612>] dev_watchdog+0x262/0x270
Jun 21 13:02:05 picimaci-laptop kernel: [21553.040220] [<ffffffff81094378>] ? tick_dev_program_event+0x68/0xd0
Jun 21 13:02:05 picimaci-laptop kernel: [21553.040229] [<ffffffff810397a9>] ? default_spin_lock_flags+0x9/0x10
Jun 21 13:02:05 picimaci-laptop kernel: [21553.040238] [<ffffffff81540cfe>] ? _spin_lock+0xe/0x20
Jun 21 13:02:05 picimaci-laptop kernel: [21553.040244] [<ffffffff814763b0>] ? dev_watchdog+0x0/0x270
Jun 21 13:02:05 picimaci-laptop kernel: [21553.040252] [<ffffffff81077697>] run_timer_softirq+0x197/0x340
Jun 21 13:02:05 picimaci-laptop kernel: [21553.040259] [<ffffffff81094378>] ? tick_dev_program_event+0x68/0xd0
Jun 21 13:02:05 picimaci-laptop kernel: [21553.040267] [<ffffffff8106e3a7>] __do_softirq+0xb7/0x1e0
Jun 21 13:02:05 picimaci-laptop kernel: [21553.040274] [<ffffffff810c4de0>] ? handle_IRQ_event+0x60/0x170
Jun 21 13:02:05 picimaci-laptop kernel: [21553.040282] [<ffffffff810142ec>] call_softirq+0x1c/0x30
Jun 21 13:02:05 picimaci-laptop kernel: [21553.040288] [<ffffffff81015cb5>] do_softirq+0x65/0xa0
Jun 21 13:02:05 picimaci-laptop kernel: [21553.040293] [<ffffffff8106e245>] irq_exit+0x85/0x90
Jun 21 13:02:05 picimaci-laptop kernel: [21553.040300] [<ffffffff81545ed5>] do_IRQ+0x75/0xf0
Jun 21 13:02:05 picimaci-laptop kernel: [21553.040306] [<ffffffff81013b13>] ret_from_intr+0x0/0x11
Jun 21 13:02:05 picimaci-laptop kernel: [21553.040309] <EOI> [<ffffffff8130d492>] ? acpi_idle_enter_simple+0x117/0x14b
Jun 21 13:02:05 picimaci-laptop kernel: [21553.040324] [<ffffffff8130d48b>] ? acpi_idle_enter_simple+0x110/0x14b
Jun 21 1...

Read more...

Revision history for this message
Maci (gergo-nemeth) wrote : apport information

AlsaVersion: Advanced Linux Sound Architecture Driver Version 1.0.21.
Architecture: amd64
AudioDevicesInUse:
 USER PID ACCESS COMMAND
 /dev/snd/controlC0: picimaci 1747 F.... pulseaudio
 /dev/snd/pcmC0D0p: picimaci 1747 F...m pulseaudio
CRDA: Error: [Errno 2] Nincs ilyen fájl vagy könyvtár
Card0.Amixer.info:
 Card hw:0 'SIS966'/'HDA SIS966 at 0xfddf8000 irq 18'
   Mixer name : 'Realtek ALC660-VD'
   Components : 'HDA:10ec0660,10430000,00100001 HDA:10573055,10431316,00100700'
   Controls : 13
   Simple ctrls : 10
DistroRelease: Ubuntu 10.04
Frequency: Once a day.
HibernationDevice: RESUME=UUID=b98a9731-db1e-4899-8e0e-c23b891337a8
MachineType: ASUSTeK Computer Inc. F5SL
NonfreeKernelModules: fglrx
Package: linux-image-2.6.32-22-generic 2.6.32-22.36
PackageArchitecture: amd64
ProcCmdLine: BOOT_IMAGE=/vmlinuz-2.6.32-22-generic root=UUID=619267c0-4b1b-464c-8a39-2025711050f1 ro quiet splash elevator=deadline
ProcEnviron:
 LANG=hu_HU.utf8
 SHELL=/bin/bash
ProcVersionSignature: Ubuntu 2.6.32-22.36-generic 2.6.32.11+drm33.2
Regression: Yes
RelatedPackageVersions: linux-firmware 1.34
Reproducible: No
RfKill:
 0: phy0: Wireless LAN
  Soft blocked: no
  Hard blocked: yes
Tags: lucid networking regression-release needs-upstream-testing
Uname: Linux 2.6.32-22-generic x86_64
UserGroups: adm admin audio cdrom dialout floppy fuse lpadmin netdev plugdev sambashare tape vboxusers video
WpaSupplicantLog:

dmi.bios.date: 10/27/2008
dmi.bios.vendor: American Megatrends Inc.
dmi.bios.version: 306
dmi.board.asset.tag: To Be Filled By O.E.M.
dmi.board.name: F5SL
dmi.board.vendor: ASUSTeK Computer Inc.
dmi.board.version: 1.0
dmi.chassis.type: 10
dmi.chassis.vendor: ASUSTeK Computer Inc.
dmi.modalias: dmi:bvnAmericanMegatrendsInc.:bvr306:bd10/27/2008:svnASUSTeKComputerInc.:pnF5SL:pvr1.0:rvnASUSTeKComputerInc.:rnF5SL:rvr1.0:cvnASUSTeKComputerInc.:ct10:cvr:
dmi.product.name: F5SL
dmi.product.version: 1.0
dmi.sys.vendor: ASUSTeK Computer Inc.

tags: added: apport-collected
Revision history for this message
Maci (gergo-nemeth) wrote : AlsaDevices.txt

apport information

Revision history for this message
Maci (gergo-nemeth) wrote : AplayDevices.txt

apport information

Revision history for this message
Maci (gergo-nemeth) wrote : ArecordDevices.txt

apport information

Revision history for this message
Maci (gergo-nemeth) wrote : BootDmesg.txt

apport information

Revision history for this message
Maci (gergo-nemeth) wrote : Card0.Amixer.values.txt

apport information

Revision history for this message
Maci (gergo-nemeth) wrote : Card0.Codecs.codec.0.txt

apport information

Revision history for this message
Maci (gergo-nemeth) wrote : Card0.Codecs.codec.1.txt

apport information

Revision history for this message
Maci (gergo-nemeth) wrote : CurrentDmesg.txt

apport information

Revision history for this message
Maci (gergo-nemeth) wrote : Dependencies.txt

apport information

Revision history for this message
Maci (gergo-nemeth) wrote : IwConfig.txt

apport information

Revision history for this message
Maci (gergo-nemeth) wrote : Lspci.txt

apport information

Revision history for this message
Maci (gergo-nemeth) wrote : Lsusb.txt

apport information

Revision history for this message
Maci (gergo-nemeth) wrote : PciMultimedia.txt

apport information

Revision history for this message
Maci (gergo-nemeth) wrote : ProcCpuinfo.txt

apport information

Revision history for this message
Maci (gergo-nemeth) wrote : ProcInterrupts.txt

apport information

Revision history for this message
Maci (gergo-nemeth) wrote : ProcModules.txt

apport information

Revision history for this message
Maci (gergo-nemeth) wrote : UdevDb.txt

apport information

Revision history for this message
Maci (gergo-nemeth) wrote : UdevLog.txt

apport information

Revision history for this message
Maci (gergo-nemeth) wrote : WifiSyslog.txt

apport information

Revision history for this message
Jeremy Foshee (jeremyfoshee) wrote :

This bug report was marked as Incomplete and has not had any updated comments for quite some time. As a result this bug is being closed. Please reopen if this is still an issue in the current Ubuntu development release http://cdimage.ubuntu.com/daily-live/current/ . Also, please be sure to provide any requested information that may have been missing. To reopen the bug, click on the current status under the Status column and change the status back to "New". Thanks.

[This is an automated message. Apologies if it has reached you inappropriately; please just reply to this message indicating so.]

tags: added: kj-expired
Changed in linux (Ubuntu):
status: Incomplete → Expired
To post a comment you must log in.