KTorrent hangs whole system when rescaning data

[Pavel Malyshev]

Binary package hint: ktorrent

I have 2 computers both with kubuntu 9.10. Amd64 arch.
Ktorrent hangs whole system (mouse cursor stops, keyboard doesn't respond (pressing num lock, caps lock), alt-ctrl-f1 does nothing, hdd led doesn' t blink) when it rescaning downloaded data. I've tried a lot of different files. Size does not matter (I've tried 300M files up to 10G).
It seems that ktorrent hangs more often when it tries to rescan in background (hidden to tray).

$ lsb_release -rd
Description: Ubuntu 9.10
Release: 9.10

$ LANG=C apt-cache policy ktorrent
ktorrent:
  Installed: 3.2.4+dfsg.1-1ubuntu1
  Candidate: 3.2.4+dfsg.1-1ubuntu1
  Version table:
 *** 3.2.4+dfsg.1-1ubuntu1 0
        500 http://192.168.1.10 karmic/main Packages
        500 http://192.168.1.10 karmic/main Packages
        100 /var/lib/dpkg/status

Ubuntu 10.04 still has this bug. Ktorrent hangs system.
Ktorrent downloading to ~/Private and got corrupted files there, it makes several rechecks and redownloads.

[Pavel Malyshev (afunix)]

It looks like video driver error.
Both computers where ktorrent hangs have nvidia video with nvidia-185 drivers.
Another computer with ati video works fine with ktorrent.
And it seems that system is not totally dead. HDD led blinks rarely.

[Andrej Mernik (r33d3m33r-deactivatedaccount)]

Hi,

I have the same problem. I did some manual integrity checks and it does not hang every time. But when it does, the progress bar stops at first -> the mouse is still movable. After few seconds the whole system freezes. I'm using Kubuntu 9.10 i386 with nVidia Geforce 4 Ti 4200 AGP 8x. The installed drivers are 96.43.13-0ubuntu6 from the repository.

$ lsb_release -rd
Description: Ubuntu 9.10
Release: 9.10

$ LANG=C apt-cache policy ktorrent
ktorrent:
  Installed: 3.2.4+dfsg.1-1ubuntu1
  Candidate: 3.2.4+dfsg.1-1ubuntu1
  Version table:
 *** 3.2.4+dfsg.1-1ubuntu1 0
        500 http://si.archive.ubuntu.com karmic/main Packages
        100 /var/lib/dpkg/status

[Christian Andersson (christian-ld-andersson)]

- Same problem for me too, but only on the newer of my two computers, running AMD64 (and using ATI graphics). On the older machine, a Pentium 4 with an nVidia GeForce FX 5700, it seems to work. Hence, probably not a graphics driver issue, as was suggested.

- Neither do I think this is actually a problem in ktorrent; the two datachecker implementations (single- and multi-) both look straightforward and correct (?).

- I suspected a 64-bit issue on some level (kernel, qt, kdelibs, boost,...), until R33D3M33R confirmed the problem on i386.

- On my problem machine, I've upgraded to KDE 4.3.3 from the kubuntu-ppa. Don't know if that caused the trouble.

[Pavel Malyshev (afunix)]

Both computers where ktorrent hangs have amd64 arch. One has kde4.3.2, another one has kde4.3.3

[Christian Andersson (christian-ld-andersson)]

Do you leech to encrypted filesystems? On my problem machine, I do
(encrypted XFS partition, in an LVM volume), and I've noticed quite a severe
performance bottleneck, particularly when allocating (large) files (before
leech) and when checking data (after).

On Fri, Nov 20, 2009 at 5:02 PM, afunix <email address hidden> wrote:

> Both computers where ktorrent hangs have amd64 arch. One has kde4.3.2,
> another one has kde4.3.3
>
> --
> KTorrent hangs whole system when rescaning data
> https://bugs.launchpad.net/bugs/482509
> You received this bug notification because you are a direct subscriber
> of the bug.
>

--
Ch

[Christian Andersson (christian-ld-andersson)]

I'm sorry for the bloat in #5. Just tried the mail-in-comment feature. (It works... kinda'!)

[Pavel Malyshev (afunix)]

I have LVM on both computers. My /home is ext3, I'm using encrypted ~/Private and some downloaded files are there.

[Andrej Mernik (r33d3m33r-deactivatedaccount)]

>Do you leech to encrypted filesystems? On my problem machine, I do
>(encrypted XFS partition, in an LVM volume), and I've noticed quite a severe
>performance bottleneck, particularly when allocating (large) files (before
>leech) and when checking data (after).

Yes. I do have my /home partition encrypted (that's where the downloads go) and I have also noticed that the allocating takes longer than with my previous system (8.04, Ktorrent 2.2.5 (or something like that), /home wasn't encrypted).

[Christian Andersson (christian-ld-andersson)]

[SOLVED]

After having reorganized my file systems, I've used ktorrent against a NON-encrypted partition for ten days without any problems at all. I'm concluding empirically that the ecryptfs doesn't work very well with bittorrent's random access to file blocks, which is perhaps not surprising after all.

And, BTW, I've upgraded to ktorrent 3.3.1 from vanilla sources. The build/install instructions from http://ktorrent.org/?q=downloads work out-of-the-box (steps prepended by a 'sudo apt-get build-dep ktorrent'). And 3.3.1 just works perfectly, with a series of improvement made to its interface. Greetings and salutations to the author(s) for a great piece of software!

[Andrej Mernik (r33d3m33r-deactivatedaccount)]

Hi, so you are using 3.3.1 on an unencrypted partition now?

[Andrej Mernik (r33d3m33r-deactivatedaccount)]

I have fixed my problem by using qBittorrent. Works out of the box - no freezing when rechecking data and no slowdowns while allocating space.

[Siegie (siegie)]

Comfirmed on a encrypted partition.
But i don't see a reason for the whole system to crash due to a program behaver.

[Siegie (siegie)]

It's on a encrypted home using ecryptfs.
Maybe assign this bug to them?

[Pavel Malyshev (afunix)]

I've noticed another problems with ktorrent & encryptfs and I think it's the same bug.

Files which ktorrent downloads to encryptfs are corrupted! The same files downloaded out of encryptfs are ok.

I'm enabling data checking after download and ktorrent often get ~30-50% corrupted parts.

[Andrej Mernik (r33d3m33r-deactivatedaccount)]

afunix: I have noticed that too. Several rechecks were needed to correctly download the file, but as I already wrote, using qBittorrent removed all the problems.

[Christian Andersson (christian-ld-andersson)]

I can also confirm the problem described by afunix and Siegfried.

@R33D...: Changing to another client may well be a solution for you, but not for ktorrent. Neither is leeching to an unencrypted partition, so my speaking about SOLVED above was clearly a mistake. It's not a fix, it's a workaround. If not just qBittorrent but also other clients other than ktorrent work correctly with encrypted working directories, our suspicions against ktorrent itself will of course strenthen further (and exclude those against ecryptfs and so on).

The question is whether the developers of ktorrent are at all considering this issue, or if they're merely focusing at the moment on feature extensions in the 4.0-branch. qBittorrent explicity rips parts of ktorrent code into its implementation (e.g. ETA estimation), which is of course perfectly ok. Perhaps it's "payback time", at least inspirationally. The two programs are after all somewhat related (C++, Qt, GPL, ok that might be it; I think it is, really).

[Jamie (solowinter)]

I'm experiencing the same problems and had no idea before reading this that they were related (I recently replaced my hard drive and just started using an encrypted home directory). Specifically:

1) Pre-allocating space takes longer than "normal";

2) I am seeing a really high percentage of corrupt chunks. Only recently have I started having Ktorrent rescan torrents after they've completed downloading (after I noticed a few were corrupted). Now almost every file that completes has a large number of corrupt chunks, so the torrents have to be re-downloaded several times before I get a good copy.

3) I started experiencing the lock-ups over the past few days and have been banging my head on the desk trying to figure out where the problem is. I haven't been able to spot anything that stands out in the logs and can't think of any updates that may have caused the problems.

[Andrej Mernik (r33d3m33r-deactivatedaccount)]

@Christian Andersson: Because Ktorrent is not usable for ~3 months now, I think people deserve to know that there is an alternative that works. Hard lock is something different from application crash, you can lose a lot of data with that. There is also no sign of any developer watching this bug report :/.

[amichair (amichai2)]

I suffer from the same lock up described here. I noticed it happens both when checking the files, and when ktorrent is configured to "Fully reserve disk space" before the download begins. But then, I've seen similar symptoms when there's a whole lot of disk thrashing unrelated to ktorrent as well (long backup, etc.) Having a mouse cursor freeze due to disk activity is a serious problem in any case - responsiveness should be completely decoupled from disk activity. Maybe other Bittorrent clients buffer disk activity differently which alleviates the symptoms, or maybe there's more than one issue at hand.

For the record, I have a Q9550, 4G RAM (both with ample space when the problem occurs), Karmic amd64, KDE 4.3.5, KTorrent 3.2.4, and do not use any encryption.

What I do think might be related is the speed of the disk which makes the IO bottleneck worse, causing this problem. Can others confirm whether they're writing to a slower green/5400RPM/low power/high-capacity media disk? Or do you see this also on fast disks (to the same degree)?

[Pavel Malyshev (afunix)]

Does your ktorrent download to encrypted partition? ~/Private?
I've found that downloading to ~/Private or to encrypted home dir results in lockups and corrupted files. When I'm trying to download same file to unencrypted directory ktorrent works well.
It looks like ecryptfs problem (maybe amd64 only?).

[amichair (amichai2)]

As I mentioned, I do not use any encryption anywhere. I haven't noticed any corruption problems, only lockups - but corruption and lockups sound like two separate issues in any case... (just guessing there)

[Siegie (siegie)]

Maybe this could be related to problems i had with gimp.
http://ubuntuforums.org/showthread.php?p=9127712

Both programs have large disk activity at the moment off the crash.
https://bugs.launchpad.net/ubuntu/+source/gimp/+bug/356308

[sk1887 (sk1887)]

It happened to me too. Using transmission as torrent downloader.
My file system is approx. 7 GB. I was downloading a torrent that had a file large 13 GB, of course that file was unchecked. transmission probably failed at reorganizing parts and added that part I didn't need. I am running Ubuntu 10.04, latest kernel and I doubt it's related to the graphic drivers.
So I went to analyze the data used, and /home/me/.Private was 3 GB.

[Pavel Malyshev (afunix)]

So this bug is still there in my KUbuntu 10.04 on AMD64 with encrypted ~/Private.
Lockups and corrupted files.

[Xoke (xokesoru)]

(Trying not to make this a 'me too').

Me too! Ubuntu 10.04 (upgraded from 9.10) and ktorrent hanging entire system when checking. Happened three times today, last time I was running nothing else (well, obviously background stuff, but no other apps open). Had a sudden thought about disk space so opened a terminal but it hung just before terminal opened.

Encrypted home drive, and yes I didn have plenty of space (around 20 gigs).

Most annoying thing was I was trying to download the Ubuntu ISO via the torrent :P

Transmission worked fine though so if anyone is having problems that is at least a work around.

[Evgeny Brazgin (xapienz)]

I have the similar problem too. The system freezes, keyboard and mouse don't respond.
I don't use encrypted fs, don't know whether ktorrent tries to rescan downloaded data.
But I get some messages in /var/log/messages. Something like this:

kernel: [176880.392142] ktorrent D f38bbb5c 0 10352 1 0x00000000
kernel: [176880.392153] f38bbb6c 00000086 00000002 f38bbb5c 00011200 c05d8a20 c08c3700 c08c3700
kernel: [176880.392167] c3c9e1b6 0000a0c0 c08c3700 c08c3700 c3c62877 0000a0c0 00000000 c08c3700
kernel: [176880.392180] c08c3700 ea198000 00000001 ea198000 c2921700 f38bbbb8 f38bbb7c c05c69b1
kernel: [176880.392193] Call Trace:
kernel: [176880.392209] [<c05c69b1>] io_schedule+0x61/0xa0
kernel: [176880.392219] [<c023d668>] sync_buffer+0x38/0x40
kernel: [176880.392226] [<c05c717d>] __wait_on_bit+0x4d/0x70
kernel: [176880.392233] [<c023d630>] ? sync_buffer+0x0/0x40
kernel: [176880.392240] [<c023d630>] ? sync_buffer+0x0/0x40
kernel: [176880.392248] [<c05c724b>] out_of_line_wait_on_bit+0xab/0xc0
kernel: [176880.392257] [<c0165e60>] ? wake_bit_function+0x0/0x50
kernel: [176880.392264] [<c023d62e>] __wait_on_buffer+0x2e/0x30
kernel: [176880.392271] [<c023e602>] __bread+0x72/0xa0
kernel: [176880.392279] [<c0277d02>] ext3_get_branch+0x72/0xf0
kernel: [176880.392287] [<c0279763>] ext3_get_blocks_handle+0x83/0x4a0
kernel: [176880.392297] [<c033f842>] ? submit_bio+0x72/0x100
kernel: [176880.392304] [<c0279c30>] ext3_get_block+0xb0/0x100
kernel: [176880.392313] [<c0245af6>] do_mpage_readpage+0x1c6/0x730
kernel: [176880.392322] [<c01e2cd9>] ? ____pagevec_lru_add+0x179/0x190
kernel: [176880.392329] [<c01e2d37>] ? __lru_cache_add+0x47/0x60
kernel: [176880.392337] [<c01da07c>] ? add_to_page_cache_lru+0x5c/0x70
kernel: [176880.392345] [<c0246197>] mpage_readpages+0xc7/0x100
kernel: [176880.392352] [<c0279b80>] ? ext3_get_block+0x0/0x100
kernel: [176880.392361] [<c01dfcc9>] ? __alloc_pages_nodemask+0xd9/0x1c0
kernel: [176880.392368] [<c0278590>] ? ext3_readpages+0x0/0x20
kernel: [176880.392375] [<c02785ae>] ext3_readpages+0x1e/0x20
kernel: [176880.392382] [<c0279b80>] ? ext3_get_block+0x0/0x100
kernel: [176880.392390] [<c01e1f6c>] __do_page_cache_readahead+0x14c/0x210
kernel: [176880.392398] [<c01e2056>] ra_submit+0x26/0x30
kernel: [176880.392406] [<c01db75f>] filemap_fault+0x3cf/0x400
kernel: [176880.392413] [<c035a1d9>] ? copy_to_user+0x39/0x130
kernel: [176880.392422] [<c01f25a0>] __do_fault+0x40/0x570
kernel: [176880.392429] [<c01f4566>] handle_mm_fault+0x146/0x400
kernel: [176880.392438] [<c0546a40>] ? udp_ioctl+0x0/0x80
kernel: [176880.392447] [<c05cbed6>] do_page_fault+0x146/0x440
kernel: [176880.392456] [<c04e7d35>] ? sys_socketcall+0x175/0x2a0
kernel: [176880.392464] [<c05cbd90>] ? do_page_fault+0x0/0x440
kernel: [176880.392471] [<c05c9307>] error_code+0x73/0x78
kernel: [176880.392479] [<c05c0000>] ? identify_cpu+0xc4/0x229

It appears many times, every time starting with "kernel: [...] ktorrent D f38bbb5c 0 10352 1 0x00000000", so it is somehow related to ktorrent.

Sometimes between this lines the info about memory usage is shown:
kernel: [177192.740206] Mem...

[Jason Straight (jason-jeetkunedomaster)]

Having possibly the same type of problem. EcryptFS, Ktorrent hanging while allocating disk space, causing whichever core it's running on to spike to 100%, there is no disk activity though. strace tells me ktorrent stops running right after ftruncate64() call.

[Jason Straight (jason-jeetkunedomaster)]

Also, I'm on 32 bit.

[Arcadie M. Cracan (acracan)]

I'm also having this problem: system hangs when KTorrent allocates diskspace. I'm on Kubuntu 10.10, amd64 system and don't use any encrypted filesystems. I was wondering if there is a way to better trace this behavior.

[Pavel Malyshev (afunix)]

I don't belive, that someone will fix this bug. It's unassigned for 1.5 years...

[Joseph Crowell (smokexjc)]

I'm having issues with this bug on openSuse 11.4 too.

[anders (andreslucena)]

I'm having this issue in Natty (Ubuntu 11.04) with ecryptfs, i686.

I don't suffer the hang up stuff, only "Allocating disk space" for too long. No IO wait activity, it seems that the bigger the torrent it's more time the "Allocating" is.