Kindle Paperwhite 3 unmounted

Bug #1843328 reported by Mithrandir
14
This bug affects 3 people
Affects Status Importance Assigned to Milestone
calibre
Invalid
Undecided
Unassigned

Bug Description

Hello, I think I need some help, please.
Everytime I mount my Kindle Paperwhite under Gentoo Linux, it is unmounted after few secconds.
trying to send books, Calibre reports:

calibre, version 3.47.1
ERROR: Error: Error communicating with device

[Errno 5] Input/output error
Traceback (most recent call last):
  File "/usr/lib64/calibre/calibre/gui2/device.py", line 90, in run
    self.result = self.func(*self.args, **self.kwargs)
  File "/usr/lib64/calibre/calibre/gui2/device.py", line 594, in _upload_books
    metadata=metadata, end_session=False)
  File "/usr/lib64/calibre/calibre/devices/usbms/driver.py", line 325, in upload_books
    filepath = self.put_file(infile, filepath, replace_file=True)
  File "/usr/lib64/calibre/calibre/devices/usbms/cli.py", line 53, in put_file
    dest, actual_path = case_preserving_open_file(path)
  File "/usr/lib64/calibre/calibre/utils/filenames.py", line 205, in case_preserving_open_file
    os.fsync(ans.fileno())
OSError: [Errno 5] Input/output error

Also, fsck reports that the dirty bit was set.
I was using Calibre with the same device some months ago without any problem.
Thank you in advance.

Revision history for this message
Mithrandir (diego-samuelle) wrote :

I found this on my dmesg, I think is relevant

[15683.346202] sd 4:0:0:0: [sdb] tag#0 access beyond end of device
[15683.346207] print_req_error: I/O error, dev sdb, sector 713324 flags 801
[15687.160688] sd 4:0:0:0: [sdb] tag#0 access beyond end of device
[15687.160692] print_req_error: I/O error, dev sdb, sector 17 flags 100001
[15687.160694] Buffer I/O error on dev sdb1, logical block 1, lost async page write
[15687.160716] sd 4:0:0:0: [sdb] tag#0 access beyond end of device
[15687.160718] print_req_error: I/O error, dev sdb, sector 2266 flags 100001
[15687.160719] Buffer I/O error on dev sdb1, logical block 2250, lost async page write
[15687.160725] sd 4:0:0:0: [sdb] tag#0 access beyond end of device
[15687.160726] print_req_error: I/O error, dev sdb, sector 2494 flags 100001
[15687.160727] Buffer I/O error on dev sdb1, logical block 2478, lost async page write
[15687.160743] sd 4:0:0:0: [sdb] tag#0 access beyond end of device
[15687.160744] print_req_error: I/O error, dev sdb, sector 5381 flags 100001
[15687.160745] Buffer I/O error on dev sdb1, logical block 5365, lost async page write
[15687.160747] sd 4:0:0:0: [sdb] tag#0 access beyond end of device
[15687.160748] print_req_error: I/O error, dev sdb, sector 5609 flags 100001
[15687.160748] Buffer I/O error on dev sdb1, logical block 5593, lost async page write
[15695.770884] sd 4:0:0:0: [sdb] tag#0 access beyond end of device
[15695.770887] print_req_error: I/O error, dev sdb, sector 17 flags 0
[15695.770893] FAT-fs (sdb1): bread failed in fat_clusters_flush
[15695.770909] sd 4:0:0:0: [sdb] tag#0 access beyond end of device
[15695.770910] print_req_error: I/O error, dev sdb, sector 8380 flags 1
[15695.770912] Buffer I/O error on dev sdb1, logical block 8364, lost async page write

description: updated
description: updated
Revision history for this message
Kovid Goyal (kovid) wrote : Re: calibre bug 1843328

Reset your kindle, and I/O error means the device is failing.

 status invalid

Changed in calibre:
status: New → Invalid
Revision history for this message
Mithrandir (diego-samuelle) wrote :

Thank you very much for your answer. This is what I tried after it:

1) After running "sudo fsck.vfat -r /dev/sdb1" and connecting under Windows, the problem is not present.
2) On linux, running "sync" in a loop, the problem is not present (and fsck does not report any problem.
3) Without the "sync" loop, the unmounting happens again and the dirty bit is set too.
4) Also, if I just mount the device and access through the file manager (Calibre not running), the problem does not happen either.

In my little understanding, it looks suspiciously similar to bug 1834641.
Do you have any advice?
Thank you in advance

Revision history for this message
Kovid Goyal (kovid) wrote :

The fsync bug in the kernel was workaround in calibre a while ago.

Revision history for this message
Mithrandir (diego-samuelle) wrote :

Yes, I read the related kernel bug too.

I am just trying to find out why this is happening.
So far I have:

while true; do
    sleep 10
    sync
done

running with calibre and it works, but is probably a nasty workaround.

Maybe, since there is so many variables in Gentoo, maybe I am missing something (kernel connfiguration or so).

I understand you cannot dedicate much time to an issue happening in a sigle computer.

However, since I ran out of ideas I need to ask for some hint (if not about the problem, about where to ask).

Thank you

Revision history for this message
Kovid Goyal (kovid) wrote :

try mounting the kindle with the nobarrier option that might help.

Revision history for this message
Dries Kimpe (p-dries) wrote :

I can confirm this problem.
Retried with known good USB cable (can also read the whole kindle partition using dd without problems) and with latest kernel (5.4.7).

Calibre 4.3

Somehow calibre triggers the vfat module to access beyond the end of the device.

This does *not* happen when mounting the kindle under the same circumstances (same kernel, cable, port, etc.) and performing normal filesystem accesses (including copying the *same book files* to the *same* directory on the kindle but without calibre).

I can reproduce this problem with calibre every time (and the kindle's filesystem ends up being corrupted).

Revision history for this message
Dries Kimpe (p-dries) wrote :

And the same still happens with calibre 4.8

Revision history for this message
Kovid Goyal (kovid) wrote :

Since calibre uses bog standard posix file apis to do file i/o it cannot "somehow trigger the vfat module" to do anything, unless there are bugs in said module. And I cannot replicate this on my linux system running a 5.5.3 kernel. I'm afraid you will need to take this up with the developers of the vfat module.

Revision history for this message
Leo Alekseyev (leo-alekseyev+ubuntuforums) wrote :

I was experiencing this bug with an older Calibre version on a 5.3.0 kernel, but an upgrade to Calibre 4.18 made the problem go away.

Revision history for this message
Fabiano Konzen (0bi-w6n-k3nobi) wrote :

The problem is not over.
I am running on Fedora 33, Caliber 4.26 and have the same problem.

Every time the Calibre is running and you plug the Kindle into the USB cable.
This happens after Calibre tries to identify e-books that are already on Kindle.

However, I FIND THE REASON and THE SOLUTION.

I noticed that when you mount the Kindle before running Calibre, through the file manager in /run/media/USER/Kindle, the error DOES NOT HAPPEN.

But, when you plug the Kindle when the Calibre is already running, then it mount to /media/Kindle, the ERROR HAPPENS.

So, what's the difference:
the "flush" parameter in the mount options.
That's right - just add the "flush" parameter.

Then, in the "udisks.py" file add:

    def mount(self, device_node_path):
        d = self.device(device_node_path)
        try:
            return unicode_type(d.FilesystemMount('',
                ['auth_no_user_interaction', 'rw', 'noexec', 'nosuid',
                 'nodev', 'uid=%d'%os.geteuid(), 'gid=%d'%os.getegid(), 'flush'])) <---

Revision history for this message
Kovid Goyal (kovid) wrote :

Nonsense, flush is added by default when mounting FAT volumes via udisks
https://github.com/storaged-project/udisks/blob/7f690b06329adde019b0df6b69cdb3b25c90b6f6/src/udiskslinuxfilesystem.c#L383

You have presumably configured udisks not to do that, for some
misbegotten reason.

Revision history for this message
Fabiano Konzen (0bi-w6n-k3nobi) wrote :

Well. Here it happens exactly as I described.
When Calibre mount Kindle it DOES NOT USE the "flush" option.
When the file-manager mount it uses "flush" ...

If you can tell me where this UDISKS option is, I can check.

Revision history for this message
Kovid Goyal (kovid) wrote :

And here is the actual UDisks documentation on the same:
http://storaged.org/doc/udisks2-api/latest/mount_options.html

Revision history for this message
Fabiano Konzen (0bi-w6n-k3nobi) wrote :

Yes. Really on the Fedora version too; however inserted within the code.
I will do tests on a fresh installation, and also on the VM.

Revision history for this message
Fabiano Konzen (0bi-w6n-k3nobi) wrote :

Well, I did the tests in a fresh installation.

At first everything was ok. So I verified that only UDisks version 2 was installed.
This version does not use DBUS, only version 1. So I found the problem.

Then you can easily reproduce the problem.
With UDisks version 1, you execute the commands:

$ udisks --mount /dev/sdb1
Mounted /org/freedesktop/UDisks/devices/sdb1 at /media/Kindle

$ grep Kindle /proc/mounts
/dev/sdb1 /media/Kindle vfat rw,nosuid,nodev,relatime,uid=1000,gid=1000,fmask=0022,dmask=0077,codepage=437,iocharset=ascii,shortname=mixed,showexec,utf8,errors=remount-ro 0 0

$ udisks --unmount /dev/sdb1

Now, using UDisks version 2:

$ udisksctl mount -b /dev/sdb1
Mounted /dev/sdb1 at /run/media/user/Kindle

$ grep Kindle /proc/mounts
/dev/sdb1 /run/media/user/Kindle vfat rw,nosuid,nodev,relatime,uid=1000,gid=1000,fmask=0022,dmask=0022,codepage=437,iocharset=ascii,shortname=mixed,showexec,utf8,flush,errors=remount-ro 0 0

$ udisksctl unmount -b /dev/sdb1
Unmounted /dev/sdb1.

As you can see only in version 2, UDisks adds "flush" to the mount options.

My Versions:
OS: Fedora 33
UDisks 1: udisks-1.0.5
UDisks 2: udisks2-2.9.1

Yes. I tested it on a notebook and desktop.

Revision history for this message
Kovid Goyal (kovid) wrote :

So dont use udisks1, udisks2 was released over a decade ago.

Revision history for this message
Fabiano Konzen (0bi-w6n-k3nobi) wrote :

Kovid, good morning.

I understand his point of view.
But, if Calibre has the code for UDisks1 / D-BUS, wouldn't it be convenient to work on any OS?

So, what I propose is: add the parameter "flush" for this case or completely remove the code for UDisks1.

Well, this is my point of view. I believe that you want the end user to have the best experience possible.

Grateful for your attention.

Revision history for this message
Kovid Goyal (kovid) wrote :

The best user experience is not overriding system settings for how
the user wants to mount his devices.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.