Calibre 5.6.0 - silent crashes on MacOS 10.15.7

[Stephen Silk]

Calibre 5.6.0 repeatedly crashes after running for an unknown period of time on MacOS 10.15.7 . Server log gives the following for each crash:

Unhandled exception in state: State: write_buf Client: 92.17.20.39:44203 Request: GET / HTTP/1.1
Traceback (most recent call last):
  File "calibre/srv/loop.py", line 606, in tick
  File "calibre/srv/http_response.py", line 582, in write_buf
  File "calibre/srv/http_response.py", line 378, in write
  File "calibre/srv/sendfile.py", line 90, in sendfile_to_socket_async
OSError: (57, 'Socket is not connected')

I leave calibre open full-time, but since I upgraded it to 5.6 I come back to my system to find it is no longer running. There is no indication as to when or why - it is just no longer in my dock or on my screen.

[Kovid Goyal (kovid)]

If calibre is crashing, there should be a crash report, attach it.

 status incomplete

[Stephen Silk (sasilk)]

I'd be happy to do so if I could find any idea of where such a thing might be kept or what it might be called. The documentation has nothing, the MobileRead forum has nothing, and even multiple straight google searches show nothing.

[Kovid Goyal (kovid)]

This is an Apple crash report that macOS pops up whenever an application
crashes. You can get historical crash reports from console.app

[Stephen Silk (sasilk)]

I'm not getting any crash reports, and there's nothing in console.app - calibre seems to be just silently shutting down. So 'crash' is probably the wrong term, but I couldn't think of what else to call it.

Is there anything else I can provide that might help?

[Kovid Goyal (kovid)]

Restart calibre in debug mode (there's a menu entry for it in the
Preferences menu) and post the log after a quit

[Stephen Silk (sasilk)]

OK, restarted in debug mode and looked into the configuration directory after calibre quit. Nothing I can find that appears to be a log file. The only log files I could find are server-log.txt and server-access-log.txt, which seem to be for the web server. (I've attached these.) I did find out that whenever I try to clear the logs, calibre immediately quits.

[Kovid Goyal (kovid)]

Click the Preferences menu item, choose "Restart in debug mode" when
calibre restarts you get a message about the debug log file, that is the
file you need to attach, after an unexpected quit.

[Stephen Silk (sasilk)]

I'll try, but calibre has *never* successfully restarted for me - I just shuts down and I have to start it manually.

[Kovid Goyal (kovid)]

then run it from Terminal.app as

/Applications/calibre.app/Contents/MacOS/calibre-debug -g

[Stephen Silk (sasilk)]

Nope - no restart, as usual. I left it 10 minutes, just in case it was slower in debug mode - nothing. Tried the menu command as well as the keyboard shortcut - nothing. Any other options?

[Kovid Goyal (kovid)]

see above

[Stephen Silk (sasilk)]

OK, I modified that command line to take into account that calibre is in a subfolder rather than directly in Applications. It started, but still gave no indication of where or what the log file you're asking for is/will be. So I guess I now wait intil it quits and see if it tells me anything then.

[Kovid Goyal (kovid)]

it will print out things to the terminal itself Though given all your
other issues, crashing when showing logs, not restarting, etc its likely
the problem is systemic and the debug output wont show anything useful.

[Stephen Silk (sasilk)]

Terminal output

[Kovid Goyal (kovid)]

Yeah there is nothing useful there, as I suspected. One other thing you
can try, is changing th elocation of the calibre temp directory. By
default it is in /tmp and macOS stupidly deletes things from /tmp after prolonged
inactivity, which can break calibre. Set the CALIBRE_TEMP_DIR
environment variable as described here https://manual.calibre-ebook.com/customize.html#environment-variables
to some directory other than /tmp

[Kovid Goyal (kovid)]

Also in 5.7 I have changed calibre to not use sockets in /tmp for IPC, this might also help.

[Stephen Silk (sasilk)]

Further information:

I changed the temp directory as you suggested, but calibre still kept quitting randomly.

5.7 doesn't appear to have fixed it.

But I've found out, I believe, where the issue is. By shutting down the server function, I've had no quits in around 48 hours, when previously I could go more than 3-4.

I hope this helps.

[Kovid Goyal (kovid)]

Then post the log from the server after an unexpected quit. You get it
from Preferences->Sharing over the net

[Kovid Goyal (kovid)]

See https://bugs.launchpad.net/calibre/+bug/1902100

You can fix it by turning off the monitor sleep setting in you macOS settings.

[Stephen Silk (sasilk)]

Thank you.

I had just re-enabled the server and was witing for a quit to send tou the log.

Is this something that you can fix, or is it a MacOS issue?

[Kovid Goyal (kovid)]

see the linked ticket for discussion

[Stephen Silk (sasilk)]

Sorry, I missed the link.

Thanks for the effort on this - it's been frustrating at my end so it must have been far worse for you.

[Stephen Silk (sasilk)]

Unfortunately, that fix doesn't seem to have worked - I just had another quit. I know my Mac's display hasn't slept as I've been using it the whole time. I have these settings:

Display set to never sleep
Prevent computer from sleeping automatically when the display is off ON
Wake for network access ON
Enable Power Nap OFF

I hadn't restarted after making the settings changes, but have done so now and will see what happens.

I've attached the calibre error and server logs.

[Stephen Silk (sasilk)]

OK, I upgraded to 5.8 and enabled the server again, while leaving the Mac's power Save and such settings the same.The display and system do not sleep at all.

Calibre has now quit several times, twice while I was actually watching it (with me not doing anything on the system); once while it was in the background while I was using a browser, and once in the middle of the night.Ihave included the server logs.

[Kovid Goyal (kovid)]

Sorry, at a loss then, silent exits with no errors and no crash reports should basically be impossible :) Only other things I can think of are third party calibre plugins, which you can eliminate by running calibre as:

/Applications/calibre.app/Contents/MacOS/calibre --ignore-plugins

[Stephen Silk (sasilk)]

An update.

I completely uninstalled removed calibre, rebooted and then installed the last version that I know worked (5.6), with no plugins.. The issue persisted (which does tend to show that the issue isn't with calibre, as you said.
I upgraded to Big Sur and rebooted - same issue.
I upgraded calibre 5.8.1 (on Big Sur) and rebooted - same issue.
I tested a lot of settings that may have had some effect, with no success.

I've found a workaround for the moment, by writing an AppleScript that restarts calibre if it ever closes, so it's not as urgent from my point of view now. I'm posting this to add to your data about this issue, just in case it's useful.

Now, if I can get the Stop Server button in Preference not to crash calibre completely... Start Server and Test Server work fine, but not Stop Server.

[jay Lincoln (xjlincoln)]

I search the web from key words like:

"Unhandled exception in state: State: write_buf Client: 92.17.20.39:44203 Request: GET / HTTP/1.1
Traceback (most recent call last):
  File "calibre/srv/loop.py", line XXX, in tick:..........."

I use the calibre-server and found the bug.it happend when I get the book from the calibre-server in the web page.

this is the question..... calibre/srv/loop.py "calibre/srv/http_response.py" calibre/srv/sendfile.py

[Kovid Goyal (kovid)]

I finally figured out the cause for this, it is fixed by https://github.com/kovidgoyal/bypy/commit/156d2b550491deb0be5a6d4db767519d7b5de05c
which will be in the next release.