Changing title or author to uppercase/lowercase causes book to disappear (French Language)

[freikland]

Hello,

error: impossible to visualize
"Book X" does not have a format available

Calibre 3.16 (today 3.20) ; Ubuntu 16.04.4 LTS

When I edit the metadata of a book in .epub format and manually edit entries (or download the metadata), I sometimes get this by wanting to browse the book after modifications :

error: impossible to visualize
"Book Title" does not have a format available

In this case, the book, however editable and perfectly legible before, disappears purely and simply from its directory !
The file is therefore deffinitively lost ...
Only the .opf file remains.
I already noticed some problems when the case is not respected, for example capital letters replaced by tiny ones ... but not only ?

How to fix this bug ?

Thank you in advance for your help.

[gf (gf-interlinks-deactivatedaccount)]

Hello Freikland,
Thank you for submitting this bug and reporting a problem with Calibre. You made this bug report some time ago and Ubuntu and Calibre have been updated since then.

Could you confirm that this is no longer a problem and that we can close the ticket?
If it is still a problem, are you still interested in finding a solution to this bug?
If you are, could you let us know and, in the current version, could you run the following (only once):
apport-collect 1758548
and upload the updated logs and and any other logs that are relevant for this particular issue.

Thank you again for helping make Ubuntu and Calibre better.
G

[freikland (freikland)]

Hello,

Yes, I confirm that the problem still occurs, but randomly.
Under Ubuntu 16.04 LTS up to date and Calibre 3.29 in any case.

I just upgraded Calibre version 3.30 at the moment, so I do not have enough perspective for it.

Of course, I would like this concern to be solved, because when the phenomenon occurs, the book concerned is simply removed from the library ...
Please excuse my ignorance, but can you explain to me where and how to do this:

"intake-collect 1758548
and upload the updated logs and any other documents that are relevant for this particular issue."

Thank you in advance for your understanding.

[gf (gf-interlinks-deactivatedaccount)]

Hello Freikland,
Thanks for the reply.

Are you translating from another language? could you tell me your language?

Does this problem happen when you give the book a new title and use special characters in the name?

Would you run this command:

apport-collect 1758548

Thanks
Have a good day.
G

[freikland (freikland)]

Hello,

My language is French.
So, I translate French into English (with an AZERTY keyboard).

Indeed, the problem arises when I import a book and when I want to modify the metadata that are wrong and / or WHEN I CHANGE THE BREAK (capital letter in lower case, for example).

Then, by validating the modifications, the concerned book disappears from the library!
This even when the book is already converted by several formats (epub, mobi, azw3 and Original_epub).

Example: title "MY BOOK" that I modify in "My book" or the author "Firstname NAME" in "Name Firstname".

I hope to be clearer in my explanations.

Have a good day.

[gf (gf-interlinks-deactivatedaccount)]

Hello Freikland,
Thanks for the reply.
Are you using special characters in the new name of the book?
For example, are you using any of these: è, é, ê, ë
Thanks

:)

G

[gf (gf-interlinks-deactivatedaccount)]

Hello again, Freikland,
I found this other bug report regarding French characters and the AZERTY keyboard. Will you go to that bug report and see if it answers your question and solves the problem?

https://bugs.launchpad.net/calibre/+bug/1739888

Please tell me if this helps.
Thanks

:)
G

[freikland (freikland)]

Hi,

First, thank you very much for your help.
As for the special characters in French (é, è, à, ç, ë, ê ...) do not seem to be the cause of the problem. Or did not I notice?

On the other hand, it is the change in the case of characters that is decisive:
Indeed, I did a test on a book saved in epub and ORIGINAL_epub formats.
Then, I copied this same title in the same library in epub format as well.

So, in duplicate. When opening in the editor for reading, no problem.

Then, I change the name of the author (First Name Last Name > First Name Last NAME) on a copy and I save.
Well, the cover has disappeared, ditto for the book (both copies!?)
Only the opf file remains. If I try to open it, I get the following error:

 calibre, version 3.30.0
ERROR: Unable to open eBook: Failed to read the book, / media / m**** / S***_livres / Library calibre / Claude-Catherine Kiejman / Eleanor Roosevelt _ First lady and rebel (22254) /metadata.opf click "Show details " for more information

Traceback (most recent call last):
  File "site-packages / template / utils / ipc / simple_worker.py", line 289, in main
  File "site-packages / template / ebooks / oeb / iterator / book.py", line 65, in extract_book
  File "site-packages / caliber / customize / conversion.py", line 246, in __call__
  File "site-packages / template / ebooks / conversion / plugins / html_input.py", line 90, in convert
  File "site-packages / template / ebooks / conversion / plumber.py", line 1311, in create_oebbook
  File "site-packages / template / ebooks / oeb / reader.py", line 72, in __call__
  File "site-packages / template / ebooks / oeb / reader.py", line 704, in _all_from_opf
  File "site-packages / caliber / ebooks / oeb / reader.py", line 349, in _spine_from_opf
OEBError: Spine is empty

A priori, the problem originates from the management of lowercase and uppercase by Calibre (Title > TITLE, First name Lastname > Firstname LastNAME).

Have a great day.

[gf (gf-interlinks-deactivatedaccount)]

Thanks Freikland, for the extra steps and details. That helps clarify the problem to upper case or lower case on the title or author name.

I have tried to edit the title and author of a book I downloaded from the web but I cannot replicate the problem. I am using version 3.28 at the moment.

I have attached a picture of the edit metadata screen in Calibre.

Just to clarify, are you editing the information in the "Title" box or the "Title Sort" box?
Or, when you edit the author, are you editing the information in the "Author(s)" box, rather than the "Author sort"?

I wonder if it is related to the calibre program searching for the cover image based on title and author name? Does this only happen with purchased books or does it also happen with Epubs you have created from regular text or Word documents?

I am new to bug triage (triaging) and I am trying to clarify the source of the problem so the programmers know what area to focus their efforts.

Thanks for your patience
Have a good day.
Glady

[freikland (freikland)]

I edit only the information in the "Title" and "Author" boxes. I'm using others (sorting) only to check colors, green or red.

I have various sources, commercial or homemade (Word, LibreOffice ...), I confess not to have paid attention to this detail that may not be one.

Thank you for your patience.

cordially

[gf (gf-interlinks-deactivatedaccount)]

Hi Freikland,
You have had Calibre 3.30 for a week. Can you tell me if the problem occurs in that new version?

Also, I have changed the title of this bug report. Please tell me if you think the title is sufficient.

Just to make it clear for the programmers, here is a summary. Please tell me if corrections are required.
-------------
SUMMARY:
Ubuntu version 16.04 LTS fully updated
Calibre version 3.29
Problem is reproducable by the bug reporter.
Bug triager is unable to reproduce the bug.

Steps to reproduce:
1. Open Calibre
2. Download an EPUB ebook
3. Right click on the EPUB ebook
4. Select "Edit Meta-data"
5. Select "Edit Meta-data individually"
6. In the "Title" box, change the title of the book to all upper/lower case letters (by editing the title in the box) or, in the "Author(s)" box, change the author name to all upper/lower case letters.
7. Click "OK" in the lower right-hand corner of the screen

Expected behaviour: the book's title or author name is changed in the book list.

Actual behaviour: The cover and the book disappear. The ".opf" file remains, which Calibre cannot open (creates error).
---------------

Thanks
G

[freikland (freikland)]

Good evening,

Indeed, the problem persists with Calibre version 3.30.
The title change is indeed more appropriate than the original.

your resume is good.

I add that books added with metadata already with lowercase and uppercase letters open quite normally. But are contrary to the norms and conventions in force. Which explains why I want to modify them accordingly.

Regards,

[gf (gf-interlinks-deactivatedaccount)]

Thanks for your reply, Freikland.
Just to clarify, when you change the case on the title or author name, are you typing the edits manually on the keyboard?

Or, are you using the case change feature, by right-click on a word in the title box, select "Change Case" and pick the type you want?

See attached screen print.

[gf (gf-interlinks-deactivatedaccount)]

Hi again Freikland,
I also found on this website, several people having difficulty with the capitalization of the words in the title, in Calibre, due to different languages.
It does not solve your problem but I just thought I would send you the link, for your information:

https://www.mobileread.com/forums/showthread.php?t=143137

Thanks
G

[freikland (freikland)]

Hi gf,

I write the changes directly to the azerty (french) keyboard.
Thank you for the link provided, even if it does not give the answer, it makes you think about the problem and can help to find ideas to solve it.

I don't use right-click.

Unfortunately, not everyone is familiar with plugin programming ...

I specify that downloaded metadata whose title and / or author do not suit me (lowercase, capital letters) without modification on my part perfectly open the book in the editor.
The disappearance occurs if I manually modify the meta. But not always ???

[gf (gf-interlinks-deactivatedaccount)]

Hi Freikland,
Thank you for your patience on this issue.

When the error happens again, could you try to identify the characteristics of the ebook that might be the problem ( since it is not happening with all ebooks). For example, language, epub or other format, a specific vendor or download site, ebook created by you or by someone else, cover picture downloads with book or generate cover with calibre later, size of book.

Could you try the right click and see if there is a difference from the behaviour when you type the edits?

Could you give me an example of a specific ebook that I could download, that has the problem?

Thanks again.
G

[freikland (freikland)]

Hi gf,

I just encountered this error again.
I will try to describe precisely the events (not always easy to explain):

1 - This is a book in epub format downloaded.
2 - The author appears in the form "first name, NAME."
3 - I try to modify with right click -> change case -> lower case, then Shift to each word.
4 - I validate with OK, then I try to edit the book, but I get this error:

"Can not view, (book title) does not have a format available."
Only the opf file remains ?

5 - I then use the original copy of the file (no modification) which I manually rename the "NAME" "of the author in" Name "(correct for me).
6- This time, the "First Name Name" appears in the Calibre database and the book opens normally.
7 - Finally, I download the missing metadata with Calibre, which completes the synopsis and others. From then on, the file works correctly.

Thank you for your patience.

Please find attached a file to help you understand, include original and modified file.

[gf (gf-interlinks-deactivatedaccount)]

Searched for duplicates of this bug - found none

G

[gf (gf-interlinks-deactivatedaccount)]

Hi Freikland,
Thanks for the epub book to test on. I have tried it on my computer and am not able to replicate the problem. That is OK.
I believe there is enough information now and I have sent it to the programmers. For their ease, I will re-post the summary.
Thanks again for your patience
:)
G

[gf (gf-interlinks-deactivatedaccount)]

Kovid:

SUMMARY:
Ubuntu version 16.04 LTS fully updated
Calibre version 3.29
Problem is reproducable by the bug reporter.
Bug triager is unable to reproduce the bug.

Steps to reproduce:
1. Open Calibre
2. Download an EPUB ebook
3. Right click on the EPUB ebook
4. Select "Edit Meta-data"
5. Select "Edit Meta-data individually"
6. In the "Title" box, change the title of the book to all upper/lower case letters (by editing the title in the box) or, in the "Author(s)" box, change the author name to all upper/lower case letters.
7. Click "OK" in the lower right-hand corner of the screen

Expected behaviour: the book's title or author name is changed in the book list.

Actual behaviour: The cover and the book disappear. The ".opf" file remains, which Calibre cannot open (creates error).

Note: Books added with metadata already with lowercase and uppercase letters open quite normally. It is only when editing the meta-data that the book seems to get scrambled/lost. French language only (calibre v3.30) - unable to replicate on an English computer (calibre v3.30).

Reporter has attached a sample ebook with results, above.

[Kovid Goyal (kovid)]

The only way this can possibly happen is if the calibre library is on a broken filesystem such as a network share or a folder shared using a file sync service such as dropbox/gdrive/etc.

Move your calibre library to a correctly functioning ext4 filesystem that is not modified by programs other than calibre and you will be fine.

https://manual.calibre-ebook.com/faq.html#i-am-getting-errors-with-my-calibre-library-on-a-networked-drive-nas

[gf (gf-interlinks-deactivatedaccount)]

Thanks for the fast reply, Kovid. :)

Freikland,
Per Kovid's comment, is your calibre library on network share or a sync service such as dropbox or gdrive?
Thanks
G

[freikland (freikland)]

Hello everyone,

First, thank you Kovid for your wise and quick answer.
I take good note.
Indeed, my Calibre library is currently located on a Synology NAS with root permissions. When adding books, synchronization is done automatically via the "Cloud station drive" application.

So, I am copying the Calibre library to a local hard drive (ext3) before any other changes.

@ gf
After copying done on the local network, I think to modify the path of the library towards this one. I hope the behavior will improve when renaming metadata.

What do you think, is my approach the right one?

[gf (gf-interlinks-deactivatedaccount)]

Hi Freikland,
Yes, I think that is a great approach. :)
He did mention a ext4 file system but try your ext3 file system and see if that works.

And I believe you can set the location of the library in the settings screen of the Calibre application.

I don't know how you will organize you syncing after that. I have no knowledge of syncing.

Please let me know how it goes.
Have a great day!
:)
G

[Kovid Goyal (kovid)]

ext3 is fine.

[gf (gf-interlinks-deactivatedaccount)]

Thanks for yhe clarification, Kovid! That's good to know.
:)
G

[jeffmcneill (jeffmcneill)]

I've had this same issue in a 2.x version and now 3.30 version of Calibre. I'm running this on Debian 9 (LMDE3) and it appears that the Insync Google Drive synchronization tool causes/contributes to this issue. In other words, when Insync is paused, and I edit the Author field of any file (which changes the folder name of where the files reside) then there is no problem, but when Insyc is active, both the cover and the file disappear (are not in the Trash nor elsewhere in the folder structure).

While the workaround is to pause Insync, it would be preferred if Calibre were robust enough to operate as expected with sync/network drives. Since this appears to be a long-standing issue, my guess is it has to do with how file operations are performed by Calibre (specifically, renaming).

[gf (gf-interlinks-deactivatedaccount)]

Hi Jeff. Thanks for the update. As Kovid indicates, if you make sure your libray is stored on your hard drive, the issue is resolved.
Take care
:)
G

[jeffmcneill (jeffmcneill)]

Hi G,

My library is stored on my local disk. However, I have a sync utility running which synchronizes with Gdrive. The work-around is pausing the sync utility while I have Calibre running. However, I have to do this manually. There is definitely some kind of race condition (or otherwise named) where what Calibre is doing allows the sync utility to interfere in a way that files are lost.

[Kovid Goyal (kovid)]

filesystems have no concept of atomic operations. You cannot have
multiple software accessing the same set of files at the same time and
expect things to magically work. If you put your calibrelibrary on a NAS
or a folder that is modified by other software, things *will* break.

https://manual.calibre-ebook.com/faq.html#i-am-getting-errors-with-my-calibre-library-on-a-networked-drive-nas

[jeffmcneill (jeffmcneill)]

Actually, every other piece of software I use has no problem with being synched to gdrive. Calibre is the only thing that breaks. Clearly this has to do with a difference in the way Calibre functions, rather than NAS or file synchronization operations. Everything else is quite robust, including about a dozen file and media editors, from Libre Office, to video, audio, and image editors, to a variety of file editors, to KeePassXC. No one else complains about these file sync utilities or NAS drives, or has worked around any issues. Things do magically work, though not unfortunately for Calibre.

[Kovid Goyal (kovid)]

Because all of those utilities are editors, that work on one file at a
time. Anyway, I dont really care whether you believe me or not. File
systems are not atomic, that is a fact. Waving your hands and saying X,
Y or Z magically work, just means that you have been lucky enough to
avoid running into a situation where they will not work so far.

It's your data, feel free to risk it with broken ass applications like
gdrive.

[jeffmcneill (jeffmcneill)]

Yeah, I risk my data with broken ass applications like Calibre all the time. That is why I have Gdrive as a backup. Get real.