import error of epub after imported into iBooks 1.1 on a Mac

[claas]

I tried to do the following on 2 different Macs and the same result:
Debug log attached; I could provide both ePub files since I took them for testing from http://gutenberg.org

1. Import an ePub into iBooks 1.1 on Mac OS X Yosemite latest Version but not Beta. Calibre 2.15
2. Connect a folder or directly take the ePub file which is in the iBooks Library and import it into calibre
3. Then following error appeared:
calibre, version 2.15.0
WARNING: Errors while adding: There were problems adding some files, click "Show details" for more information

----------------------------------------------------------------------
Failed to add the file /Users/username/Downloads/4C723FB1615B399DD71076C8078F5BEA.epub to the book: /Users/username/Downloads/4C723FB1615B399DD71076C8078F5BEA
With error:
Traceback (most recent call last):
  File "site-packages/calibre/gui2/add.py", line 396, in add_formats
  File "site-packages/calibre/db/cache.py", line 1335, in add_format
  File "site-packages/calibre/startup.py", line 183, in local_open
IOError: [Errno 21] Is a directory: u'/Users/username/Downloads/4C723FB1615B399DD71076C8078F5BEA.epub'

4. If I tried to import the original ePub, not imported into iBooks, then it worked without a problem.

Somehow iBook is converting the ePub file. Have you heard about this and do you know how to solve the problem?
The file gets even 3 times the size after importing into iBook then the original ePub.

Would be great to have some help.

If you need more information please let me know
Thanks.

Claas.

PS:
I just ran a ls -la on that directory and see what:
drwxr-xr-x@ 5 claas staff 170 7 Jan 19:45 4C723FB1615B399DD71076C8078F5BEA.epub

It's indeed a directory, just the OS GUI wants to tell me it's a normal ePub by the extension and the behavior of the supposed opening possibility in other applications.

A quote from this page: http://www.robertsetiadi.net/getting-our-books-back-itunes-yosemite/

"To make things worse, iBooks unzipped the EPUB files into folders. In case you didn’t know, EPUB format is actually a compressed file that contains structures of files and folders inside. Storing those books in uncompressed format means: it takes more space."

So the only workaround might be a plugin creating again real epubs from the content of the folders I assume.

[Kovid Goyal (kovid)]

iBooks does not store epub files as epub files, it is not possible to
import files stored by iBooks into calibre. Import the originals
instead.

 status invalid

[claas (pcinfoc)]

For those like me who don't have the originals anymore here is a script to convert the epub and ibooks files (actually folder) which are not encrypted into real Calibre compatible epubs

[Kovid Goyal (kovid)]

Thanks, that might come in handy. You might want to post it in the Apple
devices and/or calibre related tools forums over at mobileread.com

[georges.mouse (georges-mouse)]

@class
i can not get the script running. (As i do not have any clue what i am doing this is not surprising.)

After I copy your script to the script editor and push play I get the error massage:
Lineend expected identifier found

What is my mistake?

[claas (pcinfoc)]

Hello Georges-Mouse,

thanks for your post. I try to explain the procedure and hope you will have success after it.

- Save the script somewhere, e.g. on the Desktop, don't copy the text in any file, you can just use the save functionality from the browser and you will have the runEpub.sh saved.
- Create a folder on somewhere, e.g. on the Desktop, called convertediBooks
- Open the terminal (the nice command line application)
- navigate to the location where the runEpub.sh ist saved (use the command 'cd' for that)
- run the script with following command: 'bash runEpub.sh /home/myUserName/Desktop/convertediBooks /home/myUserName/Desktop/runEpub.log'

The script knows where iBooks is storing the books, it will convert them and then put the converted books as ePub into the folder which you have created. The log file is optional.

Just keep in mind that this doesn't work with DRM protected files as far as I remember.

[georges.mouse (georges-mouse)]

@ claas

1. thanks for the (almost) fearsome fast replay
2. thanks for the working solution it is great!

[claas (pcinfoc)]

I haven't tried the latest version yet, but can it be that it solved the issue of not being able to open epub books from Apple iBooks?

Here the release notes excerpt:

"Edit Book: Allow editing an unzipped EPUB (a folder) as a book
Useful if you wish to store your ebook in a version control system and edit it directly with the calibre ebook editor (File->Open folder as book)"

[claas (pcinfoc)]

Again, my previous comment was regarding version: Release: 2.41 [16 Oct, 2015]

I haven't tried the latest version yet, but can it be that it solved the issue of not being able to open epub books from Apple iBooks?

Here the release notes excerpt:

"Edit Book: Allow editing an unzipped EPUB (a folder) as a book
Useful if you wish to store your ebook in a version control system and edit it directly with the calibre ebook editor (File->Open folder as book)"

[Kovid Goyal (kovid)]

No, this has nothing to do with importing books. You can always use the script attached here to convert ibooks folders into epub files before importing.

[Chris (lavaflake)]

I have just joined this site after finding this thread using a google search. I am having having this exact issue with trying to move an ePub from iBooks to Calibre. I got excited when I saw this thread and that there was a script to fix it. Being total out of my depth I followed the guide that claas (pcinfoc)posted. However following those instructions I get the message in terminal:
Error: You must supply the first argument as a directory!
#########################################################
Usage: runEpub.sh {1 2}
Usage: runEpub.sh {Exportfolder debug-log}
Usage: example runEpub.sh {/Users/*****/Documents/Exportfolder /Users/***/Destktop/runEpub.sh/debug.log}

This script will read the iTunes books from /Users/****/Library/Containers/com.apple.BKAgentService/Data/Documents/iBooks/Books
#########################################################

Following the example I tried the command runEpub.sh {/home/***/Desktop/convertediBooks /home/***/Desktop/runEpub.log}
but that results in
-bash: runEpub.sh: command not found

So I tried
/Users/***/Desktop/runEpub.sh {/home/***/Desktop/convertediBooks /home/***/Desktop/runEpub.log}

and got

-bash: /Users/***/Desktop/runEpub.sh: Permission denied

So I tired
sudo /Users/***/Desktop/runEpub.sh {/home/***/Desktop/convertediBooks /home/***/Desktop/runEpub.log}

And got

sudo: /Users/***/Desktop/runEpub.sh: command not found

The last post in this was a very long time ago. I am not sure if the issue is that I am doing something wrong or if because of software update the script no longer works. I not sure if anyone will see this but I would greatly appreciate some assistance. I am completely out of my depth here.

[claas (pcinfoc)]

Sorry for the confusion, was my first script ever I guess, not really professional.

Please try it this way

bash /Users/***/Desktop/runEpub.sh /home/***/Desktop/convertediBooks /home/***/Desktop/runEpub.log

The folder convertediBooks should already exist, it will not be created as far as I see the previous comments.
The script does not need to run as sudo. Just the curly braces as prompted by the script should not be written out.

Hope that helps.

[Chris (lavaflake)]

Thank you for the prompt response. Unfortunately, doing it as

bash /Users/USERNAME/Desktop/runEpub.sh /home/USERNAME/Desktop/convertediBooks /home/USERNAME/Desktop/runEpub.log

Still results in

Error: You must supply the first argument as a directory!
#########################################################
Usage: /Users/USERNAME/Desktop/runEpub.sh {1 2}
Usage: /Users/USERNAME/Desktop/runEpub.sh {Exportfolder debug-log}
Usage: example /Users/USERNAME/Desktop/runEpub.sh {/Users/USERNAME/Documents/Exportfolder /Users/USERNAME/Destktop//Users/USERNAME/Desktop/runEpub.sh/debug.log}

This script will read the iTunes books from /Users/USERNAME/Library/Containers/com.apple.BKAgentService/Data/Documents/iBooks/Books
#########################################################

runEpub.sh and the folder "convertediBooks"already exist on the desktop. Terminal is in the desktop directory. Atleast I believe it is, the text that sits before you type your command is MacBook-Pro:desktop USERNAME$

[claas (pcinfoc)]

Hi Chris,

thanks for the reply, okay please try this:

1. Is there a space in the full path for the "convertediBooks" folder? If so then it should be in double quotes.

2. Is it correct with the "/home/" directory? That sounds for me like a linux structure, I have tested this script only on macOS.

3. Should not be the reason but have you checked for too many spaces between the runEpub.sh and the following parameters?

4. Try drag and trop of the "convertediBooks" folder into the Terminal and it will know itself how to escape any special characters/whitespaces.

Regards,
Claas

[Chris (lavaflake)]

"/home/"was not correct and was the issue. I am using Mac OS
I used the command

bash /Users/USERNAME/Desktop/runEpub.sh /Users/USERNAME/Desktop/convertediBooks /Users/USERNAME/Desktop/runEpub.log

and it worked perfectly.

I had no idea what I was doing I was following the instructions/examples exactly, that was where I got the /home/

Thank you so much. I could kiss you. Spend the whole day trying fix this

[claas (pcinfoc)]

Glad it worked out in the end. You're very welcome.

[mamgozh (mamgozh)]

Hi I have the same problem :

I use mac book pro 2010 with mac os sierra 10.12.5
Ibook 1.10
Kobo aura one
Calibre 2.85.1

First i download a free epub book on my mac
Then I try to download on calibre and I have this message :

Échec à ajouter le fichier /Users/gwenolalemeur/Desktop/livre kobo/Jamais plus.epub au livre : /Users/gwenolalemeur/Desktop/livre kobo/Jamais plus
Avec l'erreur :
Traceback (most recent call last):
  File "site-packages/calibre/gui2/add.py", line 430, in add_formats
  File "site-packages/calibre/db/cache.py", line 1385, in add_format
  File "site-packages/calibre/startup.py", line 122, in local_open
IOError: [Errno 21] Is a directory: u'/Users/gwenolalemeur/Desktop/livre kobo/Jamais plus.epub'

I undertand there is a problem with epub files download on mac.
I'm novice, can you help me ?

I try to do what is explain behind, but I dont understarnd everything.

Thanksb for your help !!

[claas (pcinfoc)]

Hi mamgozh,

I understand correctly that you have downloaded a book with iBook?

Have you executed the script of post #3 after that?

[mamgozh (mamgozh)]

Hi,

I try but can you tell where i copy the script ?
a the began or the end ?

[Norman Margolus (toyuniverses)]

Don't try to import directly from the hidden iCloud Books directory!!!

The iBooks program will automatically convert any epubs it contains into normal zipped archives if you just drag them from the iBooks program into a folder. These can then be added to the Cailbre library without any issues.

This doesn't remove any DRM, if the epub originally had DRM, but books that start out without DRM will remain free of it.

[claas (pcinfoc)]

Hi Norman, thanks for your valuable comment. I haven't used iBooks since v1 where this procedure didn't exist AFAIK. Good it it was added