support tar'ed image stacks

[mangtronix]

Support image stacks within tar instead of zip

[mangtronix (mang)]

There also exist jpg.zip image stacks.

[mangtronix (mang)]

Needed when image stack is >2gb (we make a tar image stack instead of zip).

[mangtronix (mang)]

"For ACPL, we currently have 29 reels that have the PDF file, but do not
have a "Read Online" option.

Below are five examples:

http://www.archive.org/details/fifteenthcensus00reel2134

http://www.archive.org/details/fifteenthcensus00reel2138

http://www.archive.org/details/fifteenthcensus00reel2142

http://www.archive.org/details/fifteenthcensus00reel2145

http://www.archive.org/details/fifteenthcensus00reel2149"

This is now urgent by request from Ken.

[mangtronix (mang)]

These books may use tar files since the image stacks are >2gb. Unix "tar" was too slow so we're using 7z instead (which, awesomely, uses seeking between the tar record headers).

Doing some cleanup but should be ready for testing soon.

[mangtronix (mang)]

The code is ready for testing on www-testflip.archive.org

For example, here are the books you mentioned.

http://www-testflip.archive.org/details/fifteenthcensus00reel2134

http://www-testflip.archive.org/details/fifteenthcensus00reel2138

http://www-testflip.archive.org/details/fifteenthcensus00reel2142

http://www-testflip.archive.org/details/fifteenthcensus00reel2145

http://www-testflip.archive.org/details/fifteenthcensus00reel2149

[mangtronix (mang)]

The first time you load an image in one of these books it may take a long time (>10s). Once the file cache is hot access should be nice and snappy.

I added two feature requests to make this better for the user:
Spinny icon while image loading: https://bugs.edge.launchpad.net/bookreader/+bug/511389
Don't pre-cache invisible images before visible ones loaded: https://bugs.edge.launchpad.net/bookreader/+bug/511391

Since these seem to require dependably catching the image loaded event cross-browser, I'm pushing them to a future release, and making the books accessible now.

[mangtronix (mang)]

>430,000 texts with Tar image stacks and scandata.

http://www.archive.org/search.php?query=mediatype%3A%28texts%29%20AND%20format%3A%28Processed%20Tar%29%20AND%20format%3A%28Scandata%29

[mangtronix (mang)]

Ah but only a few (40) of those don't also have zips. It looks like we usually store the original jp2s in .tar

http://www.archive.org/search.php?query=mediatype%3A%28texts%29%20AND%20format%3A%28Processed%20Tar%29%20AND%20format%3A%28Scandata%29%20-format%3A%28Processed%20Zip%29

[mangtronix (mang)]

Testers - please check that books with tar image stacks work, and that a random pick of some other books still work. There's a link to books with tar image stacks above.

To access the testing host replace www.archive.org with www-testflip.archive.org

There may be a long delay accessing the first page. I've added some feature requests for making that better in comment #6.

[raj (raj-archive)]

Testing in OSX 10.5 / Safari 4.0.1

These tar image stack books work:
http://www-testflip.archive.org/stream/acpl_california_04_reel04
http://www-testflip.archive.org/stream/fifteenthcensus00reel1969
http://www-testflip.archive.org/stream/fifteenthcensus00reel1972
http://www-testflip.archive.org/stream/villanovan06_02_reel02

This book works but has a color card for the last page:
http://www-testflip.archive.org/stream/revisedstatutes197007uoft

After the initial delay, the bookreader seems pretty snappy!

[raj (raj-archive)]

The above links all work in OSX 10.5 / FF 3.5.3 as well.

[mangtronix (mang)]

Great. The book with colour card has the card included as addToAccessFormats true in the scandata. Looks like operator error.