Maximum TIFF file size 4GB

[kaefert]

Hi there!

After running for 2 days the enblend binary that I compiled myself from the enblend sources revision 1049 failed with this error:

enblend: info: writing final output
enblend: error: Maximum TIFF file size exceeded

enblend: an exception occured
enblend:
Postcondition violation!
exportImage(): Unable to write TIFF data.
(/build/buildd/libvigraimpex-1.9.0+dfsg/src/impex/tiff.cxx:811)

enblend: info: remove invalid output image "x1.tif"
===================================================

I've used these parameters: --compression=LZW --verbose --output=x1.tif

The output pixel dimensions would have been 119598 * 17362 =
2076460476 which is still well within the 2^31 pixel count limit =
2147483648

In the bug report #685105 I've mentioned that I've run into this problem before, and back then Jeff Hungerford (hungerf3) told me:
"If you are seeing that error, either the TIFF library you linked against doesn't support bigTIFF, or for some reason it decided to only use the old format."

It would be very kind of you if you could guide my way how to compile enblend (and libtiff?) in a way that enblend will not fail when the output exceeds 4GB in file size.

Or is there maybe a way to use another output format then tif to workaround this problem? (With Gimp I like to use png for images that are too big for 4GB tif, it has the same problem of failing to save >4GB tiffs for me)

[kaefert (kaefert)]

Okey, so I found this:
http://drgutman.org/compiling-libtiff-on-ubuntu/

and will try to follow this guide when I'll get home tonight. I won't be able to tell you if it worked for a few days, since it takes a while to make such a huge picture to run into the 4GB limit.

[kaefert (kaefert)]

Hmm, okey. so downloading and building http://download.osgeo.org/libtiff/tiff-4.0.3.zip did work, and the new build does seems to get used, since the output of enblend changed a little bit, since those two lines get printed which did not get printed before:

/home/kaefert/src/enblend/enblend.build_1049/bin/enblend: /usr/local/lib/libtiff.so.5: no version information available (required by /home/kaefert/src/enblend/enblend.build_1049/bin/enblend)
/home/kaefert/src/enblend/enblend.build_1049/bin/enblend: /usr/local/lib/libtiff.so.5: no version information available (required by /usr/lib/libvigraimpex.so.4)

Though sadly it still fails the same way:
enblend: error: Maximum TIFF file size exceeded

[kaefert (kaefert)]

I've asked on the libtiff mailing list if they can help me, and this is the response I got from user John:

libtiff4 will write a smalltiff file by default. Bigtiff is not a
compatible format, you need to specifically ask for it. Swap:

   output = TIFFOpen( output_filename, "w" );

for

   output = TIFFOpen( output_filename, "w8" );

enblend will need to set this flag and I guess they'll need a bit of
UI to let people turn it on or off.

[Christoph Spiel (cspiel)]

        This is not an Enblend (or Enfuse)
problem. Both applications rely on the Vigra
library for I/O of images of _any_ format, i.e.
the functions described in
        https://ukoethe.github.io/vigra/doc/vigra/group__VigraImpex.html
This is in fact one of the main reasons to use
an imaging library as e.g. Vigra.

You could ask the Vigra developers for
integration of BigTIFF into Vigra or even supply
the necessary patch yourself.

[kaefert (kaefert)]

Okey, so Ulli (a Vigra developer) wrote on the Vigra mailing-list that he fixed / implemented bigtiff support:

"I uploaded a fix to 'master' on github. BigTIFF now seems to work."

To enable bigtiff he said
"you have to specify mode "w8" in ImageExportInfo (C++) or writeImage() (Python)."

Does enblend do this? specifying mode "w8"? Or could you add it? It would probably be best if there was a command line switch for enblend that triggers this, since in many cases users would probably like to have "normal" tiff files.

Kind Regards,

[tmodes (tmodes)]

Implemented in changeset 37e4d310b065. For BigTIFF output you need to compile enblend/vigra with libtiff with BigTIFF support and add `--parameter=export-bigtiff´ to command line.