enblend running out of temp space: poor diagnostics and configuration possibilities

[Richard Z.]

Hi,

this is mostly a duplicate of https://bugs.launchpad.net/enblend/+bug/1220523.

I would like to add that the diagnostic messages and documentation could be a lot better in this situation.

Instead of "enblend: No space left on device" it would be much more helpfull to say *which device/dricetory* as this does not appear documented anywhere.

Furthermore, is this device hardcoded in the source code or is it configurable somewhere? Again, did not find anything in the documentation.

Finally, this just happened to me with
$ df /tmp/
Filesystem 1K-blocks Used Available Use% Mounted on
tmpfs 1028148 924 1027224 1% /tmp

and trying to stitch together 31 images. It would seem that similar situations may become increasingly common in the near future.

Regards

[Yuv (yuv)]

Hi Richard,

Thanks for reporting.

On 14-10-23 07:27 AM, Richard Z. wrote:
> this is mostly a duplicate of
> https://bugs.launchpad.net/enblend/+bug/1220523.

Let's find out.
(1) what system are you on? the bug being filed in Enblend and
hugin(Ubuntu) leads me to believe it is Ubuntu, but there is a new
version of Ubuntu out every six months, and it would also be helpful to
know on which architecture you are on.

(2) where did you get Enblend from? which version is it? `enblend -V`

if the version of enblend on your system is one with the image cache
activated like in the bug report you point to, it is obsolete. I have
been away for a few years from enblend, but I think I have seen a
message in the flow saying that the image cache will be phased out
because it does not play nicely with other features of enblend.

if you built it yourself, look at the comment preceding the last block
of commands in
<http://wiki.panotools.org/Hugin_Compiling_Ubuntu#Building_Enblend>.

> I would like to add that the diagnostic messages and documentation could
> be a lot better in this situation.

Nobody is perfect, sorry for that. Patches are always welcome. In a
similar spirit, you will forgive my pun, but there is too much analogy
between enblend's diagnostic messages and your bug report:

I would like to add that the diagnostic messages in your bug report
could be a lot better in this situation too ;-)

> Instead of "enblend: No space left on device" it would be much more
> helpfull to say *which device/dricetory* as this does not appear
> documented anywhere.

Instead of jumping to the conclusion that this is most likely a
duplicate of a bug that has been declared invalid (which would lead the
developers to declare this bug invalid too for the exact same reason) it
would be much more helpful to say *which system/enblend version* as this
does not appear documented anywhere.

> Furthermore, is this device hardcoded in the source code or is it
> configurable somewhere? Again, did not find anything in the
> documentation.
>
> Finally, this just happened to me with
> $ df /tmp/
> Filesystem 1K-blocks Used Available Use% Mounted on
> tmpfs 1028148 924 1027224 1% /tmp
>
> and trying to stitch together 31 images. It would seem that similar
> situations may become increasingly common in the near future.

`df -h` would have been user friendlier. So you have 1GB of /tmp space.
 Have you tried the option "-m 1000000MB" ? How many megapixel each
image? What bit depth? 1GB of image cache would not be enough to stich
31 of the images I used to stitch a few years back.

Anyway, I hope you take this reply with humour. If you want to help
solve the problem, look at the Wiki page linked above, follow the
instructions to build your own version of enblend without image cache
and report back how it does. I see that there has not been any newer
version of enblend in the Hugin PPA
<https://launchpad.net/~hugin/+archive/ubuntu/hugin-builds> and maybe it
is time that we put one out there.

Thanks for bearing with me.
Regards,
Yuv

[Christoph Spiel (cspiel)]

The environment variable `TMPDIR' controls the placement of temporary files
as documented in section "Tuning Memory Usage" of the Enblend and Enfuse
manuals.

The actual, assigned directory for the ImageCache can easily be inquired by
calling Enblend or Enfuse with the option pair `--verbose --version' or shorter
`-Vv', and the relevant part of answer looks like this:
    Extra feature: image cache: yes
      - environment variable TMPDIR not set, cache file in default directory "/tmp"
or
    Extra feature: image cache: yes
      - environment variable TMPDIR set, cache file located in "/work"
if
    TMPDIR=/work

Both options, `--verbose' and `--version' are documented, too.

The user is responsible for setting up TMPDIR (if required at all) as well as
for feeding properly overlapping images with suitable alpha channel into
Enblend or Enfuse.

[Richard Z. (rz-e)]

Thank you both for your suggestions.

I am using enblend with the hugin frontend on Fedora.

enblend-4.1.1-3.fc19.i686

$ enblend -V
enblend 4.1.1

I noticed the 4.2 version in the mercurial repo appears to have more verbose output and more documentation which would have helped a lot in my case.

The TMPDIR ought also be mentioned in the manpage, attaching a patch.

However, looking at the source code I am wondering if tmpdir gets used at all - or is it dead code now?

In my case I had additional screwup as hugin discarded the preset TMPDIR and when setting the hugin tmpdir via gui somehow this setting has been reset on one occasion

Did not try "-m 1000000MB".

[Christoph Spiel (cspiel)]

OK -- even at the danger of loosing focus let me
try to explain each your points.

> enblend-4.1.1-3.fc19.i686

        The latest release in the Stable
Branch is 4.1.3. We are already preparing 4.1.4.
OTOH, the Development Branch of course holds
much more goodies.

> The TMPDIR ought also be mentioned in the
> manpage, attaching a patch.

        THX for your patch! We have a pretty
similar one in rev188e286e471d, which I just
back-ported to the Stable Branch. It will show
up in 4.1.4.

> However, looking at the source code I am
> wondering if TMPDIR gets used at all - or is
> it dead code now?

(i) Enblend and Enfuse only refer to the
    environment variable `TMPDIR' if they were
    compiled with the ImageCache feature.
    Otherwise they store everything in core and
    thus don't need a reference to a scratch
    directory.

(ii) The ImageCache feature was withdrawn in
    4.2 because of the spurious problems it
    causes; see, e.g.
        https://bugs.launchpad.net/enblend/+bug/807439
    Moreover, the ImageCache is incompatible
    with OpenMP, our main parallelization
    technology. Therefore, current Development
    Branch does not use `TMPDIR', either.

(iii) The `mmap_view' branch
        http://hg.code.sf.net/p/enblend/code/rev/7a3964af671a
    uses a different "caching scheme" to offload
    image data from core to disk. It needs to
    know where to store the backing-files for
    the images and thus again refers to
    `TMPDIR'.

> In my case I had additional screw up as hugin
> discarded the preset TMPDIR and when setting
> the hugin tmpdir via gui somehow this setting
> has been reset on one occasion.

        This looks more like a Hugin issue to
me, although we can think about adding a
command-line option to set/override `TMPDIR'.

You could easily work around the problem by
wrapping the calls to Enblend and Enfuse in
shell scripts that override `TMPDIR'. I imagine
something like

    #! /bin/dash
    export TMPDIR=/work
    exec /usr/local/bin/enblend "$@"

[Richard Z. (rz-e)]

An afterthought, much my trouble was caused because hugin did override a valid TMPDIR enviroment variable even though I did not enter any value for the tempdir in hugin GUI preferences..

Did not test if this still happens with newest hugin.

[Christoph Spiel (cspiel)]

Clarified in the documentation of 4.2.