enblend with imagecache creates artifacts

[moeeeep]

Hi
I'm using hugin-0.8.0_rc1 with libpano13-2.9.14_rc1
And it creates a lot of artifacts on huge panoramas.
http://downloads.moeeeep.com/panoramabug/Bildschirmfoto-pano5.png-1.0%20(RGB,%201%20Ebene)%2061638x10876%20%E2%80%93%20GIMP.png
http://downloads.moeeeep.com/panoramabug/Bildschirmfoto-pano5.png-1.0%20(RGB,%201%20Ebene)%2061638x10876%20%E2%80%93%20GIMP-3.png
And a line in the middle
http://downloads.moeeeep.com/panoramabug/Bildschirmfoto-pano5.png-1.0%20(RGB,%201%20Ebene)%2061638x10876%20%E2%80%93%20GIMP-1.png

The image: http://downloads.moeeeep.com/panoramabug/Bildschirmfoto-pano5.png-1.0%20(RGB,%201%20Ebene)%2061638x10876%20%E2%80%93%20GIMP-2.png

I had to create screenshots since gimp didn't allow me to save the image (not enough ram).
Here's the file hugin created: http://downloads.moeeeep.com/panoramabug/pano5.png (402mb --> needs at least 6.6gb ram to display). It has a size of 61638x10876 pixels

[moeeeep (moeeeep)]

The file ar1.jpg was added: Artifacts

[moeeeep (moeeeep)]

The file ar2.jpg was added: Artifacts

[moeeeep (moeeeep)]

The file line.jpg was added: Line in the middle

[nobody (nobody-users)]

I have the same problem, you can consult bug #2772325 ...

[Bruno Postle (brunopostle)]

Setting group to enblend, sorry no solution for this yet

[Yuv (yuv)]

did you compile Hugin from source? what operating system are you on? did you try the --fine-mask option in enblend? does it render artifacts also if you render at a smaller size, say 30000x5000? if you render twice the same project, are the artifacts exactly the same and in the same position?

[moeeeep (moeeeep)]

I didn't want to keep you waiting, until hugin is finished:

> did you compile Hugin from source?
yes
> what operating system are you on?
Gentoo 64bit

$ emerge --info
Portage 2.1.6.13 (default/linux/amd64/2008.0, gcc-4.3.3, glibc-2.10.1-r0, 2.6.30-gentoo-r1 x86_64)
=================================================================
System uname: Linux-2.6.30-gentoo-r1-x86_64-Intel-R-_Core-TM-2_Duo_CPU_E8500_@_3.16GHz-with-gentoo-2.0.1
Timestamp of tree: Sat, 18 Jul 2009 08:30:01 +0000
app-shells/bash: 4.0_p24
dev-java/java-config: 1.3.7-r1, 2.1.8-r1
dev-lang/python: 2.4.4-r6, 2.5.4-r2, 2.6.2-r1
dev-python/pycrypto: 2.0.1-r8
dev-util/cmake: 2.6.4-r1
sys-apps/baselayout: 2.0.1
sys-apps/openrc: 0.4.3-r3
sys-apps/sandbox: 2.0
sys-devel/autoconf: 2.13, 2.63-r1
sys-devel/automake: 1.5, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2, 1.10.2, 1.11
sys-devel/binutils: 2.19.1-r1
sys-devel/gcc-config: 1.4.1
sys-devel/libtool: 2.2.6a
virtual/os-headers: 2.6.30
ACCEPT_KEYWORDS="amd64 ~amd64"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=core2 -msse3 -O2 -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/kde/3.5/env /usr/kde/3.5/share/config /usr/kde/3.5/shutdown /usr/share/config"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c /etc/udev/rules.d"
CXXFLAGS="-march=core2 -msse3 -O2 -pipe"
DISTDIR="/usr/portage/distfiles"
FEATURES="distlocks fixpackages parallel-fetch protect-owned sandbox sfperms strict unmerge-orphans userfetch"
GENTOO_MIRRORS="http://mirror.switch.ch/ftp/mirror/gentoo/ http://gentoo.inode.at/ http://gentoo.mirror.solnet.ch/"
LANG="de_CH.UTF-8"
LDFLAGS="-Wl,-O1"
LINGUAS="en de"
MAKEOPTS="-j3"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage"
SYNC="rsync://rsync.gentoo.org/gentoo-portage"
USE="X Xaw3d a52 aac acl acpi alsa amarok amd64 arts artworkextra automount autoomount avahi berkdb bzip2 cairo cdaudio cddb cdr cli cracklib crypt cups curl daap dbus directfb dri dts dvb dvd dvdr dvdread encode exif fastcgi ffmpeg flac fortran ftp fuse gd gdbm gdm gif gimp git glitz gnome gphoto2 gpm gps gstreamer gtk gtk2 hal iconv id3tag ipod ipv6 isdnlog ithreads jabber jack java jbig jpeg jpeg2k kde kde4 lame latex lcd ldap libdvdread logitech-mouse mad matroska midi mmx mono mp2 mp3 mp4 mpeg mplayer mudflap multilib musicbrainz nas nautilus ncurses netplay nfs nls nptl nptlonly ogg opengl openmp pam pcre pdf perl plotutils png postscript pppd python qt3support qt4 quicktime rar readline reflection rtsp ruby samba scanner session skins spell spl sqlite3 sse sse2 ssl ssse3 subtitles suspend svg sysfs tcpd theora threads tiff truetype unicode usb vorbis wmf wxwindows x264 xcb xinerama xorg xosd xulrunner xv xvid xvmc zlib zoran" ALSA_CARDS="ali545...

[Yuv (yuv)]

looking forward for your results. thanks for testing. Yuv

[moeeeep (moeeeep)]

i'm sorry, but i have bad news for you:
> did you try the --fine-mask option in enblend?
for some reason, it wouldn't finish the project: I think there was an issue with enblend, since it stopped during the "create blending mask" process.

i think this issue could have something to do with the large amount of images (71, 12mp) and the fact that i shot all these images by hand. But i didn't have these artifacts before 0.8 rc1: like this panorama i created with 0.7-r1 and which consits of 75 hand shot images: http://downloads.moeeeep.com/panoramabug/pano14.jpg (140 mb; it would be perfect without the stupid line in the middle, but i believe that's another bug).

Should i try 0.8rc1 with the above panorama instead? Should i upgrade to rc5?

I'm really sorry,but i probably won't be able to post an update before the weekend, since i'm currently extremely busy preparing for exams.

[Yuv (yuv)]

thanks for the report. we're not in a hurry. do your exams first. good luck with them. ping this ticket back when you have time to continue.

when you updated to 0.8 rc1 has anything else changed on your system?

do you know what version of enblend-enfuse is on your system?

do I guess correctly that you got them with portage? can you point me to the ebuilds that you used? for both 0.7-r1 and 0.8 rc1?

at this moment I would not upgrade or change anything before establishing a thorough diagnosis.

FYI I've stitched a panorama of 294 images with a version from trunk that corresponds to one of the 0.8 rc and I did not have these artifacts.

again: good luck with the exams and let's debug this later when you have time.

[moeeeep (moeeeep)]

> when you updated to 0.8 rc1 has anything else changed on your system?
I don't think so, but i upgraded quite some time ago.

> do you know what version of enblend-enfuse is on your system?
I use version 3.2

> do I guess correctly that you got them with portage? can you point me to
> the ebuilds that you used? for both 0.7-r1 and 0.8 rc1?
hugin-0.8.0_rc1: http://gentoo-portage.com/AJAX/Ebuild/88466 or
http://sources.gentoo.org/viewcvs.py/gentoo-x86/media-gfx/hugin/hugin-0.8.0_rc1.ebuild?view=markup
hugin-0.7.0-r1: http://gentoo-portage.com/AJAX/Ebuild/57981 or
http://sources.gentoo.org/viewcvs.py/gentoo-x86/media-gfx/hugin/hugin-0.7.0-r1.ebuild?view=markup
(you can find all the ebuilds here: http://sources.gentoo.org/viewcvs.py/gentoo-x86/media-gfx/hugin/ )
enblend-3.2: http://gentoo-portage.com/AJAX/Ebuild/58122 or
http://sources.gentoo.org/viewcvs.py/gentoo-x86/media-gfx/enblend/enblend-3.2.ebuild?view=markup

> FYI I've stitched a panorama of 294 images with a version from trunk that
> corresponds to one of the 0.8 rc and I did not have these artifacts.
hugin complains, that the images do not fit together very well. I think some of these patches are created, when some images do not fit together: I created a screenshot of the preview:
http://downloads.moeeeep.com/panoramabug/Bildschirmfoto-Panoramavorschau-1.png notice the post (?pole) on the right side of the image. this resulted in
http://downloads.moeeeep.com/panoramabug/holz.png

> if you render twice the same
> project, are the artifacts exactly the same and in the same position?
No, not exactly. the artifacts happen in the same places, but the black dots are on a different position (the amount of them varies too):
http://downloads.moeeeep.com/panoramabug/Bildschirmfoto-4.png I tried to corelate the new image with the old image: The new one is on the right side and the old image is on the left side.

> does it render artifacts also if
> you render at a smaller size, say 30000x5000?
I don't know. Since at some point the "create blendmask process" crashes with all the smaller image sizes: It eats up all memory and crashes. --fine-mask was not enabled.
I tried with: 15000x2647px, 28000x4941px and 29000x5115px
Log from one of the crashes:
http://downloads.moeeeep.com/panoramabug/hugin_crashlog.log

[Yuv (yuv)]

Hallo Pascal, danke fuer den ausfuerhlichen Bericht.

from your description ("artifacts happen in the same places, but the black
dots are on a different position") I suspect that this is a memory issue. an overflow somewhere. Enblend-enfuse has not been very stable recently.

http://sources.gentoo.org/viewcvs.py/gentoo-x86/media-gfx/enblend/enblend-3.2.ebuild?view=markup is the ebuild you want to modify.

your best bet currently is to use the staging branch at https://code.launchpad.net/~cspiel/enblend/staging

you will need to use bzr to fetch it.

the instructions for Ubuntu are at http://wiki.panotools.org/Hugin_Compiling_Ubuntu#Building_Enblend
for gentoo I am not sure. You will want to replace SRC_URI="mirror://sourceforge/enblend/${PN}-enfuse-${PV}.tar.gz"
 with something that gets enblend-enfuse with bzr from Launchpad, and you won't need the src_unpack() stuff because it is already unpacked.

Your best option, if you are not knowledgeable enough to do that by yourself, is http://wiki.panotools.org/Hugin_Compiling_Gentoo or ask a question on the hugin-ptx mailing list. http://groups.google.com/group/hugin-ptx/

[nobody (nobody-users)]

Ok. I'm trying it out now :)
An ebuild is already available through the panotools overlay ( http://pani.webhop.org/git/panotools-overlay.git ). It can be installed through layman (see: http://wiki.panotools.org/Hugin_Compiling_Gentoo ).

[moeeeep (moeeeep)]

It seems to me that the staging version of enblend removed quite a lot of these artifacts: http://downloads.moeeeep.com/panoramabug/pano5_staging_cropped.jpg (20816x4584px -> it might kill your browser).
The issue with the line ine the middle wasn't resolved so far: http://downloads.moeeeep.com/panoramabug/enblend_and_enblend_staging_compared.png (left side: enblend staging, right side: enblend <=3.2). I did however change the horizon in the middle to get a better crop, I'll try to do another run with the original parameters.
Original: http://downloads.moeeeep.com/panoramabug/pano_hochfuegen_enblend_3.2.jpg (huge file)
Staging: http://downloads.moeeeep.com/panoramabug/pano_hochfuegen_enblend_staging.jpg (huge file)

[rew (r-e-wolff)]

A friend found a workaround to reduce/remove the stripes in the images: Increase the imagecache size. This reduces the amount of artifacts. In his case to "none". this is usable for those blends where RAM is sufficent to hold everything (but not the default 1Gb).

This is a hint that the problem lies in the imagecache code.

[rew (r-e-wolff)]

It happens with 4.0 TOO.

I have blended my panorama now with a 4.0 build on a 64bit machine. I just let the OS swap. It required over 12Gb of RAM/swap, and finished the blend just fine!

I'm now pretty sure that it is "imagecache" that causes the corruption or crashes.
I chose to NOT use imagecache to test this and sure enough the blend went fine.

[rew (r-e-wolff)]

Imagecache ALSO affects enfuse.

I have just verified that the crash I had was caused by imagecache. Just turning imagecache off fixes the problem.

[rew (r-e-wolff)]

I was going to add my PTO, but I can reproduce it with:

enfuse -w -o dsc_3210-dsc_3254_stack_ldr_0005.tif dsc_3210-dsc_3254_exposure_layers_0018.tif dsc_3210-dsc_3254_exposure_layers_0019.tif dsc_3210-dsc_3254_exposure_layers_0020.tif

The source images have just one small triangle of data wrapping around the 360 degree boundary.

[rew (r-e-wolff)]

To reproduce you can find my source images at:
http://prive.bitwizard.nl/dsc_3210-dsc_3254_exposure_layers_0018.tif
http://prive.bitwizard.nl/dsc_3210-dsc_3254_exposure_layers_0019.tif
http://prive.bitwizard.nl/dsc_3210-dsc_3254_exposure_layers_0020.tif

It happens with the current version of enblend/enfuse, just pulled from HG.