ABI break in libraw 0.19.3 causing crashes in freeimage, and probably elsewhere

[Daniel van Vugt]

The i386 testAPI in freeimage 3.18.0+ds2 crashes with proposed libraw 0.19.3-1:

free(): invalid pointer
Aborted (core dumped)

Originally reported at:

http://autopkgtest.ubuntu.com/packages/f/freeimage/eoan/i386

[Daniel van Vugt (vanvugt)]

(gdb) bt
#0 0xb7fd485d in __kernel_vsyscall ()
#1 0xb7a3cb36 in __libc_signal_restore_set (set=0xbfffeedc)
    at ../sysdeps/unix/sysv/linux/internal-signals.h:84
#2 __GI_raise (sig=6) at ../sysdeps/unix/sysv/linux/raise.c:48
#3 0xb7a26394 in __GI_abort () at abort.c:79
#4 0xb7a7fbac in __libc_message (action=<optimised out>, fmt=<optimised out>)
    at ../sysdeps/posix/libc_fatal.c:181
#5 0xb7a86c9f in malloc_printerr (
    str=str@entry=0xb7b94a47 "free(): invalid pointer") at malloc.c:5352
#6 0xb7a881f3 in _int_free (av=0xb7beb7a0 <main_arena>, p=<optimised out>,
    have_lock=0) at malloc.c:4181
#7 0xb7d9d44c in operator delete(void*) ()
   from /usr/lib/i386-linux-gnu/libstdc++.so.6
#8 0xb7d9d47c in operator delete(void*, unsigned int) ()
   from /usr/lib/i386-linux-gnu/libstdc++.so.6
#9 0xb7842435 in LibRaw::~LibRaw() ()
   from /usr/lib/i386-linux-gnu/libraw.so.19
#10 0xb78424ae in LibRaw::~LibRaw() ()
   from /usr/lib/i386-linux-gnu/libraw.so.19
#11 0xb7f6cb6b in Validate (io=<optimised out>, handle=<optimised out>)
    at Source/FreeImage/PluginRAW.cpp:644
#12 0xb7f4d212 in FreeImage_ValidateFIF (fif=FIF_RAW, io=0xbffff43c,
    handle=0x80666a0) at Source/FreeImage/Plugin.cpp:813
#13 0xb7f3f170 in FreeImage_GetFileTypeFromHandle (io=0xbffff43c,
    handle=0x80666a0, size=0) at Source/FreeImage/GetType.cpp:47
#14 0x0804e23c in testStreamMultiPageOpen (input=0x8050029 "sample.tif",
    flags=0) at testMPageStream.cpp:63
#15 0x0804e630 in testStreamMultiPage (lpszPathName=0x8050029 "sample.tif")
    at testMPageStream.cpp:205
#16 0x0804b6a3 in main (argc=1, argv=0xbffff544) at MainTestSuite.cpp:85

[Daniel van Vugt (vanvugt)]

In libraw:

ec78e397a2b202f0df89737a77266041cae962cc is the first bad commit
commit ec78e397a2b202f0df89737a77266041cae962cc
Author: Alex Tutubalin <email address hidden>
Date: Tue Jul 2 21:37:02 2019 +0300

    Metadata loop prevention; 0.19.3-release

:100755 100755 c3e4012fb34c71f6a5754f00c490a663f6c15bf6 0feedbd0cd72bf3900cebf28cb20f825934f7df1 M Changelog.txt
:040000 040000 2c79d0758084eef9364842aa4a8010f65091d9cc 1514d758b030f4553bfde33fe59f562393feab7e M dcraw
:040000 040000 f5ea6748292050c95895fc323b2492c2d7e7c105 46ba4fc7b6fb9a8d33bc6990a31b40c8cdf7d3a6 M internal
:040000 040000 f56fc59be89d49d57c91cfa3497c578a3942b786 44be58fba98c6dcfcfdfe73430ec393a7a17a547 M libraw

[Daniel van Vugt (vanvugt)]

==8515== Invalid write of size 4
==8515== at 0x47A0A97: LibRaw::identify() (dcraw_common.cpp:17756)
==8515== by 0x47BF8F3: LibRaw::open_datastream(LibRaw_abstract_datastream*) (libraw_cxx.cpp:2014)
==8515== by 0x409BB44: Validate(FreeImageIO*, void*) (PluginRAW.cpp:638)
==8515== by 0x407C211: FreeImage_ValidateFIF (Plugin.cpp:813)
==8515== by 0x406E16F: FreeImage_GetFileTypeFromHandle (GetType.cpp:47)
==8515== by 0x406E1F4: FreeImage_GetFileType (GetType.cpp:71)
==8515== by 0x804D9F6: testLockDeleteMultiPage(char const*) (testMPage.cpp:142)
==8515== by 0x804DB40: testMultiPage(char const*) (testMPage.cpp:189)
==8515== by 0x804B696: main (MainTestSuite.cpp:82)
==8515== Address 0x68a8a4c is 0 bytes after a block of size 539,996 alloc'd
==8515== at 0x4035F0B: operator new(unsigned int, std::nothrow_t const&) (in /usr/lib/i386-linux-gnu/valgrind/vgpreload_memcheck-x86-linux.so)
==8515== by 0x409BAAD: Validate(FreeImageIO*, void*) (PluginRAW.cpp:629)
==8515== by 0x407C211: FreeImage_ValidateFIF (Plugin.cpp:813)
==8515== by 0x406E16F: FreeImage_GetFileTypeFromHandle (GetType.cpp:47)
==8515== by 0x406E1F4: FreeImage_GetFileType (GetType.cpp:71)
==8515== by 0x804D9F6: testLockDeleteMultiPage(char const*) (testMPage.cpp:142)
==8515== by 0x804DB40: testMultiPage(char const*) (testMPage.cpp:189)
==8515== by 0x804B696: main (MainTestSuite.cpp:82)

[Daniel van Vugt (vanvugt)]

OK, the problem is a dumb ABI break. LibRaw version 0.19.3 added a new member to class LibRaw but did not flag it as an ABI change. So existing users of the library will now construct LibRaw objects slightly too small, leading to the above heap overrun/corruption when assigning field 'metadata_blocks'.

https://github.com/LibRaw/LibRaw/commit/ec78e397a2

The only fix is to just rebuild users of LibRaw, such as freeimage, and then the crash goes away.

[Daniel van Vugt (vanvugt)]

It looks like we're just going to have to do a rebuild of everything depending on libraw19 against 0.19.3, and then release those rebuilds atomically with libraw 0.19.3.

I think that's possible and have seen it done before, but I don't know what the process is...

libraw19
Reverse Depends:
  Depends: libraw-dev (= 0.19.3-1)
  Depends: deepin-image-viewer (>= 0.19.0)
  Depends: siril (>= 0.16.0)
  Depends: nomacs (>= 0.19.0)
  Depends: luminance-hdr (>= 0.16.0)
  Depends: libraw-bin (>= 0.19.0)
  Depends: libopenimageio2.0 (>= 0.16.0)
  Depends: libkf5kdcraw5 (>= 0.19.0)
  Depends: libgegl-0.4-0 (>= 0.16.0)
  Depends: libfreeimage3 (>= 0.19.0)
  Depends: libevas-loaders (>= 0.16.0)
  Depends: kstars (>= 0.16.0)
  Depends: krita (>= 0.19.0)
  Depends: gthumb (>= 0.16.0)
  Depends: fotoxx (>= 0.16.0)
  Depends: entangle (>= 0.16.0)
  Depends: libraw-dev (= 0.19.2-2)
  Depends: shotwell (>= 0.16.0)

[Steve Langasek (vorlon)]

On Wed, Jul 31, 2019 at 03:16:48AM -0000, Daniel van Vugt wrote:
> It looks like we're just going to have to do a rebuild of everything
> depending on libraw19 against 0.19.3, and then release those rebuilds
> atomically with libraw 0.19.3.
> I think that's possible and have seen it done before, but I don't know
> what the process is...

Because libraw19 is already in a stable release of Ubuntu, the binary
package should be renamed to libraw19debian1 so that the replacement is
atomic not only when publishing to the eoan release, but also when users are
upgrading.

'apt search --names-only debian1' shows you the prior art here.

--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer https://www.debian.org/
<email address hidden> <email address hidden>

[Daniel van Vugt (vanvugt)]

Alternatively... upstream are now proposing a fix for the ABI break, which would go into 0.19.4.

[Daniel van Vugt (vanvugt)]

Could someone reject/remove libraw 0.19.3-1 from proposed?

A fix will be in version 0.19.4, and we should wait for upstream and Debian on that. Because it's an ABI un-break that's not something we'd want to put in a patch.

[Sebastien Bacher (seb128)]

Thanks Daniel; I've removed the buggy version from proposed, let's wait for the next one to be in Debian&synced