Segfault on different language

[Kuzemko Aleksandr]

Some bug report.
When I recognize my test image (0096.tif) with English language it recognize it without segfaults (but ukrainian text text is show as english letters).
When I recognize my test image (0096.tif) with -l rus or -l ukr options it recognize it with segfaults.
When I recognize my test image (0096.tif) with -l ruseng options it recognize it without segfaults.
But when I use another test image (0005.tif)I can recognize it fine (with proper language and without segfaults )

[Jussi Pakkanen (jpakkane)]

What platform are you running on? It works without crashes on OS X.

[Alex Samorukov (samm-os2)]

Confirmed on linux-amd64:
samm@samm-laptop:~/src/cuneiform-linux/builddir$ ./cuneiform 0096.tif -l ukr
Cuneiform for Linux 0.3.1
The image depth is 1 at this point.
Segmentation fault

(gdb) run 0096.tif -l ukr
Starting program: /home/samm/src/cuneiform-linux/builddir/cuneiform 0096.tif -l ukr
[Thread debugging using libthread_db enabled]
Cuneiform for Linux 0.3.1
The image depth is 1 at this point.
[New Thread 0x7f3cc140b6f0 (LWP 5062)]

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7f3cc140b6f0 (LWP 5062)]
0x00007f3cbce62610 in make_white_hist (pint=0x7f3cbba1dca0 "", height=54)
    at /home/samm/src/cuneiform-linux/cuneiform_src/Kern/rstr/src/diffrv.c:3056
3056 while((segm++)->segwhite != 0x9000);
Current language: auto; currently c

(gdb) bt
#0 0x00007f3cbce62610 in make_white_hist (pint=0x7f3cbba1dca0 "", height=54)
    at /home/samm/src/cuneiform-linux/cuneiform_src/Kern/rstr/src/diffrv.c:3056
#1 0x00007f3cbce577ea in r_criteria (c=0x8a9170, gl=0x7fffc9440990) at /home/samm/src/cuneiform-linux/cuneiform_src/Kern/rstr/src/diffrv.c:335
#2 0x00007f3cbce7467f in dmiBOX (A=0x8a9170, GL=0x7fffc9440990, fl2=1) at /home/samm/src/cuneiform-linux/cuneiform_src/Kern/rstr/src/dmbox.c:582
#3 0x00007f3cbce75ed0 in dmBOX (BC=0x8a9170, GL=0x7fffc9440990) at /home/samm/src/cuneiform-linux/cuneiform_src/Kern/rstr/src/dmbox.c:905
#4 0x00007f3cbce1caeb in one_glue (n=2, S=0x7fffc9440b90, tol=220) at /home/samm/src/cuneiform-linux/cuneiform_src/Kern/rstr/src/cg_main.c:1115
#5 0x00007f3cbce1bb55 in process_frame (WB=0x8a98f0, WE=0x8aa070) at /home/samm/src/cuneiform-linux/cuneiform_src/Kern/rstr/src/cg_main.c:818
#6 0x00007f3cbce1b7fd in process_word (WB=0x8a98f0, WE=0x8aa070) at /home/samm/src/cuneiform-linux/cuneiform_src/Kern/rstr/src/cg_main.c:746
#7 0x00007f3cbce1a9f3 in cuts_glues () at /home/samm/src/cuneiform-linux/cuneiform_src/Kern/rstr/src/cg_main.c:468
#8 0x00007f3cbcea3d0a in pass3 (ln=0xb4fd10, lout=0xca6ac0) at /home/samm/src/cuneiform-linux/cuneiform_src/Kern/rstr/src/pass3.c:876
#9 0x00007f3cbcec52c4 in RSTRRecognizeMain (lin=0xb4fd10, lino=0xca6ac0) at /home/samm/src/cuneiform-linux/cuneiform_src/Kern/rstr/src/rcm.c:1702
#10 0x00007f3cbcec4dc6 in RSTRRecognize (lin=0xb4fd10, lino=0xca6ac0) at /home/samm/src/cuneiform-linux/cuneiform_src/Kern/rstr/src/rcm.c:1383
#11 0x00007f3cbcec5f74 in RSTR_Recog (lini=0xb4fd10, lino=0xca6ac0) at /home/samm/src/cuneiform-linux/cuneiform_src/Kern/rstr/src/rcm.c:2109
#12 0x00007f3cc1009355 in RecognizeStringsPass1 () at /home/samm/src/cuneiform-linux/cuneiform_src/Kern/puma/c/partrecog.cpp:250
#13 0x00007f3cc1009e32 in Recognize () at /home/samm/src/cuneiform-linux/cuneiform_src/Kern/puma/c/partrecog.cpp:623
#14 0x00007f3cc100b6dc in PUMA_XFinalRecognition () at /home/samm/src/cuneiform-linux/cuneiform_src/Kern/puma/main/puma.cpp:595
#15 0x0000000000402926 in main (argc=4, argv=0x7fffc9441868) at /home/samm/src/cuneiform-linux/cuneiform_src/Kern/cuneiform-cli.cpp:346

in rcriteria() language is checked, i think this is the reason why it does not crash on eng language.

[Alex Samorukov (samm-os2)]

the bug is related to amd64 - test works fine on linux i386

[Jussi Pakkanen (jpakkane)]

I pushed a fix to trunk. Please check whether it works for you.

[Alex Samorukov (samm-os2)]

it still crash, but the error is different:
(gdb) bt
#0 0x00007f352c42b095 in raise () from /lib/libc.so.6
#1 0x00007f352c42caf0 in abort () from /lib/libc.so.6
#2 0x00007f352c465a7b in ?? () from /lib/libc.so.6
#3 0x00007f352c4e6607 in __fortify_fail () from /lib/libc.so.6
#4 0x00007f352c4e65d0 in __stack_chk_fail () from /lib/libc.so.6
#5 0x00007f352fad09b3 in check_futuris_aa (rst=0x7fff3c0939b0) at /home/samm/src/cuneiform-linux/cuneiform_src/Kern/rstr/src/diffrv.c:2572
#6 0x00007f352facfdf5 in check_oa (c=0x8a93b0, let=174 '�', rst=0x7fff3c0939b0) at /home/samm/src/cuneiform-linux/cuneiform_src/Kern/rstr/src/diffrv.c:2437
#7 0x00007f352fac80da in r_criteria (c=0x8aaeb0, gl=0x7fff3c094230) at /home/samm/src/cuneiform-linux/cuneiform_src/Kern/rstr/src/diffrv.c:486
#8 0x00007f352fae467f in dmiBOX (A=0x8aaeb0, GL=0x7fff3c094230, fl2=1) at /home/samm/src/cuneiform-linux/cuneiform_src/Kern/rstr/src/dmbox.c:582
#9 0x00007f352fae5ed0 in dmBOX (BC=0x8aaeb0, GL=0x7fff3c094230) at /home/samm/src/cuneiform-linux/cuneiform_src/Kern/rstr/src/dmbox.c:905
#10 0x00007f352fa96d73 in full_recog (B1=0x8aaeb0, gl0=0x7fff3c094230, trs=220, tol=2) at /home/samm/src/cuneiform-linux/cuneiform_src/Kern/rstr/src/cg_main.c:4185
#11 0x00007f352fa929b7 in recog_one (B0=0x8aaeb0, GL=0x7fff3c094230, tol=220, vers=0x7fff3c094a60) at /home/samm/src/cuneiform-linux/cuneiform_src/Kern/rstr/src/cg_main.c:2847
#12 0x00007f352fa92bac in recog_set (C=0x8a9530, org_cells=0x7fff3c0945a8, N=2, cut_fl=16 '\020', cut_fine=32 ' ', inc=10000, tolbox=220, gvar=0x7fff3c094b4c "\001 \020",
    vers=0x7fff3c094a60, width=0x7fff3c094b3a) at /home/samm/src/cuneiform-linux/cuneiform_src/Kern/rstr/src/cg_main.c:2902
#13 0x00007f352fa921ba in recogij (C=0x8a9530, org_cells=0x7fff3c0945a8, N=2, cut_fl=16 '\020', cut_fine=32 ' ', inc=10000, roi=0x7fff3c094af0, gvar=0x7fff3c094b4c "\001 \020",
    vers=0x7fff3c094a60, width=0x7fff3c094b3a) at /home/samm/src/cuneiform-linux/cuneiform_src/Kern/rstr/src/cg_main.c:2698
#14 0x00007f352fa909a8 in accept_segment (C=0x8a9530, r0=0x7fff3c0ae180, cut_list=0x7fff3c0ac140, vers_list=0x7fff3c0ab940, i1=0, i0=6, mode=0 '\0')
    at /home/samm/src/cuneiform-linux/cuneiform_src/Kern/rstr/src/cg_main.c:2223
#15 0x00007f352fa8f784 in addij (C=0x8a9530, r0=0x7fff3c0ae180, cut_list=0x7fff3c0ac140, vers_list=0x7fff3c0ab940, ncut=17, i1=0, i0=6, mode=0 '\0')
    at /home/samm/src/cuneiform-linux/cuneiform_src/Kern/rstr/src/cg_main.c:1988
#16 0x00007f352fa992b4 in dp_pass0 (CP=0x8a9530, rp=0x7fff3c0ae180, cutp=0x7fff3c0ac140, versp=0x7fff3c0ab940, n=17)
    at /home/samm/src/cuneiform-linux/cuneiform_src/Kern/rstr/src/cg_smcut.c:417
#17 0x00007f352fa8d0dc in cut_glue (LC=0x8a9530, E=0x8a92f0, ovfl=0 '\0') at /home/samm/src/cuneiform-linux/cuneiform_src/Kern/rstr/src/cg_main.c:1245
#18 0x00007f352fa8c65d in process_frame (WB=0x8a9470, WE=0x8a9170) at /home/samm/src/cuneiform-linux/cuneiform_src/Kern/rstr/src/cg_main.c:1009
#19 0x00007f352fa8b7fd in process_word (WB=0x8a9470, WE=0x8a9170) at /home/samm/src/cuneiform-linux/cuneiform_src/Kern/rstr/src/cg_main.c:746
#20 0x00007f352fa8a9f3 in cuts_glues () at /home/samm/src/cuneiform-linux/cuneiform_src/Kern/rst...

[Kuzemko Aleksandr (kuzemkoa-rambler)]

I have another bug trace in debuger:
(gbb) run
Reading symbols from /lib64/ld-linux-x86-64.so.2...done.
Reading symbols from system-supplied DSO at 0x7fff509fd000...done.
Reading symbols from /home/starterkit/libusb-driver.so...done.
Reading symbols from /home/starterkit/cuneiform/cuneiform-linux/builddir/libpuma.so...done.
Reading symbols from /usr/lib64/libMagick++.so.1...done.
Reading symbols from /usr/lib64/libMagickCore.so.1...done.
Reading symbols from /home/starterkit/cuneiform/cuneiform-linux/builddir/librfrmt.so...done.
Reading symbols from /home/starterkit/cuneiform/cuneiform-linux/builddir/librout.so...done.
Reading symbols from /home/starterkit/cuneiform/cuneiform-linux/builddir/libced.so...done.
Reading symbols from /home/starterkit/cuneiform/cuneiform-linux/builddir/librmarker.so...done.
Reading symbols from /home/starterkit/cuneiform/cuneiform-linux/builddir/librneg.so...done.
Reading symbols from /home/starterkit/cuneiform/cuneiform-linux/builddir/librselstr.so...done.
Reading symbols from /home/starterkit/cuneiform/cuneiform-linux/builddir/librstuff.so...done.
Reading symbols from /home/starterkit/cuneiform/cuneiform-linux/builddir/librimage.so...done.
Reading symbols from /home/starterkit/cuneiform/cuneiform-linux/builddir/librline.so...done.
Reading symbols from /home/starterkit/cuneiform/cuneiform-linux/builddir/librshelllines.so...done.
Reading symbols from /home/starterkit/cuneiform/cuneiform-linux/builddir/librverline.so...done.
Reading symbols from /home/starterkit/cuneiform/cuneiform-linux/builddir/libcimage.so...done.
Reading symbols from /home/starterkit/cuneiform/cuneiform-linux/builddir/libcfio.so...done.
Reading symbols from /home/starterkit/cuneiform/cuneiform-linux/builddir/librblock.so...done.
Reading symbols from /home/starterkit/cuneiform/cuneiform-linux/builddir/librcorrkegl.so...done.
Reading symbols from /home/starterkit/cuneiform/cuneiform-linux/builddir/librpic.so...done.
Reading symbols from /home/starterkit/cuneiform/cuneiform-linux/builddir/libcpage.so...done.
Reading symbols from /home/starterkit/cuneiform/cuneiform-linux/builddir/librreccom.so...done.
Reading symbols from /home/starterkit/cuneiform/cuneiform-linux/builddir/libexc.so...done.
Reading symbols from /home/starterkit/cuneiform/cuneiform-linux/builddir/libloc.so...done.
Reading symbols from /home/starterkit/cuneiform/cuneiform-linux/builddir/librpstr.so...done.
Reading symbols from /home/starterkit/cuneiform/cuneiform-linux/builddir/librstr.so...done.
Reading symbols from /home/starterkit/cuneiform/cuneiform-linux/builddir/liblns32.so...done.
Reading symbols from /home/starterkit/cuneiform/cuneiform-linux/builddir/libcline.so...done.
Reading symbols from /home/starterkit/cuneiform/cuneiform-linux/builddir/librcutp.so...done.
Reading symbols from /home/starterkit/cuneiform/cuneiform-linux/builddir/libsmetric.so...done.
Reading symbols from /home/starterkit/cuneiform/cuneiform-linux/builddir/librbal.so...done.
Reading symbols from /home/starterkit/cuneiform/cuneiform-linux/builddir/libleo.so...done.
Reading symbols from /home/starterkit/cuneiform/cuneiform-linux/builddir/libevn32.so...done.
Reading symbols from /home/starterkit/cuneiform/cu...

[Jussi Pakkanen (jpakkane)]

Valgrind tells me that someone somewhere overwrites the stack return address. Hints on detecting where that happens are welcome.

[Alex Samorukov (samm-os2)]

I think the error may come from
make_seg_line, but i don`t now how to fix this

/home/samm/src/cuneiform-linux/cuneiform_src/Kern/evn32/src/v0compgl.c: In function ‘make_seg_line’:
/home/samm/src/cuneiform-linux/cuneiform_src/Kern/evn32/src/v0compgl.c:111: warning: overflow in implicit constant conversion
/home/samm/src/cuneiform-linux/cuneiform_src/Kern/evn32/src/v0compgl.c:113: warning: overflow in implicit constant conversion
/home/samm/src/cuneiform-linux/cuneiform_src/Kern/evn32/src/v0compgl.c:114: warning: overflow in implicit constant conversion
/home/samm/src/cuneiform-linux/cuneiform_src/Kern/evn32/src/v0compgl.c:116: warning: overflow in implicit constant conversion
/home/samm/src/cuneiform-linux/cuneiform_src/Kern/evn32/src/v0compgl.c:117: warning: overflow in implicit constant conversion
/home/samm/src/cuneiform-linux/cuneiform_src/Kern/evn32/src/v0compgl.c:118: warning: overflow in implicit constant conversion
/home/samm/src/cuneiform-linux/cuneiform_src/Kern/evn32/src/v0compgl.c:129: warning: overflow in implicit constant conversion
/home/samm/src/cuneiform-linux/cuneiform_src/Kern/evn32/src/v0compgl.c:130: warning: overflow in implicit constant conversion
/home/samm/src/cuneiform-linux/cuneiform_src/Kern/evn32/src/v0compgl.c:131: warning: overflow in implicit constant conversion
/home/samm/src/cuneiform-linux/cuneiform_src/Kern/evn32/src/v0compgl.c:132: warning: overflow in implicit constant conversion
/home/samm/src/cuneiform-linux/cuneiform_src/Kern/evn32/src/v0compgl.c:149: warning: overflow in implicit constant conversion
/home/samm/src/cuneiform-linux/cuneiform_src/Kern/evn32/src/v0compgl.c:167: warning: overflow in implicit constant conversion
/home/samm/src/cuneiform-linux/cuneiform_src/Kern/evn32/src/v0compgl.c:169: warning: overflow in implicit constant conversion
/home/samm/src/cuneiform-linux/cuneiform_src/Kern/evn32/src/v0compgl.c:170: warning: overflow in implicit constant conversion
/home/samm/src/cuneiform-linux/cuneiform_src/Kern/evn32/src/v0compgl.c:171: warning: overflow in implicit constant conversion
/home/samm/src/cuneiform-linux/cuneiform_src/Kern/evn32/src/v0compgl.c:172: warning: overflow in implicit constant conversion
/home/samm/src/cuneiform-linux/cuneiform_src/Kern/evn32/src/v0compgl.c:188: warning: overflow in implicit constant conversion
/home/samm/src/cuneiform-linux/cuneiform_src/Kern/evn32/src/v0compgl.c:205: warning: overflow in implicit constant conversion
/home/samm/src/cuneiform-linux/cuneiform_src/Kern/evn32/src/v0compgl.c:225: warning: overflow in implicit constant conversion
/home/samm/src/cuneiform-linux/cuneiform_src/Kern/evn32/src/v0compgl.c:242: warning: overflow in implicit constant conversion

[Alex Samorukov (samm-os2)]

this lines are coming from replacement
 #define setp(b,w) *((int*)p)=(w<<16)|b;
with
#define setp(b,w) *((int16_t*)p)=(w<<16)|b;

This patch cause regression, because now much more files giving this error

[Jussi Pakkanen (jpakkane)]

What happens if you replace int16_t with int32_t in the patch?

[Alex Samorukov (samm-os2)]

with int32_t it works the same as with int.

[Jussi Pakkanen (jpakkane)]

The define touches this struct:

struct bw_segment
{
 INT b;
 INT w;
 BOX * box;
};

On Windows int is 16 bits, so the (int*) cast in the original code means a cast to a pointer to a 16 bit integer. However the definition of setp has "(w<<16)|b", which shifts w out. So effectively the #define reduces to p->b = b;

This becomes all the more confusing by the next #define, which is

#define setpo(b,w) p->b=b;p->w=w;

My current hunch is that there are two bugs, which cancel each other out in most, but not all cases. Changing int* to uint16_t* fixed one, but now causes more of the other bugs to be triggered.

[Jussi Pakkanen (jpakkane)]

I reverted the patch for release 0.4. as it caused problems.

Could you try replacing the define with this:

#define setpo(bb,ww) p->b=bb;p->w=ww;

[Jussi Pakkanen (jpakkane)]

Sorry, that should be

#define setp(bb,ww) p->b=bb;p->w=ww;

[Andrey Vasilenko (freexen)]

Confirmed on Ubuntu 8.04 amd64 with 2.6.24-21-generic Linux kernel.
It is attempts of recognition of the Russian text:

$ cuneiform -l rus -f html -o ~/testanalyze.html ~/tfkp_voprosnik.bmp
Cuneiform for Linux 0.4.0
The image depth is 8 at this point.
Segmentation fault

$ cuneiform -l rus -f html -o ~/testanalyze.html ~/tfkp_voprosnik.bmp
Cuneiform for Linux 0.4.0
The image depth is 1 at this point.
Warning: RSL said that the lines don't need to be erased from the picture.
Segmentation fault

[Alex Samorukov (samm-os2)]

I tried to compile with
#define setp(bb,ww) p->b=bb;p->w=ww;
but it segfaults on the same place as before (make_white_hist())