IP/ROHC client crashes when transfering a large file with scp

[Didier Barvaux]

Reported by Matthew in question #228952:

My test case.

1. Start up rohc server and client communicating via private ip tunnel using ip addresses 10.0.0.1 -> 10.0.0.11
2. Send a large amount of data client -> server.
3. Client core dumps.

Now I can do all manner of other things successfully, for instance I can ssh from client to server and have no problem. Here is the text from the client. Note: nothing of interest shows up in /var/log/messages when the crash occurs.

The command I use on the server side in this particular test was "scp rohcuser@10.0.0.11:/home/rohcuser/test.tar.bz2"

*** glibc detected *** /home/rohcuser/workspace/iprohc/client/iprohc_client: double free or corruption (out): 0x00007f27e402b180 ***
======= Backtrace: =========
/lib64/libc.so.6[0x345e8760e6]
/lib64/libc.so.6[0x345e878c13]
/home/rohcuser/workspace/iprohc/common/libiprohc_common.so(tun2raw+0x40a)[0x7f27eb6cb370]
/home/rohcuser/workspace/iprohc/common/libiprohc_common.so(new_tunnel+0x5a5)[0x7f27eb6cab7b]
/lib64/libpthread.so.0[0x3dc4407851]
/lib64/libc.so.6(clone+0x6d)[0x345e8e890d]
======= Memory map: ========
00400000-00406000 r-xp 00000000 08:03 1726032 /home/rohcuser/workspace/iprohc/client/iprohc_client
00605000-00606000 rw-p 00005000 08:03 1726032 /home/rohcuser/workspace/iprohc/client/iprohc_client
012ce000-0130f000 rw-p 00000000 00:00 0 [heap]
3032c00000-3032c30000 r-xp 00000000 08:03 1472141 /usr/lib64/librohc_comp.so.0.1.0
3032c30000-3032e30000 ---p 00030000 08:03 1472141 /usr/lib64/librohc_comp.so.0.1.0
3032e30000-3032e31000 rw-p 00030000 08:03 1472141 /usr/lib64/librohc_comp.so.0.1.0
3033000000-303300a000 r-xp 00000000 08:03 1471908 /usr/lib64/librohc_common.so.0.1.0
303300a000-303320a000 ---p 0000a000 08:03 1471908 /usr/lib64/librohc_common.so.0.1.0
303320a000-303320b000 rw-p 0000a000 08:03 1471908 /usr/lib64/librohc_common.so.0.1.0
3033400000-3033430000 r-xp 00000000 08:03 1484267 /usr/lib64/librohc_decomp.so.0.1.0
3033430000-3033630000 ---p 00030000 08:03 1484267 /usr/lib64/librohc_decomp.so.0.1.0
3033630000-3033631000 rw-p 00030000 08:03 1484267 /usr/lib64/librohc_decomp.so.0.1.0
345e000000-345e020000 r-xp 00000000 08:03 916117 /lib64/ld-2.12.so
345e21f000-345e220000 r--p 0001f000 08:03 916117 /lib64/ld-2.12.so
345e220000-345e221000 rw-p 00020000 08:03 916117 /lib64/ld-2.12.so
345e221000-345e222000 rw-p 00000000 00:00 0
345e400000-345e402000 r-xp 00000000 08:03 916126 /lib64/libdl-2.12.so
345e402000-345e602000 ---p 00002000 08:03 916126 /lib64/libdl-2.12.so
345e602000-345e603000 r--p 00002000 08:03 916126 /lib64/libdl-2.12.so
345e603000-345e604000 rw-p 00003000 08:03 916126 /lib64/libdl-2.12.so
345e800000-345e98a000 r-xp 00000000 08:03 916118 /lib64/libc-2.12.so
345e98a000-345eb89000 ---p 0018a000 08:03 916118 /lib64/libc-2.12.so
345eb89000-345eb8d000 r--p 00189000 08:03 916118 /lib64/libc-2.12.so
345eb8d000-345eb8e000 rw-p 0018d000 08:03 916118 /lib64/libc-2.12.so
345eb8e000-345eb93000 rw-p 00000000 00:00 0
345fc00000-345fc15000 r-xp 00000000 08:03 916125 /lib64/libz.so.1.2.3
345fc15000-345fe14000 ---p 00015000 08:03 916125 /lib64/libz.so.1.2.3
345fe14000-345fe15000 r--p 00014000 08:03 916125 /lib64/libz.so.1.2.3
345fe15000-345fe16000 rw-p 00015000 08:03 916125 /lib64/libz.so.1.2.3
346ac00000-346ac16000 r-xp 00000000 08:03 916140 /lib64/libgcc_s-4.4.7-20120601.so.1
346ac16000-346ae15000 ---p 00016000 08:03 916140 /lib64/libgcc_s-4.4.7-20120601.so.1
346ae15000-346ae16000 rw-p 00015000 08:03 916140 /lib64/libgcc_s-4.4.7-20120601.so.1
346dc00000-346dc03000 r-xp 00000000 08:03 916141 /lib64/libgpg-error.so.0.5.0
346dc03000-346de02000 ---p 00003000 08:03 916141 /lib64/libgpg-error.so.0.5.0
346de02000-346de03000 r--p 00002000 08:03 916141 /lib64/libgpg-error.so.0.5.0
346de03000-346de04000 rw-p 00003000 08:03 916141 /lib64/libgpg-error.so.0.5.0
3470800000-3470872000 r-xp 00000000 08:03 916142 /lib64/libgcrypt.so.11.5.3
3470872000-3470a71000 ---p 00072000 08:03 916142 /lib64/libgcrypt.so.11.5.3
3470a71000-3470a72000 r--p 00071000 08:03 916142 /lib64/libgcrypt.so.11.5.3
3470a72000-3470a75000 rw-p 00072000 08:03 916142 /lib64/libgcrypt.so.11.5.3
3472000000-3472010000 r-xp 00000000 08:03 1441128 /usr/lib64/libtasn1.so.3.1.6
3472010000-347220f000 ---p 00010000 08:03 1441128 /usr/lib64/libtasn1.so.3.1.6
347220f000-3472210000 rw-p 0000f000 08:03 1441128 /usr/lib64/libtasn1.so.3.1.6
3bb3a00000-3bb3a9c000 r-xp 00000000 08:03 1441114 /usr/lib64/libgnutls.so.26.14.12
3bb3a9c000-3bb3c9c000 ---p 0009c000 08:03 1441114 /usr/lib64/libgnutls.so.26.14.12
3bb3c9c000-3bb3ca3000 rw-p 0009c000 08:03 1441114 /usr/lib64/libgnutls.so.26.14.12
3dc4400000-3dc4417000 r-xp 00000000 08:03 923980 /lib64/libpthread-2.12.so
3dc4417000-3dc4617000 ---p 00017000 08:03 923980 /lib64/libpthread-2.12.so
3dc4617000-3dc4618000 r--p 00017000 08:03 923980 /lib64/libpthread-2.12.so
3dc4618000-3dc4619000 rw-p 00018000 08:03 923980 /lib64/libpthread-2.12.so
3dc4619000-3dc461d000 rw-p 00000000 00:00 0
7f27e4000000-7f27e4037000 rw-p 00000000 00:00 0
7f27e4037000-7f27e8000000 ---p 00000000 00:00 0
7f27eacb0000-7f27eacb1000 ---p 00000000 00:00 0
7f27eacb1000-7f27eb6b8000 rw-p 00000000 00:00 0
7f27eb6c6000-7f27eb6c8000 rw-p 00000000 00:00 0
7f27eb6c8000-7f27eb6d1000 r-xp 00000000 08:03 1725987 /home/rohcuser/workspace/iprohc/common/libiprohc_common.so
7f27eb6d1000-7f27eb8d0000 ---p 00009000 08:03 1725987 /home/rohcuser/workspace/iprohc/common/libiprohc_common.so
7f27eb8d0000-7f27eb8d1000 rw-p 00008000 08:03 1725987 /home/rohcuser/workspace/iprohc/common/libiprohc_common.so
7f27eb8d1000-7f27eb8d2000 rw-p 00000000 00:00 0
7fffd3daf000-7fffd3dc4000 rw-p 00000000 00:00 0 [stack]
7fffd3dff000-7fffd3e00000 r-xp 00000000 00:00 0 [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall]
./RunClient: line 1: 10245 Aborted (core dumped) /home/rohcuser/workspace/iprohc/client/iprohc_client --remote $1 --port 3126 --dev rohcclient --debug --p12 /home/rohcuser/certificate/ia.p12

p.s. in the logs for the client and the server I get this message a lot:
"localhost rsyslogd-2177: imuxsock lost 708 messages from pid 6429 due to rate-limiting"

pid 6429 is the process id of my iprohc-server.

[matthew (xcalibre0)]

This was just a tar zip of all the files that ldd iprohc_client showed. That means it copied the linked files more than once, but this way i cant get it wrong.

Also it has the core file and the client exe.

[matthew (xcalibre0)]

Also if I just ssh over through the tunnel and I just do something like VI a file or run ifconfig it also crashes.

[matthew (xcalibre0)]

Looks like the core file i sent you got truncated in the transit, here is a zip of the full core.

[Didier Barvaux (didier-barvaux)]

1/ Discard incoming packets that results in packets larger than the internal buffers and/or the MTU of the underlying device. See http://bazaar.launchpad.net/~didier-barvaux/rohc/iprohc/revision/107
2/ Handle the MTU of the underlying and tunnel interfaces gracefully. See http://bazaar.launchpad.net/~didier-barvaux/rohc/iprohc/revision/108

[Didier Barvaux (didier-barvaux)]

Matthew, please try to reproduce the problem with the latest IP/ROHC version (revision 196). Tell me if the problem is fixed or not.

[matthew (xcalibre0)]

Oh baby, it works. We are back in business

thanks

[Didier Barvaux (didier-barvaux)]

Great! Thank you for the confirmation :)

Didier

[Mohammad (shaidur)]

no problem still available on latest version. if i ssh from server or to server and running some command ..its stack on terminal but ping between server to client or client to server still working .

[Didier Barvaux (didier-barvaux)]

Mohammad, what version are you using ?

[Mohammad (shaidur)]

Hi Dider

i am using iprohc 0.6 (server in centos 6 & client on debian)

regards

Mohammad

[Didier Barvaux (didier-barvaux)]

Mohammad, the bug is not fixed in version 0.6. It is fixed on the dev branch, and will be released with the next 0.7 release. If you encounter the problem, please use the dev branch.