GNU gdb (GDB) Red Hat Enterprise Linux (7.2-90.el6) Copyright (C) 2010 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-redhat-linux-gnu". For bug reporting instructions, please see: ... Reading symbols from /opt/epics/extensions/bin/linux-x86_64/gateway...(no debugging symbols found)...done. [New Thread 5330] [New Thread 5328] [New Thread 5428] [New Thread 5425] [New Thread 5329] [New Thread 5426] [New Thread 5427] [New Thread 5326] [New Thread 5327] [New Thread 5324] Reading symbols from /opt/epics/base/lib/linux-x86_64/libcas.so.3.14...Reading symbols from /usr/lib/debug/opt/epics/base/lib/linux-x86_64/libcas.so.3.14.debug...done. done. Loaded symbols for /opt/epics/base/lib/linux-x86_64/libcas.so.3.14 Reading symbols from /opt/epics/base/lib/linux-x86_64/libgdd.so.3.14...Reading symbols from /usr/lib/debug/opt/epics/base/lib/linux-x86_64/libgdd.so.3.14.debug...done. done. Loaded symbols for /opt/epics/base/lib/linux-x86_64/libgdd.so.3.14 Reading symbols from /opt/epics/base/lib/linux-x86_64/libasHost.so.3.14...Reading symbols from /usr/lib/debug/opt/epics/base/lib/linux-x86_64/libasHost.so.3.14.debug...done. done. Loaded symbols for /opt/epics/base/lib/linux-x86_64/libasHost.so.3.14 Reading symbols from /opt/epics/base/lib/linux-x86_64/libdbStaticHost.so.3.14...Reading symbols from /usr/lib/debug/opt/epics/base/lib/linux-x86_64/libdbStaticHost.so.3.14.debug...done. done. Loaded symbols for /opt/epics/base/lib/linux-x86_64/libdbStaticHost.so.3.14 Reading symbols from /opt/epics/base/lib/linux-x86_64/libregistryIoc.so.3.14...Reading symbols from /usr/lib/debug/opt/epics/base/lib/linux-x86_64/libregistryIoc.so.3.14.debug...done. done. Loaded symbols for /opt/epics/base/lib/linux-x86_64/libregistryIoc.so.3.14 Reading symbols from /opt/epics/base/lib/linux-x86_64/libca.so.3.14...Reading symbols from /usr/lib/debug/opt/epics/base/lib/linux-x86_64/libca.so.3.14.debug...done. done. Loaded symbols for /opt/epics/base/lib/linux-x86_64/libca.so.3.14 Reading symbols from /opt/epics/base/lib/linux-x86_64/libCom.so.3.14...Reading symbols from /usr/lib/debug/opt/epics/base/lib/linux-x86_64/libCom.so.3.14.debug...done. done. Loaded symbols for /opt/epics/base/lib/linux-x86_64/libCom.so.3.14 Reading symbols from /usr/lib64/libstdc++.so.6...(no debugging symbols found)...done. Loaded symbols for /usr/lib64/libstdc++.so.6 Reading symbols from /lib64/libm.so.6...(no debugging symbols found)...done. Loaded symbols for /lib64/libm.so.6 Reading symbols from /lib64/libgcc_s.so.1...(no debugging symbols found)...done. Loaded symbols for /lib64/libgcc_s.so.1 Reading symbols from /lib64/libc.so.6...(no debugging symbols found)...done. Loaded symbols for /lib64/libc.so.6 Reading symbols from /lib64/libpthread.so.0...(no debugging symbols found)...done. [Thread debugging using libthread_db enabled] Loaded symbols for /lib64/libpthread.so.0 Reading symbols from /lib64/libreadline.so.6...(no debugging symbols found)...done. Loaded symbols for /lib64/libreadline.so.6 Reading symbols from /lib64/librt.so.1...(no debugging symbols found)...done. Loaded symbols for /lib64/librt.so.1 Reading symbols from /lib64/libdl.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/libdl.so.2 Reading symbols from /opt/epics/base/lib/linux-x86_64/libdbStaticIoc.so.3.14...Reading symbols from /usr/lib/debug/opt/epics/base/lib/linux-x86_64/libdbStaticIoc.so.3.14.debug...done. done. Loaded symbols for /opt/epics/base/lib/linux-x86_64/libdbStaticIoc.so.3.14 Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/ld-linux-x86-64.so.2 Reading symbols from /lib64/libtinfo.so.5...(no debugging symbols found)...done. Loaded symbols for /lib64/libtinfo.so.5 Reading symbols from /lib64/libnss_files.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/libnss_files.so.2 Reading symbols from /lib64/libnss_dns.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/libnss_dns.so.2 Reading symbols from /lib64/libresolv.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/libresolv.so.2 Core was generated by `/opt/epics/extensions/bin/linux-x86_64/gateway -sip 10.16.99.100 -cip 10.16.3.2'. Program terminated with signal 11, Segmentation fault. #0 0x00007f67e7a3ed26 in assertIdenticalMutex (this=0x0, guard=..., chan=..., sidIn=4294967295, typeIn=65535, countIn=0) at ../../../include/epicsGuard.h:81 81 assert ( _pTargetMutex == & mutexToVerify ); Missing separate debuginfos, use: debuginfo-install epics-extension-gateway-2.1.0-1.el6.x86_64 [?1034h(gdb) thread apply all bt Thread 10 (Thread 0x7f67e7586720 (LWP 5324)): #0 0x00000031114e1523 in select () from /lib64/libc.so.6 #1 0x00007f67e77d1ea1 in fdManager::process (this=0x628380, delay=) at ../../../src/libCom/fdmgr/fdManager.cpp:125 #2 0x000000000041359f in ?? () #3 0x00000000004065a2 in ?? () #4 0x0000000000408350 in ?? () #5 0x000000311141ed1d in __libc_start_main () from /lib64/libc.so.6 #6 0x0000000000405919 in ?? () #7 0x00007ffd26c9ad48 in ?? () #8 0x000000000000001c in ?? () #9 0x000000000000000f in ?? () #10 0x00007ffd26c9c6d0 in ?? () #11 0x00007ffd26c9c6ff in ?? () #12 0x00007ffd26c9c704 in ?? () #13 0x00007ffd26c9c711 in ?? () #14 0x00007ffd26c9c716 in ?? () #15 0x00007ffd26c9c722 in ?? () #16 0x00007ffd26c9c727 in ?? () #17 0x00007ffd26c9c733 in ?? () #18 0x00007ffd26c9c73b in ?? () #19 0x00007ffd26c9c743 in ?? () #20 0x00007ffd26c9c74c in ?? () #21 0x00007ffd26c9c756 in ?? () #22 0x00007ffd26c9c75f in ?? () #23 0x00007ffd26c9c76c in ?? () #24 0x00007ffd26c9c774 in ?? () #25 0x0000000000000000 in ?? () Thread 9 (Thread 0x7f67e68ed700 (LWP 5327)): #0 0x0000003111c0ba5e in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f67e77e4704 in condTimedwait (pevent=0x24167b0, timeout=0.0015257859999999999) at ../../../src/libCom/osi/os/posix/osdEvent.c:65 #2 epicsEventWaitWithTimeout (pevent=0x24167b0, timeout=0.0015257859999999999) at ../../../src/libCom/osi/os/posix/osdEvent.c:156 #3 0x00007f67e77ddb4c in epicsEvent::wait (this=, timeOut=) at ../../../src/libCom/osi/epicsEvent.cpp:72 #4 0x00007f67e77ea0b3 in timerQueueActive::run (this=0x24164c0) at ../../../src/libCom/timer/timerQueueActive.cpp:95 #5 0x00007f67e77dd249 in epicsThreadCallEntryPoint (pPvt=0x2416590) at ../../../src/libCom/osi/epicsThread.cpp:83 #6 0x00007f67e77e3ed3 in start_routine (arg=0x24169e0) at ../../../src/libCom/osi/os/posix/osdThread.c:389 #7 0x0000003111c07aa1 in start_thread () from /lib64/libpthread.so.0 #8 0x00000031114e8aad in clone () from /lib64/libc.so.6 Thread 8 (Thread 0x7f67e6aee700 (LWP 5326)): #0 0x00000031114df283 in poll () from /lib64/libc.so.6 #1 0x000000311300c086 in __libc_res_nsend () from /lib64/libresolv.so.2 #2 0x0000003113008811 in __libc_res_nquery () from /lib64/libresolv.so.2 #3 0x00007f67e5d8f112 in _nss_dns_gethostbyaddr2_r () from /lib64/libnss_dns.so.2 #4 0x00007f67e5d8f2a5 in _nss_dns_gethostbyaddr_r () from /lib64/libnss_dns.so.2 #5 0x0000003111503018 in gethostbyaddr_r@@GLIBC_2.2.5 () from /lib64/libc.so.6 #6 0x0000003111502dbc in gethostbyaddr () from /lib64/libc.so.6 #7 0x00007f67e77e10de in ipAddrToHostName (pAddr=, pBuf=0x24158f0 "10.16.3.254:39677", bufSize=1024) at ../../../src/libCom/osi/os/posix/osdSock.c:147 #8 0x00007f67e77e1539 in ipAddrToA (paddr=0x7f67e6aedd70, pBuf=0x24158f0 "10.16.3.254:39677", bufSize=1024) at ../../../src/libCom/osi/osiSock.c:96 #9 0x00007f67e77db0b7 in ipAddrToAsciiEnginePrivate::run (this=0x24158e0) at ../../../src/libCom/misc/ipAddrToAsciiAsynchronous.cpp:257 #10 0x00007f67e77dd249 in epicsThreadCallEntryPoint (pPvt=0x2415d38) at ../../../src/libCom/osi/epicsThread.cpp:83 #11 0x00007f67e77e3ed3 in start_routine (arg=0x24160a0) at ../../../src/libCom/osi/os/posix/osdThread.c:389 #12 0x0000003111c07aa1 in start_thread () from /lib64/libpthread.so.0 #13 0x00000031114e8aad in clone () from /lib64/libc.so.6 Thread 7 (Thread 0x7f67d7fff700 (LWP 5427)): #0 0x00000031114e985c in recv () from /lib64/libc.so.6 #1 0x00007f67e7a400ef in recv (this=0x7f67d8000c60, pBuf=0x7f67e61ca06c, nBytesInBuf=16384, stat=...) at /usr/include/bits/socket2.h:45 #2 tcpiiu::recvBytes (this=0x7f67d8000c60, pBuf=0x7f67e61ca06c, nBytesInBuf=16384, stat=...) at ../tcpiiu.cpp:304 #3 0x00007f67e7a42682 in fillFromWire (this=0x7f67d8000d48) at ../comBuf.h:182 #4 tcpRecvThread::run (this=0x7f67d8000d48) at ../tcpiiu.cpp:474 #5 0x00007f67e77dd249 in epicsThreadCallEntryPoint (pPvt=0x7f67d8000d50) at ../../../src/libCom/osi/epicsThread.cpp:83 #6 0x00007f67e77e3ed3 in start_routine (arg=0x7f67d80171f0) at ../../../src/libCom/osi/os/posix/osdThread.c:389 #7 0x0000003111c07aa1 in start_thread () from /lib64/libpthread.so.0 #8 0x00000031114e8aad in clone () from /lib64/libc.so.6 Thread 6 (Thread 0x7f67e6343700 (LWP 5426)): #0 0x0000003111c0b68c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f67e77e47a9 in condWait (pevent=0x7f67d800e720) at ../../../src/libCom/osi/os/posix/osdEvent.c:75 #2 epicsEventWait (pevent=0x7f67d800e720) at ../../../src/libCom/osi/os/posix/osdEvent.c:137 #3 0x00007f67e77ddbac in epicsEvent::wait (this=) at ../../../src/libCom/osi/epicsEvent.cpp:63 #4 0x00007f67e7a434fc in tcpSendThread::run (this=0x7f67d8000a00) at ../tcpiiu.cpp:84 #5 0x00007f67e77dd249 in epicsThreadCallEntryPoint (pPvt=0x7f67d8000a08) at ../../../src/libCom/osi/epicsThread.cpp:83 #6 0x00007f67e77e3ed3 in start_routine (arg=0x7f67d800a370) at ../../../src/libCom/osi/os/posix/osdThread.c:389 #7 0x0000003111c07aa1 in start_thread () from /lib64/libpthread.so.0 #8 0x00000031114e8aad in clone () from /lib64/libc.so.6 Thread 5 (Thread 0x7f67e676b700 (LWP 5329)): #0 0x0000003111c0ba5e in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f67e77e4704 in condTimedwait (pevent=0x241b370, timeout=9.9949381220000006) at ../../../src/libCom/osi/os/posix/osdEvent.c:65 #2 epicsEventWaitWithTimeout (pevent=0x241b370, timeout=9.9949381220000006) at ../../../src/libCom/osi/os/posix/osdEvent.c:156 #3 0x00007f67e77ddb4c in epicsEvent::wait (this=, timeOut=) at ../../../src/libCom/osi/epicsEvent.cpp:72 #4 0x00007f67e77ea0b3 in timerQueueActive::run (this=0x241b080) at ../../../src/libCom/timer/timerQueueActive.cpp:95 #5 0x00007f67e77dd249 in epicsThreadCallEntryPoint (pPvt=0x241b150) at ../../../src/libCom/osi/epicsThread.cpp:83 #6 0x00007f67e77e3ed3 in start_routine (arg=0x241b5a0) at ../../../src/libCom/osi/os/posix/osdThread.c:389 #7 0x0000003111c07aa1 in start_thread () from /lib64/libpthread.so.0 #8 0x00000031114e8aad in clone () from /lib64/libc.so.6 Thread 4 (Thread 0x7f67e6544700 (LWP 5425)): #0 0x00000031114e985c in recv () from /lib64/libc.so.6 #1 0x00007f67e7a400ef in recv (this=0x7f67d80008c0, pBuf=0x7f67e61c604c, nBytesInBuf=16384, stat=...) at /usr/include/bits/socket2.h:45 #2 tcpiiu::recvBytes (this=0x7f67d80008c0, pBuf=0x7f67e61c604c, nBytesInBuf=16384, stat=...) at ../tcpiiu.cpp:304 #3 0x00007f67e7a42682 in fillFromWire (this=0x7f67d80009a8) at ../comBuf.h:182 #4 tcpRecvThread::run (this=0x7f67d80009a8) at ../tcpiiu.cpp:474 #5 0x00007f67e77dd249 in epicsThreadCallEntryPoint (pPvt=0x7f67d80009b0) at ../../../src/libCom/osi/epicsThread.cpp:83 #6 0x00007f67e77e3ed3 in start_routine (arg=0x7f67d8009ea0) at ../../../src/libCom/osi/os/posix/osdThread.c:389 #7 0x0000003111c07aa1 in start_thread () from /lib64/libpthread.so.0 #8 0x00000031114e8aad in clone () from /lib64/libc.so.6 Thread 3 (Thread 0x7f67e5d8b700 (LWP 5428)): #0 0x0000003111c0b68c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f67e77e47a9 in condWait (pevent=0x7f67d801ba70) at ../../../src/libCom/osi/os/posix/osdEvent.c:75 #2 epicsEventWait (pevent=0x7f67d801ba70) at ../../../src/libCom/osi/os/posix/osdEvent.c:137 #3 0x00007f67e77ddbac in epicsEvent::wait (this=) at ../../../src/libCom/osi/epicsEvent.cpp:63 #4 0x00007f67e7a434fc in tcpSendThread::run (this=0x7f67d8000da0) at ../tcpiiu.cpp:84 #5 0x00007f67e77dd249 in epicsThreadCallEntryPoint (pPvt=0x7f67d8000da8) at ../../../src/libCom/osi/epicsThread.cpp:83 #6 0x00007f67e77e3ed3 in start_routine (arg=0x7f67d80176c0) at ../../../src/libCom/osi/os/posix/osdThread.c:389 #7 0x0000003111c07aa1 in start_thread () from /lib64/libpthread.so.0 #8 0x00000031114e8aad in clone () from /lib64/libc.so.6 Thread 2 (Thread 0x7f67e67ec700 (LWP 5328)): #0 0x0000003111c0b68c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f67e77e47a9 in condWait (pevent=0x2417720) at ../../../src/libCom/osi/os/posix/osdEvent.c:75 #2 epicsEventWait (pevent=0x2417720) at ../../../src/libCom/osi/os/posix/osdEvent.c:137 #3 0x00007f67e77cea90 in errlogThread () at ../../../src/libCom/error/errlog.c:491 #4 0x00007f67e77e3ed3 in start_routine (arg=0x241acd0) at ../../../src/libCom/osi/os/posix/osdThread.c:389 #5 0x0000003111c07aa1 in start_thread () from /lib64/libpthread.so.0 #6 0x00000031114e8aad in clone () from /lib64/libc.so.6 Thread 1 (Thread 0x7f67e666a700 (LWP 5330)): #0 0x00007f67e7a3ed26 in assertIdenticalMutex (this=0x0, guard=..., chan=..., sidIn=4294967295, typeIn=65535, countIn=0) at ../../../include/epicsGuard.h:81 #1 tcpiiu::installChannel (this=0x0, guard=..., chan=..., sidIn=4294967295, typeIn=65535, countIn=0) at ../tcpiiu.cpp:1911 #2 0x00007f67e7a2c2bb in cac::transferChanToVirtCircuit (this=, cid=, sid=4294967295, typeCode=65535, count=0, minorVersionNumber=13, addr=..., currentTime=...) at ../cac.cpp:639 #3 0x00007f67e7a3a4a0 in udpiiu::searchRespAction (this=, msg=, addr=, currentTime=) at ../udpiiu.cpp:690 #4 0x00007f67e7a3a5c2 in udpiiu::postMsg (this=0x242d760, net_addr=..., pInBuf=, blockSize=48, currentTime=...) at ../udpiiu.cpp:857 #5 0x00007f67e7a3c681 in udpRecvThread::run (this=0x243db88) at ../udpiiu.cpp:394 #6 0x00007f67e77dd249 in epicsThreadCallEntryPoint (pPvt=0x243dba8) at ../../../src/libCom/osi/epicsThread.cpp:83 #7 0x00007f67e77e3ed3 in start_routine (arg=0x243de20) at ../../../src/libCom/osi/os/posix/osdThread.c:389 #8 0x0000003111c07aa1 in start_thread () from /lib64/libpthread.so.0 #9 0x00000031114e8aad in clone () from /lib64/libc.so.6 (gdb) quit