[CentOS-devel] Observing deadlock between SIPhandler and SDL Router in CUCM.

Observing deadlock between SIPhandler and SDL Router in CUCM.
Looks like this is fixed in glibc 2.17.162 version, but we see the issue still

https://bugzilla.redhat.com/show_bug.cgi?id=906468

Need help to identify the  root cause and add a preventive fix.

Here is the error messages:

***********************************************************************************************************************************************************************************************************************************
 gdb) thread 40
[Switching to thread 40 (Thread 0xaca90b40 (LWP 5935))]
#3 0xf67fdbd1 in malloc () from /usr/lib/libc.so.6
(gdb) bt
#0 0xf775a430 in __kernel_vsyscall ()
#1 0xf688d462 in __lll_lock_wait_private () from /usr/lib/libc.so.6
#2 0xf68006c0 in _L_lock_14246 () from /usr/lib/libc.so.6
#3 0xf67fdbd1 in malloc () from /usr/lib/libc.so.6
#4 0xf6a68907 in operator new(unsigned int) () from /usr/local/cm/lib/libstlport.so.5.2
#5 0x0936c3f9 in operator() (this=0xaca8dd70) at /view/BLD-cm_12_5_1-cct-ccm-d/vob/ccm/Common/Include/Utilities/Member.hpp:52
#6 c_str (this=0xaca8dd70) at /view/BLD-cm_12_5_1-cct-ccm-d/vob/ccm/Common/Include/CallManager/TDCLCpShares.hpp:862
#7 assign (len=12, digits=0xaca8de34 "+19570676705", this=0xaca8dd70) at /view/BLD-cm_12_5_1-cct-ccm-d/vob/ccm/Common/Include/CallManager/TDCLCpShares.hpp:1252
#8 CdPNumStringVar::CdPNumStringVar (this=0xaca8dd70, digits=0xaca8de34 "+19570676705", len=12) at /view/BLD-cm_12_5_1-cct-ccm-d/vob/ccm/Common/Include/CallManager/TDCLCpShares.hpp:846
#9 0x092efdf8 in SIPCdpc::processSIPSetupMsg (this=this at entry=0x7de0c8e0, sipContainerWrapper=sipContainerWrapper at entry=0x8e966e78) at ProcessSIPCdpc.cpp:2142
#10 0x0934086d in SIPCdpc::processSIPSetupInd (this=this at entry=0x7de0c8e0) at ProcessSIPCdpc.cpp:3300
#11 0x0934ec08 in SIPCdpc::inCall_waitRSVPRes_PolicyAndCACRegisterRes (this=0x7de0c8e0, s=
Name : "PolicyAndCACRegisterRes",
Sender : spy:PID (8,100,152,1) "ReservationMgr",
Receiver : spy:PID (8,100,180,995813) "SIPCdpc",
Priority : kNormalPriority) at ProcessSIPCdpc.cpp:26471
#12 0x09e69c7b in SdlProcessBase::inputSignal (this=0x7de0c8e0, rSignal=0x807d8d80, traceType=SdlSystemLog::SignalRouterThread, highPriority=0, normalPriority=0, lowPriority=0, veryLowPriority=0, lazyPriority=0,
dbUpdatePriority=0) at SdlProcessBase.cpp:417
#13 0x09e6ce8a in SdlRouter::callProcess (this=this at entry=0xcfff0e0, _sdlSignal=_sdlSignal at entry=0x807d8d80, _deleteSignal=@0xaca8fdb3: true, _traceType=_traceType at entry=SdlSystemLog::SignalRouterThread, _hp=0,
_np=0, _lp=0, _vlp=0, _lzp=0, _dbp=0) at SdlRouter.cpp:257
#14 0x09e6d60c in SdlRouter::scheduler (this=this at entry=0xcfff0e0) at SdlRouter.cpp:164
#15 0x09e6dcad in SdlRouter::schedulerInit (sdlRouter=0xcfff0e0) at SdlRouter.cpp:107
#16 0xf69beb3c in start_thread () from /usr/lib/libpthread.so.0
#17 0xf687f44e in clone () from /usr/lib/libc.so.6

(gdb) thread 1
[Switching to thread 1 (Thread 0xa9136b40 (LWP 5964))]
#0 0xf67fb876 in _int_malloc () from /usr/lib/libc.so.6
(gdb) bt
#0 0xf67fb876 in _int_malloc () from /usr/lib/libc.so.6
#1 0xf67fdbda in malloc () from /usr/lib/libc.so.6
#2 0xf6a68907 in operator new(unsigned int) () from /usr/local/cm/lib/libstlport.so.5.2
#3 0x09cd4ce2 in sip_get_sipspi_message () at sip_common_spi.cpp:20059
#4 0x09cf94d1 in sipTransportPostSendMessage (msg=0x8a20a150, gcb=gcb at entry=0x8015d208, tcb_context=..., connId=24, destAddr=0x8015d228, destPort=5061, transport=SIP_TRANSPORT_TLS_OVER_TCP, emit_debugs=1 '\001',
ip_sig_tos=0 '\000') at sip_common_transport.cpp:3032
#5 0x09cf9946 in sipTransportLogicSendMsg (gcb=gcb at entry=0x8015d208, sw_transport=sw_transport at entry=0 '\000') at sip_common_transport.cpp:2367
#6 0x09cfab6f in sipSPITransportSendMessage (msg=msg at entry=0x8a20a150, context=context at entry=0x8015d208, addr=addr at entry=0xa91355dc, port=port at entry=5061, sentBy_port=sentBy_port at entry=0,
is_request=is_request at entry=1 '\001', transport=transport at entry=SIP_TRANSPORT_TLS_OVER_TCP, sw_transport=0 '\000', func=func at entry=0x9cb0080 <sentAck(_sipGenericCB*)>, tcb_context=...)
at sip_common_transport.cpp:3323
#7 0x09cbe638 in sipSPISendAck (ccb=0x8015d208, flags=flags at entry=0, func=0x9cb0080 <sentAck(_sipGenericCB*)>) at sip_common_send_msg.cpp:2992
#8 0x09ccc910 in sipSPISendAckForMidCallInvite (ccb=0x8015d208, flag=flag at entry=0) at sip_common_spi.cpp:8745
#9 0x09ccdb37 in sact_sent_mid_invite_new_message_response (ccb=0x8015d208, response=0x89305d58) at sip_common_spi.cpp:9278
#10 0x09cda815 in ccsip_spi_process_event (freeEvent=0xa91359df "\001", event=0x7ca43348) at sip_common_spi.cpp:19333
#11 ccsip_process_sipspi_queue_event (event=event at entry=0x7ca43348) at sip_common_spi.cpp:19997
#12 0x093ef758 in SIPHandler::wait_SIPNormalizeRes (this=0xac1c05e8, s=
Name : "SIPNormalizeRes",
Sender : spy:PID (8,100,184,1) "SIPNormalization",
Receiver : spy:PID (8,100,183,1) "SIPHandler",
Priority : kNormalPriority) at ProcessSIPHandler.cpp:1683
#13 0x09e69c7b in SdlProcessBase::inputSignal (this=this at entry=0xac1c05e8, rSignal=rSignal at entry=0x7d2f8a78, traceType=SdlSystemLog::SignalThreadedWithPriorities, highPriority=0, normalPriority=0, lowPriority=0,
veryLowPriority=0, lazyPriority=0, dbUpdatePriority=0) at SdlProcessBase.cpp:417
#14 0x09e8d2fd in SdlThreadedProcess::threadQueueReader (this=this at entry=0xac1c05e8) at SdlThreadedProcess.cpp:110
#15 0x09e8d56d in SdlThreadedProcess::threadQueueReaderInit (sdlThreadedProcess=0xac1c05e8) at SdlThreadedProcess.cpp:75
#16 0xf69beb3c in start_thread () from /usr/lib/libpthread.so.0
#17 0xf687f44e in clone () from /usr/lib/libc.so.6

BOth the threads are allocated memory at the same time , eventually resulting in deadlock.

-Thanks
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.centos.org/pipermail/centos-devel/attachments/20190125/c65d99ef/attachment.html>