[CentOS] Application Hang issue in RHEL6.4

Thu Feb 6 06:39:32 UTC 2014
Miraj Mohamed <Miraj.Mohamed at radisys.com>

Hi All,


We see our application Hanging at shmctl (SHM_LOCK).



Kernel :  2.6.32-358.el6.x86_64

OS : Red Hat Enterprise Linux Server release 6.4 (Santiago)





1.       When it hangs, this is what strace gives:



[RHEL6.4 at 10.201.0.221 bin]# strace -p 23793 Process 23793 attached - interrupt to quit [ Process PID=23793 runs in 32 bit mode. ]

shmget(0x110c1be, 16777216, 0666)       = 3801140

shmat(3801140, 0, 0)                    = 0x85c2c70e7755000

shmctl(3801140, IPC_64|SHM_LOCK, 0^C <unfinished ...>







2.       When we kill the application and generate a core file, Here is what  gdb shows:



Core was generated by ` <our application> 12 12 10.33.0.230 eth1 -1 0 3 fd5d:d50a:8c17:3300::e6'.

Program terminated with signal 6, Aborted.

#0  0x007c2430 in __kernel_vsyscall ()

Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.107.el6.i686 libgcc-4.4.7-3.el6.i686 libstdc++-4.4.7-3.el6.i686

(gdb) bt

#0  0x007c2430 in __kernel_vsyscall ()

#1  0x008a8d8e in shmctl@@GLIBC_2.2 () from /lib/libc.so.6

#2  0x0847c13e in MpcMpSwms_SwibIfShmSegOpen (ps=<value optimized out>, ctl=<value optimized out>)

   at src/MpcMpSwms_SwibIfShm.c:271

#3  0x0847c551 in MpcMpSwms_SwibIfShmOpen (pvs=0xb0e8984, pvctl=0xffc35354) at src/MpcMpSwms_SwibIfShm.c:379

#4  0x0847bb9f in MpcMpSwms_SwibIfOpen (ps=0xb0e7dc4, ctl=0xffc35354, swib_if=SWIB_IF_SHM) at src/MpcMpSwms_SwibIf.c:126

#5  0x080b792a in MpcMpSwms_SwibOpen (ps=0xa019da8, ctl=0xffc357d8) at src/MpcMpSwms_Swib.c:253

#6  0x080b63a0 in main (argc=9, argv=0xffc35924) at src/MpcMpSwms_Main.c:1364

(gdb)





What could be the reason for hanging?

Any solution?



-miraj