[CentOS] Race Condition In PHP Session Handler

Fri Feb 17 21:51:57 UTC 2012
Adam Tauno Williams <awilliam at whitemice.org>

php-5.3.3-3.el6_2.6.x86_64
2.6.32-220.4.2.el6.x86_64

After some testing we moved our production PHP intranet site from an old
server to a new CentOS 6.2 instance running the mentioned versions.  At
first it worked well and then user's started to experience a hung site.
Upon some digging it appears that the PHP module is getting stuck in a
race condition regarding the sessions file -

[root at yannigan-orange fd]# strace -p  22607
Process 22607 attached - interrupt to quit
flock(53, LOCK_EX^C <unfinished ...>

where file handle 53 is 

...
lr-x------. 1 root root 64 Feb 17 15:24 52
-> /opt/informix/msg/en_us/0333/cli.iem
lrwx------. 1 root root 64 Feb 17 15:24 53
-> /var/lib/php/session/sess_ulgsl9joplobb9o4fue3n2e1k0
l-wx------. 1 root root 64 Feb 17 15:24 6 -> pipe:[660246]
....

The PHP session file.  Originally I had a tmpfs mounted for the session
files.  I removed that and used the underlying filesystem.  Same issue.
I disabled APC.  Same issue.

Switching the the memcache session handler resolved the issue,  although
that may be less optimal.  But this seems like a PHP bug.

Tips, comments, pointers?

Backtrace of a hung httpd -
#0  0x00007fa175af9be7 in flock () from /lib64/libc.so.6
#1  0x00007fa16cb59dfc in ?? () from /etc/httpd/modules/libphp5.so
#2  0x00007fa16cb5a047 in ?? () from /etc/httpd/modules/libphp5.so
#3  0x00007fa16cb56e4c in php_session_start ()
from /etc/httpd/modules/libphp5.so
#4  0x00007fa16cb57559 in ?? () from /etc/httpd/modules/libphp5.so
#5  0x00007fa16cc9afe8 in ?? () from /etc/httpd/modules/libphp5.so
#6  0x00007fa16cc72320 in execute () from /etc/httpd/modules/libphp5.so
#7  0x00007fa16cc4ca5d in zend_execute_scripts ()
from /etc/httpd/modules/libphp5.so
#8  0x00007fa16cbfacf8 in php_execute_script ()
from /etc/httpd/modules/libphp5.so
#9  0x00007fa16ccd59a5 in ?? () from /etc/httpd/modules/libphp5.so
#10 0x00007fa17751da60 in ap_run_handler ()
#11 0x00007fa17752131e in ap_invoke_handler ()
#12 0x00007fa17752c990 in ap_process_request ()
#13 0x00007fa177529858 in ?? ()
#14 0x00007fa177525568 in ap_run_process_connection ()
#15 0x00007fa177531767 in ?? ()
#16 0x00007fa177531a7a in ?? ()
#17 0x00007fa177531dab in ap_mpm_run ()
#18 0x00007fa177509900 in main ()



-- 
System & Network Administrator [ LPI & NCLA ]
<http://www.whitemiceconsulting.com>
OpenGroupware Developer <http://www.opengroupware.us>
Adam Tauno Williams
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part
URL: <http://lists.centos.org/pipermail/centos/attachments/20120217/df89ab7e/attachment-0004.sig>