Hi,
We have seen this messages on one of our server.
OS: CentOS 5.5 Processor: Xeon E5520 @ 2.27GHz Memory: 24 GB localhost ]# rpm -qa | grep httpd httpd-2.2.3-43.el5.centos.3 system-config-httpd-1.3.3.3-1.el5 httpd-devel-2.2.3-43.el5.centos.3 httpd-devel-2.2.3-43.el5.centos.3
Need help in collecting data and solve the issue.
Below are the details
message kernel: httpd[2090]: segfault at 00007fff3bacfec4 rip 00002ba6bbc50855 rsp 00007fff3bacfe50 error 6
(gdb) bt full #0 0x00002b30a5a91b00 in __accept_nocancel () from /lib64/libpthread.so.0 No symbol table info available. #1 0x00002b30a58772f4 in apr_socket_accept () from /usr/lib64/libapr-1.so.0 No symbol table info available. #2 0x00002b30a3fbcc31 in unixd_accept () No symbol table info available. #3 0x00002b30a3fbb6d7 in ?? () No symbol table info available. #4 0x00002b30a3fbba1a in ?? () No symbol table info available. #5 0x00002b30a3fbc27d in ap_mpm_run () No symbol table info available. #6 0x00002b30a3f96e48 in main () No symbol table info available.
(gdb) run Starting program: /usr/sbin/httpd [Thread debugging using libthread_db enabled] (98)Address already in use: make_sock: could not bind to address [::]:80 (98)Address already in use: make_sock: could not bind to address 0.0.0.0:80 no listening sockets available, shutting down Unable to open logs
Program exited with code 01.
Please advice.
Thank you, Sandeil
On Sat, Dec 25, 2010 at 3:16 PM, Sandeil Tenebro mayukmok00@yahoo.com wrote:
Hi,
We have seen this messages on one of our server.
OS: CentOS 5.5 Processor: Xeon E5520 @ 2.27GHz Memory: 24 GB localhost ]# rpm -qa | grep httpd httpd-2.2.3-43.el5.centos.3 system-config-httpd-1.3.3.3-1.el5 httpd-devel-2.2.3-43.el5.centos.3 httpd-devel-2.2.3-43.el5.centos.3
Need help in collecting data and solve the issue.
Below are the details
message kernel: httpd[2090]: segfault at 00007fff3bacfec4 rip 00002ba6bbc50855 rsp 00007fff3bacfe50 error 6
(gdb) bt full #0 0x00002b30a5a91b00 in __accept_nocancel () from /lib64/libpthread.so.0 No symbol table info available. #1 0x00002b30a58772f4 in apr_socket_accept () from /usr/lib64/libapr-1.so.0 No symbol table info available. #2 0x00002b30a3fbcc31 in unixd_accept () No symbol table info available. #3 0x00002b30a3fbb6d7 in ?? () No symbol table info available. #4 0x00002b30a3fbba1a in ?? () No symbol table info available. #5 0x00002b30a3fbc27d in ap_mpm_run () No symbol table info available. #6 0x00002b30a3f96e48 in main () No symbol table info available.
(gdb) run Starting program: /usr/sbin/httpd [Thread debugging using libthread_db enabled] (98)Address already in use: make_sock: could not bind to address [::]:80 (98)Address already in use: make_sock: could not bind to address 0.0.0.0:80 no listening sockets available, shutting down Unable to open logs
Program exited with code 01.
Please advice.
Thank you, Sandeil
CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Seems like there's stuff causing httpd to crash and hang. Before starting httpd again you need to kill the hanging processes (ps aux|grep http or lsof -iTCP:80). What exactly are you serving off that web server? Do you have any manually installed/compiled programs (e.g. php) or from 3rd party repos?
Hello Lucian,
We are hosting sites and yes, we are using Webtatic repository. By tracing Zend is also installed from tar.gz package.
Thank you, Sandeil
Seems like there's stuff causing httpd to crash and hang. Before starting httpd again you need to kill the hanging processes (ps aux|grep http or lsof -iTCP:80). What exactly are you serving off that web server? Do you have any manually installed/compiled programs (e.g. php) or from 3rd party repos?
Hi list,
As an update I have the latest core dump and did a back trace. Please see below.
Thank you, Sandeil
===== *** thread apply all bt
#20131 0x00002ac96feba29a in ?? () from /lib64/libpcre.so.0 #20132 0x00002ac96feba1d9 in ?? () from /lib64/libpcre.so.0 #20133 0x00002ac96feba29a in ?? () from /lib64/libpcre.so.0 #20134 0x00002ac96feb6c4b in ?? () from /lib64/libpcre.so.0 #20135 0x00002ac96febb6c1 in pcre_exec () from /lib64/libpcre.so.0 #20136 0x00002ac97a155806 in php_pcre_replace_impl () from /etc/httpd/modules/libphp5.so #20137 0x00002ac97a1569b2 in ?? () from /etc/httpd/modules/libphp5.so #20138 0x00002ac97a157075 in ?? () from /etc/httpd/modules/libphp5.so #20139 0x00002ac97f8d19a3 in ?? () from /usr/local/Zend/lib/Optimizer-3.3.3/php-5.2.x/ZendOptimizer.so #20140 0x00002ac97f8d4188 in ?? () from /usr/local/Zend/lib/Optimizer-3.3.3/php-5.2.x/ZendOptimizer.so #20141 0x00002ac97f8d506c in ?? () from /usr/local/Zend/lib/Optimizer-3.3.3/php-5.2.x/ZendOptimizer.so #20142 0x00002ac97f8d3b74 in ?? () from /usr/local/Zend/lib/Optimizer-3.3.3/php-5.2.x/ZendOptimizer.so #20143 0x00002ac97f92f311 in zend_oe () from /usr/local/Zend/lib/Optimizer-3.3.3/php-5.2.x/ZendOptimizer.so #20144 0x00002ac97f8d434f in ?? () from /usr/local/Zend/lib/Optimizer-3.3.3/php-5.2.x/ZendOptimizer.so #20145 0x00002ac97f8d506c in ?? () from /usr/local/Zend/lib/Optimizer-3.3.3/php-5.2.x/ZendOptimizer.so #20146 0x00002ac97f8d3b74 in ?? () from /usr/local/Zend/lib/Optimizer-3.3.3/php-5.2.x/ZendOptimizer.so #20147 0x00002ac97f92f311 in zend_oe () from /usr/local/Zend/lib/Optimizer-3.3.3/php-5.2.x/ZendOptimizer.so #20148 0x00002ac97f8d434f in ?? () from /usr/local/Zend/lib/Optimizer-3.3.3/php-5.2.x/ZendOptimizer.so #20149 0x00002ac97f8d506c in ?? () from /usr/local/Zend/lib/Optimizer-3.3.3/php-5.2.x/ZendOptimizer.so ---Type <return> to continue, or q <return> to quit--- #20150 0x00002ac97f8d3b74 in ?? () from /usr/local/Zend/lib/Optimizer-3.3.3/php-5.2.x/ZendOptimizer.so #20151 0x00002ac97f92f311 in zend_oe () from /usr/local/Zend/lib/Optimizer-3.3.3/php-5.2.x/ZendOptimizer.so #20152 0x00002ac97f8d434f in ?? () from /usr/local/Zend/lib/Optimizer-3.3.3/php-5.2.x/ZendOptimizer.so #20153 0x00002ac97f8d506c in ?? () from /usr/local/Zend/lib/Optimizer-3.3.3/php-5.2.x/ZendOptimizer.so #20154 0x00002ac97f8d3b74 in ?? () from /usr/local/Zend/lib/Optimizer-3.3.3/php-5.2.x/ZendOptimizer.so #20155 0x00002ac97f92f311 in zend_oe () from /usr/local/Zend/lib/Optimizer-3.3.3/php-5.2.x/ZendOptimizer.so #20156 0x00002ac97a29e0af in ?? () from /etc/httpd/modules/libphp5.so #20157 0x00002ac97f8d3b74 in ?? () from /usr/local/Zend/lib/Optimizer-3.3.3/php-5.2.x/ZendOptimizer.so #20158 0x00002ac97f92f311 in zend_oe () from /usr/local/Zend/lib/Optimizer-3.3.3/php-5.2.x/ZendOptimizer.so #20159 0x00002ac97a277cb3 in zend_execute_scripts () from /etc/httpd/modules/libphp5.so #20160 0x00002ac97a23798a in php_execute_script () from /etc/httpd/modules/libphp5.so #20161 0x00002ac97a2fc805 in ?? () from /etc/httpd/modules/libphp5.so #20162 0x00002ac96f7e0a4a in ap_run_handler () #20163 0x00002ac96f7e3ed8 in ap_invoke_handler () #20164 0x00002ac96f7ee78a in ap_internal_redirect () #20165 0x00002ac978628bc0 in ap_make_dirstr_parent () from /etc/httpd/modules/mod_rewrite.so #20166 0x00002ac96f7e0a4a in ap_run_handler () #20167 0x00002ac96f7e3ed8 in ap_invoke_handler () #20168 0x00002ac96f7ee938 in ap_process_request () #20169 0x00002ac96f7ebb70 in ?? () #20170 0x00002ac96f7e7cd2 in ap_run_process_connection () #20171 0x00002ac96f7f2789 in ?? () #20172 0x00002ac96f7f2a1a in ?? () #20173 0x00002ac96f7f327d in ap_mpm_run () #20174 0x00002ac96f7cde48 in main () (gdb) (gdb)
*** bt #20135 0x00002ac96febb6c1 in pcre_exec () from /lib64/libpcre.so.0 #20136 0x00002ac97a155806 in php_pcre_replace_impl () from /etc/httpd/modules/libphp5.so #20137 0x00002ac97a1569b2 in ?? () from /etc/httpd/modules/libphp5.so #20138 0x00002ac97a157075 in ?? () from /etc/httpd/modules/libphp5.so #20139 0x00002ac97f8d19a3 in ?? () from /usr/local/Zend/lib/Optimizer-3.3.3/php-5.2.x/ZendOptimizer.so #20140 0x00002ac97f8d4188 in ?? () from /usr/local/Zend/lib/Optimizer-3.3.3/php-5.2.x/ZendOptimizer.so #20141 0x00002ac97f8d506c in ?? () from /usr/local/Zend/lib/Optimizer-3.3.3/php-5.2.x/ZendOptimizer.so #20142 0x00002ac97f8d3b74 in ?? () from /usr/local/Zend/lib/Optimizer-3.3.3/php-5.2.x/ZendOptimizer.so #20143 0x00002ac97f92f311 in zend_oe () from /usr/local/Zend/lib/Optimizer-3.3.3/php-5.2.x/ZendOptimizer.so #20144 0x00002ac97f8d434f in ?? () from /usr/local/Zend/lib/Optimizer-3.3.3/php-5.2.x/ZendOptimizer.so #20145 0x00002ac97f8d506c in ?? () from /usr/local/Zend/lib/Optimizer-3.3.3/php-5.2.x/ZendOptimizer.so #20146 0x00002ac97f8d3b74 in ?? () from /usr/local/Zend/lib/Optimizer-3.3.3/php-5.2.x/ZendOptimizer.so #20147 0x00002ac97f92f311 in zend_oe () from /usr/local/Zend/lib/Optimizer-3.3.3/php-5.2.x/ZendOptimizer.so #20148 0x00002ac97f8d434f in ?? () from /usr/local/Zend/lib/Optimizer-3.3.3/php-5.2.x/ZendOptimizer.so #20149 0x00002ac97f8d506c in ?? () from /usr/local/Zend/lib/Optimizer-3.3.3/php-5.2.x/ZendOptimizer.so #20150 0x00002ac97f8d3b74 in ?? () from /usr/local/Zend/lib/Optimizer-3.3.3/php-5.2.x/ZendOptimizer.so #20151 0x00002ac97f92f311 in zend_oe () from /usr/local/Zend/lib/Optimizer-3.3.3/php-5.2.x/ZendOptimizer.so ---Type <return> to continue, or q <return> to quit--- #20152 0x00002ac97f8d434f in ?? () from /usr/local/Zend/lib/Optimizer-3.3.3/php-5.2.x/ZendOptimizer.so #20153 0x00002ac97f8d506c in ?? () from /usr/local/Zend/lib/Optimizer-3.3.3/php-5.2.x/ZendOptimizer.so #20154 0x00002ac97f8d3b74 in ?? () from /usr/local/Zend/lib/Optimizer-3.3.3/php-5.2.x/ZendOptimizer.so #20155 0x00002ac97f92f311 in zend_oe () from /usr/local/Zend/lib/Optimizer-3.3.3/php-5.2.x/ZendOptimizer.so #20156 0x00002ac97a29e0af in ?? () from /etc/httpd/modules/libphp5.so #20157 0x00002ac97f8d3b74 in ?? () from /usr/local/Zend/lib/Optimizer-3.3.3/php-5.2.x/ZendOptimizer.so #20158 0x00002ac97f92f311 in zend_oe () from /usr/local/Zend/lib/Optimizer-3.3.3/php-5.2.x/ZendOptimizer.so #20159 0x00002ac97a277cb3 in zend_execute_scripts () from /etc/httpd/modules/libphp5.so #20160 0x00002ac97a23798a in php_execute_script () from /etc/httpd/modules/libphp5.so #20161 0x00002ac97a2fc805 in ?? () from /etc/httpd/modules/libphp5.so #20162 0x00002ac96f7e0a4a in ap_run_handler () #20163 0x00002ac96f7e3ed8 in ap_invoke_handler () #20164 0x00002ac96f7ee78a in ap_internal_redirect () #20165 0x00002ac978628bc0 in ap_make_dirstr_parent () from /etc/httpd/modules/mod_rewrite.so #20166 0x00002ac96f7e0a4a in ap_run_handler () #20167 0x00002ac96f7e3ed8 in ap_invoke_handler () #20168 0x00002ac96f7ee938 in ap_process_request () #20169 0x00002ac96f7ebb70 in ?? () #20170 0x00002ac96f7e7cd2 in ap_run_process_connection () #20171 0x00002ac96f7f2789 in ?? () #20172 0x00002ac96f7f2a1a in ?? () #20173 0x00002ac96f7f327d in ap_mpm_run () #20174 0x00002ac96f7cde48 in main ()
*** bt full
---Type <return> to continue, or q <return> to quit--- #20130 0x00002ac96feba1d9 in ?? () from /lib64/libpcre.so.0 No symbol table info available. #20131 0x00002ac96feba29a in ?? () from /lib64/libpcre.so.0 No symbol table info available. #20132 0x00002ac96feba1d9 in ?? () from /lib64/libpcre.so.0 No symbol table info available. #20133 0x00002ac96feba29a in ?? () from /lib64/libpcre.so.0 No symbol table info available. #20134 0x00002ac96feb6c4b in ?? () from /lib64/libpcre.so.0 No symbol table info available. #20135 0x00002ac96febb6c1 in pcre_exec () from /lib64/libpcre.so.0 No symbol table info available. #20136 0x00002ac97a155806 in php_pcre_replace_impl () from /etc/httpd/modules/libphp5.so No symbol table info available. #20137 0x00002ac97a1569b2 in ?? () from /etc/httpd/modules/libphp5.so No symbol table info available. #20138 0x00002ac97a157075 in ?? () from /etc/httpd/modules/libphp5.so No symbol table info available. #20139 0x00002ac97f8d19a3 in ?? () from /usr/local/Zend/lib/Optimizer-3.3.3/php-5.2.x/ZendOptimizer.so No symbol table info available. #20140 0x00002ac97f8d4188 in ?? () from /usr/local/Zend/lib/Optimizer-3.3.3/php-5.2.x/ZendOptimizer.so No symbol table info available. #20141 0x00002ac97f8d506c in ?? () from /usr/local/Zend/lib/Optimizer-3.3.3/php-5.2.x/ZendOptimizer.so No symbol table info available. #20142 0x00002ac97f8d3b74 in ?? () from /usr/local/Zend/lib/Optimizer-3.3.3/php-5.2.x/ZendOptimizer.so No symbol table info available. #20143 0x00002ac97f92f311 in zend_oe () from /usr/local/Zend/lib/Optimizer-3.3.3/php-5.2.x/ZendOptimizer.so No symbol table info available. #20144 0x00002ac97f8d434f in ?? () from /usr/local/Zend/lib/Optimizer-3.3.3/php-5.2.x/ZendOptimizer.so No symbol table info available. #20145 0x00002ac97f8d506c in ?? () from /usr/local/Zend/lib/Optimizer-3.3.3/php-5.2.x/ZendOptimizer.so No symbol table info available. #20146 0x00002ac97f8d3b74 in ?? () from /usr/local/Zend/lib/Optimizer-3.3.3/php-5.2.x/ZendOptimizer.so No symbol table info available. #20147 0x00002ac97f92f311 in zend_oe () from /usr/local/Zend/lib/Optimizer-3.3.3/php-5.2.x/ZendOptimizer.so No symbol table info available. #20148 0x00002ac97f8d434f in ?? () from /usr/local/Zend/lib/Optimizer-3.3.3/php-5.2.x/ZendOptimizer.so No symbol table info available. #20149 0x00002ac97f8d506c in ?? () from /usr/local/Zend/lib/Optimizer-3.3.3/php-5.2.x/ZendOptimizer.so No symbol table info available. #20150 0x00002ac97f8d3b74 in ?? () from /usr/local/Zend/lib/Optimizer-3.3.3/php-5.2.x/ZendOptimizer.so No symbol table info available. #20151 0x00002ac97f92f311 in zend_oe () from /usr/local/Zend/lib/Optimizer-3.3.3/php-5.2.x/ZendOptimizer.so No symbol table info available. ---Type <return> to continue, or q <return> to quit--- #20152 0x00002ac97f8d434f in ?? () from /usr/local/Zend/lib/Optimizer-3.3.3/php-5.2.x/ZendOptimizer.so No symbol table info available. #20153 0x00002ac97f8d506c in ?? () from /usr/local/Zend/lib/Optimizer-3.3.3/php-5.2.x/ZendOptimizer.so No symbol table info available. #20154 0x00002ac97f8d3b74 in ?? () from /usr/local/Zend/lib/Optimizer-3.3.3/php-5.2.x/ZendOptimizer.so No symbol table info available. #20155 0x00002ac97f92f311 in zend_oe () from /usr/local/Zend/lib/Optimizer-3.3.3/php-5.2.x/ZendOptimizer.so No symbol table info available. #20156 0x00002ac97a29e0af in ?? () from /etc/httpd/modules/libphp5.so No symbol table info available. #20157 0x00002ac97f8d3b74 in ?? () from /usr/local/Zend/lib/Optimizer-3.3.3/php-5.2.x/ZendOptimizer.so No symbol table info available. #20158 0x00002ac97f92f311 in zend_oe () from /usr/local/Zend/lib/Optimizer-3.3.3/php-5.2.x/ZendOptimizer.so No symbol table info available. #20159 0x00002ac97a277cb3 in zend_execute_scripts () from /etc/httpd/modules/libphp5.so No symbol table info available. #20160 0x00002ac97a23798a in php_execute_script () from /etc/httpd/modules/libphp5.so No symbol table info available. #20161 0x00002ac97a2fc805 in ?? () from /etc/httpd/modules/libphp5.so No symbol table info available. #20162 0x00002ac96f7e0a4a in ap_run_handler () No symbol table info available. #20163 0x00002ac96f7e3ed8 in ap_invoke_handler () No symbol table info available. #20164 0x00002ac96f7ee78a in ap_internal_redirect () No symbol table info available. #20165 0x00002ac978628bc0 in ap_make_dirstr_parent () from /etc/httpd/modules/mod_rewrite.so No symbol table info available. #20166 0x00002ac96f7e0a4a in ap_run_handler () No symbol table info available. #20167 0x00002ac96f7e3ed8 in ap_invoke_handler () No symbol table info available. #20168 0x00002ac96f7ee938 in ap_process_request () No symbol table info available. #20169 0x00002ac96f7ebb70 in ?? () No symbol table info available. #20170 0x00002ac96f7e7cd2 in ap_run_process_connection () No symbol table info available. #20171 0x00002ac96f7f2789 in ?? () No symbol table info available. #20172 0x00002ac96f7f2a1a in ?? () No symbol table info available. #20173 0x00002ac96f7f327d in ap_mpm_run () No symbol table info available. ---Type <return> to continue, or q <return> to quit--- #20174 0x00002ac96f7cde48 in main () No symbol table info available.
*** bt all No symbol table is loaded. Use the "file" command.
=====
-
Lucian -
Sandeil Tenebro