[CentOS] since CentOS 5.6 upgrade, squid crashes

Mon May 9 08:10:59 UTC 2011
Laurent CREPET <lct at megrapet.fr>

On Fri, 6 May 2011 11:06:18 +0200
Laurent CREPET <lct at megrapet.fr> wrote:

> squid's logs are rotated on a weekly basis by logrorate. squid crashed 2 times, with one week between.
> 
> Now, I have switched to daily/rotate 15, and squid did not crash since the change (2 days ago).

Today, squid crashed again.

Below last logs:
[root at stargate squid]# zcat cache.log.1.gz && cat cache.log
2011/05/08 04:04:27| storeDirWriteCleanLogs: Starting...
2011/05/08 04:04:27|     65536 entries written so far.
2011/05/08 04:04:27|    131072 entries written so far.
2011/05/08 04:04:27|    196608 entries written so far.
2011/05/08 04:04:27|    262144 entries written so far.
2011/05/08 04:04:27|    327680 entries written so far.
2011/05/08 04:04:27|    393216 entries written so far.
2011/05/08 04:04:27|    458752 entries written so far.
2011/05/08 04:04:27|    524288 entries written so far.
2011/05/08 04:04:27|    589824 entries written so far.
2011/05/08 04:04:27|    655360 entries written so far.
2011/05/08 04:04:27|    720896 entries written so far.
2011/05/08 04:04:27|    786432 entries written so far.
2011/05/08 04:04:27|    851968 entries written so far.
2011/05/08 04:04:27|    917504 entries written so far.
2011/05/08 04:04:27|    983040 entries written so far.
2011/05/08 04:04:27|   1048576 entries written so far.
2011/05/08 04:04:27|   1114112 entries written so far.
2011/05/08 04:04:27|   1179648 entries written so far.
2011/05/08 04:04:27|   1245184 entries written so far.
2011/05/08 04:04:27|   Finished.  Wrote 1266215 entries.
2011/05/08 04:04:27|   Took 0.4 seconds (3398869.9 entries/sec).
2011/05/08 04:04:27| aioSync: flushing pending I/O operations
2011/05/08 04:04:27| aioSync: done
2011/05/08 04:04:27| logfileRotate: /var/log/squid/store.log
2011/05/08 04:04:27| logfileRotate: /var/log/squid/access.log
2011/05/09 04:02:11| storeDirWriteCleanLogs: Starting...
2011/05/09 04:02:11| WARNING: Closing open FD   15
2011/05/09 04:02:11| commSetEvents: epoll_ctl(EPOLL_CTL_DEL): failed on fd=15: (1) Operation not permitted
2011/05/09 04:02:11|     65536 entries written so far.
2011/05/09 04:02:11|    131072 entries written so far.
2011/05/09 04:02:11|    196608 entries written so far.
2011/05/09 04:02:11|    262144 entries written so far.
2011/05/09 04:02:11|    327680 entries written so far.
2011/05/09 04:02:11|    393216 entries written so far.
2011/05/09 04:02:11|    458752 entries written so far.
2011/05/09 04:02:11|    524288 entries written so far.
2011/05/09 04:02:11|    589824 entries written so far.
2011/05/09 04:02:11|    655360 entries written so far.
2011/05/09 04:02:11|    720896 entries written so far.
2011/05/09 04:02:11|    786432 entries written so far.
2011/05/09 04:02:11|    851968 entries written so far.
2011/05/09 04:02:11|    917504 entries written so far.
2011/05/09 04:02:11|    983040 entries written so far.
2011/05/09 04:02:11|   1048576 entries written so far.
2011/05/09 04:02:11|   1114112 entries written so far.
2011/05/09 04:02:11|   1179648 entries written so far.
2011/05/09 04:02:11|   1245184 entries written so far.
2011/05/09 04:02:11|   Finished.  Wrote 1265715 entries.
2011/05/09 04:02:11|   Took 0.4 seconds (3220452.2 entries/sec).
FATAL: logfileWrite: /var/log/squid/store.log: (0) Success

Squid Cache (Version 2.6.STABLE21): Terminated abnormally.
CPU Usage: 512.797 seconds = 298.711 user + 214.086 sys
Maximum Resident Size: 936720 KB
Page faults with physical i/o: 2
Memory usage for squid via mallinfo():
	total space in arena:  231496 KB
	Ordinary blocks:       220724 KB   1045 blks
	Small blocks:               0 KB      0 blks
	Holding blocks:          4972 KB      3 blks
	Free Small blocks:          0 KB
	Free Ordinary blocks:   10771 KB
	Total in use:          225696 KB 95%
	Total free:             10771 KB 5%
2011/05/09 04:02:14| Starting Squid Cache version 2.6.STABLE21 for x86_64-redhat-linux-gnu...
2011/05/09 04:02:14| Process ID 20015
2011/05/09 04:02:14| With 1024 file descriptors available
2011/05/09 04:02:14| Using epoll for the IO loop
2011/05/09 04:02:14| DNS Socket created at 0.0.0.0, port 52427, FD 6
2011/05/09 04:02:14| Adding domain fr.atempo.network from /etc/resolv.conf
2011/05/09 04:02:14| Adding nameserver 172.16.15.130 from /etc/resolv.conf
2011/05/09 04:02:14| Adding nameserver 172.16.17.13 from /etc/resolv.conf
2011/05/09 04:02:14| Adding nameserver 172.16.16.100 from /etc/resolv.conf
2011/05/09 04:02:14| Adding nameserver 172.16.10.200 from /etc/resol2011/05/09 09:08:07| Starting Squid Cache version 2.6.STABLE21 for x86_64-redhat-linux-gnu...
2011/05/09 09:08:07| Process ID 31894
2011/05/09 09:08:07| With 1024 file descriptors available
2011/05/09 09:08:07| Using epoll for the IO loop
2011/05/09 09:08:07| DNS Socket created at 0.0.0.0, port 45845, FD 6
2011/05/09 09:08:07| Adding domain fr.atempo.network from /etc/resolv.conf
2011/05/09 09:08:07| Adding nameserver 172.16.15.130 from /etc/resolv.conf
2011/05/09 09:08:07| Adding nameserver 172.16.17.13 from /etc/resolv.conf
2011/05/09 09:08:07| Adding nameserver 172.16.16.100 from /etc/resolv.conf
2011/05/09 09:08:07| Adding nameserver 172.16.10.200 from /etc/resolv.conf
2011/05/09 09:08:07| User-Agent logging is disabled.
2011/05/09 09:08:07| Referer logging is disabled.
2011/05/09 09:08:07| Unlinkd pipe opened on FD 11
2011/05/09 09:08:07| Swap maxSize 102400000 + 8192 KB, estimated 7877553 objects
2011/05/09 09:08:07| Target number of buckets: 393877
2011/05/09 09:08:07| Using 524288 Store buckets
2011/05/09 09:08:07| Max Mem  size: 8192 KB
2011/05/09 09:08:07| Max Swap size: 102400000 KB
2011/05/09 09:08:07| Local cache digest enabled; rebuild/rewrite every 3600/3600 sec
2011/05/09 09:08:07| Rebuilding storage in /proxies/squid (DIRTY)
2011/05/09 09:08:07| Using Least Load store dir selection
2011/05/09 09:08:07| Set Current Directory to /var/spool/squid
2011/05/09 09:08:07| Loaded Icons.
2011/05/09 09:08:07| Accepting proxy HTTP connections at 0.0.0.0, port 3128, FD 15.
2011/05/09 09:08:07| Accepting proxy HTTP connections at 0.0.0.0, port 8080, FD 16.
2011/05/09 09:08:07| Accepting ICP messages at 0.0.0.0, port 3130, FD 17.
2011/05/09 09:08:07| WCCP Disabled.
2011/05/09 09:08:07| Ready to serve requests.
2011/05/09 09:08:08| Store rebuilding is  0.3% complete
2011/05/09 09:08:12| Done reading /proxies/squid swaplog (1265714 entries)
2011/05/09 09:08:12| Finished rebuilding storage from disk.
2011/05/09 09:08:12|   1265714 Entries scanned
2011/05/09 09:08:12|         0 Invalid entries.
2011/05/09 09:08:12|         0 With invalid flags.
2011/05/09 09:08:12|   1265713 Objects loaded.
2011/05/09 09:08:12|         0 Objects expired.
2011/05/09 09:08:12|         0 Objects cancelled.
2011/05/09 09:08:12|         0 Duplicate URLs purged.
2011/05/09 09:08:12|         1 Swapfile clashes avoided.
2011/05/09 09:08:12|   Took 4.6 seconds (274455.1 objects/sec).
2011/05/09 09:08:12| Beginning Validation Procedure
2011/05/09 09:08:12|    262144 Entries Validated so far.
2011/05/09 09:08:12|    524288 Entries Validated so far.
2011/05/09 09:08:12|    786432 Entries Validated so far.
2011/05/09 09:08:12|   1048576 Entries Validated so far.
2011/05/09 09:08:12|   Completed Validation Procedure
2011/05/09 09:08:12|   Validated 1265713 Entries
2011/05/09 09:08:12|   store_swap_size = 92157784k
2011/05/09 09:08:12| storeLateRelease: released 0 objects
2011/05/09 09:11:09| squidaio_queue_request: WARNING - Queue congestion
2011/05/09 09:19:50| CACHEMGR: <unknown>@127.0.0.1 requesting 'utilization'
[root at stargate squid]# 


-- 
Laurent CREPET <lct at megrapet.fr>