Hi, my ip_conntrack table is filling up and now my server is dropping packets. I'm running CentOS release 4.4 (Final) on a fairly busy webserver. The table is full of various connections, including a lot of "ESTABLISHED" tcp connections from my webserver (the src is my webserver ip), and some other random connections to my webserver, and many "ASSURED" connections. So why is it filling up? I changed the default timeout value like so:
echo 36000 > /proc/sys/net/ipv4/netfilter/ip_conntrack_tcp_timeout_established
but I don't think that's had any effect. any thoughts? what additional info can I provide that would be helpful? I did find a script that clears out some of the stale connections using hping2, but I don't know if that's really a great solution to this problem.
cat /proc/sys/net/ipv4/ip_conntrack_max # 34576
after cleaning out the ip_conntrack table using an hping2 script: cat /proc/net/ip_conntrack | wc -l # 3702 -- this number was around 34000 before I cleared it out because it was dropping packets. rebooting the machine, of course, clears it out.
I've spent many hours banging my head against the wall trying to figure this out, reading in google groups and in various forums, to no avail. My webserver does send out emails to a few thousand registered users (if they opt it for the email) every day.
Thank you for your time! I hope I sent this to the right list. This looked like the right one. Sorry in advance if I made a mistake.
Michelson
Hi Michelson, I have that problem also on one of my FW box. What i did is i created a cronjob that reload the iptables rule. In this case you dont drop any connections and you dont need to reboot your box. So far its working on our production deployed FW.
Note: You need to find out how frequent you do this on a weeks.
Cheers!
On 6/12/07, yossarian1@gmail.com yossarian1@gmail.com wrote:
Hi, my ip_conntrack table is filling up and now my server is dropping packets. I'm running CentOS release 4.4 (Final) on a fairly busy webserver. The table is full of various connections, including a lot of "ESTABLISHED" tcp connections from my webserver (the src is my webserver ip), and some other random connections to my webserver, and many "ASSURED" connections. So why is it filling up? I changed the default timeout value like so:
echo 36000 > /proc/sys/net/ipv4/netfilter/ip_conntrack_tcp_timeout_established
but I don't think that's had any effect. any thoughts? what additional info can I provide that would be helpful? I did find a script that clears out some of the stale connections using hping2, but I don't know if that's really a great solution to this problem.
cat /proc/sys/net/ipv4/ip_conntrack_max # 34576
after cleaning out the ip_conntrack table using an hping2 script: cat /proc/net/ip_conntrack | wc -l # 3702 -- this number was around 34000 before I cleared it out because it was dropping packets. rebooting the machine, of course, clears it out.
I've spent many hours banging my head against the wall trying to figure this out, reading in google groups and in various forums, to no avail. My webserver does send out emails to a few thousand registered users (if they opt it for the email) every day.
Thank you for your time! I hope I sent this to the right list. This looked like the right one. Sorry in advance if I made a mistake.
Michelson _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
If your server isn't having a problem, then why not bump up the conntrack number? I've bumped mine up to 2097152. I can't remember where, but I remember reading a pdf article on iptables and how many connections a specific server with X amount of CPU's and X amount of memory can handle.
[root@firewall1 ~]# cat /proc/sys/net/ipv4/ip_conntrack_max 2097152
-matt
On 6/15/07, Michael Calizo mike.calizo@gmail.com wrote:
Hi Michelson, I have that problem also on one of my FW box. What i did is i created a cronjob that reload the iptables rule. In this case you dont drop any connections and you dont need to reboot your box. So far its working on our production deployed FW.
Note: You need to find out how frequent you do this on a weeks.
Cheers!
On 6/12/07, yossarian1@gmail.com yossarian1@gmail.com wrote:
Hi, my ip_conntrack table is filling up and now my server is dropping packets. I'm running CentOS release 4.4 (Final) on a fairly busy webserver. The table is full of various connections, including a lot of "ESTABLISHED" tcp connections from my webserver (the src is my webserver ip), and some other random connections to my webserver, and many "ASSURED" connections. So why is it filling up? I changed the default timeout value like so:
echo 36000 >
/proc/sys/net/ipv4/netfilter/ip_conntrack_tcp_timeout_established
but I don't think that's had any effect. any thoughts? what additional info can I provide that would be helpful? I did find a script that clears out some of the stale connections using hping2, but I don't know if that's really a great solution to this problem.
cat /proc/sys/net/ipv4/ip_conntrack_max # 34576
after cleaning out the ip_conntrack table using an hping2 script: cat /proc/net/ip_conntrack | wc -l # 3702 -- this number was around 34000 before I cleared it out because it was dropping packets. rebooting the machine, of course, clears it out.
I've spent many hours banging my head against the wall trying to figure this out, reading in google groups and in various forums, to no avail. My webserver does send out emails to a few thousand registered users (if they opt it for the email) every day.
Thank you for your time! I hope I sent this to the right list. This looked like the right one. Sorry in advance if I made a mistake.
Michelson _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
-- Mike Calizo Registered Linux User # 365113
Even the longest journey has to start with a small first-step
CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
On 6/12/07, yossarian1@gmail.com yossarian1@gmail.com wrote:
Hi, my ip_conntrack table is filling up and now my server is dropping packets. I'm running CentOS release 4.4 (Final) on a fairly busy webserver. The table is full of various connections, including a lot of "ESTABLISHED" tcp connections from my webserver (the src is my webserver ip), and some other random connections to my webserver, and many "ASSURED" connections. So why is it filling up? I changed the default timeout value like so:
echo 36000 > /proc/sys/net/ipv4/netfilter/ip_conntrack_tcp_timeout_established
but I don't think that's had any effect. any thoughts? what additional info can I provide that would be helpful? I did find a script that clears out some of the stale connections using hping2, but I don't know if that's really a great solution to this problem.
I have seen this in connection with some dreadful internet worm affecting Windows stations in the last hours. This particular worm seems related to DEL.EXE file modifications. :(
-
Eduardo Grosclaude -
Matt Shields -
Michael Calizo -
yossarian1＠gmail.com