Hi,
I'm experiencing the most perplexing problem with iptables on CentOS 5.2. I'm hoping someone can point out what I must be missing here.
I have memcached set up on several nodes on an internal network. I have the following rules set up to allow traffic between memcached nodes:
IPTABLES -A INPUT -i bond0 -p tcp -m tcp -s 192.168.1.0/24 -d 192.168.1.0/24 --dport 11211 -j ACCEPT IPTABLES -A FORWARD -i bond0 -p tcp -m tcp -s 192.168.1.0/24 -d 192.168.1.0/24 --dport 11211 -j ACCEPT IPTABLES -A OUTPUT -o bond0 -p tcp -m tcp -s 192.168.1.0/24 -d 192.168.1.0/24 --dport 11211 -j ACCEPT IPTABLES -A FORWARD -o bond0 -p tcp -m tcp -s 192.168.1.0/24 -d 192.168.1.0/24 --dport 11211 -j ACCEPT
And yet, iptables continues to drop memcached packets. It appears to simply skip these rules and fall through to the default CATCH-ALL policy. Here are some iptables log messages notifying of the dropped packets:
Dec 12 20:33:53 s1 kernel: DROP -- Catch All: IN= OUT=bond0 SRC=192.168.1.1 DST=192.168.1.2 LEN=40 TOS=0x00 PREC=0x00 TTL=64 ID=0 DF PROTO=TCP SPT=11211 DPT=47567 WINDOW=0 RES=0x00 RST URGP=0 Dec 12 20:33:59 s1 kernel: DROP -- Catch All: IN= OUT=bond0 SRC=192.168.1.1 DST=192.168.1.2 LEN=40 TOS=0x00 PREC=0x00 TTL=64 ID=0 DF PROTO=TCP SPT=11211 DPT=47771 WINDOW=0 RES=0x00 RST URGP=0 Dec 12 20:34:00 s1 kernel: DROP -- Catch All: IN= OUT=bond0 SRC=192.168.1.1 DST=192.168.1.2 LEN=40 TOS=0x00 PREC=0x00 TTL=64 ID=0 DF PROTO=TCP SPT=11211 DPT=35600 WINDOW=0 RES=0x00 RST URGP=0 Dec 12 20:34:12 s1 kernel: DROP -- Catch All: IN= OUT=bond0 SRC=192.168.1.1 DST=192.168.1.4 LEN=40 TOS=0x00 PREC=0x00 TTL=64 ID=0 DF PROTO=TCP SPT=11211 DPT=52823 WINDOW=0 RES=0x00 RST URGP=0 Dec 12 20:34:40 s1 kernel: DROP -- Catch All: IN= OUT=bond0 SRC=192.168.1.1 DST=192.168.1.4 LEN=40 TOS=0x00 PREC=0x00 TTL=64 ID=0 DF PROTO=TCP SPT=11211 DPT=44946 WINDOW=0 RES=0x00 RST URGP=0 Dec 12 20:34:42 s1 kernel: DROP -- Catch All: IN= OUT=bond0 SRC=192.168.1.1 DST=192.168.1.3 LEN=40 TOS=0x00 PREC=0x00 TTL=64 ID=0 DF PROTO=TCP SPT=11211 DPT=45046 WINDOW=0 RES=0x00 RST URGP=0 Dec 12 20:37:40 s1 kernel: DROP -- Catch All: IN= OUT=bond0 SRC=192.168.1.1 DST=192.168.1.4 LEN=40 TOS=0x00 PREC=0x00 TTL=64 ID=0 DF PROTO=TCP SPT=11211 DPT=58825 WINDOW=0 RES=0x00 RST URGP=0 Dec 12 20:37:49 s1 kernel: DROP -- Catch All: IN= OUT=bond0 SRC=192.168.1.1 DST=192.168.1.2 LEN=40 TOS=0x00 PREC=0x00 TTL=64 ID=0 DF PROTO=TCP SPT=11211 DPT=44183 WINDOW=0 RES=0x00 RST URGP=0
I am not seeing anything like this with any of the other filtered ports in my ruleset. Is there something obvious here that I am just missing? Or am I running into a possible kernel bug of some sort?
Thanks in advance for any insights.
Hi,
On Fri, Dec 12, 2008 at 15:45, Art Age Software artagesw@gmail.com wrote:
IPTABLES -A XXX -i bond0 -p tcp -m tcp -s 192.168.1.0/24 -d 192.168.1.0/24 --dport 11211 -j ACCEPT
Dec 12 20:33:53 s1 kernel: DROP -- Catch All: IN= OUT=bond0 SRC=192.168.1.1 DST=192.168.1.2 LEN=40 TOS=0x00 PREC=0x00 TTL=64 ID=0 DF PROTO=TCP SPT=11211 DPT=47567 WINDOW=0 RES=0x00 RST URGP=0
The packages it's dropping are with *source* port 11211, they are the replies.
Either configure your firewall in stateful mode (-m state, --state NEW, --state ESTABLISHED, etc.) or add rules to allow the replies from that source port.
HTH, Filipe
Thanks for your reply. I originally had stateful rules in place and packets were being dropped. I had just switched to stateless rules in an attempt to fix the problem.
I will go back to stateful and update this thread with the new log messages.
Thanks.
Sam
On Fri, Dec 12, 2008 at 2:33 PM, Filipe Brandenburger filbranden@gmail.com wrote:
Hi,
On Fri, Dec 12, 2008 at 15:45, Art Age Software artagesw@gmail.com wrote:
IPTABLES -A XXX -i bond0 -p tcp -m tcp -s 192.168.1.0/24 -d 192.168.1.0/24 --dport 11211 -j ACCEPT
Dec 12 20:33:53 s1 kernel: DROP -- Catch All: IN= OUT=bond0 SRC=192.168.1.1 DST=192.168.1.2 LEN=40 TOS=0x00 PREC=0x00 TTL=64 ID=0 DF PROTO=TCP SPT=11211 DPT=47567 WINDOW=0 RES=0x00 RST URGP=0
The packages it's dropping are with *source* port 11211, they are the replies.
Either configure your firewall in stateful mode (-m state, --state NEW, --state ESTABLISHED, etc.) or add rules to allow the replies from that source port.
HTH, Filipe _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
OK, here are the original (stateful) rules reinstated:
IPTABLES -A INPUT -i bond0 -p tcp -m tcp -s 192.168.1.0/24 -d 192.168.1.0/24 --dport 11211 -m state --state NEW -j ACCEPT IPTABLES -A FORWARD -i bond0 -p tcp -m tcp -s 192.168.1.0/24 -d 192.168.1.0/24 --dport 11211 -m state --state NEW -j ACCEPT IPTABLES -A OUTPUT -o bond0 -p tcp -m tcp -s 192.168.1.0/24 -d 192.168.1.0/24 --dport 11211 -m state --state NEW -j ACCEPT IPTABLES -A FORWARD -o bond0 -p tcp -m tcp -s 192.168.1.0/24 -d 192.168.1.0/24 --dport 11211 -m state --state NEW -j ACCEPT
And here's a sampling of iptables dropping packets with stateful rules in place:
s1 kernel: DROP -- Catch All: IN=bond0 OUT= SRC=192.168.1.2 DST=192.168.1.1 LEN=52 TOS=0x00 PREC=0x00 TTL=64 ID=6467 DF PROTO=TCP SPT=51837 DPT=11211 WINDOW=202 RES=0x00 ACK FIN URGP=0 s1 kernel: DROP -- Catch All: IN=bond0 OUT= SRC=192.168.1.2 DST=192.168.1.1 LEN=52 TOS=0x00 PREC=0x00 TTL=64 ID=9158 DF PROTO=TCP SPT=50690 DPT=11211 WINDOW=339 RES=0x00 ACK FIN URGP=0 s1 kernel: DROP -- Catch All: IN=bond0 OUT= SRC=192.168.1.4 DST=192.168.1.1 LEN=52 TOS=0x00 PREC=0x00 TTL=64 ID=22155 DF PROTO=TCP SPT=53800 DPT=11211 WINDOW=113 RES=0x00 ACK FIN URGP=0 s1 kernel: DROP -- Catch All: IN= OUT=bond0 SRC=192.168.1.1 DST=192.168.1.3 LEN=1369 TOS=0x00 PREC=0x00 TTL=64 ID=17238 DF PROTO=TCP SPT=58539 DPT=11211 WINDOW=501 RES=0x00 ACK PSH URGP=0 s1 kernel: DROP -- Catch All: IN= OUT=bond0 SRC=192.168.1.1 DST=192.168.1.3 LEN=1086 TOS=0x00 PREC=0x00 TTL=64 ID=49105 DF PROTO=TCP SPT=49535 DPT=11211 WINDOW=501 RES=0x00 ACK PSH URGP=0
Any ideas?
On Fri, Dec 12, 2008 at 3:10 PM, Art Age Software artagesw@gmail.com wrote:
Thanks for your reply. I originally had stateful rules in place and packets were being dropped. I had just switched to stateless rules in an attempt to fix the problem.
I will go back to stateful and update this thread with the new log messages.
Thanks.
Sam
On Fri, Dec 12, 2008 at 2:33 PM, Filipe Brandenburger filbranden@gmail.com wrote:
Hi,
On Fri, Dec 12, 2008 at 15:45, Art Age Software artagesw@gmail.com wrote:
IPTABLES -A XXX -i bond0 -p tcp -m tcp -s 192.168.1.0/24 -d 192.168.1.0/24 --dport 11211 -j ACCEPT
Dec 12 20:33:53 s1 kernel: DROP -- Catch All: IN= OUT=bond0 SRC=192.168.1.1 DST=192.168.1.2 LEN=40 TOS=0x00 PREC=0x00 TTL=64 ID=0 DF PROTO=TCP SPT=11211 DPT=47567 WINDOW=0 RES=0x00 RST URGP=0
The packages it's dropping are with *source* port 11211, they are the replies.
Either configure your firewall in stateful mode (-m state, --state NEW, --state ESTABLISHED, etc.) or add rules to allow the replies from that source port.
HTH, Filipe _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Hi,
On Fri, Dec 12, 2008 at 18:28, Art Age Software artagesw@gmail.com wrote:
IPTABLES -A INPUT -i bond0 -p tcp -m tcp -s 192.168.1.0/24 -d 192.168.1.0/24 --dport 11211 -m state --state NEW -j ACCEPT
s1 kernel: DROP -- Catch All: IN=bond0 OUT= SRC=192.168.1.2 DST=192.168.1.1 LEN=52 TOS=0x00 PREC=0x00 TTL=64 ID=6467 DF PROTO=TCP SPT=51837 DPT=11211 WINDOW=202 RES=0x00 ACK FIN URGP=0
To use stateful rules, you must have rules for state ESTABLISHED too, otherwise it will only allow the first packet and not the all others. The first rule should always be:
iptables -A ... -i bond0 -m state --state ESTABLISHED,RELATED -j ACCEPT
That's the point of stateful rules, you match the rules further down for new connections, but existing connections will always match the first rule.
You should google for "iptables stateful" and try to get a better explanation of what it is and how it works.
HTH, Filipe
I do have such a rule at the top of the rule set. Sorry I did not provide more context. The basic structure of my rule set is:
1. Deny some known bad packet types. 2. Accept ESTABLISHED, RELATED traffic. 3. Accept rules of various sorts (such as the memcached rules). 4. Deny all (catch all rule).
I'm not an iptables newbie - I have used it for years and understand its basic operation. All my other rules work perfectly. This is why I am perplexed as to the memcached rules not working as expected and why I suspect a bug of some sort.
Thanks,
Sam
On Sat, Dec 13, 2008 at 7:14 AM, Filipe Brandenburger filbranden@gmail.com wrote:
Hi,
On Fri, Dec 12, 2008 at 18:28, Art Age Software artagesw@gmail.com wrote:
IPTABLES -A INPUT -i bond0 -p tcp -m tcp -s 192.168.1.0/24 -d 192.168.1.0/24 --dport 11211 -m state --state NEW -j ACCEPT
s1 kernel: DROP -- Catch All: IN=bond0 OUT= SRC=192.168.1.2 DST=192.168.1.1 LEN=52 TOS=0x00 PREC=0x00 TTL=64 ID=6467 DF PROTO=TCP SPT=51837 DPT=11211 WINDOW=202 RES=0x00 ACK FIN URGP=0
To use stateful rules, you must have rules for state ESTABLISHED too, otherwise it will only allow the first packet and not the all others. The first rule should always be:
iptables -A ... -i bond0 -m state --state ESTABLISHED,RELATED -j ACCEPT
That's the point of stateful rules, you match the rules further down for new connections, but existing connections will always match the first rule.
You should google for "iptables stateful" and try to get a better explanation of what it is and how it works.
HTH, Filipe _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
-
Art Age Software -
Filipe Brandenburger