On Wed, 24 Apr 2019 at 06:01, likun kun.li@ucarinc.com wrote:
Hi，guys.
There is a wierd problem with iptables recently, hopes somebody can help me.
I have installed Centos 7.2.1511 on a bare metal Dell server these days, disabled firewalld and enabled iptables.services, and setup a group of very simple rules, as the following:
I believe you are saying both 10.3.3.3 and 10.3.3.4 have this same firewall but I am not sure.
# iptables-save
# Generated by iptables-save v1.4.21 on Tue Apr 23 09:15:14 2019
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [2449555:327804572]
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A INPUT -p icmp -j ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -s 172.22.0.0/16 -p tcp -m tcp --dport 49152:49664 -m conntrack --ctstate NEW -j ACCEPT
-A INPUT -s 10.3.3.0/25 -p tcp -m tcp --dport 49152:49664 -m conntrack --ctstate NEW -j ACCEPT
-A INPUT -s 172.22.0.0/16 -p tcp -m tcp --dport 24007 -m conntrack --ctstate NEW -j ACCEPT
-A INPUT -s 10.3.3.0/25 -p tcp -m tcp --dport 24007 -m conntrack --ctstate NEW -j ACCEPT
-A INPUT -s 10.3.3.0/25 -p tcp -m tcp --dport 22 -m conntrack --ctstate NEW -j ACCEPT
-A INPUT -s 10.3.7.0/25 -p tcp -m tcp --dport 22 -m conntrack --ctstate NEW -j ACCEPT
-A INPUT -j REJECT --reject-with icmp-host-prohibited
-A FORWARD -j REJECT --reject-with icmp-host-prohibited
COMMIT
# Completed on Tue Apr 23 09:15:14 2019
From time to time, when this server, say 10.3.3.3, trying to connect to port 24007 of another server 10.3.3.4, it will fail sometimes. from tcpdump output, you can see these packages:
22:49:05.992737 IP 10.3.3.3.49149 > 10.3.3.4.24007: Flags [S], seq 2454712274, win 29200, options [mss 1460,sackOK,TS val 24055648 ecr 0,nop,wscale 7], length 0
22:49:05.992847 IP 10.3.3.4.24007 > 10.3.3.3.49149: Flags [S.], seq 3127562073, ack 2454712275, win 28960, options [mss 1460,sackOK,TS val 17803660 ecr 24055648,nop,wscale 7], length 0
22:49:05.992872 IP 10.3.3.3 > 10.3.3.4: ICMP host 10.3.3.3 unreachable - admin prohibited, length 68
So it looks like the packet getting rejected wasn't just a SYN packet but had something extra tagged NONE tagged in it [S.]. Do the packets which don't trigger an ICMP return contain a [SA] or a [S.] ?
My guess is that something in that return packet is making the cstate not be NEW or ESTABLISHED and so it is failing down to the drop. If the packets which are accepted look the same then I would look to see if the conntrack table is overflowing.. if the memory is tight and the traffic is high, streams will get pushed out and you will get similar failures. That however usally puts something in dmesg when it happens. Sorry I don't have a better answer and I am looking forward to someone correcting my mistakes here :).
The package back from 10.3.3.4 is prohibited by 10.3.3.3, why?
Obviously, the package that's been prohibited is the SYNC package that in response of the SYNC sent out by 10.3.3.3, it should match the iptables rule -m state --state RELATED,ESTABLISHED -j ACCEPT, but still be prohibited by the rule -j REJECT --reject-with icmp-host-prohibited.
Also, the iptables -nvL output:
# iptables -nvL
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
8258 615K ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED
1 84 ACCEPT icmp -- * * 0.0.0.0/00.0.0.0/0
0 0 ACCEPT all -- lo * 0.0.0.0/00.0.0.0/0
0 0 ACCEPT tcp -- * * 172.22.0.0/160.0.0.0/0 tcp dpts:49152:49664 ctstate NEW
0 0 ACCEPT tcp -- * * 10.3.3.0/250.0.0.0/0 tcp dpts:49152:49664 ctstate NEW
0 0 ACCEPT tcp -- * * 172.22.0.0/160.0.0.0/0 tcp dpt:24007 ctstate NEW
918 55080 ACCEPT tcp -- * * 10.3.3.0/25 0.0.0.0/0 tcp dpt:24007 ctstate NEW
1 60 ACCEPT tcp -- * * 10.3.3.0/250.0.0.0/0 tcp dpt:22 ctstate NEW
0 0 ACCEPT tcp -- * * 10.3.7.0/250.0.0.0/0 tcp dpt:22 ctstate NEW
244 14640 REJECT all -- * * 0.0.0.0/0 0.0.0.0/0 reject-with icmp-host-prohibited
Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
0 0 REJECT all -- * * 0.0.0.0/00.0.0.0/0 reject-with icmp-host-prohibited
Chain OUTPUT (policy ACCEPT 7378 packets, 957K bytes)
pkts bytes target prot opt in out source destination
I noticed that the 0.0.0.0/0 0.0.0.0/0 reject-with icmp-host-prohibited one increase fast.
Is this a bug of iptables, or is there something that I have not noticed?
Any hints will be appreciated.
Likun
CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos