I've got small numbers of connections moving through a load balancer configured in NAT mode. So I've got an iptables table called "nat", which has in it a line "-A POSTROUTING -o eth0 -j MASQUERADE" (lan connect is eth0, private lan inside the cluster is eth1).
The load balancer is working; connections made to the virtual ip on that host do get routed to one of the real servers behind this load load balancer.
But I want to observe the connections on the load balancer.
My first attempt was to use netstat with the --masquerade switch. This produced the result "netstat: no support for `ip_masquerade' on this system." Consistent with this, there is no /proc/net/ip_masquerade.
On the other hand, the load balancer *IS* working; those connections *are* getting NATted and routed.
Also, lsmod shows varous relevant modules loaded: iptable_nat 40773 1 ip_nat 53101 2 ipt_MASQUERADE,iptable_nat ip_conntrack 91237 5 xt_state,ip_conntrack_netbios_ns,ipt_MASQUERADE,iptable_nat,ip_nat nfnetlink 40457 2 ip_nat,ip_conntrack ip_tables 55329 2 iptable_filter,iptable_nat x_tables 50377 7 xt_state,ipt_REJECT,xt_tcpudp,ipt_MASQUERADE,xt_multiport,iptable_nat,ip_tables
So, netstat just isn't somehow the right monitoring tool, right? So what is the right monitoring tool? I need to know the source IP and real-server IP of connections being handled by the load balancer. I don't need a lot showing exactly how each one was handled, but I'd like to be able to determine the state of any connection currently active. How can I do this?
David Dyer-Bennet wrote:
So, netstat just isn't somehow the right monitoring tool, right? So what is the right monitoring tool? I need to know the source IP and
Shot in the dark since I've never used LVS but perhaps /proc/net/ip_conntrack
If that is right then there is a program called netstat-nat that is out there, not sure if there is a ready-made package for CentOS or if it's included by default but here is the debian version(source code on the right)
http://packages.debian.org/lenny/netstat-nat
I statically compiled it for a ipcop firewall recently and it worked pretty well.
nate
Quoting David Dyer-Bennet dd-b@dd-b.net:
I've got small numbers of connections moving through a load balancer configured in NAT mode. So I've got an iptables table called "nat", which has in it a line "-A POSTROUTING -o eth0 -j MASQUERADE" (lan connect is eth0, private lan inside the cluster is eth1).
The load balancer is working; connections made to the virtual ip on that host do get routed to one of the real servers behind this load load balancer.
But I want to observe the connections on the load balancer.
My first attempt was to use netstat with the --masquerade switch. This produced the result "netstat: no support for `ip_masquerade' on this system." Consistent with this, there is no /proc/net/ip_masquerade.
On the other hand, the load balancer *IS* working; those connections *are* getting NATted and routed.
Also, lsmod shows varous relevant modules loaded: iptable_nat 40773 1 ip_nat 53101 2 ipt_MASQUERADE,iptable_nat ip_conntrack 91237 5 xt_state,ip_conntrack_netbios_ns,ipt_MASQUERADE,iptable_nat,ip_nat nfnetlink 40457 2 ip_nat,ip_conntrack ip_tables 55329 2 iptable_filter,iptable_nat x_tables 50377 7
xt_state,ipt_REJECT,xt_tcpudp,ipt_MASQUERADE,xt_multiport,iptable_nat,ip_tables
So, netstat just isn't somehow the right monitoring tool, right? So what is the right monitoring tool? I need to know the source IP and real-server IP of connections being handled by the load balancer. I don't need a lot showing exactly how each one was handled, but I'd like to be able to determine the state of any connection currently active. How can I do this?
ipvsadm -L -c -n should do the trick. Also, you shouldn't need that MASQ rule unless you need to MASQ traffic originating from inside your private network. LVS handles all LVS related NATing.
Be careful .. you must use the lower case 'c' in this command as the uppercase 'C' will CLEAR your ipvs table and break things.
Hope this helps.
Barry
Barry Brimer wrote:
Quoting David Dyer-Bennet dd-b@dd-b.net:
But I want to observe the connections on the load balancer.
ipvsadm -L -c -n should do the trick. Also, you shouldn't need that MASQ rule unless you need to MASQ traffic originating from inside your private network. LVS handles all LVS related NATing.
Ah, yes, ipvsadm, had forgotten that, or I'm sure the man page would have given me the rest (downside of using web-based config, I don't learn the local tools as well). I do need to MASQ traffic originating in the private network, the services running there have to connect out to get to the database, and since the default route on those boxes points to the load-balancer to make LVS work.
Be careful .. you must use the lower case 'c' in this command as the uppercase 'C' will CLEAR your ipvs table and break things.
That'd be exciting :-).
On Wed, March 25, 2009 17:40, Barry Brimer wrote:
ipvsadm -L -c -n should do the trick.
Just following up, now that I'm back at work and have tried it. Yep, excellent. Using that with "watch" gives me a nice display.
(I'm load-balancing a rather small load of rather compute-heavy web services across a small cluster, so a connection persists for long enough to be noticed on the screen often, and there are few enough of them to keep track of. This lets me observe directly (and hence resolve other people's doubts) that the load really is being spread across the cluster for example. Monitoring directly on each server is harder, plus they're running Windows now so they're harder to monitor remotely.)
Thanks again!
-
Barry Brimer -
David Dyer-Bennet -
nate