I've been trying to do traffic shaping on one of my public servers and after reading up, it seems like the way to do so is via tc/htb. However, most of the documentation seems at least half a decade old with nothing new recently.
Furthermore, trying to get documentation on tc filters turned up a blank. man tc refers to a tc-filters (8) but trying to man that gives a no such page/section error. Googling on this seems to imply that the documentation was never created. The author also seems to have stop updating his blog/company site since 2007 based on the last login date on netherlabs.nl
So I'm wondering is tc the current and recommended method for traffic shaping on CentOS or is there some newer method that has superceded it?
On 09/09/10 8:51 PM, Emmanuel Noobadmin wrote:
So I'm wondering is tc the current and recommended method for traffic shaping on CentOS or is there some newer method that has superceded it?
welcome to the truly absymal state of linux documentation. the TLDP site should be slammed off the net entirely and nuked, 12 year old HOWTO's talking about patches to 2.3.test aren't doing anyone any favors.
put a pfsense based router on your network border, and use that to do the shaping :deal:
On 9/10/10, John R Pierce pierce@hogranch.com wrote:
On 09/09/10 8:51 PM, Emmanuel Noobadmin wrote:
So I'm wondering is tc the current and recommended method for traffic shaping on CentOS or is there some newer method that has superceded it?
welcome to the truly absymal state of linux documentation. the TLDP site should be slammed off the net entirely and nuked, 12 year old HOWTO's talking about patches to 2.3.test aren't doing anyone any favors.
To be fair, what's on it still works, at least as far as I could tell. Just that I get quite uncomfortable not knowing what/why am I seeing errors when I use tc to display the qdisc stats and I don't know if just doing a filter match on dport 80 is enough or should I use the often found method of marking in iptables and filtering on the mark.
put a pfsense based router on your network border, and use that to do the shaping :deal:
Actually using pfsense was my original intention after reading up on some past discussion here. Except I didn't realize I couldn't run pfsense on top of a normal CentOS distribution and I don't have the option of putting another machine in that particular setup for a dedicated router/firewall box.
On Fri, September 10, 2010 13:20, Emmanuel Noobadmin wrote:
On 9/10/10, John R Pierce pierce@hogranch.com wrote:
On 09/09/10 8:51 PM, Emmanuel Noobadmin wrote:
So I'm wondering is tc the current and recommended method for traffic shaping on CentOS or is there some newer method that has superceded it?
welcome to the truly absymal state of linux documentation. the TLDP site should be slammed off the net entirely and nuked, 12 year old HOWTO's talking about patches to 2.3.test aren't doing anyone any favors.
To be fair, what's on it still works, at least as far as I could tell. Just that I get quite uncomfortable not knowing what/why am I seeing errors when I use tc to display the qdisc stats and I don't know if just doing a filter match on dport 80 is enough or should I use the often found method of marking in iptables and filtering on the mark.
put a pfsense based router on your network border, and use that to do the shaping :deal:
Actually using pfsense was my original intention after reading up on some past discussion here. Except I didn't realize I couldn't run pfsense on top of a normal CentOS distribution and I don't have the option of putting another machine in that particular setup for a dedicated router/firewall box. _______________________________________________
Note that you will only be able to control the flow of outgoing traffic to your system if you place the bandwidth control on the server endpoint. Incoming traffic needs an in-line box to so that you can access the other interface and control it's outgoing traffic (your servers incoming traffic).
On 9/10/10, Giles Coochey giles@coochey.net wrote:
Note that you will only be able to control the flow of outgoing traffic to your system if you place the bandwidth control on the server endpoint. Incoming traffic needs an in-line box to so that you can access the other interface and control it's outgoing traffic (your servers incoming traffic).
I understand that problem from the reading done so far. Fortunately in this particular situation, all I really need is the outgoing traffic shaping since the ISP would be limiting what's incoming anyway. The client just want to make sure certain key interactive services doesn't get drowned out when somebody is pulling a huge document from the server.
-----Original Message----- From: centos-bounces@centos.org [mailto:centos-bounces@centos.org] On Behalf Of Emmanuel Noobadmin Sent: Friday, September 10, 2010 11:16 AM To: CentOS mailing list Subject: Re: [CentOS] Traffic shaping on CentOS
On 9/10/10, Giles Coochey giles@coochey.net wrote:
Note that you will only be able to control the flow of outgoing
traffic to
your system if you place the bandwidth control on the server
endpoint.
Incoming traffic needs an in-line box to so that you can access the
other
interface and control it's outgoing traffic (your servers incoming traffic).
I understand that problem from the reading done so far. Fortunately in this particular situation, all I really need is the outgoing traffic shaping since the ISP would be limiting what's incoming anyway. The client just want to make sure certain key interactive services doesn't get drowned out when somebody is pulling a huge document from the server.
You may still be hosed since the bottleneck is in front of your server.
New client requests -> InternetConnection -> Router/FW -> Server
If your new client requests are coming into an internet connection that's saturated, I'm pretty sure they won't even make it to the server to get rate limited. Your client would start seeing error rates/retransmits and you'd be effectively DOS'd. If you were running with an ISP that let you burst, then used a Router/FW that let you start throttling traffic you may do better, but I don't think you're going to get good results out of that system.
Any reason you don't buy a hosted solution and put your static content (manuals, long downloads, etc) up there for people to pull? You could also get pay as you go caching thru a few limelight/level3/akami/etc for your domain.
In the past I've used tc to do testing for crappy network links. Here are the two links that I found helpful * http://www.linuxfoundation.org/collaborate/workgroups/networking/netem * http://lartc.org/howto/lartc.ratelimit.single.html
Good luck, Patrick
You may still be hosed since the bottleneck is in front of your server. New client requests -> InternetConnection -> Router/FW -> Server
If your new client requests are coming into an internet connection that's saturated, I'm pretty sure they won't even make it to the server to get rate limited. Your client would start seeing error rates/retransmits and you'd be effectively DOS'd. If you were running with an ISP that let you burst, then used a Router/FW that let you start throttling traffic you may do better, but I don't think you're going to get good results out of that system.
True, but based on the traffic graphs so far, the uplink is usually saturated first during normal usage. If for some reason the incoming requests are coming so fast as to saturate the downlink, I don't think having another inline server/router would help, would it? This is because it seems to me that since the ISP's the one controlling that end of the traffic and if they don't shape/police, whatever they let in would be choking out the legit users anyway regardless of what I have waiting to receive the packets.
Any reason you don't buy a hosted solution and put your static content (manuals, long downloads, etc) up there for people to pull? You could also get pay as you go caching thru a few limelight/level3/akami/etc for your domain.
Unfortunately, they are not static content. Their webapp generates certain documents, then generate links to their clients to download. Some of these contain quite a number of images so the size can add up quite quickly even at 50kb~100kb per pic. So when they get a few clients doing this, they basically get snail paced access to the webapp.
In the past I've used tc to do testing for crappy network links. Here are the two links that I found helpful
Thanks for the links :)
I've been trying to do traffic shaping on one of my public servers and after reading up, it seems like the way to do so is via tc/htb. However, most of the documentation seems at least half a decade old with nothing new recently.
Furthermore, trying to get documentation on tc filters turned up a blank. man tc refers to a tc-filters (8) but trying to man that gives a no such page/section error. Googling on this seems to imply that the documentation was never created. The author also seems to have stop updating his blog/company site since 2007 based on the last login date on netherlabs.nl
So I'm wondering is tc the current and recommended method for traffic shaping on CentOS or is there some newer method that has superceded it?
Yes, it still work pretty on CentOS. I don't remember what documentation I used when I set up traffic shaping on my server, but there was no problems. So I think you can try to use the documentation that you have found.
2010/9/10 Mintairov Mikhail mikxalich@yandex.ru:
I've been trying to do traffic shaping on one of my public servers and after reading up, it seems like the way to do so is via tc/htb. However, most of the documentation seems at least half a decade old with nothing new recently.
Furthermore, trying to get documentation on tc filters turned up a blank. man tc refers to a tc-filters (8) but trying to man that gives a no such page/section error. Googling on this seems to imply that the documentation was never created. The author also seems to have stop updating his blog/company site since 2007 based on the last login date on netherlabs.nl
So I'm wondering is tc the current and recommended method for traffic shaping on CentOS or is there some newer method that has superceded it?
Yes, it still work pretty on CentOS. I don't remember what documentation I used when I set up traffic shaping on my server, but there was no problems. So I think you can try to use the documentation that you have found.
maybe this script/documentation helps:
http://lartc.org/wondershaper/
br, -- Eero, RHCE
On Fri, September 10, 2010 05:51, Emmanuel Noobadmin wrote:
I've been trying to do traffic shaping on one of my public servers and after reading up, it seems like the way to do so is via tc/htb. However, most of the documentation seems at least half a decade old with nothing new recently.
Furthermore, trying to get documentation on tc filters turned up a blank. man tc refers to a tc-filters (8) but trying to man that gives a no such page/section error. Googling on this seems to imply that the documentation was never created. The author also seems to have stop updating his blog/company site since 2007 based on the last login date on netherlabs.nl
So I'm wondering is tc the current and recommended method for traffic shaping on CentOS or is there some newer method that has superceded it? _______________________________________________
I bought this book a long time ago http://www.amazon.com/Policy-Routing-Using-Linux-Professional/dp/0672320525
I have to say it helped me a lot - even if it is quite antiquated.
Unfortunately, online / up to date information and documentation for Linux in general is pretty scarce, particularly distribution-centric documentation. The reason for this is that developers tend to hate documentation and Linux itself is a moving target - documentation needs constant updating.
I wonder how many people would pay for good documentation?
From: Emmanuel Noobadmin centos.admin@gmail.com
So I'm wondering is tc the current and recommended method for traffic shaping on CentOS or is there some newer method that has superceded it?
Once I tried http://sourceforge.net/projects/cbqinit/ and it made it very easy...
JD
On 9/10/10, John Doe jdmls@yahoo.com wrote:
From: Emmanuel Noobadmin centos.admin@gmail.com
So I'm wondering is tc the current and recommended method for traffic shaping on CentOS or is there some newer method that has superceded it?
Once I tried http://sourceforge.net/projects/cbqinit/ and it made it very easy...
The oddity here is from my reading so far, CBQ is an older queue discipline compared to HTB and importantly having more archaic syntax. Or am I mistaken?
From: Emmanuel Noobadmin centos.admin@gmail.com
The oddity here is from my reading so far, CBQ is an older queue discipline compared to HTB and importantly having more archaic syntax. Or am I mistaken?
http://luxik.cdi.cz/~devik/qos/htb/old/htbmeas1.htm
JD
On 9/10/10, John Doe jdmls@yahoo.com wrote:
From: Emmanuel Noobadmin centos.admin@gmail.com
The oddity here is from my reading so far, CBQ is an older queue discipline compared to HTB and importantly having more archaic syntax. Or am I mistaken?
Just to confirm we're understanding the same thing. My reading of that link is that HTB is a newer qdisc and offers more precise control over bandwidth, the drawback being slightly more computationally intensive and therefore higher chances of delay compared to CBQ.
Digging around, a newer page dated 2002 http://luxik.cdi.cz/~devik/qos/htb/htb3perf/cbqhtb3perf.htm seems to conclude similarly except it provides more precise data which I understand to mean that the "new" HTB implementation performs better with lesser (<400) active classes while CBQ does better with more active classes.
Based on that information, in my situation where bandwidth control is a primary consideration (client has a limited budget for bandwidth but wants certain services prioritized) and the system in question has processing capacity to spare, HTB would appear be the better solution since I only have a few active class.
Please correct me if I'm reading those pages and data wrongly.
Emmanuel Noobadmin wrote:
I've been trying to do traffic shaping on one of my public servers and after reading up, it seems like the way to do so is via tc/htb. However, most of the documentation seems at least half a decade old with nothing new recently.
Furthermore, trying to get documentation on tc filters turned up a blank. man tc refers to a tc-filters (8) but trying to man that gives a no such page/section error. Googling on this seems to imply that the documentation was never created. The author also seems to have stop updating his blog/company site since 2007 based on the last login date on netherlabs.nl
So I'm wondering is tc the current and recommended method for traffic shaping on CentOS or is there some newer method that has superceded it?
I use DummyNet[1] for any traffic shaping and bottleneck testing I need. It is distributed as part of the FreeBSD system, although you may need to recompiled the kernel to enable it.
Bob McConnell N2SPP
-
Bob McConnell -
Eero Volotinen -
Emmanuel Noobadmin -
Flaherty, Patrick -
Giles Coochey -
John Doe -
John R Pierce -
Mintairov Mikhail