I do indeed have stats @ http://stats.btnchicago.com/usage_201001.html
Most of the traffic came from Chinese addresses in the 114.249.219.0, 121.41.181.0, 221.0.0.0, 123.118.107.0 and 218.1.7.200.0 subnets. According to GeoIP these originate from Beijing, Zhejiang, Hubei, Beijing and Fujian respectively. Each downloaded approximately 23TB, 17TB, 10TB, 10TB, 10TB and exhibited similar repetitive patterns of the same file.
The stats analysis is via webalizer (however I also ran analog on the server logs and it confirms the amount of traffic is correct) On the 15th I began blocking several of the offending /8 subnets (drastic, I know) while we determined what to do next hence the deep drop off in hits. On the 16th I removed the blocks and tried qdisc throttling and bandwidth management tools via Apache which as you can see only moderately control the rate, not the requests. It isn’t so much the traffic that we're worried about now as much as the attention being drawn to our network. The upper management at FOX is not interested in being a target hence the request to terminate the mirror (which has been shut down and will remain down).
We are extremely grateful for all the time and effort put forth by all the CentOS volunteers and offer our sincerest apologies for backing out of the mirror hosting. It was a decision not made lightly.
Thank you all for the insight and assistance, We hope to find a way to give back to the CentOS project in some other way in the near future.
Cheers,
Scott Adametz Systems Engineer Big Ten Network 600 W Chicago Ave. Chicago, IL 60654 scott.adametz@bigtennetwork.com O:312.665.0787 C:708.214.4232
-----Original Message----- From: centos-mirror-bounces@centos.org [mailto:centos-mirror-bounces@centos.org] On Behalf Of Karanbir Singh Sent: Thursday, January 21, 2010 3:22 PM To: Mailing list for CentOS mirrors. Subject: Re: [CentOS-mirror] Please remove from mirrorlist: mirrors.bigtennetwork.com/CentOS
On 21/01/10 20:06, Scott Adametz wrote:
Due to an inordinate amount of Chinese based traffic from only a handful of IP addresses (over 145 TB transferred to just *12* IP addresses in only 15 days since we started hosting the mirror) we are forced to cease our participation in the CentOS mirror project.
Do you have some details on what this traffic is ? as in details to the number of times they hit your mirror, and what url they were getting from there ?