On Thu, 21 Jan 2010, Scott Adametz wrote: > Most of the traffic came from Chinese addresses in the 114.249.219.0, > 121.41.181.0, 221.0.0.0, 123.118.107.0 and 218.1.7.200.0 subnets. > According to GeoIP these originate from Beijing, Zhejiang, Hubei, > Beijing and Fujian respectively. Each downloaded approximately 23TB, > 17TB, 10TB, 10TB, 10TB and exhibited similar repetitive patterns of the > same file. We had a similar issue at the centos (and other stuff) mirror at ftp.iitm.ac.in some months ago. We have solved it effectively using per ip connection limit and fail2ban. It appears that the traffic originates via a download accerlator that is popular in china. We used to get the similar thousands of ranged requests for the iso image files of centos and other linux distributions. We have put a per-ip connection limit of 5 using the limitipconn module. Connection attempts over 5 get logged in the apache error log. fail2ban package is used to monitor this log file; when any single ip generates more than 5 error message in a minute (meaning that ip has tried to open more than 5 connections more than 5 times in a minute), the fail2ban package inserts an iptables firewall rule that blocks ALL connection requests from this IP for the next one hour. After a few minutes, the 5 existing (ranged download request) connections complete their download and the offending IP is locked out for the rest of the hour. Works very very effectively. We saw our hit rate drop from about 700,000 per day to below 100,000 per day. We continue to server the centos (and other) mirror community. Scott, I would urge you to seriously consider this type of solution instead of dropping out of the mirror network. I will be happy to provide any further assistance in this regard. -- sriram