[CentOS-mirror] Chinese addresses requesting excessive iso's?

Thu Apr 28 10:21:20 UTC 2022
Alex Iribarren <alex.m.lists3 at gmail.com>

Hi,

We've seen this sort of traffic as well for quite a while now. We didn't 
want to block just by user-agent, but we did notice that all the 
requests were for a random Range, not for the entire iso.

We drop the traffic completely with the following ModSecurity rule:

SecRule REQUEST_BASENAME \.iso$ "id:1,phase:1,chain,drop,msg:'Drop weird 
Chinese traffic'"
SecRule REQUEST_HEADERS_NAMES Range "chain"
SecRule REQUEST_HEADERS:User-Agent "^Mozilla/"


This behavior is pretty weird, I don't understand why they would be 
doing it. I don't think it's Stephen's first theory because the 
downloads continue for the entire ISO if ignore the Range server-side, 
so there is no firewall cutting off the connection in the middle. The 
second theory doesn't make sense either, if your goal is to saturate the 
network connection, why download just a small part of the ISO instead of 
the full thing over and over again? The clients are also only active for 
for 20.5 hours of the day, they basically stop between 11:30 GMT and 
15:00 GMT. Bots need to rest too, I guess...

Cheers,
Alex

On 4/28/22 06:40, Russell Jones wrote:
>     This is an old problem, I have already re-posted the solution once -
>     the original author was the TUNA Mirror Team.
>     https://lists.centos.org/pipermail/centos-mirror/2020-October/024445.html
>     <https://lists.centos.org/pipermail/centos-mirror/2020-October/024445.html>Maybe
>     it would be a good idea to add this info to the CentOS wiki
>     https://wiki.centos.org/HowTos/CreatePublicMirrors
>     <https://wiki.centos.org/HowTos/CreatePublicMirrors> , so it
>     wouldn't be "loop" asked again.
>     By the way, if a mirror/firewall can't handle a few 403 requests
>     from a few hosts then it's really a big problem. ;)
> 
> 
> Thanks for the info. Blocking China solved the problem for me. The 403's 
> that are now being generated from me blocking China wasn't the issue - 
> Having 50+ hosts all requesting 8GB iso files over and over again was 
> the issue. ;)
> 
> On Wed, Apr 27, 2022 at 3:17 PM Quantum Mirror <root at quantum-mirror.hu 
> <mailto:root at quantum-mirror.hu>> wrote:
> 
>     This is an old problem, I have already re-posted the solution once -
>     the original author was the TUNA Mirror Team.
> 
>     https://lists.centos.org/pipermail/centos-mirror/2020-October/024445.html
>     <https://lists.centos.org/pipermail/centos-mirror/2020-October/024445.html>
> 
>     Maybe it would be a good idea to add this info to the CentOS wiki
>     https://wiki.centos.org/HowTos/CreatePublicMirrors
>     <https://wiki.centos.org/HowTos/CreatePublicMirrors> , so it
>     wouldn't be "loop" asked again.
> 
>     By the way, if a mirror/firewall can't handle a few 403 requests
>     from a few hosts then it's really a big problem. ;)
> 
>     Have a nice day!
> 
> 
>     Cheers,
> 
>     Peter
> 
> 
>     On 2022. 04. 27. 20:55, Paul Mezzanini wrote:
>>     We've been noticing the exact same behaviour and are still
>>     discussing internally the best way to address it.
>>
>>     On Wed, Apr 27, 2022 at 2:28 PM Stephen Smoogen
>>     <ssmoogen at redhat.com <mailto:ssmoogen at redhat.com>> wrote:
>>
>>
>>
>>         On Wed, 27 Apr 2022 at 14:16, Russell Jones
>>         <arjones85 at gmail.com <mailto:arjones85 at gmail.com>> wrote:
>>
>>             So, for whatever reason my mirror seems to be getting
>>             targeted by China:
>>
>>             [root at repos ~]# tail -f access.log | grep 403
>>             112.22.135.89 - - [27/Apr/2022:13:10:52 -0500] "GET
>>             /centos/7.9.2009/isos/x86_64/CentOS-7-x86_64-Everything-2009.iso
>>             HTTP/1.1" 403 153 "-" "curl/7.29.0"
>>
>>
>>         <deleted>
>>
>>             I geoblocked the country about a week ago, but the
>>             requests haven't stopped. It was at the level that it was
>>             maxing out my 1gbit/sec link until I did something.
>>
>>             Anyone else seeing anything similar?
>>
>>
>>         I have seen this going for about 10 years with different
>>         mirrors. The connections are one of three things:
>>         1. Automated downloaders getting blocked by Great-Firewall
>>         configurations getting to a certain point
>>         2. Malware installed on a lot of systems being commanded to
>>         download the software and desist. This is usually done to
>>         cause bandwidth issues all through the stack. They are either
>>         getting stopped by firewalls or just stopping the connections
>>         themselves as part of the badness.
>>
>>         From mirror managing Fedora, number 2 seems to be more likely
>>         as a lot of the IP addresses doing this never show up on
>>         asking mirrormanager for downloads. Instead they seem to have
>>         gotten a list of mirrors from some third party and are being
>>         commanded to do the infinite downloads. I don't know if this
>>         is similar with what is going on now.
>>
>>
>>
>>             _______________________________________________
>>             CentOS-mirror mailing list
>>             CentOS-mirror at centos.org <mailto:CentOS-mirror at centos.org>
>>             https://lists.centos.org/mailman/listinfo/centos-mirror
>>             <https://lists.centos.org/mailman/listinfo/centos-mirror>
>>
>>
>>
>>         -- 
>>         Stephen Smoogen, Red Hat Automotive
>>         Let us be kind to one another, for most of us are fighting a
>>         hard battle. -- Ian MacClaren
>>         _______________________________________________
>>         CentOS-mirror mailing list
>>         CentOS-mirror at centos.org <mailto:CentOS-mirror at centos.org>
>>         https://lists.centos.org/mailman/listinfo/centos-mirror
>>         <https://lists.centos.org/mailman/listinfo/centos-mirror>
>>
>>
>>     _______________________________________________
>>     CentOS-mirror mailing list
>>     CentOS-mirror at centos.org  <mailto:CentOS-mirror at centos.org>
>>     https://lists.centos.org/mailman/listinfo/centos-mirror  <https://lists.centos.org/mailman/listinfo/centos-mirror>
>     _______________________________________________
>     CentOS-mirror mailing list
>     CentOS-mirror at centos.org <mailto:CentOS-mirror at centos.org>
>     https://lists.centos.org/mailman/listinfo/centos-mirror
>     <https://lists.centos.org/mailman/listinfo/centos-mirror>
> 
> 
> _______________________________________________
> CentOS-mirror mailing list
> CentOS-mirror at centos.org
> https://lists.centos.org/mailman/listinfo/centos-mirror