Theo Morra kirjoitti 1.12.2018 klo 9.03:
Heya,
Does anyone know the useragent(s) for the crawlers?
We're using Mirrorbits internally but it's causing us to look like we're down on mirror-status.c.o so I was going to selectively proxy for just the crawlers
I'm not sure if that is the correct approach. Your mirror should be available for all user agents.
Now that I tested your mirror, I got
502 Bad Gateway nginx/1.12.2
on Chrome, and the following on a CentOS 6 host that I happened to have available:
$ wget http://centos.mirrors.theom.nz/timestamp.txt --2018-12-01 07:50:08-- http://centos.mirrors.theom.nz/timestamp.txt Resolving centos.mirrors.theom.nz... 203.109.248.50 Connecting to centos.mirrors.theom.nz|203.109.248.50|:80... connected. HTTP request sent, awaiting response... 502 Bad Gateway 2018-12-01 07:50:11 ERROR 502: Bad Gateway.
$ curl http://centos.mirrors.theom.nz/timestamp.txt <html> <head><title>502 Bad Gateway</title></head> <body bgcolor="white"> <center><h1>502 Bad Gateway</h1></center> <hr><center>nginx/1.12.2</center> </body> </html>
I'd say that you should fix that first. Such failures will also affect the mirror-status.c.o status for your mirror.