[CentOS-mirror] HTTP logs of mirror server

Mon Oct 15 13:30:05 UTC 2012
Nick Olsen <nick at flhsi.com>

Looking at my log, 90% of the entries are what you're seeing. So that should be normal traffic. Even see them in IPv6.

72.43.14.218 - - [15/Oct/2012:08:57:29 -0400] "GET /centos/5.8/os/i386/repodata/repomd.xml HTTP/1.1" 200 1140 "-" "urlgrabber/3.1.0 yum/3.2.22"
216.52.195.141 - - [15/Oct/2012:08:57:29 -0400] "GET /centos/5.8/extras/i386/repodata/repomd.xml HTTP/1.1" 200 1953 "-" "urlgrabber/3.1.0 yum/3.2.22"
168.85.177.70 - - [15/Oct/2012:08:57:30 -0400] "GET /centos/5.7/os/x86_64/repodata/repomd.xml HTTP/1.1" 404 319 "-" "urlgrabber/3.1.0 yum/3.2.22"
216.59.32.140 - - [15/Oct/2012:08:57:30 -0400] "GET /centos/5.8/updates/x86_64/repodata/repomd.xml HTTP/1.1" 200 1953 "-" "urlgrabber/3.1.0 yum/3.2.22"
66.7.207.80 - - [15/Oct/2012:08:57:31 -0400] "GET /centos/5.8/extras/x86_64/repodata/repomd.xml HTTP/1.1" 200 1953 "-" "urlgrabber/3.1.0 yum/3.2.22"
2607:fe50:0:f201:21e:68ff:fe49:9d70 - - [15/Oct/2012:08:57:31 -0400] "GET /centos/5/extras/x86_64/repodata/repomd.xml HTTP/1.1" 200 1953 "-" "urlgrabber/3.1.0 yum/3.2.22"
2607:fe50:0:f201:21e:68ff:fe49:9d70 - - [15/Oct/2012:08:57:31 -0400] "GET /centos/5/updates/x86_64/repodata/repomd.xml HTTP/1.1" 200 1953 "-" "urlgrabber/3.1.0 yum/3.2.22"
66.232.36.2 - - [15/Oct/2012:08:57:32 -0400] "GET /centos/5.8/extras/i386/repodata/repomd.xml HTTP/1.1" 200 1953 "-" "urlgrabber/3.1.0 yum/3.2.22"

Nick Olsen
Network Operations (855) FLSPEED  x106

----------------------------------------
 From: fakrul at dhakacom.com
Sent: Sunday, October 14, 2012 9:18 AM
To: "Mailing list for CentOS mirrors." <centos-mirror at centos.org>
Subject: Re: [CentOS-mirror] HTTP logs of mirror server

Hi Brian,

Thanks for your response. I am glad that people are using my mirror.  But I am not sure whether these are legitimate request. All request is only fetching repomd.xml and urlgrabber attached to it from randomized IP. 

If it is normal behavior, than it is ok. I am also hosting other public mirror, but there is no such http request for those flavors.

Regards,
Fakrul  
Sent from my BlackBerry® smartphone - Airtel

-----Original Message-----
From: Brian Rak <brak at gameservers.com>
Date: Sat, 13 Oct 2012 17:56:14 
To: Mailing list for CentOS mirrors.<centos-mirror at centos.org>
Cc: Fakrul Alam<fakrul at dhakacom.com>
Subject: Re: [CentOS-mirror] HTTP logs of mirror server

If your mirror is public (and listed with CentOS), then you're going to 
see traffic from random hosts.. that's kind of the point of the mirror.

Did I misunderstand what you are asking?  It seems that you're confused 
that people are actually using your mirror.

On 10/13/2012 12:38 PM, Fakrul Alam wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Hi,
>
> In my CentOS mirror server I am getting huge HTTP request from random
> IP's. Bellow are few sample logs:
>
> 14.140.162.178 - - [13/Oct/2012:22:35:13 +0600] "GET
> /centos/5.8/os/x86_64/repodata/repomd.xml HTTP/1.1" 200 1140 "-"
> "urlgrabber/3.1.0 yum/3.2.19"
> 124.7.36.144 - - [13/Oct/2012:22:35:16 +0600] "GET
> /centos/5.8/os/i386/repodata/repomd.xml HTTP/1.1" 200 1140 "-"
> "urlgrabber/3.1.0 yum/3.2.22"
> 124.7.36.144 - - [13/Oct/2012:22:35:18 +0600] "GET
> /centos/5.8/extras/i386/repodata/repomd.xml HTTP/1.1" 200 1953 "-"
> "urlgrabber/3.1.0 yum/3.2.22"
> 122.166.231.199 - - [13/Oct/2012:22:35:21 +0600] "GET
> /centos/5.8/os/x86_64/repodata/repomd.xml HTTP/1.1" 200 1140 "-"
> "urlgrabber/3.1.0 yum/3.2.22"
> 27.251.30.86 - - [13/Oct/2012:22:35:21 +0600] "GET
> /centos/5.8/extras/x86_64/repodata/repomd.xml HTTP/1.1" 200 1953 "-"
> "urlgrabber/3.1.0 yum/3.2.22"
> 122.166.231.199 - - [13/Oct/2012:22:35:23 +0600] "GET
> /centos/5.8/extras/x86_64/repodata/repomd.xml HTTP/1.1" 200 1953 "-"
> "urlgrabber/3.1.0 yum/3.2.22"
> 117.58.244.90 - - [13/Oct/2012:22:35:25 +0600] "GET
> /centos/5.8/os/x86_64/repodata/repomd.xml HTTP/1.1" 200 1140 "-"
> "urlgrabber/3.1.0 yum/3.2.22"
> 117.58.244.90 - - [13/Oct/2012:22:35:25 +0600] "GET
> /centos/5.8/extras/x86_64/repodata/repomd.xml HTTP/1.1" 200 1953 "-"
> "urlgrabber/3.1.0 yum/3.2.22"
> 117.58.244.90 - - [13/Oct/2012:22:35:25 +0600] "GET
> /centos/5.8/updates/x86_64/repodata/repomd.xml HTTP/1.1" 200 1953 "-"
> "urlgrabber/3.1.0 yum/3.2.22"
> 180.179.39.9 - - [13/Oct/2012:22:35:35 +0600] "GET
> /centos/5.8/os/i386/repodata/repomd.xml HTTP/1.1" 200 1140 "-"
> "urlgrabber/3.1.0 yum/3.2.22"
> 115.112.108.130 - - [13/Oct/2012:22:35:38 +0600] "GET
> /centos/5.8/os/x86_64/repodata/repomd.xml HTTP/1.1" 200 1140 "-"
> "urlgrabber/3.1.0 yum/3.2.22"
> 115.112.108.130 - - [13/Oct/2012:22:35:38 +0600] "GET
> /centos/5.8/extras/x86_64/repodata/repomd.xml HTTP/1.1" 200 1953 "-"
> "urlgrabber/3.1.0 yum/3.2.22"
> 180.179.39.9 - - [13/Oct/2012:22:35:40 +0600] "GET
> /centos/5.8/extras/i386/repodata/repomd.xml HTTP/1.1" 200 1953 "-"
> "urlgrabber/3.1.0 yum/3.2.22"
> 180.179.39.9 - - [13/Oct/2012:22:35:42 +0600] "GET
> /centos/5.8/updates/i386/repodata/repomd.xml HTTP/1.1" 200 1953 "-"
> "urlgrabber/3.1.0 yum/3.2.22"
> 182.71.223.79 - - [13/Oct/2012:22:35:43 +0600] "GET
> /centos/5.8/os/i386/repodata/repomd.xml HTTP/1.1" 200 1140 "-"
> "urlgrabber/3.1.0 yum/3.2.22"
> 182.71.223.79 - - [13/Oct/2012:22:35:44 +0600] "GET
> /centos/5.8/extras/i386/repodata/repomd.xml HTTP/1.1" 200 1953 "-"
> "urlgrabber/3.1.0 yum/3.2.22"
>
> Is it normal phenomena or there is some thing wrong?
>
> Fakrul
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG/MacGPG2 v2.0.18 (Darwin)
> Comment: GPGTools - http://gpgtools.org
>
> iQEcBAEBAgAGBQJQeZkXAAoJEMrRrLU8Ar25vHwH/1Bwh7ZnNFaeADc8MkeyStRg
> 5OKz6DeTqRw3r9vUiQtU4qefjdZz7OJElhzIqaMhU5xBoWvyPzp3bsdEG5mBsD3s
> 9Oama6HSAJgt9luz6iR9Nq5P6AGKqRj0mOu/r+monlfV+rBDBEcY2JfyF11dmbJP
> JkqyQcmGKHUYjmr5CtYSr0p+oL/9q58vfh8aIGjf2+IPur/Bq6Rm7zQZf0zaqhIH
> lc3MX4abaB66jUFgMxaVLJUpaa6al+7uQfAm3BtmQ1dC9LEkm1z6r4mxLB77lsA3
> tqGL4itcLq+PyYkeRMIASkW8I+6MazAGxbXpAothlx//7GkUQfI0C5zRinbpl/Y=
> =lAVa
> -----END PGP SIGNATURE-----
> _______________________________________________
> CentOS-mirror mailing list
> CentOS-mirror at centos.org
> http://lists.centos.org/mailman/listinfo/centos-mirror

_______________________________________________
CentOS-mirror mailing list
CentOS-mirror at centos.org
http://lists.centos.org/mailman/listinfo/centos-mirror

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.centos.org/pipermail/centos-mirror/attachments/20121015/2067666f/attachment-0006.html>