Are there any known speed issues with us-msync.centos.org? We've been noticing issues with syncs taking longer than normal and causing issues with our round-robin while the sync is still running on our primary mirror.
Thanks!
On 28/07/2020 23:54, Lance Albertson wrote:
Are there any known speed issues with us-msync.centos.org http://us-msync.centos.org? We've been noticing issues with syncs taking longer than normal and causing issues with our round-robin while the sync is still running on our primary mirror.
Thanks!
Hi Lance,
No particular "known" issue, but because there are more and more mirrors pulling from us, and that it still runs from donated machines (so even just connected at 100Mbits in the US, and even *less* in some regions like Asia), it depends on which node you land, how many updates you're pulling through rsync and also people hitting it through httpd too.
It looks like the offending slow server is 64.150.179.24. Do you notice any issues with that particular node? This only seems to happen sometimes.
On Fri, Aug 7, 2020 at 6:25 AM Fabian Arrotin arrfab@centos.org wrote:
On 28/07/2020 23:54, Lance Albertson wrote:
Are there any known speed issues with us-msync.centos.org http://us-msync.centos.org? We've been noticing issues with syncs taking longer than normal and causing issues with our round-robin while the sync is still running on our primary mirror.
Thanks!
Hi Lance,
No particular "known" issue, but because there are more and more mirrors pulling from us, and that it still runs from donated machines (so even just connected at 100Mbits in the US, and even *less* in some regions like Asia), it depends on which node you land, how many updates you're pulling through rsync and also people hitting it through httpd too.
-- Fabian Arrotin The CentOS Project | https://www.centos.org gpg key: 17F3B7A1 | twitter: @arrfab
CentOS-mirror mailing list CentOS-mirror@centos.org https://lists.centos.org/mailman/listinfo/centos-mirror
This problem i still happening to this IP address and causing issues to our downstream users since our sync is taking so long to complete. Is there any update on this?
Thanks!
On Fri, Aug 7, 2020 at 1:31 PM Lance Albertson lance@osuosl.org wrote:
It looks like the offending slow server is 64.150.179.24. Do you notice any issues with that particular node? This only seems to happen sometimes.
On Fri, Aug 7, 2020 at 6:25 AM Fabian Arrotin arrfab@centos.org wrote:
On 28/07/2020 23:54, Lance Albertson wrote:
Are there any known speed issues with us-msync.centos.org http://us-msync.centos.org? We've been noticing issues with syncs taking longer than normal and causing issues with our round-robin while the sync is still running on our primary mirror.
Thanks!
Hi Lance,
No particular "known" issue, but because there are more and more mirrors pulling from us, and that it still runs from donated machines (so even just connected at 100Mbits in the US, and even *less* in some regions like Asia), it depends on which node you land, how many updates you're pulling through rsync and also people hitting it through httpd too.
-- Fabian Arrotin The CentOS Project | https://www.centos.org gpg key: 17F3B7A1 | twitter: @arrfab
CentOS-mirror mailing list CentOS-mirror@centos.org https://lists.centos.org/mailman/listinfo/centos-mirror
-- Lance Albertson Director Oregon State University | Open Source Lab
On 13/08/2020 18:34, Lance Albertson wrote:
This problem i still happening to this IP address and causing issues to our downstream users since our sync is taking so long to complete. Is there any update on this?
Thanks!
Hi Lance,
I had a look at our graphs for that node and on average on the last days it's pushing out at ~66Mbit/s with peaks to 585Mbits/s , so even connected over gbit these days.
Wondering if that's a routing/peering issue from OSUOSL but I can confirm that it's happening from some locations in the US that I tested : seems capp'ed around ~500kbit/s ... while from another server in the US, I could get the following stats for the same file :
100%[=============================================================================>] 558,002,176 50.3MB/s in 11s
So clearly far better. :-)
That's what I think that it's probably some peering issue. I currently don't have time to investigate that , but if that impacts you at that level (also bad luck to fall on that particular node while there are other in the same pool), can you investigate at your side with your provider, which routes, etc .. maybe that would help ?
PS: for the time being, I've just removed it from the msync.centos.org DNS pool, but node is still online and up2date and can be used for some tests.
Here is the file I tested : http://centosv4.centos.org/centos/8/BaseOS/x86_64/os/images/install.img
On Fri, Aug 14, 2020 at 2:23 AM Fabian Arrotin arrfab@centos.org wrote:
On 13/08/2020 18:34, Lance Albertson wrote:
This problem i still happening to this IP address and causing issues to our downstream users since our sync is taking so long to complete. Is there any update on this?
Thanks!
Hi Lance,
I had a look at our graphs for that node and on average on the last days it's pushing out at ~66Mbit/s with peaks to 585Mbits/s , so even connected over gbit these days.
Wondering if that's a routing/peering issue from OSUOSL but I can confirm that it's happening from some locations in the US that I tested : seems capp'ed around ~500kbit/s ... while from another server in the US, I could get the following stats for the same file :
100%[=============================================================================>] 558,002,176 50.3MB/s in 11s
So clearly far better. :-)
It actually seems OK right now doing the test myself: 100%[=============================================================================>] 558,002,176 31.8MB/s in 18s
That's what I think that it's probably some peering issue. I currently don't have time to investigate that , but if that impacts you at that level (also bad luck to fall on that particular node while there are other in the same pool), can you investigate at your side with your provider, which routes, etc .. maybe that would help ?
Yeah, let me consult with our provider and see if they notice anything. We've had problems with peering before impacting performance so it might be that.
PS: for the time being, I've just removed it from the msync.centos.org
DNS pool, but node is still online and up2date and can be used for some tests.
Thank you for doing that! I'll let you know what we find out.
Here is the file I tested : http://centosv4.centos.org/centos/8/BaseOS/x86_64/os/images/install.img
-- Fabian Arrotin The CentOS Project | https://www.centos.org gpg key: 17F3B7A1 | twitter: @arrfab
CentOS-mirror mailing list CentOS-mirror@centos.org https://lists.centos.org/mailman/listinfo/centos-mirror
-
Fabian Arrotin -
Lance Albertson