On Tue, Apr 02, 2024 at 05:08:33PM -0700, Kevin Fenzi wrote:
In general clients should pull the repomd.xml file and check if it's changed, if not, use their cached versions.
Does this also apply to this file? /fedora-epel/7/x86_64/repodata/f25bccaa54b08e731162aec431dd82887a5ed1abce8eb6f892ad41454f07e1f3-primary.sqlite.bz2
I have five hosts which each downloaded 4 GByte of this file in the span of 24 hours. The delay between two downloads varies, from a few seconds to a few minutes.
On Wed, 3 Apr 2024 at 04:33, Carsten Otto otto@informatik.rwth-aachen.de wrote:
On Tue, Apr 02, 2024 at 05:08:33PM -0700, Kevin Fenzi wrote:
In general clients should pull the repomd.xml file and check if it's changed, if not, use their cached versions.
Does this also apply to this file?
/fedora-epel/7/x86_64/repodata/f25bccaa54b08e731162aec431dd82887a5ed1abce8eb6f892ad41454f07e1f3-primary.sqlite.bz2
I have five hosts which each downloaded 4 GByte of this file in the span of 24 hours. The delay between two downloads varies, from a few seconds to a few minutes.
Can you give the full httpd string (remove the ip address). This might help figure out if it is a specific version of a tool which is the problem or if it is a generic issue.
-- Dr. Carsten Otto http://verify.rwth-aachen.de/otto/ _______________________________________________ CentOS-mirror mailing list CentOS-mirror@centos.org https://lists.centos.org/mailman/listinfo/centos-mirror
Dear Mailing list for CentOS mirrors. , Thank you for your Exascale NOC Email enquiry. This is an auto-generated message as the result of a newly created ticket regarding "Re: [CentOS-mirror] Abusive repodata downloads" There is no need to reply to this message right now, a ticket has automatically been created with Ref # of: 2695 A member of the Exascale NOC team will review your request and reply via Email within 24 business hours. Thank you! To view the status of your ticket or add comments at any time, please visit: https://helpdesk.exascale.co.uk/helpdesk/tickets/2695 Best Regards, Exascale NOC Team
On Wed, Apr 03, 2024 at 08:38:17AM -0400, Stephen Smoogen wrote:
Can you give the full httpd string (remove the ip address). This might help figure out if it is a specific version of a tool which is the problem or if it is a generic issue.
I don't log user agents, sorry. I could temporarily enable that if helps?
On Wed, 3 Apr 2024 at 08:42, Carsten Otto otto@informatik.rwth-aachen.de wrote:
On Wed, Apr 03, 2024 at 08:38:17AM -0400, Stephen Smoogen wrote:
Can you give the full httpd string (remove the ip address). This might
help
figure out if it is a specific version of a tool which is the problem or
if
it is a generic issue.
I don't log user agents, sorry. I could temporarily enable that if helps?
If you for debugging purposes and then turn it off again it would be helpful. There are multiple tools which can download this data from a 'client' point of view and some of them are going to be 'buggy'.
-- Dr. Carsten Otto http://verify.rwth-aachen.de/otto/
On Wed, Apr 03, 2024 at 08:45:57AM -0400, Stephen Smoogen wrote:
If you for debugging purposes and then turn it off again it would be helpful. There are multiple tools which can download this data from a 'client' point of view and some of them are going to be 'buggy'.
Downloads a few files about once a minute:
libdnf (Rocky Linux 8.8; generic; Linux.x86_64)
/fedora-epel/8/Everything/x86_64/repodata/repomd.xml /fedora-epel/8/Everything/x86_64/repodata/0e32b3c58c14790acdb973f20a4b2207e24720222513b0bc63297a649a9884a8-comps-Everything.x86_64.xml /fedora-epel/8/Everything/x86_64/repodata/cf47b564c8fa629069ef0f0836dda95fff05e077068eb3e9ed554c343581be84-prestodelta.xml.xz /fedora-epel/8/Everything/x86_64/repodata/6f0dd08e0df456c5c28d4f776aa71b4bb6d0e4bce4e4bf3423260efdd4b7af2b-updateinfo.xml.bz2 /fedora-epel/8/Everything/x86_64/repodata/1f3bb463d1f38d18f6af974e8353b7e8ad7f368686a462c94d664541f00c7c39-primary.xml.gz /fedora-epel/8/Everything/x86_64/repodata/20761194528ef5ba68ccab20ed7ab3f9303c81ebc562f9dca772a7f20db59dd8-filelists.xml.gz
Similar issue, different user:
libdnf (Rocky Linux 9.3; generic; Linux.x86_64)
/fedora-epel/9/Everything/x86_64/repodata/repomd.xml /fedora-epel/9/Everything/x86_64/repodata/8768391f4f5cb0daebfd4f6103704b325933733c458df8f261b1e47601f81afc-comps-Everything.x86_64.xml.gz /fedora-epel/9/Everything/x86_64/repodata/881a09d92cdb4078eed6995d8e2c48304c4a36011b6fdc695bc19705f86ab2fe-updateinfo.xml.bz2 /fedora-epel/9/Everything/x86_64/repodata/a38ae61a2abfb200cdc4a796f5478c8bcbba4b6a222ab46e7893d9879760734a-primary.xml.gz /fedora-epel/9/Everything/x86_64/repodata/b2cd153314a9aed419fa6adf682c85503be7b6045772bf6cfffef2a4f6fd19fb-filelists.xml.gz
This user downloads other files, also about once a minute (via HTTP 2.0):
urlgrabber/3.10 yum/3.4.3
/fedora-epel/7/x86_64/repodata/repomd.xml /fedora-epel/7/x86_64/repodata/b067b70cf45ef66ab62dc5661b802249f45756ef620528ccf9843d04ca6da83b-updateinfo.xml.bz2 /fedora-epel/7/x86_64/repodata/11ce2dc3bd977b1cbb9acccf51629f6d07ee9c191fecd960ee82df1d307ef3cb-primary.sqlite.bz2
I'll provide more examples as soon as they pour in.
Every few minutes:
libdnf (Red Hat Enterprise Linux 8.6; generic; Linux.x86_64)
/fedora-epel/8/Everything/x86_64/repodata/repomd.xml /fedora-epel/8/Everything/x86_64/repodata/0e32b3c58c14790acdb973f20a4b2207e24720222513b0bc63297a649a9884a8-comps-Everything.x86_64.xml /fedora-epel/8/Everything/x86_64/repodata/cf47b564c8fa629069ef0f0836dda95fff05e077068eb3e9ed554c343581be84-prestodelta.xml.xz /fedora-epel/8/Everything/x86_64/repodata/6f0dd08e0df456c5c28d4f776aa71b4bb6d0e4bce4e4bf3423260efdd4b7af2b-updateinfo.xml.bz2 /fedora-epel/8/Everything/x86_64/repodata/1f3bb463d1f38d18f6af974e8353b7e8ad7f368686a462c94d664541f00c7c39-primary.xml.gz /fedora-epel/8/Everything/x86_64/repodata/20761194528ef5ba68ccab20ed7ab3f9303c81ebc562f9dca772a7f20db59dd8-filelists.xml.gz
More examples (similar patterns to those already shown):
libdnf (AlmaLinux 8.8; generic; Linux.x86_64) libdnf (Rocky Linux 8.8; generic; Linux.x86_64) libdnf (Rocky Linux 8.9; generic; Linux.x86_64) libdnf (Rocky Linux 9.2; generic; Linux.x86_64) urlgrabber/3.10 yum/3.4.3 (HTTP 2.0)
-
Carsten Otto -
Exascale NOC -
Stephen Smoogen