Hi guys, CentOS 5.10 is currently being synced to mirrors.
While monitoring the synchronization status, I noticed that there are a number of mirrors that apparently don't use the -H flag to preserve hard links. Using that flag would have multiple benefits:
1) Reduced hard disk space. Some i386 packages are also present in the x86_64 tree. These are hard links with their counterparts in the i386 directory. By using the hard links, those files would need to be stored only once on your hard disk.
2) Faster sync times at release. Almost every package in 5.10 is already in a 5.9 directory -- either cr, fasttrack, os or updates. Those not using the rsync -H flag will now need to download the 5.10 os RPMs, instead of just making more hard links to the already existing files. Obviously this does not apply to the .iso files, those will need to be downloaded in any case.
3) Bandwidth consumption would also be reduced, both from your side and from the CentOS msync service side. The msync servers are under a very heavy burden at release times, therefore anything that reduces the traffic would help everyone.
For reference: http://wiki.centos.org/HowTos/CreatePublicMirrors
If I add that to my CRON, will it remove the duplicates I already have or would I need to delete and resync my entire mirror?
-----Original Message----- From: centos-mirror-bounces@centos.org [mailto:centos-mirror-bounces@centos.org] On Behalf Of Anssi Johansson Sent: Saturday, October 19, 2013 9:56 AM To: Mailing list for CentOS mirrors.; Mailing list for CentOS mirrors. Subject: [CentOS-mirror] rsync -H flag
Hi guys, CentOS 5.10 is currently being synced to mirrors.
While monitoring the synchronization status, I noticed that there are a number of mirrors that apparently don't use the -H flag to preserve hard links. Using that flag would have multiple benefits:
1) Reduced hard disk space. Some i386 packages are also present in the x86_64 tree. These are hard links with their counterparts in the i386 directory. By using the hard links, those files would need to be stored only once on your hard disk.
2) Faster sync times at release. Almost every package in 5.10 is already in a 5.9 directory -- either cr, fasttrack, os or updates. Those not using the rsync -H flag will now need to download the 5.10 os RPMs, instead of just making more hard links to the already existing files. Obviously this does not apply to the .iso files, those will need to be downloaded in any case.
3) Bandwidth consumption would also be reduced, both from your side and from the CentOS msync service side. The msync servers are under a very heavy burden at release times, therefore anything that reduces the traffic would help everyone.
For reference: http://wiki.centos.org/HowTos/CreatePublicMirrors _______________________________________________ CentOS-mirror mailing list CentOS-mirror@centos.org http://lists.centos.org/mailman/listinfo/centos-mirror
Thomas kirjoitti:
If I add that to my CRON, will it remove the duplicates I already have or would I need to delete and resync my entire mirror?
Yes, you can simply add the -H flag to your rsync command line. There is no need to delete any files manually. The next time you sync, rsync will take care of the duplicate files and remove the extra ones. If you use the -v (verbose) flag while syncing, you'll see something like this when rsync creates the links:
5.9/os/x86_64/CentOS/zlib-devel-1.2.3-7.el5.i386.rpm => 5.10/os/x86_64/CentOS/zlib-devel-1.2.3-7.el5.i386.rpm
-
Anssi Johansson -
Thomas