Hi,
I've already noticed that almost 30G of data is saved on CentOS mirrors (only counting release 4/5, i386 and x86_64 archs), just by using hardlinks for similar packages.
But I have gone further, and checked if something else could be saved, and my hardlink report is this:
Directories 404 Objects 45722 IFREG 45231 Mmaps 307 Comparisons 308 Would link 300 Would save 78958592
Another 300 files, 79Mbytes could be saved. Not as much as what is already saved, but some (38) of these files are RPMs, so it makes me wonder if the hardlink process is not automatic at all, and who should I contact in such cases, if not this list.
Would link ./5.4/os/i386/CentOS/libwpd-devel-0.8.7-3.el5.i386.rpm to ./5.4/os/x86_64/CentOS/libwpd-devel-0.8.7-3.el5.i386.rpm Would link ./5.4/os/i386/CentOS/libsilc-devel-1.0.2-2.fc6.i386.rpm to ./5.4/os/x86_64/CentOS/libsilc-devel-1.0.2-2.fc6.i386.rpm Would link ./5.4/os/i386/CentOS/libgpod-devel-0.4.0-1.el5.i386.rpm to ./5.4/os/x86_64/CentOS/libgpod-devel-0.4.0-1.el5.i386.rpm Would link ./5.4/os/i386/CentOS/libgpod-0.4.0-1.el5.i386.rpm to ./5.4/os/x86_64/CentOS/libgpod-0.4.0-1.el5.i386.rpm Would link ./5.4/os/i386/CentOS/libwpd-devel-0.8.7-3.el5.i386.rpm to ./5.4/extras/i386/RPMS/libwpd-devel-0.8.7-3.el5.i386.rpm, would save 12894 Would link ./5.4/os/i386/CentOS/libsilc-devel-1.0.2-2.fc6.i386.rpm to ./5.4/extras/i386/RPMS/libsilc-devel-1.0.2-2.fc6.i386.rpm, would save 220674 Would link ./5.4/os/i386/CentOS/libgpod-0.4.0-1.el5.i386.rpm to ./5.4/extras/i386/RPMS/libgpod-0.4.0-1.el5.i386.rpm, would save 99870 Would link ./4.8/updates/i386/RPMS/tzdata-2009o-2.el4.noarch.rpm to ./4.8/updates/x86_64/RPMS/tzdata-2009o-2.el4.noarch.rpm, would save 468167 Would link ./4.8/updates/i386/RPMS/kernel-doc-2.6.9-89.0.15.EL.noarch.rpm to ./4.8/updates/x86_64/RPMS/kernel-doc-2.6.9-89.0.15.EL.noarch.rpm, would save 2369828 Would link ./4.8/updates/i386/RPMS/kernel-doc-2.6.9-89.0.16.EL.noarch.rpm to ./4.8/updates/x86_64/RPMS/kernel-doc-2.6.9-89.0.16.EL.noarch.rpm, would save 2369859 Would link ./4.8/centosplus/i386/RPMS/kernel-doc-2.6.9-89.0.15.plus.c4.noarch.rpm to ./4.8/centosplus/x86_64/RPMS/kernel-doc-2.6.9-89.0.15.plus.c4.noarch.rpm, would save 2370306 Would link ./4.8/centosplus/i386/RPMS/kmod-xfs-0.4-2.el4.i686.rpm to ./4.8/extras/i386/RPMS/kmod-xfs-0.4-2.el4.i686.rpm Would link ./4.8/centosplus/i386/RPMS/kmod-xfs-xenU-0.4-2.el4.i686.rpm to ./4.8/extras/i386/RPMS/kmod-xfs-xenU-0.4-2.el4.i686.rpm Would link ./4.8/centosplus/i386/RPMS/kmod-xfs-hugemem-0.4-2.el4.i686.rpm to ./4.8/extras/i386/RPMS/kmod-xfs-hugemem-0.4-2.el4.i686.rpm Would link ./4.8/centosplus/i386/RPMS/kmod-xfs-smp-0.4-2.el4.i686.rpm to ./4.8/extras/i386/RPMS/kmod-xfs-smp-0.4-2.el4.i686.rpm Would link ./4.8/centosplus/x86_64/RPMS/kmod-xfs-0.4-2.el4.x86_64.rpm to ./4.8/extras/x86_64/RPMS/kmod-xfs-0.4-2.el4.x86_64.rpm Would link ./4.8/centosplus/x86_64/RPMS/kmod-xfs-largesmp-0.4-2.el4.x86_64.rpm to ./4.8/extras/x86_64/RPMS/kmod-xfs-largesmp-0.4-2.el4.x86_64.rpm Would link ./4.8/centosplus/x86_64/RPMS/kmod-xfs-xenU-0.4-2.el4.x86_64.rpm to ./4.8/extras/x86_64/RPMS/kmod-xfs-xenU-0.4-2.el4.x86_64.rpm Would link ./4.8/centosplus/x86_64/RPMS/kmod-xfs-smp-0.4-2.el4.x86_64.rpm to ./4.8/extras/x86_64/RPMS/kmod-xfs-smp-0.4-2.el4.x86_64.rpm Would link ./4.8/centosplus/i386/RPMS/kmod-xfs-0.4-2.el4.i686.rpm to ./4.7/extras/i386/RPMS/kmod-xfs-0.4-2.el4.i686.rpm Would link ./4.8/centosplus/i386/RPMS/kmod-xfs-xenU-0.4-2.el4.i686.rpm to ./4.7/extras/i386/RPMS/kmod-xfs-xenU-0.4-2.el4.i686.rpm Would link ./4.8/centosplus/i386/RPMS/kmod-xfs-hugemem-0.4-2.el4.i686.rpm to ./4.7/extras/i386/RPMS/kmod-xfs-hugemem-0.4-2.el4.i686.rpm Would link ./4.8/centosplus/i386/RPMS/kmod-xfs-smp-0.4-2.el4.i686.rpm to ./4.7/extras/i386/RPMS/kmod-xfs-smp-0.4-2.el4.i686.rpm Would link ./4.8/centosplus/x86_64/RPMS/kmod-xfs-0.4-2.el4.x86_64.rpm to ./4.7/extras/x86_64/RPMS/kmod-xfs-0.4-2.el4.x86_64.rpm Would link ./4.8/centosplus/x86_64/RPMS/kmod-xfs-largesmp-0.4-2.el4.x86_64.rpm to ./4.7/extras/x86_64/RPMS/kmod-xfs-largesmp-0.4-2.el4.x86_64.rpm Would link ./4.8/centosplus/x86_64/RPMS/kmod-xfs-xenU-0.4-2.el4.x86_64.rpm to ./4.7/extras/x86_64/RPMS/kmod-xfs-xenU-0.4-2.el4.x86_64.rpm Would link ./4.8/centosplus/x86_64/RPMS/kmod-xfs-smp-0.4-2.el4.x86_64.rpm to ./4.7/extras/x86_64/RPMS/kmod-xfs-smp-0.4-2.el4.x86_64.rpm Would link ./5.4/os/i386/CentOS/libwpd-devel-0.8.7-3.el5.i386.rpm to ./5.3/os/i386/CentOS/libwpd-devel-0.8.7-3.el5.i386.rpm, would save 12894 Would link ./5.4/os/i386/CentOS/libsilc-devel-1.0.2-2.fc6.i386.rpm to ./5.3/os/i386/CentOS/libsilc-devel-1.0.2-2.fc6.i386.rpm, would save 220674 Would link ./5.4/os/i386/CentOS/libgpod-devel-0.4.0-1.el5.i386.rpm to ./5.3/os/i386/CentOS/libgpod-devel-0.4.0-1.el5.i386.rpm, would save 45327 Would link ./5.4/os/i386/CentOS/libgpod-0.4.0-1.el5.i386.rpm to ./5.3/os/i386/CentOS/libgpod-0.4.0-1.el5.i386.rpm, would save 99870 Would link ./5.4/os/i386/CentOS/libwpd-devel-0.8.7-3.el5.i386.rpm to ./5.3/os/x86_64/CentOS/libwpd-devel-0.8.7-3.el5.i386.rpm Would link ./5.4/os/i386/CentOS/libsilc-devel-1.0.2-2.fc6.i386.rpm to ./5.3/os/x86_64/CentOS/libsilc-devel-1.0.2-2.fc6.i386.rpm Would link ./5.4/os/i386/CentOS/libgpod-devel-0.4.0-1.el5.i386.rpm to ./5.3/os/x86_64/CentOS/libgpod-devel-0.4.0-1.el5.i386.rpm Would link ./5.4/os/i386/CentOS/libgpod-0.4.0-1.el5.i386.rpm to ./5.3/os/x86_64/CentOS/libgpod-0.4.0-1.el5.i386.rpm Would link ./5.4/os/i386/CentOS/libwpd-devel-0.8.7-3.el5.i386.rpm to ./5.3/extras/i386/RPMS/libwpd-devel-0.8.7-3.el5.i386.rpm, would save 12894 Would link ./5.4/os/i386/CentOS/libsilc-devel-1.0.2-2.fc6.i386.rpm to ./5.3/extras/i386/RPMS/libsilc-devel-1.0.2-2.fc6.i386.rpm, would save 220674 Would link ./5.4/os/i386/CentOS/libgpod-0.4.0-1.el5.i386.rpm to ./5.3/extras/i386/RPMS/libgpod-0.4.0-1.el5.i386.rpm, would save 99870
João Carlos Mendes Luís jonny@jonny.eng.br writes:
I've already noticed that almost 30G of data is saved on CentOS
mirrors (only counting release 4/5, i386 and x86_64 archs), just by using hardlinks for similar packages.
...snip...
Another 300 files, 79Mbytes could be saved. Not as much as what is already saved, but some (38) of these files are RPMs, so it makes me wonder if the hardlink process is not automatic at all, and who should I contact in such cases, if not this list.
Could somebody on the CentOS mirror team please comment on this?
I also wonder why no hardlink script is run on the CentOS master mirrors. I run a script periodically on our mirror, and as João says this saves a significant amount of disk space.
If this were done on the master mirrors it would save bandwidth while mirrors are syncing. Especially when a new minor release is rolled out, as several RPMs in the new release is identical to the old one. This of course only works if everybody runs rsync with -H.
Kind regards,
Pär Andersson