Hello,
some weeks ago I looked briefly at the drpms subdirectories for Centos-6.4-x86_64. All updtes took about 4GB whereas the drpms subdir had already 27GB. Listing the drpms directory by size showed e.g. libreoffice updates taking e.g. 35MB whereas the full update rpm is 37MB.
Is it possible to move the drpms directories to special mirrors only. Similar to the debuginfo/source/oldreleases? Maybe also some tuning is possible to generate drpms only if they really give a huge download saving.
Sorry, I don't have detailed numbers anymore, I've just disabled to mirror the drpms subdir for now. (All numbers above are from memory, my local mirror doesn't have drpms anymore.)
Keeping the official CentOS mirror rather small should be a real plus for the large volume of mirror sites and the real good distribution of CentOS.
Thanks a lot for CentOS,
Florian La Roche
On Tue, 2014-01-07 at 07:05 +0100, Florian La Roche wrote:
Hello,
some weeks ago I looked briefly at the drpms subdirectories for Centos-6.4-x86_64. All updtes took about 4GB whereas the drpms subdir had already 27GB. Listing the drpms directory by size showed e.g. libreoffice updates taking e.g. 35MB whereas the full update rpm is 37MB.
<snip>
The problem is that CentOS seems to currently be generating deltarpms for all possible combinations of rpms, whereas in Fedora deltarpms are only generated from base -> current and current-1 -> current.
To use a current example, the updates/x86_64/Packages directory for CentOS 6.5 is 873MB, while the drpms directory is 8.2GB (almost 10x the rpm size). In comparison, the packages in the updates directory for Fedora 20 use roughly 10GB, while the drpms directory is only 2.4 GB. (roughly 1/4 the rpm size).
Rather than completely ditching drpms, I'd suggest pruning the old drpms.
Jonathan
On 07/01/14 07:05, Florian La Roche wrote:
Hello,
some weeks ago I looked briefly at the drpms subdirectories for Centos-6.4-x86_64. All updtes took about 4GB whereas the drpms subdir had already 27GB. Listing the drpms directory by size showed e.g. libreoffice updates taking e.g. 35MB whereas the full update rpm is 37MB.
Yeah, that's a discussion we had just after the last kernel update push : 2.8Gb were pushed (kernel/kernel-firmware/kernel-docs,etc) all being each time drpm'ized. We noticed that by the fact that several mirrors (slow mirrors) were still rsync'ing after several hours :-(
Is it possible to move the drpms directories to special mirrors only. Similar to the debuginfo/source/oldreleases? Maybe also some tuning is possible to generate drpms only if they really give a huge download saving.
Well, Johnny modified the numbers of deltarpms created now (so up to 5 drpms per package now) so the total size occupied by those drpms should be less than before (was verified yesterday, with last rsync between mirrors deleting quite some files) Problem is that we can't move those drpms to another specific server, as they are declared in the repodata on each mirror : it's a matter of "is yum-presto installed and enabled" that will trigger the use of those drpms (or not).
Sorry, I don't have detailed numbers anymore, I've just disabled to mirror the drpms subdir for now. (All numbers above are from memory, my local mirror doesn't have drpms anymore.)
And you'll not have errors if yum-presto ins't installed/enabled. Even if it is, it will try to fetch the drpms and will fail to find them so it will go with the normal packages (iirc)
Keeping the official CentOS mirror rather small should be a real plus for the large volume of mirror sites and the real good distribution of CentOS.
Thanks a lot for CentOS,
Florian La Roche
Well, Johnny modified the numbers of deltarpms created now (so up to 5 drpms per package now) so the total size occupied by those drpms should be less than before (was verified yesterday, with last rsync between mirrors deleting quite some files)
Perfect! I'll re-add drpms for the mirror again...
Thanks a lot,
Florian La Roche
On Tue, 2014-01-07 at 09:17 +0100, Fabian Arrotin wrote:
On 07/01/14 07:28, Jonathan Dieter wrote:
<snip> > > Rather than completely ditching drpms, I'd suggest pruning the old > drpms. > > Jonathan >
Which normally was done by Johnny last week-end ... :-)
And I've just updated my local mirror. From 8.2GB down to 1.5GB! Absolutely beautiful. Thanks for this,
Jonathan
Hello Fabian and Johnny,
On Tue, Jan 07, 2014 at 09:16:50AM +0100, Fabian Arrotin wrote:
Well, Johnny modified the numbers of deltarpms created now (so up to 5 drpms per package now) so the total size occupied by those drpms should be less than before (was verified yesterday, with last rsync between mirrors deleting quite some files)
Seems there are still way more delta-rpms out there and quite huge "drpms"-subdirectories out there on the mirrors.
Looking at the drpms-Dir listed by size: http://msync.centos.org/centos-6/6.5/updates/x86_64/drpms/?C=S;O=A And comparing that to the list of packages: http://msync.centos.org/centos-6/6.5/updates/x86_64/Packages/
E.g. firefox has 4 normal rpm-Packages with about 47MB out there, those 4 rpm-Packages result in 18 drpm-packages with each 41MB.
This remains a huge burden for drpms...
best regards,
Florian La Roche
-
Fabian Arrotin -
Florian La Roche -
Jonathan Dieter