On Thu, 26 Sep 2019 at 05:29, Simon Matter via CentOS-devel centos-devel@centos.org wrote:
On 26/09/2019 10:06, Kaj Niemi wrote:
Hi,
Is it on purpose that under AppStream, BaseOS, PowerTools (but not
under
extras, fasttrack and centosplusplus) there is an os and a kickstart directory for each architecture? On a quick glance the directory
layout
and contents seem rather identical.
Asking as “we” do a local mirror of releases and started wondering
about
the amount of disk space 8 takes.
Hi,
kickstart is a snapshot of os at GA/release time. So that permits people to deploy with exactly same content without having a moving target, as BaseOS/AppStream will be including updates
as
they land.
WRT disk space, hopefully, as explained on how to mirror correctly (https://wiki.centos.org/HowTos/CreatePublicMirrors) , you use -H for rsync, as we heavily use hardlinks in our trees so that it reduces
space
and also used bandwidth to sync mirror content
Hi,
I was just wondering why it wasn't implemented with symlinks? They have the advantage that they can be recognized easily IMHO. You could then go a step further and even create an updates/ directory with only symlinks to the corresponding files. It would allow to still have an updates/ directory which some of us are missing now.
From dealing with mirroring issues a long time, large number of symlinks in a directory have broken various filesystems that various systems use. You end up with sites with missing symlinks, broken symlinks or other anomalies. If you hardlink the mirror might download the file twice but it will be there. If you symlink hundreds of files, you are debugging client issues to a mirror you don't control and you have to make your check script even more complicated to confirm that things are there versus just look to be there.
Interesting, I never saw issues with symlinks. Was it a problem with filesystems or a problem with the sync tool?
A bit OT here but: I do have problems syncing large filesystems which contain large number of hardlinks using rsync. I get the following error and don't know how to go on. That's with rsync -aH:
building file list ... ERROR: out of memory in hashtable_node [sender] rsync error: error allocating core memory buffers (code 22) at util.c(117) [sender=3.0.6]
I've built latest rsync to try but it fails the same way. Is the only way to fix this to add more memory to the systems?
Thanks, Simon