> On Thu, 26 Sep 2019 at 05:29, Simon Matter via CentOS-devel > <centos-devel at centos.org> wrote: >> >> > On 26/09/2019 10:06, Kaj Niemi wrote: >> >> Hi, >> >> >> >> >> >> Is it on purpose that under AppStream, BaseOS, PowerTools (but not >> under >> >> extras, fasttrack and centosplusplus) there is an os and a kickstart >> >> directory for each architecture? On a quick glance the directory >> layout >> >> and contents seem rather identical. >> >> >> >> Asking as “we” do a local mirror of releases and started wondering >> about >> >> the amount of disk space 8 takes. >> >> >> > >> > Hi, >> > >> > kickstart is a snapshot of os at GA/release time. >> > So that permits people to deploy with exactly same content without >> > having a moving target, as BaseOS/AppStream will be including updates >> as >> > they land. >> > >> > WRT disk space, hopefully, as explained on how to mirror correctly >> > (https://wiki.centos.org/HowTos/CreatePublicMirrors) , you use -H for >> > rsync, as we heavily use hardlinks in our trees so that it reduces >> space >> > and also used bandwidth to sync mirror content >> >> Hi, >> >> I was just wondering why it wasn't implemented with symlinks? They have >> the advantage that they can be recognized easily IMHO. You could then go >> a >> step further and even create an updates/ directory with only symlinks to >> the corresponding files. It would allow to still have an updates/ >> directory which some of us are missing now. >> > > From dealing with mirroring issues a long time, large number of > symlinks in a directory have broken various filesystems that various > systems use. You end up with sites with missing symlinks, broken > symlinks or other anomalies. If you hardlink the mirror might download > the file twice but it will be there. If you symlink hundreds of files, > you are debugging client issues to a mirror you don't control and you > have to make your check script even more complicated to confirm that > things are there versus just look to be there. Interesting, I never saw issues with symlinks. Was it a problem with filesystems or a problem with the sync tool? A bit OT here but: I do have problems syncing large filesystems which contain large number of hardlinks using rsync. I get the following error and don't know how to go on. That's with rsync -aH: building file list ... ERROR: out of memory in hashtable_node [sender] rsync error: error allocating core memory buffers (code 22) at util.c(117) [sender=3.0.6] I've built latest rsync to try but it fails the same way. Is the only way to fix this to add more memory to the systems? Thanks, Simon