<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
</head>
<body>
<div dir="auto" style="direction: ltr; margin: 0; padding: 0; font-family: sans-serif; font-size: 11pt; color: black; ">
Thanks! I'll take a look and let you know if I have any questions.<br>
<br>
</div>
<div dir="auto" style="direction: ltr; margin: 0; padding: 0; font-family: sans-serif; font-size: 11pt; color: black; ">
<span id="OutlookSignature"></span><br>
</div>
<div dir="auto" style="direction: ltr; margin: 0; padding: 0; font-family: sans-serif; font-size: 11pt; color: black; ">
<b><font color="#000000">From:</font></b><font color="#000000"> CentOS-mirror <centos-mirror-bounces@centos.org> on behalf of Moshe M. Katz <mmkatz@umd.edu></font><br>
</div>
<div dir="auto" style="direction: ltr; margin: 0; padding: 0; font-family: sans-serif; font-size: 11pt; color: black; ">
<b><font color="#000000">Sent:</font></b><font color="#000000"> Thursday, October 15, 2020 8:49:39 PM</font><br>
</div>
<div dir="auto" style="direction: ltr; margin: 0; padding: 0; font-family: sans-serif; font-size: 11pt; color: black; ">
<b><font color="#000000">To:</font></b><font color="#000000"> Mailing list for CentOS mirrors. <centos-mirror@centos.org></font><br>
</div>
<div dir="auto" style="direction: ltr; margin: 0; padding: 0; font-family: sans-serif; font-size: 11pt; color: black; ">
<b><font color="#000000">Subject:</font></b><font color="#000000"> Re: [CentOS-mirror] Setting up mirror from scratch</font>
<br>
</div>
<div dir="auto" style="direction: ltr; margin: 0; padding: 0; font-family: sans-serif; font-size: 11pt; color: black; ">
<br>
</div>
<div dir="auto" style="direction: ltr; margin: 0; padding: 0; font-family: sans-serif; font-size: 11pt; color: black; ">
Here's a little bit about how we run <a href="http://mirror.umd.edu">mirror.umd.edu</a> (serving almost two dozen free software projects, up to 5 TB of traffic per day).
<br>
<br>
</div>
<div dir="auto" style="direction: ltr; margin: 0; padding: 0; font-family: sans-serif; font-size: 11pt; color: black; ">
Hardware: <br>
</div>
<div dir="auto" style="direction: ltr; margin: 0; padding: 0; font-family: sans-serif; font-size: 11pt; color: black; ">
We are running on a Dell PowerEdge R720xd with 10x 8TB hard drives using ZFS in a raidz2 configuration. We also have two 512GB SSDs for L2ARC caching, and two 1TB hard drives in a mirrored configuration for the operating system. We have 128GB of RAM and we
let ZFS use most of it (right now ZFS is using 90GB). <br>
<br>
</div>
<div dir="auto" style="direction: ltr; margin: 0; padding: 0; font-family: sans-serif; font-size: 11pt; color: black; ">
Software: <br>
</div>
<div dir="auto" style="direction: ltr; margin: 0; padding: 0; font-family: sans-serif; font-size: 11pt; color: black; ">
We serve files using nginx, with a pretty basic configuration that only has a few changes from the default. You can see it here:
<a href="https://github.com/umd-mirror/nginx">https://github.com/umd-mirror/nginx</a>
<br>
</div>
<div dir="auto" style="direction: ltr; margin: 0; padding: 0; font-family: sans-serif; font-size: 11pt; color: black; ">
We have symlinks in our web root directory (`/home/mirror/web`) which point to the actual storage locations (in `/pool/mirrors/NAME`).
<br>
<br>
</div>
<div dir="auto" style="direction: ltr; margin: 0; padding: 0; font-family: sans-serif; font-size: 11pt; color: black; ">
Management: <br>
</div>
<div dir="auto" style="direction: ltr; margin: 0; padding: 0; font-family: sans-serif; font-size: 11pt; color: black; ">
We use a set of Python scripts to manage syncing (mostly using rsync). You can find them here:
<a href="https://github.com/umd-mirror/scripts">https://github.com/umd-mirror/scripts</a>
<br>
</div>
<div dir="auto" style="direction: ltr; margin: 0; padding: 0; font-family: sans-serif; font-size: 11pt; color: black; ">
We use Jenkins to schedule running the python scripts because it allows us to track success/failure of the syncing over time. (I don't like this, but it was this way before I became involved and it does work pretty well). I can't share the whole Jenkins configuration,
but here is how our CentOS job is configured: <br>
</div>
<div dir="auto" style="direction: ltr; margin: 0; padding: 0; font-family: sans-serif; font-size: 11pt; color: black; ">
Run every six hours using: Build Triggers -> Build Periodically (checked) -> Schedule: "H */6 * * *"Build -> Execute Shell: "sudo -u mirror /home/mirror/scripts/run_mirror.py -v centos"
<br>
</div>
<div dir="auto" style="direction: ltr; margin: 0; padding: 0; font-family: sans-serif; font-size: 11pt; color: black; ">
In order to allow the Jenkins user to use `sudo`, we have the following content in `/etc/sudoers.d/jenkins`:
<br>
</div>
<div dir="auto" style="direction: ltr; margin: 0; padding: 0; font-family: sans-serif; font-size: 11pt; color: black; ">
<tt>jenkins ALL=(mirror) NOPASSWD: /home/mirror/scripts/run_mirror.py</tt> <br>
<br>
</div>
<div dir="auto" style="direction: ltr; margin: 0; padding: 0; font-family: sans-serif; font-size: 11pt; color: black; ">
Storage: <br>
</div>
<div dir="auto" style="direction: ltr; margin: 0; padding: 0; font-family: sans-serif; font-size: 11pt; color: black; ">
We created a ZFS ZPOOL at /pool/mirrors and then also a separate "sub pool" for each software project we mirror. That way we can manage disk space very easily and add quotas if we think a project is growing too quickly and might use too much disk space. (We
have not had to do that now that we have our big new disks, but we used to need it on our old server and we are prepared to use it again in the future if needed.) It also allows us to show the used space very quickly and easily on our web page (look at the
end of each row on our home page). To allow the `mirror` user to run ZFS commands, we created a shell script in `/usr/local/bin/get_lused.sh` which can be seen in our `scripts` repository. There is also a comment there with the necessary sudoers entry to allow
it to work. <br>
<br>
</div>
<div dir="auto" style="direction: ltr; margin: 0; padding: 0; font-family: sans-serif; font-size: 11pt; color: black; ">
Feel free to ask me for details if you have any questions about our configuration or scripts.
<br>
<br>
</div>
<div dir="auto" style="direction: ltr; margin: 0; padding: 0; font-family: sans-serif; font-size: 11pt; color: black; ">
Moshe <br>
<br>
</div>
<div dir="auto" style="direction: ltr; margin: 0; padding: 0; font-family: sans-serif; font-size: 11pt; color: black; ">
-- <br>
</div>
<div dir="auto" style="direction: ltr; margin: 0; padding: 0; font-family: sans-serif; font-size: 11pt; color: black; ">
Moshe Katz <br>
</div>
<div dir="auto" style="direction: ltr; margin: 0; padding: 0; font-family: sans-serif; font-size: 11pt; color: black; ">
<a href="mailto:mmkatz@umd.edu">mmkatz@umd.edu</a> <br>
<br>
<br>
<br>
<br>
</div>
<div dir="auto" style="direction: ltr; margin: 0; padding: 0; font-family: sans-serif; font-size: 11pt; color: black; ">
On Thu, Oct 15, 2020 at 8:10 PM Alexandre Leonenko < <a href="mailto:alex@esecuredata.com">
alex@esecuredata.com</a>> wrote: <br>
</div>
<blockquote type="cite">
<div dir="auto" style="direction: ltr; margin: 0; padding: 0; font-family: sans-serif; font-size: 11pt; color: black; ">
Hey guys, <br>
<br>
</div>
<div dir="auto" style="direction: ltr; margin: 0; padding: 0; font-family: sans-serif; font-size: 11pt; color: black; ">
I'm setting up a new mirror as the existing one is going to be eol due to using CentOS 6 as a base. (
<a href="http://mirror.esecuredata.com">mirror.esecuredata.com</a>) <br>
</div>
<div dir="auto" style="direction: ltr; margin: 0; padding: 0; font-family: sans-serif; font-size: 11pt; color: black; ">
It's also running on old Pentium and is very slow nowadays. <br>
<br>
</div>
<div dir="auto" style="direction: ltr; margin: 0; padding: 0; font-family: sans-serif; font-size: 11pt; color: black; ">
My question is this, what do you guys usually use, apache or nginx for serving http?
<br>
</div>
<div dir="auto" style="direction: ltr; margin: 0; padding: 0; font-family: sans-serif; font-size: 11pt; color: black; ">
Also if you have nifty scripts to share those would be welcome. <br>
<br>
</div>
<div dir="auto" style="direction: ltr; margin: 0; padding: 0; font-family: sans-serif; font-size: 11pt; color: black; ">
Regards, <br>
</div>
<div dir="auto" style="direction: ltr; margin: 0; padding: 0; font-family: sans-serif; font-size: 11pt; color: black; ">
Alex <br>
<br>
</div>
<div dir="auto" style="direction: ltr; margin: 0; padding: 0; font-family: sans-serif; font-size: 11pt; color: black; ">
_______________________________________________ <br>
</div>
<div dir="auto" style="direction: ltr; margin: 0; padding: 0; font-family: sans-serif; font-size: 11pt; color: black; ">
CentOS-mirror mailing list <br>
</div>
<div dir="auto" style="direction: ltr; margin: 0; padding: 0; font-family: sans-serif; font-size: 11pt; color: black; ">
<a href="mailto:CentOS-mirror@centos.org">CentOS-mirror@centos.org</a> <br>
</div>
<div dir="auto" style="direction: ltr; margin: 0; padding: 0; font-family: sans-serif; font-size: 11pt; color: black; ">
<a href="https://lists.centos.org/mailman/listinfo/centos-mirror">https://lists.centos.org/mailman/listinfo/centos-mirror</a>
<br>
</div>
</blockquote>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> CentOS-mirror <centos-mirror-bounces@centos.org> on behalf of Moshe M. Katz <mmkatz@umd.edu><br>
<b>Sent:</b> Thursday, October 15, 2020 8:49:39 PM<br>
<b>To:</b> Mailing list for CentOS mirrors. <centos-mirror@centos.org><br>
<b>Subject:</b> Re: [CentOS-mirror] Setting up mirror from scratch</font>
<div> </div>
</div>
<div>
<div dir="ltr">
<div>Here's a little bit about how we run <a href="http://mirror.umd.edu">mirror.umd.edu</a> (serving almost two dozen free software projects, up to 5 TB of traffic per day).</div>
<div><br>
</div>
<div>Hardware:</div>
<div>We are running on a Dell PowerEdge R720xd with 10x 8TB hard drives using ZFS in a raidz2 configuration. We also have two 512GB SSDs for L2ARC caching, and two 1TB hard drives in a mirrored configuration for the operating system. We have 128GB of RAM and
we let ZFS use most of it (right now ZFS is using 90GB).</div>
<div><br>
</div>
<div>Software:</div>
<div>We serve files using nginx, with a pretty basic configuration that only has a few changes from the default. You can see it here: <a href="https://github.com/umd-mirror/nginx">https://github.com/umd-mirror/nginx</a></div>
<div>We have symlinks in our web root directory (`/home/mirror/web`) which point to the actual storage locations (in `/pool/mirrors/NAME`).</div>
<div><br>
</div>
<div>Management:</div>
<div>We use a set of Python scripts to manage syncing (mostly using rsync). You can find them here: <a href="https://github.com/umd-mirror/scripts">https://github.com/umd-mirror/scripts</a></div>
<div>We use Jenkins to schedule running the python scripts because it allows us to track success/failure of the syncing over time. (I don't like this, but it was this way before I became involved and it does work pretty well). I can't share the whole Jenkins
configuration, but here is how our CentOS job is configured:</div>
<div>
<ul>
<li>Run every six hours using: Build Triggers -> Build Periodically (checked) -> Schedule: "H */6 * * *"</li><li>Build -> Execute Shell: "sudo -u mirror /home/mirror/scripts/run_mirror.py -v centos"</li></ul>
</div>
<div>In order to allow the Jenkins user to use `sudo`, we have the following content in `/etc/sudoers.d/jenkins`:</div>
<div><font face="monospace">jenkins ALL=(mirror) NOPASSWD: /home/mirror/scripts/run_mirror.py</font><br>
</div>
<div><br>
</div>
<div>Storage:</div>
<div>We created a ZFS ZPOOL at /pool/mirrors and then also a separate "sub pool" for each software project we mirror. That way we can manage disk space very easily and add quotas if we think a project is growing too quickly and might use too much disk space.
(We have not had to do that now that we have our big new disks, but we used to need it on our old server and we are prepared to use it again in the future if needed.) It also allows us to show the used space very quickly and easily on our web page (look at
the end of each row on our home page). To allow the `mirror` user to run ZFS commands, we created a shell script in `/usr/local/bin/get_lused.sh` which can be seen in our `scripts` repository. There is also a comment there with the necessary sudoers entry
to allow it to work.</div>
<div><br>
</div>
<div>Feel free to ask me for details if you have any questions about our configuration or scripts.</div>
<div><br>
</div>
<div>Moshe</div>
<div><br>
</div>
<div>
<div dir="ltr" class="x_gmail_signature">
<div dir="ltr">--<br>
Moshe Katz<br>
<a href="mailto:mmkatz@umd.edu" target="_blank">mmkatz@umd.edu</a><br>
<div><br>
</div>
<div><br>
</div>
</div>
</div>
</div>
<br>
</div>
<br>
<div class="x_gmail_quote">
<div dir="ltr" class="x_gmail_attr">On Thu, Oct 15, 2020 at 8:10 PM Alexandre Leonenko <<a href="mailto:alex@esecuredata.com">alex@esecuredata.com</a>> wrote:<br>
</div>
<blockquote class="x_gmail_quote" style="margin:0px 0px 0px 0.8ex; border-left:1px solid rgb(204,204,204); padding-left:1ex">
<div dir="ltr">
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Hey guys,</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
I'm setting up a new mirror as the existing one is going to be eol due to using CentOS 6 as a base. (<a href="http://mirror.esecuredata.com" target="_blank">mirror.esecuredata.com</a>)</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
It's also running on old Pentium and is very slow nowadays.</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
My question is this, what do you guys usually use, apache or nginx for serving http?</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Also if you have nifty scripts to share those would be welcome.</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Regards,</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Alex</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
</div>
_______________________________________________<br>
CentOS-mirror mailing list<br>
<a href="mailto:CentOS-mirror@centos.org" target="_blank">CentOS-mirror@centos.org</a><br>
<a href="https://lists.centos.org/mailman/listinfo/centos-mirror" rel="noreferrer" target="_blank">https://lists.centos.org/mailman/listinfo/centos-mirror</a><br>
</blockquote>
</div>
</div>
</body>
</html>