[CentOS-mirror] Mirror monitoring script Contrib

Our setup is mostly Python-based, it works quite well and it has been
running reliably for many years.

We currently use Jenkins to schedule the python scripts that run our
rsync jobs.
Jenkins is set to run the commands like this: sudo -u mirror
/home/mirror/scripts/run_mirror.py -v centos
(We need `sudo` because we don't want to have everything owned by the
Jenkins user. The `sudoers` configuration allows Jenkins to run that Python
script only.)

All of our Python scripts are available on GitHub here:
https://github.com/umd-mirror/scripts
(Just for completeness, our website is here:
https://github.com/umd-mirror/web )

I don't particularly like relying on Jenkins (because it is bloated and
huge and it requires Java), but it's what we were using before I took over
the mirror and I haven't had time to replace it.
We have Jenkins send the emails with the build failure notifications
because that is built in, and it also sends a "back to normal" email when
the next successful build completes. It also stores a week of successful
logs, just in case we need to look at them, as well as all the failure logs.

Just for completeness, I have attached a screenshot of (the top half of)
the Jenkins status page.

I know there was a time in the past that we did not use Jenkins, just cron
calling the Python script, so if you look around in our Python scripts you
may find email notification code that can help you as well.

Moshe

--
Moshe Katz
mmkatz at umd.edu
(301) 867-3732


On Mon, May 11, 2020 at 9:09 PM Cambo.Host Mirror Service <mirror at cambo.host>
wrote:

> Hello fellow mirrorers
>
> This morning I started writing a wrapper for rsync to monitor the syncs
> and notify me of failures, however before I continue I'd like to ask if
> anyone else has such a script they'd like to share so as not to re-
> invent the wheel.
>
> The script I'm planning will obviously run the rsync and notify me by
> email of faults, but it will also monitor rsync in case it stalls.
> This happens sometimes if by chance there's some upstream network
> maintenance and a disconnect occurs in the middle of a sync.
>
> Look forward to seeing other people's solutions.
>
> Cheers from Cambodia!
>
> _______________________________________________
> CentOS-mirror mailing list
> CentOS-mirror at centos.org
> https://lists.centos.org/mailman/listinfo/centos-mirror
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.centos.org/pipermail/centos-mirror/attachments/20200512/4c9a3f12/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: mirror jenkins screenshot.png
Type: image/png
Size: 143148 bytes
Desc: not available
URL: <http://lists.centos.org/pipermail/centos-mirror/attachments/20200512/4c9a3f12/attachment.png>