Hello fellow mirrorers
This morning I started writing a wrapper for rsync to monitor the syncs and notify me of failures, however before I continue I'd like to ask if anyone else has such a script they'd like to share so as not to re- invent the wheel.
The script I'm planning will obviously run the rsync and notify me by email of faults, but it will also monitor rsync in case it stalls. This happens sometimes if by chance there's some upstream network maintenance and a disconnect occurs in the middle of a sync.
Look forward to seeing other people's solutions.
Cheers from Cambodia!
Our setup is mostly Python-based, it works quite well and it has been running reliably for many years.
We currently use Jenkins to schedule the python scripts that run our rsync jobs. Jenkins is set to run the commands like this: sudo -u mirror /home/mirror/scripts/run_mirror.py -v centos (We need `sudo` because we don't want to have everything owned by the Jenkins user. The `sudoers` configuration allows Jenkins to run that Python script only.)
All of our Python scripts are available on GitHub here: https://github.com/umd-mirror/scripts (Just for completeness, our website is here: https://github.com/umd-mirror/web )
I don't particularly like relying on Jenkins (because it is bloated and huge and it requires Java), but it's what we were using before I took over the mirror and I haven't had time to replace it. We have Jenkins send the emails with the build failure notifications because that is built in, and it also sends a "back to normal" email when the next successful build completes. It also stores a week of successful logs, just in case we need to look at them, as well as all the failure logs.
Just for completeness, I have attached a screenshot of (the top half of) the Jenkins status page.
I know there was a time in the past that we did not use Jenkins, just cron calling the Python script, so if you look around in our Python scripts you may find email notification code that can help you as well.
Moshe
-- Moshe Katz mmkatz@umd.edu (301) 867-3732
On Mon, May 11, 2020 at 9:09 PM Cambo.Host Mirror Service mirror@cambo.host wrote:
-
Cambo.Host Mirror Service -
Moshe M. Katz