How many extra servers can you add to your setup? If I were in your shoes, I would consider building a file server/NAS with fast connection to your server(s). Then share the data to your services to the server (NFS?), export the disk (iscsi) or some combination of both. I hope someone can correct me but I think postfix has issues with use accounts in NFS partitions.
Next step is building your web/mail/etc servers -- be them as VMs or as all in the same baremetal -- as thin as possible so you can recover that quickly (ansible?), mount data fileshares, and off you go. If you are going the vm route you could ether save snapshots or build one of those setups with two servers so in case one goes boink the other takes over. This is also good for upgrading one of the VM servers: do them on different days so you can see if there are problems.
If you cannot have more than one server, do run VMs and then put them in a second set of disks so something happens to boot disk you can recover.
On Sun, Mar 14, 2021 at 1:13 AM Nicolas Kovacs info@microlinux.fr wrote:
Hi,
Last week I had a disaster which took me a few unnerving days to repair. My main Internet-facing server is a bare-metal installation with CentOS 7. It hosts four dozen web sites (or web applications) based on WordPress, Dolibarr, OwnCloud, GEPI, and quite a number of mail accounts for ten different domains. On sunday afternoon this machine had a hardware failure and proved to be unrecoverable.
The good news is, I always have backups of everything. In that case, I have a dedicated backup server (in a different datacenter in a different country). I’m using Rsnapshot for incremental backups, so I had all data: websites, mail accounts, database dumps, configurations, etc.
Now here’s the problem: it took me three and a half days of intense work to restore everything and get everything running again. Three and a half days of downtime is quite a stretch.
As far as I understand, my mistake was to use a bare-metal installation and not a virtualized solution where I could simply restore a snapshot of a VM. Correct me if I’m wrong.
Now I’m doing a lot of thinking and searching. Proxmox and Ceph look quite promising. From what I can tell, the idea is not to use a big server but a cluster of many small servers, and aggregate them like you would do with hard disks in a RAID 10 array for example, only you would do this for the whole system. And then install one or several CentOS 7 VMs on top of this setup.
Any advice from the pros before I dive head first into the documentation?
Cheers from the sunny South of France,
Niki
-- Microlinux - Solutions informatiques durables 7, place de l'église - 30730 Montpezat Site : https://www.microlinux.fr Blog : https://blog.microlinux.fr Mail : info@microlinux.fr Tél. : 04 66 63 10 32 Mob. : 06 51 80 12 12 _______________________________________________ CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos