[CentOS] RE: Keeping data on 2 servers in sync !

Thu Dec 8 18:51:09 UTC 2005
Jonathan Darton <jdarton at ibigroup.com>

On 12/7/05, Denis Croombs <denis at croombs.org> wrote:
> On 12/6/05, Jonathan Darton <jdarton at ibigroup.com> wrote:
> >> >I want to build 2 servers (both running samba) to provide file
> >> >storage to 2 offices (approx 100 miles apart, linked via DSL) but all
> >> >data writen to 1 server must also be saved to the other server.
> >> >Both servers would also allow users to access the data via a VPN thus
> >> >allowing 1 office with a failed server to access the other server via
> >> >the vpn and still see the data from both offices.
> >> >I currently have 1 server working but we want to add the second
> >> >office to the system. (Currently 1 office has 10 users and the second
> >> >office has 1 user connected via VPN ) but the second office will have
> >> >20 within 12
> >> months
> >> >and the first will have 35 soon ))
> >>
> >> >Has anyone done anything like this ?
> >>
> >> I am currently synchronizing multiple office locations using a program
> >> called unison. Unison (http://www.cis.upenn.edu/~bcpierce/unison/) is
> >> a very well written program that can perform 2 way file
> >> synchronization. There are many configurable options with unison and I
> recommend that you check it out.
> >> In each office I have a PII350 128RAM Fedora or CentOS server running
> >> unison and the files are accessed via samba. I also configure samba to
> >> hide (veto) all of the temporary files used during synchronization.
> >> For redundancy I place a slave server with each master server that
> >> backs up all the user data / file system using rsync. This way if one
> >> of my $5 PII servers catches fire I can automatically switch over with
no
> downtime for the users.
> >>
> >> The only downfall I have encountered is with Autocad files not
> >> properly reading the synchronized .dwl lock file and more than one
> >> user working on the same file. As a work around for this I have
> >> configured Unison to keep a backup of the last 20 versions of a file.
> >> This way I can always hit my backups to retreive lost data. As a side
> >> note, if anyone knows a work around for the stubborn autocad dwl lock
> file let me know :))!
> >>
> >> In any case my implementation has allowed me to synchronize file
> >> systems between 4 offices (3 in Canada, 1 in USA), using recycled
> >> hardware that was otherwise going to be donated/trashed.
> >>
> >> Let me know if you have any further questions.
> >
> >I'm about to do a Unison setup on two CentOS servers, so I'm thrilled to
> see this response. I also work with Architects >>sometimes, so I'm
> interested to hear about the dwl lock file issue.
> >
> >My one compound question: how are you invoking Unison? In batch mode,
with
> cron? How often? Wat other options did you
> >consider before settling on the scheme you use?
>
> I see the project is no longer supported, do you have rpms for it ?
>
> Thanks

>It's not that it's no longer supported, really, just that the project
>that initiated it is not an officially-funded academic project. It has
>actually been updated since the termination of the project, it looks
>like.

>I haven't installed it on CentOS yet but I got FC4 rpms from fedora
>extras (I just typed 'yum install unison' and it worked). I think it's
>in pretty common usage in RHEL too, so it can't be too hard to find
>for CentOS, right? (But let us know what you turn up :-)).

I created a script that is run by cron every minute. The script creates a
lock file for the session, rsyncs the changes to the backup server,
synchronizes offices using unison, deletes the lock file. Unison is executed
in "batch" and "silent" mode so that no output is generated and mailed by
cron.
One other thing you need to consider is the size of the file system you are
trying to synchronize. I have a very large directory structure in two
offices which required running unison a second time splitting up the
synchronization into two sessions. This can easily be done by ignoring a
certain path/paths and then synchronizing them in the next session. If you
are thinking of synchronizing between more than 2 locations, you need to
plan the fastest route (ie. A<->B<->C or B<->A<->C). I have had no problems
synchronizing between more than two locations.

For all of my implementations using Fedora or CentOS operating systems I
have downloaded the unison source, installed the ocaml compiler (easy to
find for centos/fedora use google) and compiled from source. I have never
had a problem.

Another benefit to the implementation that I have been using is the fact
that employees in each office can work on files on the server as long as the
internal network is functioning, regardless if the synchronization link /
externel connection is working. The synchronization is basically transparent
to the user, and if it happens to go down it will pick up where it left off
as soon as it can regain a connection.

Let me know if you require any further information.


A. Jonathan Darton, B.COMM CIS
System Consultant