WARSYNC v0.9.8

Warsync is the program that initiates replication between the Warsync server and it's clients. Warsync is designed to be called from the command line of the Warsync server to initiate replication on demand. It can also be called from cron on the server to automate replication.

Warsync not only replicates files from the server to the client, but can also synchronize what software packages are installed on a Debian GNU/Linux system.

Warsync determines what files to replicate by processing the /etc/warsync/filepacks.d directory on the sever. This directory contains filepacks which are simple text files that contain lists of the files and directories that should be replicated.


Download

Warsync can be downloaded from the Warsync Alioth project page at http://alioth.debian.org/projects/warsync/

It is available as a Debian package (.deb) and as a source tarball (.tar.gz)


Install

On a Debian GNU/Linux system, the best way to install Warsync is to download the Debian package. Once downloaded it can be installed easily with dpkg.

 dpkg -i warsync_0.9.8_all.deb

If you receive an error about missing dependencies, a quick call to apt-get will fix that up.

 apt-get -f install

Warsync can also be installed manual from the source tarball, although this method has not been tested. A from source installation would go something like this:

 tar zxvf warsync_0.9.8.tar.gz
 cd warsync-0.9.8
 perl Makefile.PL
 make
 make install


Warsync Configuration

First configure the Warsync server by executing the following command on the system that will serve as the Warsync server.

 warsync-config --server

Next configure Warsync clients by executing the following command on each Warsync client.

 warsync-config --client

Back on the Warsync server, execute the following command to fully register each client system with the server.

 warsync-add-client clientname

After this you will need to create filepacks in the /etc/warsync/filepacks.d directory on the Warsync server. Each filepack should contain a list of directories or files to be replicated when the filepack name is specified on the warsync command line.


Filepacks

Filepack should list each file or directory pattern to be replicated one per line. It is best to have several filepacks where each filepack replicates a group of related files and directories. For instance you may have one filepack called homes that will replicate user home directories. You could have another filepack called website that will replicate the files for your website. Another filepack called config could be used to copy configuration or /etc files that you want to keep replicated between server and client.

To match a file, specify the full absolute path to the file. For example, /etc/passwd. Wildcards ? and * may also be specified. ? matches any single character, while * will match zero or more characters. For example, /etc/pa* could match /etc/pam.conf and /etc/passwd.

To replicate a directory and all it's subdirectories, specify two asterisk, ** after the directory name. For example /home/** will replicate everything in the /home directory. If only one asterisk is specified, then only the files directly contained in the directory will be replicated. The subdirectories will not be replicated.

To exclude files from being replicated that would match the wild card in another entry, specify them on line before the include line. It should start with (a minus followed by a space). For example, - /etc/network/interfaces followed by /etc/network/** would replicate all files and subdirectories contained in the /etc/network directory, except the interfaces file.

Filepacks may also contain comments and are signified when the first non-whitespace character on a line is a #.

An example homes filepack could look like:

 # replicate home directories.
 /home/**
 # replicate root's home, but not it's .ssh directory
 # otherwise warsync will break.
 - /root/.ssh
 /root/**

Example website filepack could be as simple as:

 /var/www/**

A config filepack for replicating config files could look like:

 # copy user info
 /etc/passwd
 /etc/shadow
 /etc/group
 /etc/gshadow
 /etc/skel/**
 # apache config
 /etc/apache**
 # ssh config
 - /etc/ssh/ssh_host_dsa_key*
 - /etc/ssh/ssh_host_rsa_key*
 /etc/ssh/**

Some directories that you probably never want to completely replicate any top level directories, especially /boot, /cdrom, /dev, /floppy, /initrd, /lost+found, /mnt, /proc, /tmp, and /var. You also probably shouldn't replicate /bin, /lib, /sbin, or /usr as these directories should be managed by your operating systems package management system. Although replicating /usr/local shouldn't hurt. While you may want to replicate some of the files in /etc, you definitely do NOT want to replicate them all. Also replicating the /etc/warsync directory will break configuration on the clients. The following directories are sensitive to warsync's operation and should not be replicated unless you absolutely know what you are doing:

 /etc/warsync
 /etc/ssh
 /root/.ssh
 /var/log/warsync

Warsync makes no effort to keep you from replicating things you that you shouldn't. A good rule of thumb is to be very explicit about what to replicate and use fewer wildcards rather than more. And if you don't know what a file does, don't replicate it.


Running Warsync

To replicate a single filepack, specify it on the warsync command line like so:

 warsync foobar

The above command will replicate the files and directories specified in /etc/warsync/filepacks.d/foobar file pack. Multiple filepacks can be specified at once to replicate several filepacks one after the other:

 warsync homes website

This would replicate the homes filepack, and then the website file pack.

To replicate all filepacks, specify --all-filepacks on the command line:

 warsync --all-filepacks

You can also use -a for short to replicate all filepacks:

 warsync -a

If you want to test a filepack to see what will be replicated before actually making any real live changes, you can specify --dry-run on the command line:

 warsync --dry-run config

This will simulate replication of the config filepack. You can also use -n for short instead of --dry-run.

 warsync -n foobar

If you want to see more verbose output about what is happening then specify --verbose on the command line:

 warsync --verbose homes

You can also specify -v as a shortcut. To see even more output, you can specify multiple v's like so:

 warsync -vv config

And of course this can be combined with -n to simulate a replication with extra output:

 warsync -vn --all-filepacks

To completely limit the output produced by warsync so that it will only display output if an error occurs, add --quiet to the command line:

 warsync --quiet homes

And of course --quiet can be shortened to just -q as in:

 warsync -q foobar

By default warsync will replicate specified filepacks to all clients. If you only want to replicate to a specific warsync client, you can specify the --client option on the warsync command line;

 warsync --client=web2 website

The above command would replicate the website filepack to only the client named web2. More than one client can be specified at a time by using multiple --client options:

 warsync --client=web2 --client=web9 website

Warsync can also synchronize the software packages installed on Debian GNU/Linux systems. The Debian package synchronization will install, remove, upgrade, and/or downgrade the Debian packages on a Warsync client in order to match the packages installed on the server.

To synchronize the Debian packages installed on the client with those on the server, specify the --debian option on the warsync command line.

 warsync --debian

If --debian is specified along with filepacks, the filepacks will be replicated first before doing the Debian package synchronization.

Warsync can also be used to take a snapshot of installed Debian packages and then later used to ``restore'' back to this state. This comes in handy if you want to install a package that requires several libraries that you don't want to leave installed if you later decide to remove the package. Prior to installing the new package, take a snapshot of installed Debian packages by executing:

 warsync --debian-snapshot=debsnap

This will write the snapshot to the debsnap file in the current directory. Later when you want to restore back to package state, run the command:

 warsync --debian-from-snapshot=debsnap

The debsnap file can also be copied to another system with Warsync installed to install the same packages on another system.


Automating Warsync Via Cron

Warsync can be comfortably run from a crontab for filepack replication. In fact warsync does special handling when it is not run from a tty, such as when run from cron, and will not display any output unless changes are made, or -v has been specified on the command line. In this fashion, when warsync runs from cron, you will not receive a cron email unless changes were actually made on the server. This helps to cut down on the noise level if you want to run warsync periodically through out the day, but only want to receive a notification when a change was made. If you always want to receive output even when no changes were made, specify -v on the warsync command line in the crontab. If you want to receive no output at all, specify -q on the command line. In this case you will only receive an email if there was an error.

It should be noted that running Warsync from cron to do Debian package synchronization has not been extensively tested and should be considered experimental. This is mainly because when a new package is installed or upgraded, you may be prompted to answer configuration questions, which can yield unpredictable results when running from cron.

Because warsync can replicate individual filepacks at a time, it is beneficial to put multiple cron entries that replicate different filepacks throughout the day. For instance, you might want to automatically replicate changes to the website every thirty minutes, but only replicate user home directories every two hours. And you may want to do a full replication every night at 11 O'clock. In that case you might have a crontab that looks like this:

 15,45 * * * * /usr/sbin/warsync website
 30 */2 * * * /usr/sbin/warsync homes
 0 23 * * * /usr/sbin/warsync -a

Of course it might get pretty annoying to get emailed about every single change made to users' home directories. To quiet the noise, you'll want to change the second crontab line as shown below.

 30 */2 * * * /usr/sbin/warsync -q homes


About the Author

Paul Baker has been developing applications in Perl since 1997. He also maintained the popular ClanRing Quake Mod <http://planetquake.com/crmod/> from 1997 to 1999. Paul has been a full time Perl developer and sysadmin since 2000. In 2001, Paul began the PFARS <http://pfars.sourceforge.net/> project on SourceForge which aimed to be a useable cluster/server replication system. Warsync was started by Paul in 2004 as a complete rewrite of PFARS from the ground up.

Special Thanks to Where2GetIt <http://where2getit.com/> for employing Paul Baker and paying his bills while working on Warsync and PFARS. Thanks go to Steve Johnson for helping to test Warsync. Thanks to Vijay Avarachen for writing the HOWTO for PFARS that I never got around to including on the PFARS website and also for the idea doing Debian package version synronization. And finally thanks to Kurt Stephens for coming up with the name Warsync which is so many times better than PFARS.


Copyright

Warsync - Wrapper Around Rsync v0.9.8

Copyright (C) 2001-2004 Paul J. Baker

This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License version 2 as published by the Free Software Foundation.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.

You may contact the author via electronic mail at pbaker -at- where2getit.com.

$Revision: 1.5 $ $Date: 2004/05/29 00:12:56 $