How to do encrypted remote backup using rsync.
Posted by: pazli on Wednesday, March 26, 2003 - 10:51 AM
HOWTOs and Technical This document explains the method I use to implement a periodical backup to my office files and databases. It shows that rsync is a very handy tool for backup. The document features only my rough idea of setting up a backup system using rsync


Figure 1 :First, plan what you are going to back up.


Secure Copy from remote host:


The options that I choose:
- the data from the remote server to be in archive mode
- Rsync operates on SSH as a secure channel, send/receive only the bytes inside files that have changed since the last replication, and remove files on the destination host if those files were deleted on the source host to keep both hosts in sync.

The backup server runs on Openbsd, while the source server that has to be backed up runs on Red Hat Linux. The servers can however run on other operating systems such as Freebsd.

Steps


1.Install the rsync application. RPM or OpenBSD port package is fine. Install on both sides; on your backup server and also on the source server you intend to backup.

2.Install chroot environment.
- Get the latest version of Openssh with chroot support at http://mail.incredimail.com/howto/openssh/prebuilt/
- Uninstall the current openssh.
- Install the new chroot supported openssh.

3.Creating user for rsync. The user created here is rsyncUID. On my redhat:
adduser rsyncUID -g users


4.Run the create_chroot_env.sh script . (Thanks to the script's maker).
I have modified part of the script since I was unable to logon from my backup server. You can find my modified script here.

You can edit the APPS variable in the script to add or remove applications

./create_chroot_env.sh rsyncUID



5.Disable login to the account ( rsyncUID) , put the * at the password column of your password file. (use vipw). This to avoid any direct login to the account.

6.Add the trailing /./ to the end of user home directory.

rsyncUID:*:510:100:rsync user:/home/rsyncUID/./:/bin/bash


Note : All the above steps except no. 1 were executed on servers you want to backup.


7.Generate ssh key agent at the backup server. The key is already available at /home/rsyncUID/.ssh/ Below are the steps to create the key, this is done only once:

su - rsyncUID ( logon to the newly created rsync user)
cd /home/rsyncUID/.ssh/
ssh-keygen -t dsa ( simply press enter for any passphrase)



8.Copy the generated key with the extension *.pub to the server you want to backup.

scp id_dsa.pub [email protected]_Server_to_backup:/tmp


9.At the other server where the source data exists, do the following:

su - rsyncUID
mkdir .ssh
cd .ssh
cat /tmp/id_dsa.pub >> authorized_keys2 (transfer keys from backup server)



10.Test your connection from backup server, to make sure you are not at the top level of root tree.

# ssh -l rsyncUID IP_Source_data_Server
# cd /
# ls -l



11.Now the backup server from internal network can logon direct to rsyncUID on the source server to be backed up without any password, with chroot enabled. This part is very important since we want to schedule backup through crontab.


12.Test the rsync, command should work with no error and files copied to backup server:

At Source Server:

su - rsyncUID
mkdir source
touch source/testfile.txt


From the backup server:

rsync -av --delete -e ssh [email protected]:~/source ~/


You should see something like this:

receiving file list ... done
source/
source/testfile.txt
wrote 32 bytes read 135 bytes 66.80 bytes/sec
total size is 0 speedup is 0.00


This is my simple script to backup document webroot, databases and CVS. Rsync request comes only from the backup server.

$HOME/rsyncUID/script/backup.sh


# $Id: backup.sh,v 1.1.1.1 2003/01/24 12:04:48 pazli Exp $
#
#!/usr/local/bin/bash

echo === Begin Backup Databases on Live Machine `date`
/usr/local/bin/rsync -avz --delete -e ssh [email protected]_OF_THE_SERVER:/home/rsyncUID/database/ /home/rsyncUID/database/

echo
echo ==== Begin Backup CVSROOT `date`
/usr/local/bin/rsync -avz --delete -e ssh [email protected]:/data/cvsroot/ /home/rsyncUID/cvsbackup/cvsroot

/usr/local/bin/rsync -avz --delete -e ssh [email protected]:/data/cvsroot_msm/ /home/rsyncUID/cvsbackup/cvsroot_msm

echo
echo ==== BEGIN Backup Webroot on Live Machine `date`

backuphistoryfolder=~/previous/webroot/old.`date +%Y%m%d-%H%M`
backupfolder=~/webroot/
sourcefolder=/var/www/html/

mkdir -p $backuphistoryfolder

rsync -avrtz --exclude "usage/" --delete --backup
--backup-dir $backuphistoryfolder -e ssh
[email protected]_OF_THE_SERVER:$sourcefolder $backupfolder

# remove only empty folder
rmdir $backuphistoryfolder

echo =======End Backup Live Databases, Live Webroot , CVSROOT ======== ; echo;




13.Enable crontab to schedule backup.

su – rsyncUID
crontab -e


Add the following line to crontab:

# Backup Live Databases, CVSROOT, Webroot on Live
15 23 * * 1-5 ~/script/backup.sh >>/var/log/backup.sh.log 2>&1



14.Enable users to view the previous backup data on backup server through samba services. So later windows users can browse through Windows Network-Neigbourhood to find the files previously backed up.

Parts of my /etc/samba/smb.conf


[netfiles]
comment = netfiles syncronized
path = /home/rsyncUID/netfiles
valid users = pazli user1 user2 user3
public = no
writable = no

[previous]
comment = previous changed files
path = /home/rsyncUID/previous
valid users = pazli user1 user2 user3
public = no
writable = no

[webroot]
comment = Live server webroot
path = /home/rsyncUID/webroot
valid users = pazli user1 user2 user3
public = no
writable = no

Then, restart your samba service.

How is the database backed up?


Mysqldump will backup the database locally, and the rsync secure copy will transfer the data to back server at internal network.

Root crontab to backup database locally:

0 22 * * 1-5 /root/database/backup-database.sh


The script might look like this:

#!/bin/bash

d=`date +%Y-%m-%d_%H%M`

mysqldump -pxxxxxx wikidb > /home/rsyncUID/database/wikidb.sql
mysqldump -pxxxxxx postnuke > /home/rsyncUID/database/postnuke.sql

cd /home/rsyncUID/database
tar czf wikidb.${d}.tar.gz wikidb.sql
tar czf postnuke.${d}.tar.gz postnuke.sql

# Optionally the script will email backup database.
# /usr/bin/uuencode /home/rsyncUID/database/wikidb.${d}.tar.gz wikidb.${d}.tar.gz | /bin/mail -s 'mysqldump: Opensource wikidb' [email protected]

# /usr/bin/uuencode /root/database/postnuke.${d}.tar.gz postnuke.${d}.tar.gz | /bin/mail -s 'mysqldump: Opensource postnuke' [email protected]



How to restore the database from backup?


Example:

Restore database to a new server:


mysqladmin -p create database_name
tar xzvf database.tar.gz
mysql -p database_name < database.sql
mysql -p
GRANT select, insert, update, create, alter, delete, drop ON database_name.* TO [email protected] IDENTIFIED BY 'xxxxxx';
flush privileges;


Backing up Windows->Unix.


1.Share your folder from windows with password
eg. Your Windows Hostname: //leywin
Your shared folder is Data
2.Mount windows shared folder on unix.
Linux to mount windows:

mkdir /mnt/pazliwin98
smbmount //leywin/data /mnt/pazliwin98/


Openbsd to mount windows:
I install shlight from Openbsd Ports as smbmount substitute:

cd /usr/ports/net/sharity-light/
make install
/usr/local/sbin/shlight //leywin/data/ /mnt/pazliwin98/


Add windows hostname to /etc/hosts if the error cannot find host appear.

3.Lastly, schedule you backup with my simple rsync script.


# $Id$
#
#!/usr/local/bin/bash

backuphistoryfolder=~/personalhistory/old.`date +%Y%m%d-%H%M`
backupfolder=~/personal/
sourcefolder=/mnt/pazliwin98/

mkdir -p $backuphistoryfolder

rsync -avz --delete --backup
--backup-dir $backuphistoryfolder $sourcefolder $backupfolder

# remove only empty folder
rmdir $backuphistoryfolder


Acknowledgment:


1.To configure CHROOT
http://mail.incredimail.com/howto/openssh/
2.To backup mysql database
http://www.theprojects.org/tutorials/mysqldump.php
Ref:: http://www.asiaosc.org/article_26.html