File Synchronization With Rsync(2)

File Synchronization With Rsync - Temporary Insanity
(Page 4 of 9 )

The first (and most basic) thing you can do with rsync involves using it as a more intelligent copy command on your local system. So, let's suppose that I have the following directory tree in my home area on my Linux box,
[[email protected]] $ tree -d /home/me
|-- Desktop
| |-- Autostart
| |-- Templates
| `-- Trash
|-- bin
|-- mail
|-- public_html
| |-- cache
| `-- chatserver
| |-- chat
| | |-- admin
| | |-- config
| | |-- images
| | | |-- admin
| | | |-- smilies
| | | `-- tutorials
| | |-- install
| | | |-- database
| | | `-- languages
| | |-- lib
| | | |-- commands
| | | `-- database
| | `-- localization
| | `-- english
| `-- docs
|-- test
`-- tmp
27 directories
and let's further suppose that I need to copy my personal Web pages to a backup folder ("/mnt/zip/backup") on the same system.
[[email protected]] $ ls -l /mnt/zip/backup
total 0
With rsync, accomplishing this is a snap:
[[email protected]] $ rsync --verbose --stats --recursive public_html
/mnt/zip/backup/Number of files: 177Number of files transferred: 158Total file size: 1043209 bytesTotal transferred file size: 1043209 bytesLiteral data: 0 bytesMatched data: 1043209 bytesFile list size: 3453Total bytes written: 15469Total bytes read: 12018
As with a regular "cp" command, rsync needs to know the source and destination for the copy operation. Once it has this information, it compares the files in the two locations and updates the destination to an exact replica of the source. Let's see if it worked as advertised:
[[email protected]] $ ls /mnt/zip/backup/
Yup, it did - as you can see, my backup folder now contains a copy of my Web pages.

You can specify more than one source directory to be copied as well - let me add my "mail" directory to the list and run the command again.
[[email protected]] $ rsync --verbose --stats --recursive public_html mail
/mnt/zip/backup/Number of files: 181Number of files transferred: 161Total file size: 1043209 bytesTotal transferred file size: 1043209 bytesLiteral data: 0 bytesMatched data: 1043209 bytesFile list size: 3513Total bytes written: 15637Total bytes read: 12066wrote 15637 bytes read 12066 bytes 18468.67 bytes/sectotal size is 1043209 speedup is 37.66
Next, let's make a few changes to the original files, and see if rsync can detect them and selectively update the destination the next time I sync up.
[[email protected]] $ touch public_html/a.dat
[[email protected]] $ ls > public_html/a.dat[[email protected]] $ touch public_html/b.dat[[email protected]] $ rm public_html/cache/test.html[[email protected]] $ vi public_html/error.php
As you can see, I've added a couple of new files, deleted one old file and made changes to one PHP script. Let's sync up again and see what happens.
[[email protected]] $  rsync --verbose --stats --recursive public_html
Pretty cool, huh? rsync added the two extra files to my backup, and identified and copied the modified file as well. However, the single file I deleted from the source is still present in the backup - obviously, rsync didn't delete it.

A quick look at the rsync documentation clears that one up - by default, rsync doesn't delete files from the destination when synchronizing directories. This default behaviour can be overridden by adding the "--delete" parameter to the rsync command line.
[[email protected]] $ rsync --verbose --stats --recursive --delete public_html
And now my destination is an exact copy of my source.
[[email protected]] $ ls /mnt/zip/backup/public_html/cache/
index.php m2_h.gif m2_n.gif tmp.gif
It should be noted that the "--delete" option can cause substantial damage if used unwisely - the rsync manual suggests always performing a dry run first when using this option.
File Synchronization With Rsync - Remote Control
(Page 5 of 9 )

So that takes care of local sync - now how about remote sync?

In order to synchronize files between two hosts, it's necessary to run rsync in daemon (server) mode on one of the hosts. When running in this mode, rsync exposes a list of directories on the server; any remote host running rsync can then connect to this server and copy files to or from it.

Before you can run rsync in daemon mode, you need to configure it. This is accomplished via a configuration file, usually "/etc/rsyncd.conf" (although you can specify a different file as well, via the "--config" command-line argument). Here's an example:
 log file = /var/log/rsyncd.log
path = /home/me
comment = My Home Area
list = yes
read only = no
As you can see, this configuration file is similar to a standard Windows INI file, in that it is broken up into different sections or "modules", each containing variable-value pairs. Modules are identified by square braces around the module name, and lines beginning with semi-colons (;) or hashes (#) are treated as comments and ignored.

The first part of the file sets up global variables for rsync to use - in this case, it specifies the log file for rsync to use. It's also possible to specify, in this section, a welcome message that is displayed when a client attempts to connect to the server.
log file = /var/log/rsyncd.log
motd file = /var/log/message.txt
The second part sets up a module on the server - this is simply a directory that is available to all connecting clients. In this case, I've selected the "/home/me" directory, given it the share name "home" and set it to be writeable by all users.
path = /home/me comment = My Home Area list = yesread only = no
   path = /home/me  
option tells rsync where to locate the module on the server, while the
   list = yes  
option tells it to include the module in the list returned to connecting clients.

By default, modules on the server are not writable - that is, clients cannot upload files to the corresponding directories. This default behaviour cane be corrected via the extra
   read only = no  

Once the configuration file has been saved, it's time to start up rsync in daemon mode.
[[email protected]] $ rsync --daemon  
If the rsync daemon starts up OK, you can attempt to connect to it from another host. Let's assume that this second host is named "xanadu", and already has rsync installed on it. What I'd like to do is transfer my home directory on "olympus" - the same one I backed up on the previous page - to "xanadu". Here's how I'd go about it:
[[email protected]] $  rsync  --verbose --progress --stats --recursive
olympus::home/ .Number of files: 230Number of files transferred: 187Total file size: 1054649 bytesTotal transferred file size: 1054649 bytesLiteral data: 1054649 bytesMatched data: 0 bytesFile list size: 4318Total bytes written: 3052Total bytes read: 1066527wrote 3052 bytes read 1066527 bytes 713052.67 bytes/sectotal size is 1054649 speedup is 0.99[[email protected]] $ lsbin Desktop mail public_html test tmp
Synchronization need not be in one direction only. Using exactly the same setup as above - an rsync server on "olympus" and an rsync client on "xanadu" - it's also possible for me to transfer files in the other direction. Consider the following example, which illustrates by copying the "sql" directory to my home area on "olympus":
[[email protected]] $ rsync --verbose --progress --stats --recursive sql

File Synchronization With Rsync - Doing More
(Page 6 of 9 )

It's possible to obtain a list of all the modules available on the rsync server by omitting the module name from the command line when connecting to the server. Here's an example, and the output:
[[email protected]] $ rsync olympus::
home My Home Area
Now, if I were to add a few more modules to the configuration file,
path = /tmp comment = Temp Arealist = yes
restart the rsync server on "olympus",
[[email protected]] $ killall rsync
[[email protected]] $ rsync --daemon
and attempt to reconnect to it from "xanadu", I'd have access to the new modules as well.
[[email protected]] $ rsync olympus::
home My Home Areatemp Temp Area
You can exclude modules from being listed in this manner by specifying a
list = no  
option within the module configuration.

It's also possible to tell rsync to exclude certain files from the synchronization process, with the "--exclude" command-line option. Here's an example, which copies all the files *except* those with a ".tmp" extension from "xanadu" to "olympus":
[[email protected]] $ rsync --verbose --progress --stats --recursive
--exclude="*.tmp" olympus::home/ .
Finally, rsync's default behaviour when encountering symbolic links is to omit them - as in the following example:
[[email protected]] $ rsync --progress  --recursive  olympus::home/ .          
skipping non-regular file "public_html/config.lib.php3" skippingnon-regular file "public_html/start.php"
As you can see, when I attempt to copy the "/home/me/public_html" directory to "xanadu", every symbolic link within that directory is skipped. You can have rsync retain these links as is during the copy process by specifying the "--links" option on the command line,
[[email protected]] $ rsync --progress  --recursive  --links olympus::home/ .  
or replace the symbolic links by the actual files being referenced with the "--copy-links" option.
[[email protected]] $ rsync --progress  --recursive  --copy-links olympus::home/


บันทึกนี้เขียนที่ GotoKnow โดย  ใน นายก๊อต:รวมแหล่งความรู้

คำสำคัญ (Tags)#network#opensource#computer#it#งานพัฒนา

หมายเลขบันทึก: 23850, เขียน: 13 Apr 2006 @ 17:24 () , แก้ไข, 11 Feb 2012 @ 14:43 (),  | , สัญญาอนุญาต: สงวนสิทธิ์ทุกประการ, อ่าน: คลิก

ความเห็น (0)