Use Rsync to Backup Local and Remote Directories on Linux

Updated : Aug 20, 2016 in Coding

How To Use Rsync to Backup Local and Remote Directories on Linux

Use Rsync to Backup Local and Remote Directories on Linux
Recently while working on a project i had a requirement to regularly take backup of our source file folder to some other location for archival and Rsync utility came handy here. In this post i am going to explain how we can use Rsync to Sync or Backup Local and Remote Directories.

What Is Rsync?

Rsync is a utility that keeps copies of a file on two computer systems. It is commonly found on Unix-like systems and functions as both a file synchronization and file transfer program. It is famous for its delta-transfer algorithm, which reduces the amount of data sent over the network by sending only the differences between the source files and the existing files in the destination. Rsync is widely used for backups and mirroring and as an improved copy command for everyday use and due to its popularity its included in Most Linux distribution.

Rsync Syntax

Below is the basic syntax for Rsync. Here source is the source directory which you want to Sync or Backup and Destination is the target directory where files will be copied.

rsync options source destination

Rsync provides several options which we can use based on our requirement.

  • -r : If your source directory contains sub directory and you want to backup all files then use -r option. It will look for the changes files in all sub directory recursively however it does not maintain source file timestamp.
  • -v : Use this option to see the detailed information from Rsync
  • -a :Use this option to copy file in archive mode recursively. It preserves symlinks, permissions, ownerships and timestamps
  • -z : Use this option to compress the data while transfer
  • -h : To see the output in human readable format, use -h option
  • -n : If you want to perform a trial run with no changes made then use -n option

Local Backup Using Rsync

Now you know the syntax for Rsync, lets see how we can setup the local backup from source folder to archive folder. For this i am going to create two directories souce_file and archive

$ mkdir source_file
$ mkdir archive
$ ls
archive  source_file

Now we have created source and archive directory, lets create some dummy files in source_file directory.

$ cd source_file
$ touch sourc_file{1..20}
$ ls -ltr source_file/
total 0
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:16 sourc_file9
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:16 sourc_file8
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:16 sourc_file7
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:16 sourc_file6
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:16 sourc_file5
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:16 sourc_file4
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:16 sourc_file3
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:16 sourc_file20
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:16 sourc_file2
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:16 sourc_file19
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:16 sourc_file18
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:16 sourc_file17
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:16 sourc_file16
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:16 sourc_file15
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:16 sourc_file14
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:16 sourc_file13
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:16 sourc_file12
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:16 sourc_file11
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:16 sourc_file10
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:16 sourc_file1

Using touch command we have created 20 dummy files in source directory. Now lets run the Rsync for local backup.

$ rsync -rv source_file/ archive
sending incremental file list
sourc_file1
sourc_file10
sourc_file11
sourc_file12
sourc_file13
sourc_file14
sourc_file15
sourc_file16
sourc_file17
sourc_file18
sourc_file19
sourc_file2
sourc_file20
sourc_file3
sourc_file4
sourc_file5
sourc_file6
sourc_file7
sourc_file8
sourc_file9

sent 963 bytes  received 392 bytes  2710.00 bytes/sec
total size is 0  speedup is 0.00

$ ls -ltr archive/
total 0
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:25 sourc_file9
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:25 sourc_file8
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:25 sourc_file7
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:25 sourc_file6
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:25 sourc_file5
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:25 sourc_file4
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:25 sourc_file3
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:25 sourc_file20
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:25 sourc_file2
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:25 sourc_file19
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:25 sourc_file18
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:25 sourc_file17
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:25 sourc_file16
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:25 sourc_file15
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:25 sourc_file14
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:25 sourc_file13
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:25 sourc_file12
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:25 sourc_file11
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:25 sourc_file10
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:25 sourc_file1

In above example you can see that i have used Rsync and it send 20 files from source_file directory to archive. Here we are able to see the detailed information because we are using -v option, if you remove -v then you will not get any information on unix prompt. Here i have used -r option for copying from source to target directory. If you see the source and target file timestamp then you might have noticed they are not same, its because

Using -r option in Rsync does not maintain the source file timestamp

Now lets try using -a option in archive mode which preserves symlinks, permissions, ownerships and timestamps.

$ rsync -av source_file/ archive
sending incremental file list
./
sourc_file1
sourc_file10
sourc_file11
sourc_file12
sourc_file13
sourc_file14
sourc_file15
sourc_file16
sourc_file17
sourc_file18
sourc_file19
sourc_file2
sourc_file20
sourc_file3
sourc_file4
sourc_file5
sourc_file6
sourc_file7
sourc_file8
sourc_file9

sent 983 bytes  received 395 bytes  2756.00 bytes/sec
total size is 0  speedup is 0.00

$ ls -ltr source_file/
total 0
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:16 sourc_file9
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:16 sourc_file8
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:16 sourc_file7
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:16 sourc_file6
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:16 sourc_file5
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:16 sourc_file4
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:16 sourc_file3
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:16 sourc_file20
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:16 sourc_file2
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:16 sourc_file19
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:16 sourc_file18
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:16 sourc_file17
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:16 sourc_file16
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:16 sourc_file15
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:16 sourc_file14
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:16 sourc_file13
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:16 sourc_file12
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:16 sourc_file11
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:16 sourc_file10
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:16 sourc_file1
$ ls -ltr archive/
total 0
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:16 sourc_file9
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:16 sourc_file8
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:16 sourc_file7
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:16 sourc_file6
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:16 sourc_file5
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:16 sourc_file4
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:16 sourc_file3
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:16 sourc_file20
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:16 sourc_file2
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:16 sourc_file19
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:16 sourc_file18
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:16 sourc_file17
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:16 sourc_file16
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:16 sourc_file15
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:16 sourc_file14
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:16 sourc_file13
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:16 sourc_file12
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:16 sourc_file11
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:16 sourc_file10
-rw-rw-r-- 1 reaper reaper 0 Aug 20 03:16 sourc_file1

Now you can clearly see that both source and archived files have same timestamps.

Now lets change modify one file and rerun the Rsync

$ cat > sourc_file1
Just adding some content to empty file

$ rsync -av source_file/ archive
sending incremental file list
sourc_file1

sent 290 bytes received 31 bytes 642.00 bytes/sec
total size is 39 speedup is 0.12

Since we made changes to only sourc_file1 Rsync only copied this file to archival.

How to Dry Run Rsync

In above examples we have directly run the Rsync command for archival, but its always better to check before actual run. Rsync -n option come handy here, it will simulate the actual run without copying source files to directory, so that you can verify.

$ cat > sourc_file2
making changes to check rsync -n option
$ cat sourc_file2
making changes to check rsync -n option

$ rsync -anv source_file/ archive
sending incremental file list
sourc_file2

sent 211 bytes received 15 bytes 452.00 bytes/sec
total size is 79 speedup is 0.35 (DRY RUN)

You can see the Dry Run detail in the unix prompt, this way you can ensure what you running is correct.

Remote Backup Using Rsync

We have setup the local backup, let see  how we can use Rsync to remote backup when your backup location is present on different server. We can backup from our local source_file directory to our Backup server using below command syntax.

rsync -azP /source_file [email protected]_host:destination_directory

Here first argument will be your source and second argument will be target. Using -a option ensure that we are not loosing any metadata of original file. Since we are transferring to remote server its always better to compress the source file before transfer, and that is why we used -z option.  Using -P option will give you the progress details of the file transfer.

Please not that the above syntax will copy from local to remote directory, in case you need to copy from remote to local using Rsync then follow below Rsync command syntax.

rsync -azP [email protected]_host:/home/reaper/source_file place_to_sync_on_local_machine

Automated Backup Using Rsync

So now we know how to use Rsync for archival or backup. In my case everyday i have to take the backup of source file folder at the end of the day. We had a shell script which we used to run for executing our job using Scheduler, i added the Rsync command with proper directory and option to the script. And now everyday we have a automated backup in case any new file comes or file is modified.

You can also create a shell script with Rsync command and if you want to automate it on certain time, you can use crontab which allow you to schedule any script at certain time interval.

So these are the steps you can follow to Backup Local and Remote Directories on Linux using Rsync. For more option you can see the man rsync . 

I have tried to give a quick explanation about Rsync utility. Please feel free to share the post and also give your opinion and feedback in the comment.

Leave a Reply

Your email address will not be published. Required fields are marked *