Rsync (Remote Sync) is a most commonly used command for copying and synchronizing files and directories remotely as well as locally in Linux/Unix systems. It provides fast incremental file transfer by transferring only the differences between the source and the destination.
Rsync can be used for mirroring data, incremental backups, copying files between systems and as a replacement for scp, sftp, and cp commands for everyday use.
In this article we will discuss 13 useful rsync command examples in Linux, these examples will help specially Linux beginners to manage their sync, mirroring, and backup task more efficiently.
Installing Rsync
The rsync utility is pre-installed on most Linux distributions. You can check if it is installed on your system by typing:
$ rsync --version
Example output:
rsync version 3.1.2 protocol version 31
Copyright (C) 1996-2015 by Andrew Tridgell, Wayne Davison, and others.
Web site: http://rsync.samba.org/
Capabilities:
64-bit files, 64-bit inums, 64-bit timestamps, 64-bit long ints,
socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace,
append, ACLs, xattrs, iconv, symtimes, prealloc
rsync comes with ABSOLUTELY NO WARRANTY. This is free software, and you
are welcome to redistribute it under certain conditions. See the GNU
General Public Licence for details.
If you don’t have rsync installed on your system, you can easily install it using your distribution’s package manager.
Install Rsync on Ubuntu and Debian
$ sudo apt install rsync -y
Install Rsync on CentOS and Fedora
$ sudo yum install rsync
Rsync Command Syntax
Before going into how to use the rsync command, let’s start by reviewing the basic syntax.
Local Sync: # rsync {options} {Source} {Destination}
Remote Sync pull: # rsync {options} <User_Name>@<Remote-Host>:<Source-File-Dir> <Destination>
Remote Sync Push: # rsync <Options> <Source-Files-Dir> <User_Name>@<Remote-Host>:<Destination>
Some of the commonly used options in rsync command are listed below:
- -v, –verbose Verbose output
- -q, –quiet suppress message output
- -a, –archive archive files and directory while synchronizing ( -a equal to following options -rlptgoD)
- -r, –recursive sync files and directories recursively
- -b, –backup take the backup during synchronization
- -u, –update don’t copy the files from source to destination if destination files are newer
- -l, –links copy symlinks as symlinks during the sync
- -n, –dry-run perform a trial run without synchronization
- -e, –rsh=COMMAND mention the remote shell to use in rsync
- -z, –compress compress file data during the transfer
- -h, –human-readable display the output numbers in a human-readable format
- –progress show the sync progress during transfer
Let’s jump into the useful examples of rsync command.
[ads]
1. Copy/Sync files and directory locally
Copy/Sync a File on a Local Computer
This following command will sync a single file on a local machine from one location to another location. For example, a file name backup.tar needs to be copied or synced to /tmp/backups/ folder.
$ rsync -zvh backup.tar /tmp/backups/
Example output:
created directory /tmp/backups backup.tar sent 14.71M bytes received 31 bytes 3.27M bytes/sec total size is 16.18M speedup is 1.10
In above we have used the options like -z for compression, -v for verbose output and -h for human readable output, -a for archived.
In above example, you can see that if the destination is not already exists rsync will create a directory automatically for destination.
Copy/Sync a Directory on Local Computer (rsync -zavh)
Let’s assume we want to copy or sync rasho user’s home directory to /opt/backup folder, execute the below rsync command with options -zavh:
$ rsync -zavh /home/rasho /opt/backup
Example output:
sending incremental file list rasho/ rasho/.bash_logout rasho/.bash_profile rasho/.bashrc rasho/OpenStack-Networking.pdf sent 4.09M bytes received 96 bytes 8.19M bytes/sec total size is 6.15M speedup is 1.50
2. Copy files & directories recursively locally (rsync -zrvh or rsync -zavh)
Let’s assume we have multiple files and directories inside rasho user home directory, use below rsync command to copy files and directories recursively, either use -a or -r option to copy files and directories recursively.
Note : In rsync command -a option is used for archiving during the copy or sync and apart from archiving -a option is also used for followings:
- recursively copy files and directory
- copy symlinks as symlinks
- preserve permissions
- preserve group
- preserve modification time
- preserve ownership
$ rsync -zrvh /home/rasho /opt/backup or $ rsync -zavh /home/rasho/test /opt/backup
3. Copy a Directory from Local Server to a Remote Server
Let’s suppose we want to copy the folder “/home/rasho/test” from local machine to remote machine (192.168.1.88) under the /opt
$ rsync -zarvh /home/rasho/test root@192.168.1.88:/opt root@192.168.1.88's password: ………………………………………………………… test/OTRS/ test/OTRS/Database-Selection-OTRS-Installation-CentOS7.jpg test/OTRS/Install-OTRS-Accept-License.png test/OTRS/Install-OTRS-CentOS7-RHEL7.jpg test/OTRS/mysql-secure-installation-centos7-part2.jpg sent 34.85M bytes received 2.65K bytes 6.34M bytes/sec total size is 37.77M speedup is 1.08 ……………………………………………………………
4. Copy/Sync a Remote Directory to a Local Machine
This command will help you sync a remote directory to a local directory. Here in this example, a directory /home/remoteuser/rpmpkgs which is on a remote server is being copied in your local computer in /tmp/myrpms.
$ rsync -avzh root@192.168.1.88:/home/remoteuser/rpmpkgs /tmp/myrpms root@192.168.1.88's password: receiving incremental file list created directory /tmp/myrpms rpmpkgs/ rpmpkgs/httpd-2.2.3-82.el5.centos.i386.rpm rpmpkgs/mod_ssl-2.2.3-82.el5.centos.i386.rpm rpmpkgs/nagios-3.5.0.tar.gz rpmpkgs/nagios-plugins-1.4.16.tar.gz sent 91 bytes received 4.99M bytes 322.16K bytes/sec total size is 4.99M speedup is 1.00
5. Specify remote shell during synchronization (rsync -e ssh)
In rsync command we can specify the remote shell like ssh during synchronization, let’s assume we want to use secure communication between source to destination machine via rsync then we should use ssh as remote shell, example is shown below:
$ rsync -zarvh -e ssh root@192.168.1.88:/home/remoteuser/rpmpkgs /tmp root@192.168.1.88's password: receiving incremental file list ……………………………………………………………………………… rpmpkgs/ rpmpkgs/ rpmpkgs/httpd-2.2.3-82.el5.centos.i386.rpm rpmpkgs/mod_ssl-2.2.3-82.el5.centos.i386.rpm rpmpkgs/nagios-3.5.0.tar.gz rpmpkgs/nagios-plugins-1.4.16.tar.gz sent 91 bytes received 4.99M bytes 322.16K bytes/sec total size is 4.99M speedup is 1.0
[ads]
6. Show Progress While Transferring Data with rsync
To show the progress while transferring the data from one machine to a different machine, we can use –progress option for it. It displays the files and the time remaining to complete the transfer.
rsync -avh --progress root@192.168.1.88:/opt/rpmpkgs /tmp
root@192.168.1.88's password:
receiving incremental file list
……………………………………………………………………………………………………..
rpmpkgs/
rpmpkgs/httpd-2.4.6-88.el7.centos.x86_64.rpm
2.84M 100% 35.22MB/s 0:00:00 (xfr#6, to-chk=18/25)
rpmpkgs/httpd-tools-2.4.6-88.el7.centos.x86_64.rpm
92.50K 100% 1.13MB/s 0:00:00 (xfr#7, to-chk=17/25)
rpmpkgs/postfix-2.10.1-7.el7.x86_64.rpm
2.56M 100% 14.44MB/s 0:00:00 (xfr#17, to-chk=7/25)
.........................................
sent 484 bytes received 16.38M bytes 3.64M bytes/sec
total size is 16.37M speedup is 1.00
7. Use of –include and –exclude Options
There are some situations where we want to copy or sync files & directories of specific type and want to exclude files or directories of specific type. Rsync command supports both include and exclude options.
In the below example want to copy the files of type pdf and rpm and exclude png file types.
rsync -avz -e ssh --include '*.pdf *.rpm' --exclude '*.png' /home/rasho/test root@192.168.1.88:/opt root@192.168.1.88's password: sending incremental file list test/ test/OpenStack-Networking.pdf test/httpd-2.4.6-88.el7.centos.x86_64.rpm test/httpd-tools-2.4.6-88.el7.centos.x86_64.rpm test/postfix-2.10.1-7.el7.x86_64.rpm sent 9,469,912 bytes received 96 bytes 2,705,716.57 bytes/sec total size is 11,647,907 speedup is 1.23
8. Sync only the directory structure
There are some scenarios where we want to copy the directory structure skiping files only from local machine to remote or vice versa.
Let’s suppose we want to copy the directory structure of “/home/rasho” from local machine to remote machine (192.168.1.88) under /opt folder.
$ rsync -av -f"+ */" -f"- *" /home/rasho root@192.168.1.88:/opt/ root@192.168.1.88's password: building file list ... done ……………………………………………… rasho/ sent 43 bytes received 19 bytes 17.71 bytes/sec total size is 0 speedup is 0.00
9. Use of –delete Option
If you have already synced files from source to destination and from source you have deleted the files then you can force rsync command to delete the files on destination using the –delete option, example is shown below
$ rsync -avz --delete /opt/rpmpkgs root@192.168.1.88:/tmp/rpmpkgs root@192.168.1.88's password: sending incremental file list deleting rpmpkgs/apr-util-1.5.2-6.el7.x86_64.rpm deleting rpmpkgs/apr-1.4.8-3.el7_4.1.x86_64.rpm rpmpkgs/ sent 862 bytes received 105 bytes 276.29 bytes/sec total size is 15,947,152 speedup is 16,491.37
10. Set the max size of files to be transferred
You can specify the Max file size to be transferred or sync. You can do it with –max-size option.
Here in this example, Max file size is 200k, so this command will transfer only those files which are equal or smaller than 200k.
# rsync -avzhe ssh --max-size='200k' /var/lib/rpm/ root@192.168.1.88:/root/tmprpm root@192.168.1.88's password: sending incremental file list created directory /root/tmprpm./ Conflictname Group Installtid ................ sent 189.79K bytes received 224 bytes 13.10K bytes/sec total size is 38.08M speedup is 200.43
11. Automatically delete source files after successful transfer
Now, suppose you have a main web server and a data backup server, you created a daily backup and synced it with your backup server, now you don’t want to keep that local copy of backup in your web server.
So, will you wait for transfer to complete and then delete those local backup file manually? Of Course NO. This automatic deletion can be done using –remove-source-files option.
$rsync --remove-source-files -zvh backup.tar /tmp/backups/backup.tar sent 14.71M bytes received 31 bytes 4.20M bytes/sec total size is 16.18M speedup is 1.10 $ ll backup.tar ls: backup.tar: No such file or directory
12. Set Bandwidth Limit and Transfer File
You can set the bandwidth limit while transferring data from one machine to another machine with the the help of –bwlimit option. This options helps us to limit I/O bandwidth.
# rsync --bwlimit=100 -avzhe ssh /var/lib/rpm/ root@192.168.1.88:/root/tmprpm/ root@192.168.1.88's password: sending incremental file list sent 324 bytes received 12 bytes 61.09 bytes/sec total size is 38.08M speedup is 113347.05
13. View the difference in files & directories between source and destination
Use -i option in rsync command to list the difference in files and directories between source and destination. Example is shown below
$ rsync -avzi /home/pkumar/test root@192.168.1.88:/opt root@192.168.1.88's password: sending incremental file list .d..t...... test/ <f.st...... techi/projects.txt sent 438 bytes received 45 bytes 138.00 bytes/sec total size is 11,648,064 speedup is 24,116.07
As per above command output, there is difference in file called “projects.txt” on destination. Following are the meaning of the keywords in above output,
- d: indicates change in destination file
- f: indicates a file
- t: indicates change in timestamps
- s: indicates change in size
That’s all with rsync now, you can see man pages for more options. Stay connected with Lintutfor more exciting and interesting tutorials in future. Do leave your comments and suggestions.