rsync is used to sync data from one location to another and is used mainly for sync-copying data from/to remote systems. Rsync is faster than scp since it compares and copies only the changes betwreen source and destination and copies only the changes or difference. If for whatever reason rsync gets interrupted in the middle, running the command again will pick it up from where it was left. In this post we see rsync usage with examples
rsync is similar to scp in terms of syntax. One has to just specify the source and destination with optional arguments.
rsync two directories locally
rsync -avr /SOURCE /DESTINATIOn # -a for preserve the metadata such as permissions, time, symbolic links etc. # -v display verbose output # -r copy recursively all sub-directories and its contents
As seen in the above example, after running rsync command all files have been copied recursively to the destination.
rsync will create the destination directory if it does not already exist. In the below example, we will try the same command without the destination directory.
The above screenshot shows that rsync has created the destination directory ‘DEST’ itself and copied all the contents of ‘SOURCE’ recursively in the destination.
Note: rsync does not create the entire remote path if it does not exist. It only creates the destination directory if it does not exist. If the path does not exist for the destination directory, it will throw the below error.
The presence or absence of trailing slash [/] for the path makes a lot of difference. When the trailing slash is present, rsync copies the contents of the directory instead of the directory itself and vice versa.
In the below example, we remove the trailing slash and run the same command.
The above screenshot shows that, instead of contents of ‘SOURCE’ the directory ‘SOURCE’ itself is copied to the destination directory ‘DEST’ since the trailing slash / is absent.
rsync two directories from local to remote
rsync to a remote system can be done by just adding the destination server’s name to the command.
rsync -a /tmp/SOURCE username@remotehost:/tmp/DEST
Backup with rsync
rsync does an incremental copy by default. This can be used to run an incremental backup.
rsync -avrz /var/www/html rsync -arvz /var/www/html admin@backup-server:/backup/web_server/ # -z compress the files
Exclude specific files or directories from being rsync’d
rsync -arvz --exclude 'tmp*' /var/www/html admin@backup-server:/backup/web_server/ rsync -arvz --exclude 'tmp/jan*' /var/www/html admin@backup-server:/backup/web_server/ rsync -arvz -e ssh --exclude 'tmp/jan*' /var/www/html admin@backup-server:/backup/web_server/ #make rsync to use ssh, this way in case you have public keys exchanged, rsync works password less.
Below is an example to rsync the directory ‘SOURCE’ to destination ‘DEST’ in the same location.
The SOURCE directory has files and sub-directory, as in the below screenshot.
In the below screenshot we rsync contents of SOURCE to DEST but exclude the directory ‘subdir’
Note that ‘subdir’ has been excluded from rsync.
In the below screenshot we rsync contents of SOURCE to DEST but exclude the files of pattern ‘j*’
We see that all files starting with ‘j’ has been excluded.
rsync options - Full list
-v, --verbose increase verbosity --info=FLAGS fine-grained informational verbosity --debug=FLAGS fine-grained debug verbosity --msgs2stderr special output handling for debugging -q, --quiet suppress non-error messages --no-motd suppress daemon-mode MOTD (see manpage caveat) -c, --checksum skip based on checksum, not mod-time & size -a, --archive archive mode; equals -rlptgoD (no -H,-A,-X) --no-OPTION turn off an implied OPTION (e.g. --no-D) -r, --recursive recurse into directories -R, --relative use relative path names --no-implied-dirs don't send implied dirs with --relative -b, --backup make backups (see --suffix & --backup-dir) --backup-dir=DIR make backups into hierarchy based in DIR --suffix=SUFFIX set backup suffix (default ~ w/o --backup-dir) -u, --update skip files that are newer on the receiver --inplace update destination files in-place (SEE MAN PAGE) --append append data onto shorter files --append-verify like --append, but with old data in file checksum -d, --dirs transfer directories without recursing -l, --links copy symlinks as symlinks -L, --copy-links transform symlink into referent file/dir --copy-unsafe-links only "unsafe" symlinks are transformed --safe-links ignore symlinks that point outside the source tree --munge-links munge symlinks to make them safer (but unusable) -k, --copy-dirlinks transform symlink to a dir into referent dir -K, --keep-dirlinks treat symlinked dir on receiver as dir -H, --hard-links preserve hard links -p, --perms preserve permissions -E, --executability preserve the file's executability --chmod=CHMOD affect file and/or directory permissions -A, --acls preserve ACLs (implies --perms) -X, --xattrs preserve extended attributes -o, --owner preserve owner (super-user only) -g, --group preserve group --devices preserve device files (super-user only) --specials preserve special files -D same as --devices --specials -t, --times preserve modification times -O, --omit-dir-times omit directories from --times -J, --omit-link-times omit symlinks from --times --super receiver attempts super-user activities --fake-super store/recover privileged attrs using xattrs -S, --sparse handle sparse files efficiently --preallocate allocate dest files before writing them -n, --dry-run perform a trial run with no changes made -W, --whole-file copy files whole (without delta-xfer algorithm) -x, --one-file-system don't cross filesystem boundaries -B, --block-size=SIZE force a fixed checksum block-size -e, --rsh=COMMAND specify the remote shell to use --rsync-path=PROGRAM specify the rsync to run on the remote machine --existing skip creating new files on receiver --ignore-existing skip updating files that already exist on receiver --remove-source-files sender removes synchronized files (non-dirs) --del an alias for --delete-during --delete delete extraneous files from destination dirs --delete-before receiver deletes before transfer, not during --delete-during receiver deletes during the transfer --delete-delay find deletions during, delete after --delete-after receiver deletes after transfer, not during --delete-excluded also delete excluded files from destination dirs --ignore-missing-args ignore missing source args without error --delete-missing-args delete missing source args from destination --ignore-errors delete even if there are I/O errors --force force deletion of directories even if not empty --max-delete=NUM don't delete more than NUM files --max-size=SIZE don't transfer any file larger than SIZE --min-size=SIZE don't transfer any file smaller than SIZE --partial keep partially transferred files --partial-dir=DIR put a partially transferred file into DIR --delay-updates put all updated files into place at transfer's end -m, --prune-empty-dirs prune empty directory chains from the file-list --numeric-ids don't map uid/gid values by user/group name --usermap=STRING custom username mapping --groupmap=STRING custom groupname mapping --chown=USER:GROUP simple username/groupname mapping --timeout=SECONDS set I/O timeout in seconds --contimeout=SECONDS set daemon connection timeout in seconds -I, --ignore-times don't skip files that match in size and mod-time -M, --remote-option=OPTION send OPTION to the remote side only --size-only skip files that match in size --modify-window=NUM compare mod-times with reduced accuracy -T, --temp-dir=DIR create temporary files in directory DIR -y, --fuzzy find similar file for basis if no dest file --compare-dest=DIR also compare destination files relative to DIR --copy-dest=DIR ... and include copies of unchanged files --link-dest=DIR hardlink to files in DIR when unchanged -z, --compress compress file data during the transfer --compress-level=NUM explicitly set compression level --skip-compress=LIST skip compressing files with a suffix in LIST -C, --cvs-exclude auto-ignore files the same way CVS does -f, --filter=RULE add a file-filtering RULE -F same as --filter='dir-merge /.rsync-filter' repeated: --filter='- .rsync-filter' --exclude=PATTERN exclude files matching PATTERN --exclude-from=FILE read exclude patterns from FILE --include=PATTERN don't exclude files matching PATTERN --include-from=FILE read include patterns from FILE --files-from=FILE read list of source-file names from FILE -0, --from0 all *-from/filter files are delimited by 0s -s, --protect-args no space-splitting; only wildcard special-chars --address=ADDRESS bind address for outgoing socket to daemon --port=PORT specify double-colon alternate port number --sockopts=OPTIONS specify custom TCP options --blocking-io use blocking I/O for the remote shell --stats give some file-transfer stats -8, --8-bit-output leave high-bit chars unescaped in output -h, --human-readable output numbers in a human-readable format --progress show progress during transfer -P same as --partial --progress -i, --itemize-changes output a change-summary for all updates --out-format=FORMAT output updates using the specified FORMAT --log-file=FILE log what we're doing to the specified FILE --log-file-format=FMT log updates using the specified FMT --password-file=FILE read daemon-access password from FILE --list-only list the files instead of copying them --bwlimit=RATE limit socket I/O bandwidth --outbuf=N|L|B set output buffering to None, Line, or Block --write-batch=FILE write a batched update to FILE --only-write-batch=FILE like --write-batch but w/o updating destination --read-batch=FILE read a batched update from FILE --protocol=NUM force an older protocol version to be used --iconv=CONVERT_SPEC request charset conversion of filenames --checksum-seed=NUM set block/file checksum seed (advanced) --noatime do not alter atime when opening source files -4, --ipv4 prefer IPv4 -6, --ipv6 prefer IPv6 --version print version number (-h) --help show this help (-h is --help only if used alone)