rsync + cron: incremental synchronization between servers

Introduction

rsync is the reference Unix tool for syncing files between two locations (local↔local, local↔remote, remote↔remote). Its strengths:

• Incremental: only transfers what changed
• Delta algorithm: computes differences down to the byte
• Preserves permissions, timestamps, symlinks
• SSH-friendly: secures transfers between servers
• Battle-tested: present for 25+ years, ultra-reliable

Typical use cases:

• Copy /var/www from web VPS to backup VPS every hour
• Sync /home to a backup NAS
• Replicate media between CDN servers
• Push a static site to production server

Prerequisites

• 2 Linux servers (source + destination)
• Active SSH and public key pre-installed on destination
• rsync installed (present everywhere by default)

Step 1: Verify rsync

rsync --version

If missing:

sudo apt install -y rsync

To install on both servers (ideally same version).

Step 2: First sync

General syntax

rsync [options] source destination

Local → remote sync

rsync -avz /var/www/site/ user@remote-server:/var/www/site/

Decomposed:

• -a: archive mode (preserves permissions, timestamps, symlinks, etc.)
• -v: verbose
• -z: compression during transfer (saves bandwidth)
• source/ (with trailing slash): copies the directory content
• source (no slash): copies the directory itself

⚠️ Watch the trailing slash: it's the #1 source of rsync errors.

# These 2 commands have DIFFERENT results
rsync -av /var/www/site/ /backup/    # copies content of site/ into /backup/
rsync -av /var/www/site  /backup/    # copies into /backup/site/...

Remote → local sync

rsync -avz user@remote-server:/var/www/site/ /local-backup/

Step 3: Essential options

# Test mode (dry run) - simulates without transferring
rsync -avzn source/ destination/

# Delete on destination what no longer exists in source
rsync -avz --delete source/ destination/

# Progress bar for big transfers
rsync -avzP source/ destination/   # --progress + --partial

# Bandwidth limit (KB/s)
rsync -avz --bwlimit=10000 source/ destination/   # 10 MB/s

# Exclusions
rsync -avz --exclude='*.log' --exclude='cache/' source/ destination/

# Exclusions from file
rsync -avz --exclude-from='/etc/rsync-exclude.txt' source/ destination/

# Preserve hardlinks
rsync -avzH source/ destination/

# Specify non-standard SSH port
rsync -avz -e "ssh -p 22022" source/ user@host:dest/

Step 4: Reusable exclusion file

sudo nano /etc/rsync-exclude.txt

# Caches
*.cache
*~
.DS_Store
Thumbs.db
__pycache__/
node_modules/
.git/

# Logs
*.log
*.gz
*.bz2

# Folders to exclude
tmp/
cache/
sessions/

Usage:

rsync -avz --exclude-from=/etc/rsync-exclude.txt /var/www/ user@dst:/var/www/

Step 5: Sync over SSH with key

Verify SSH key works:

ssh user@remote-server "echo OK"
# Must return OK without asking for password

If SSH asks for password, see SSH key configuration guide on verycloud.fr.

For cron, always use a dedicated SSH key without passphrase (otherwise cron can't authenticate).

# On source
sudo ssh-keygen -t ed25519 -N "" -f /root/.ssh/id_rsync

Copy public key to destination:

sudo ssh-copy-id -i /root/.ssh/id_rsync.pub user@remote-server

Step 6: Restrict key to rsync only (security)

On destination, in ~/.ssh/authorized_keys, prefix the key:

command="rsync --server -vlogDtprze.iLsfxC . /backup/",restrict ssh-ed25519 AAAA... rsync@source

This key can only run rsync to /backup/, nothing else. If someone steals the key, they can't get a shell.

To generate the exact command rsync expects:

# On source, add -e "ssh -v" to see the launched command
rsync -avz -e "ssh -v -i /root/.ssh/id_rsync" /backup/ user@host:/backup/

Look at Sending command:. Copy-paste into command="...".

Step 7: Robust sync script

sudo nano /usr/local/bin/sync-prod-to-backup.sh

#!/bin/bash
set -e  # stop on error

# Configuration
SOURCE_DIR="/var/www/"
DEST_HOST="backup-vps"
DEST_USER="rsync"
DEST_DIR="/backup/www/"
SSH_KEY="/root/.ssh/id_rsync"
SSH_PORT="22"
LOG_FILE="/var/log/rsync-sync.log"
EXCLUDE_FILE="/etc/rsync-exclude.txt"

# Logging
exec >> "$LOG_FILE" 2>&1
echo "========================================"
echo "Sync started: $(date)"
echo "========================================"

# Lock to avoid overlaps
LOCKFILE="/tmp/rsync-sync.lock"
exec 200>$LOCKFILE
if ! flock -n 200; then
    echo "Sync already in progress, skip"
    exit 1
fi

# Sync
rsync -avz \
    --delete \
    --exclude-from="$EXCLUDE_FILE" \
    --bwlimit=20000 \
    -e "ssh -p $SSH_PORT -i $SSH_KEY -o StrictHostKeyChecking=accept-new" \
    "$SOURCE_DIR" \
    "$DEST_USER@$DEST_HOST:$DEST_DIR"

EXIT_CODE=$?

if [ $EXIT_CODE -eq 0 ]; then
    echo "Sync OK: $(date)"
else
    echo "Sync FAIL (exit $EXIT_CODE): $(date)"
    # Discord notification on failure
    curl -s -X POST -H "Content-Type: application/json" \
        -d "{\"content\": \"❌ Sync $SOURCE_DIR failed (exit $EXIT_CODE)\"}" \
        "https://discord.com/api/webhooks/YOUR_WEBHOOK"
fi

echo ""
exit $EXIT_CODE

sudo chmod +x /usr/local/bin/sync-prod-to-backup.sh

Test manually:

sudo /usr/local/bin/sync-prod-to-backup.sh

Check log:

sudo tail -50 /var/log/rsync-sync.log

Step 8: Schedule via cron

sudo crontab -e

Typical examples:

# Every hour at minute 5
5 * * * * /usr/local/bin/sync-prod-to-backup.sh

# Every 10 minutes
*/10 * * * * /usr/local/bin/sync-prod-to-backup.sh

# Daily at 2 AM
0 2 * * * /usr/local/bin/sync-prod-to-backup.sh

# Mon-Fri at 6 AM
0 6 * * 1-5 /usr/local/bin/sync-prod-to-backup.sh

Step 9: Rotation and versioning (rsync + hard links)

To keep N backup versions without duplicating space, use --link-dest:

#!/bin/bash
TODAY=$(date +%Y-%m-%d)
YESTERDAY=$(date -d "yesterday" +%Y-%m-%d)
BACKUP_ROOT="/backup/www"

# If yesterday's backup exists, use its files as hardlink base
if [ -d "$BACKUP_ROOT/$YESTERDAY" ]; then
    LINK_OPT="--link-dest=$BACKUP_ROOT/$YESTERDAY"
else
    LINK_OPT=""
fi

rsync -avz --delete $LINK_OPT \
    -e "ssh -p 22 -i /root/.ssh/id_rsync" \
    rsync@prod-vps:/var/www/ \
    "$BACKUP_ROOT/$TODAY/"

# Rotation: keep 14 days
find $BACKUP_ROOT -maxdepth 1 -type d -mtime +14 -exec rm -rf {} +

With --link-dest, identical files between 2026-05-15 and 2026-05-16 are hardlinks → single disk space used. You can restore any date.

Step 10: Bidirectional sync?

rsync is unidirectional by nature. For bidirectional, use Unison (but more complex) or Syncthing (zero-config mesh sync).

⚠️ Running rsync both ways via cron is dangerous: conflicts, unintended overwrites. Avoid.

Step 11: Performance optimization

Disable compression on gigabit LAN

-z (compression) costs CPU. On a fast link, it's counterproductive:

rsync -av --no-compress source/ dst/

Use fastest SSH encryption

rsync -av -e "ssh -c [email protected]" source/ user@host:dst/

Sync from LVM snapshot (consistency)

If you sync a DB being written to, you risk inconsistency. LVM snapshot:

sudo lvcreate -L 10G -s -n var-snapshot /dev/vg0/var
sudo mount /dev/vg0/var-snapshot /mnt/snapshot

rsync -av /mnt/snapshot/lib/mysql/ user@host:/backup/mysql/

sudo umount /mnt/snapshot
sudo lvremove -f /dev/vg0/var-snapshot

Parallel rsync (large volumes)

To sync a folder with 100k+ files, running multiple rsync in parallel is faster:

# First listing
ls /var/www | xargs -I {} -P 4 rsync -avz /var/www/{}/ user@host:/var/www/{}/

Step 12: Test and monitoring

Check what would change without doing it

sudo rsync -avzn --delete --exclude-from=/etc/rsync-exclude.txt /var/www/ user@host:/var/www/

Shows all files that would be transferred/deleted. Useful to validate a new config.

Measure time and quantity

time rsync -avz --stats source/ dst/

--stats adds at end:

Total bytes sent: 1.2M
Total bytes received: 154
Speed: 8.5MB/s
File list size: 12345
Reduction: 95% (delta effective)

Alert if sync doesn't run

Combine with healthchecks.io (free):

5 * * * * /usr/local/bin/sync-prod-to-backup.sh && curl -fsS https://hc-ping.com/YOUR-UUID

If sync fails or cron doesn't run on time, healthchecks.io alerts you via email/Discord/Slack.

Troubleshooting

"Permission denied (publickey)"

SSH key not recognized. Check:

sudo ssh -i /root/.ssh/id_rsync user@remote-host echo OK

If asks for password, public key isn't in authorized_keys of destination.

"rsync: failed to set times: Operation not permitted"

Destination user lacks right to modify timestamps. Either use root, or add --no-times to rsync.

Very slow sync

Possible causes:

• Many small files → CPU saturated on file scan
• -z compression on fast link
• Low-bandwidth network link (use higher --bwlimit)

Check throughput:

rsync -avzP source/ dst/ | grep "speed"

"rsync error: protocol incompatibility"

Different versions on source/destination. Update rsync on both sides.

"vanished file" warning

File disappeared (was deleted) during sync. Not critical for temp files. To ignore:

rsync ... 2>/dev/null || true

Useful commands

# See what will be transferred (dry run)
rsync -avzn --stats source/ dst/

# Sync only recent files
rsync -avz --max-age=30 source/ dst/

# Sync to multiple destinations
for host in backup1 backup2 backup3; do
    rsync -avz source/ user@$host:dst/ &
done
wait

# Verify a sync is complete (integrity)
rsync -avc source/ dst/    # -c uses checksum, slower but reliable

# Measure diff with --dry-run + --stats
rsync -avzn --stats source/ dst/ 2>&1 | grep -E "Total|sent|received"

# See running rsync PID
pgrep -af rsync

# Kill a stuck rsync
pkill -f rsync

Conclusion

rsync + cron is the simplest and most reliable solution to sync files between servers. Advantages:

• Zero dependencies (rsync + SSH, everywhere)
• Incremental → light on bandwidth
• Native error recovery
• Hardlinks for versioned rotation without duplication

Going further:

• Restic for encrypted and deduplicated backups (see dedicated guide)
• Syncthing for bidirectional mesh sync
• lsyncd for real-time sync (inotify)
• rclone to sync to cloud (B2, S3, OneDrive, etc.)

Resources

• rsync documentation: https://download.samba.org/pub/rsync/rsync.html
• Advanced examples: https://www.snailbook.com
• VeryCloud guide — Restic B2 backups: /docs/article/restic-b2
• VeryCloud guide — Debian setup: /docs/article/debian-setup