How to debug rsync filter rules
An rsync filter rule can hide a parent directory before a later include rule ever evaluates the file expected in the transfer. A dry run with filter debugging shows whether each path is being hidden or shown, and which pattern made that decision, before a live sync changes the destination.
Rsync applies include, exclude, and filter rules while building the file list. A file-level include cannot rescue a path if rsync never descends into its parent directory, so parent-directory includes, specific excludes, and final catch-all excludes need to be tested in the same order the real command uses.
Run the debug pass against a controlled copy of the command or a disposable test tree whenever possible. For local copies and pushes, –debug=FILTER is enough; for remote pulls, pass the option to the remote sender with -M–debug=FILTER so the side deciding what to send reports the rule matches.
Steps to debug rsync filter rules:
- Add –dry-run and –debug=FILTER to the filter rule set being tested.
$ rsync -av --dry-run --debug=FILTER --include='documents/report.txt' --exclude='*' /tmp/source/ /tmp/dest/ sending incremental file list [sender] hiding directory logs because of pattern * [sender] hiding directory documents because of pattern * sent 51 bytes received 12 bytes 126.00 bytes/sec total size is 0 speedup is 0.00 (DRY RUN)
The sample is intentionally missing an include rule for documents/, so rsync cannot descend far enough to evaluate documents/report.txt.
- Check the parent directories before changing the file include.
In the sample output, documents is hidden by the final * pattern. Because rsync does not descend into that directory, documents/report.txt is never evaluated.
- Add an include rule for each parent directory that rsync must enter.
$ rsync -av --dry-run --debug=FILTER --include='documents/' --include='documents/report.txt' --exclude='*' /tmp/source/ /tmp/dest/ sending incremental file list [sender] hiding directory logs because of pattern * [sender] showing directory documents because of pattern documents/ [sender] hiding file documents/draft.tmp because of pattern * [sender] showing file documents/report.txt because of pattern documents/report.txt documents/ documents/report.txt sent 121 bytes received 19 bytes 280.00 bytes/sec total size is 6 speedup is 0.04 (DRY RUN)
Directory include rules normally need a trailing slash, such as documents/, so they match the directory itself instead of a same-named file.
- Move specific excludes before the final catch-all exclude when the output should name the narrower pattern.
$ rsync -av --dry-run --debug=FILTER --include='documents/' --include='documents/report.txt' --exclude='*.tmp' --exclude='*' /tmp/source/ /tmp/dest/ sending incremental file list [sender] hiding directory logs because of pattern * [sender] showing directory documents because of pattern documents/ [sender] hiding file documents/draft.tmp because of pattern *.tmp [sender] showing file documents/report.txt because of pattern documents/report.txt documents/ documents/report.txt sent 121 bytes received 19 bytes 280.00 bytes/sec total size is 6 speedup is 0.04 (DRY RUN)
–include and –exclude are shorthand filter rules. Keep any existing –filter, –include-from, or –exclude-from inputs in the same relative order when adding the debug flags.
- Use remote-side debug output when the source is a remote pull.
$ rsync -av --dry-run -M--debug=FILTER host:/src/ /dest/
Modern rsync does not automatically forward non-default debug flags to the remote side, so -M–debug=FILTER asks the sender to report filter decisions during a pull.
- Repeat the dry run against the real source and destination until the debug lines explain every missing or unexpected path.
- Run the live sync only after the dry run lists the intended files and the hidden paths are matched by the intended patterns.
Remove –dry-run only after the filter output has been reviewed; a live command with –delete can remove destination-only files that the filters no longer protect.