How to debug rsync filter rules

An rsync filter rule can hide a parent directory before a later include rule ever evaluates the file expected in the transfer. A dry run with filter debugging shows whether each path is being hidden or shown, and which pattern made that decision, before a live sync changes the destination.

Rsync applies include, exclude, and filter rules while building the file list. A file-level include cannot rescue a path if rsync never descends into its parent directory, so parent-directory includes, specific excludes, and final catch-all excludes need to be tested in the same order the real command uses.

Run the debug pass against a controlled copy of the command or a disposable test tree whenever possible. For local copies and pushes, –debug=FILTER is enough; for remote pulls, pass the option to the remote sender with -M–debug=FILTER so the side deciding what to send reports the rule matches.

1. Add –dry-run and –debug=FILTER to the filter rule set being tested.

   $ rsync -av --dry-run --debug=FILTER --include='documents/report.txt' --exclude='*' /tmp/source/ /tmp/dest/
   sending incremental file list
   [sender] hiding directory logs because of pattern *
   [sender] hiding directory documents because of pattern *
   
   sent 51 bytes  received 12 bytes  126.00 bytes/sec
   total size is 0  speedup is 0.00 (DRY RUN)

   The sample is intentionally missing an include rule for documents/, so rsync cannot descend far enough to evaluate documents/report.txt.

2. Check the parent directories before changing the file include.

   In the sample output, documents is hidden by the final * pattern. Because rsync does not descend into that directory, documents/report.txt is never evaluated.

3. Add an include rule for each parent directory that rsync must enter.

   $ rsync -av --dry-run --debug=FILTER --include='documents/' --include='documents/report.txt' --exclude='*' /tmp/source/ /tmp/dest/
   sending incremental file list
   [sender] hiding directory logs because of pattern *
   [sender] showing directory documents because of pattern documents/
   [sender] hiding file documents/draft.tmp because of pattern *
   [sender] showing file documents/report.txt because of pattern documents/report.txt
   documents/
   documents/report.txt
   
   sent 121 bytes  received 19 bytes  280.00 bytes/sec
   total size is 6  speedup is 0.04 (DRY RUN)

   Directory include rules normally need a trailing slash, such as documents/, so they match the directory itself instead of a same-named file.

4. Move specific excludes before the final catch-all exclude when the output should name the narrower pattern.

   $ rsync -av --dry-run --debug=FILTER --include='documents/' --include='documents/report.txt' --exclude='*.tmp' --exclude='*' /tmp/source/ /tmp/dest/
   sending incremental file list
   [sender] hiding directory logs because of pattern *
   [sender] showing directory documents because of pattern documents/
   [sender] hiding file documents/draft.tmp because of pattern *.tmp
   [sender] showing file documents/report.txt because of pattern documents/report.txt
   documents/
   documents/report.txt
   
   sent 121 bytes  received 19 bytes  280.00 bytes/sec
   total size is 6  speedup is 0.04 (DRY RUN)

   –include and –exclude are shorthand filter rules. Keep any existing –filter, –include-from, or –exclude-from inputs in the same relative order when adding the debug flags.

5. Use remote-side debug output when the source is a remote pull.

   $ rsync -av --dry-run -M--debug=FILTER host:/src/ /dest/

   Modern rsync does not automatically forward non-default debug flags to the remote side, so -M–debug=FILTER asks the sender to report filter decisions during a pull.

6. Repeat the dry run against the real source and destination until the debug lines explain every missing or unexpected path.
7. Run the live sync only after the dry run lists the intended files and the hidden paths are matched by the intended patterns.

   Remove –dry-run only after the filter output has been reviewed; a live command with –delete can remove destination-only files that the filters no longer protect.

   Related: How to mirror a directory with rsync and delete extra files