How to download only newer files with wget

wget supports conditional transfers that skip files already present and unchanged on the local filesystem. It compares the server-supplied Last-Modified value with the local modification timestamp and leaves identical files untouched.

Enable the behaviour with the --timestamping (-N) option, which sends an If-Modified-Since header, preserves server timestamps on successful downloads, and logs “Not Modified” when nothing new is retrieved. Accurate system time is essential to avoid false positives.

Conditional checks work over HTTP, HTTPS, and FTP. Combine timestamping with recursive, no-parent, and continue in scheduled jobs to build fault-tolerant mirrors; if a server omits modification data, every request falls back to a full transfer.

1. Verify that wget is installed on the system.

   $ wget --version
   GNU Wget 1.21.4 built on linux-gnu

2. Change to the directory where updated files should be stored.

   $ cd ~/Downloads/updates

3. Download a remote file only if it is newer by adding the --timestamping (-N) flag.

   $ wget --timestamping https://example.com/archive.tar.gz
   HTTP request sent, awaiting response… 304 Not Modified

   If the server returns “304 Not Modified”, no data is transferred and the exit status remains 0.

4. Mirror an entire directory while skipping unchanged files by combining --recursive (-r), --no-parent, and --timestamping.

   $ wget --recursive --no-parent --timestamping https://example.com/repo/
   ...
   No newer files found.

   --no-parent prevents ascending above the specified directory.

5. Resume an interrupted timestamped transfer safely by adding --continue (-c).

   $ wget --timestamping --continue https://example.com/large.iso
   ...
   Continuing download…