How to add wait and random delays between wget requests

Batch wget jobs can reach the same host far more aggressively than an operator using an interactive shell. Adding deliberate spacing between requests reduces burst traffic, keeps scheduled pulls friendlier to shared services, and makes recurring retrievals easier to defend when a site publishes rate expectations.

GNU wget applies a fixed pause with --wait and adds jitter with --random-wait. Current GNU documentation defines that random window as 0.5 to 1.5 times the base wait, so a two-second wait becomes a one-to-three-second gap between retrievals instead of an exact cadence that is easy to fingerprint.

The delay applies between completed retrievals, not in the middle of a single file transfer, and --waitretry solves a different problem by spacing retries after failures. Pick a base wait that matches the remote service policy, record one run to a log, and confirm the timestamps before the command becomes part of unattended automation.

1. Create a plain-text URL list so one pacing policy applies to every retrieval in the batch.

   $ cat > urls.txt <<'EOF'
   https://downloads.example.net/releases/2026-03/ops-status-2026-03.csv
   https://downloads.example.net/releases/2026-03/platform-assets-2026.03.tar.gz
   https://downloads.example.net/releases/2026-03/compliance-handbook-2026.pdf
   EOF

   One URL per line keeps the request order explicit and makes the next unattended run easier to audit.

   Related: How to download files from a list with wget

2. Run wget with a base wait, randomized jitter, and a dedicated log file.

   $ wget --wait=2 --random-wait --input-file=urls.txt \
     --directory-prefix=downloads \
     --output-file=wget-delay.log

   Use --output-file so the timestamps survive shell scrollback and can be reviewed after the batch finishes.

3. Review the request timestamps in the log to confirm that the pauses were actually applied.

   $ grep '^--' wget-delay.log
   --2026-03-29 09:23:09--  https://downloads.example.net/releases/2026-03/ops-status-2026-03.csv
   --2026-03-29 09:23:12--  https://downloads.example.net/releases/2026-03/platform-assets-2026.03.tar.gz
   --2026-03-29 09:23:15--  https://downloads.example.net/releases/2026-03/compliance-handbook-2026.pdf

   With --wait=2 and --random-wait enabled, the request starts should vary instead of repeating at one fixed cadence, while the randomized post-transfer pause stays in the one-to-three-second range.

4. Persist the same pacing defaults in /$HOME/.wgetrc when the same account runs recurring download jobs.

   ~/.wgetrc
   wait = 2
   random_wait = on

   Account-level defaults are useful for cron jobs and ad hoc runs that should inherit the same polite request spacing.

   Related: How to configure default options in ~/.wgetrc

5. Verify that the completed batch produced the expected files before another script consumes the directory.

   $ ls -1 downloads
   compliance-handbook-2026.pdf
   ops-status-2026-03.csv
   platform-assets-2026.03.tar.gz

   A complete file list plus spaced log timestamps confirms both the transfer result and the pacing behavior.