How to limit total download size in wget

Limiting total download size in wget prevents automated transfers from exhausting bandwidth or filling disks during bulk downloads. Systems running scheduled retrievals, mirroring sites, or fetching large media sets benefit from a hard cap on how much data each invocation is allowed to store. A quota also simplifies planning for metered connections where every gigabyte matters.

The GNU wget downloader exposes a quota mechanism through the -Q and –quota options, which track the cumulative size of successfully downloaded objects in a single run. When used with a list of URLs or recursive crawling, wget sums each completed file and emits a clear message once the configured byte limit is crossed. The quota applies per invocation, so each new process starts again from zero and can be tuned independently.

Quota limits do not interrupt a file already in progress, which means a large download that starts before the threshold is reached can still complete and push the total beyond the nominal cap. Quota values are specified in bytes by default, with k, m, and g suffixes for kibibytes, mebibytes, and gibibytes, so careful sizing is important. Storing quota-controlled downloads in a dedicated directory makes inspection and cleanup easier, and helps confirm how closely the final size matches the intended limit.

1. Prepare a dedicated directory for quota-controlled downloads under the home directory.

   $ mkdir -p ~/wget-quota
   $ cd ~/wget-quota

   Keeping all quota-limited downloads and the default wget-log file in a single directory simplifies checking total size and log messages.

2. Create a text file containing one URL per line to give wget a fixed set of objects to fetch under the quota.

   $ cat > urls.txt <<'EOF'
   https://example.org/file1.iso
   https://example.org/file2.iso
   https://example.org/file3.iso
   EOF

   Any mix of HTTP, HTTPS, or FTP URLs can be listed; the quota is enforced across all successfully downloaded objects referenced by --input-file=urls.txt.

3. Run wget with a quota value so the total transferred size for this invocation is limited to a specific number of bytes.

   $ wget --quota=200m --input-file=urls.txt --continue --timestamping
   --2025-12-08 10:00:00--  https://example.org/file1.iso
   Resolving example.org (example.org)... 93.184.216.34
   Connecting to example.org (example.org)|93.184.216.34|:443... connected.
   HTTP request sent, awaiting response... 200 OK
   Length: 31457280 (30M) [application/octet-stream]
   Saving to: ‘file1.iso’
   
   file1.iso           100%[===================>]  30.0M  11.0MB/s    in 2.8s    
   
   ##### snipped #####
   Download quota (209715200 bytes) EXCEEDED!

   Quota enforcement does not terminate a file already being downloaded, so a single large object can finish after the threshold and cause the final total to exceed the configured limit.

4. Use the short -Q form for the quota option when scripting repeated jobs that require different limits.

   $ wget -Q 500m --input-file=urls.txt --directory-prefix=job1
   $ wget -Q 1g  --input-file=urls.txt --directory-prefix=job2

   Option --quota=SIZE and its short form ‑Q SIZE behave identically, so either form can be chosen based on readability preferences in scripts.

5. Confirm that the total size of downloaded files and the log entry align with the configured quota.

   $ du -sh .
   195M  .
   $ grep -i "Download quota" wget-log
   Download quota (209715200 bytes) EXCEEDED!

   Successful enforcement typically shows a directory size close to the quota value and a Download quota (... bytes) EXCEEDED! line in wget-log for that run.