Batch downloads from long URL lists are easier to manage when a single tool automates every request. Using wget against a prepared list avoids repetitive manual pasting, reduces the chance of typing mistakes, and keeps large collections of files synchronized in a repeatable way.
Option --input-file makes wget read a plain‑text file line by line, treating each nonempty entry as a separate HTTP, HTTPS, or FTP URL to fetch. Combined with options such as --directory-prefix for controlling where files land and --continue for resuming interrupted transfers, this mechanism turns a short text file into a reusable batch‑download job.
Downloading dozens or hundreds of objects from remote servers can consume a lot of bandwidth and disk space and may stress rate‑limited services. Ensuring there is enough free space, verifying that every URL is trusted and allowed by provider terms, and enabling polite throttling with options like --limit-rate and --wait keeps bulk transfers safe, predictable, and friendly to the infrastructure being used.
Steps to download files from a list using wget:
- Create a dedicated directory that will hold the URL list and downloaded files.
$ mkdir -p ~/batch-downloads $ ls -d ~/batch-downloads /home/user/batch-downloads
Keeping all inputs and outputs under a single directory simplifies later verification and cleanup.
- Change into the new directory in a shell session.
$ cd ~/batch-downloads && pwd /home/user/batch-downloads
- Create a text file named download-list.txt in the working directory, putting one complete URL per line.
$ nano download-list.txt https://www.example.com/data/image01.jpg https://www.example.com/files/largefile.zip https://downloads.example.net/data.tar.gz
wget ignores blank lines and any line beginning with #, which allows lightweight comments inside the list file.
- Run wget with the prepared list to download every URL in sequence into the current directory.
$ cd ~/batch-downloads && wget --input-file=download-list.txt --2026-01-10 05:23:33-- https://www.example.com/data/image01.jpg Resolving www.example.com (www.example.com)... 203.0.113.50 Connecting to www.example.com (www.example.com)|203.0.113.50|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 14 [image/jpeg] Saving to: 'image01.jpg' 0K 100% 742K=0s 2026-01-10 05:23:34 (742 KB/s) - 'image01.jpg' saved [14/14] --2026-01-10 05:23:34-- https://www.example.com/files/largefile.zip Connecting to www.example.com (www.example.com)|203.0.113.50|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 2097152 (2.0M) [application/zip] Saving to: 'largefile.zip' 0K .......... .......... .......... .......... .......... 2% 356M 0s 50K .......... .......... .......... .......... .......... 4% 202M 0s 100K .......... .......... .......... .......... .......... 7% 448M 0s 150K .......... .......... .......... .......... .......... 9% 455M 0s 200K .......... .......... .......... .......... .......... 12% 641M 0s 250K .......... .......... .......... .......... .......... 14% 603M 0s 300K .......... .......... .......... .......... .......... 17% 549M 0s 350K .......... .......... .......... .......... .......... 19% 354M 0s 400K .......... .......... .......... .......... .......... 21% 222M 0s 450K .......... .......... .......... .......... .......... 24% 305M 0s 500K .......... .......... .......... .......... .......... 26% 397M 0s 550K .......... .......... .......... .......... .......... 29% 291M 0s 600K .......... .......... .......... .......... .......... 31% 466M 0s 650K .......... .......... .......... .......... .......... 34% 447M 0s 700K .......... .......... .......... .......... .......... 36% 500M 0s 750K .......... .......... .......... .......... .......... 39% 236M 0s 800K .......... .......... .......... .......... .......... 41% 339M 0s 850K .......... .......... .......... .......... .......... 43% 486M 0s 900K .......... .......... .......... .......... .......... 46% 609M 0s 950K .......... .......... .......... .......... .......... 48% 284M 0s 1000K .......... .......... .......... .......... .......... 51% 561M 0s 1050K .......... .......... .......... .......... .......... 53% 571M 0s 1100K .......... .......... .......... .......... .......... 56% 613M 0s 1150K .......... .......... .......... .......... .......... 58% 300M 0s 1200K .......... .......... .......... .......... .......... 61% 305M 0s 1250K .......... .......... .......... .......... .......... 63% 243M 0s 1300K .......... .......... .......... .......... .......... 65% 441M 0s 1350K .......... .......... .......... .......... .......... 68% 546M 0s 1400K .......... .......... .......... .......... .......... 70% 596M 0s 1450K .......... .......... .......... .......... .......... 73% 327M 0s 1500K .......... .......... .......... .......... .......... 75% 457M 0s 1550K .......... .......... .......... .......... .......... 78% 569M 0s 1600K .......... .......... .......... .......... .......... 80% 447M 0s 1650K .......... .......... .......... .......... .......... 83% 661M 0s 1700K .......... .......... .......... .......... .......... 85% 612M 0s 1750K .......... .......... .......... .......... .......... 87% 653M 0s 1800K .......... .......... .......... .......... .......... 90% 625M 0s 1850K .......... .......... .......... .......... .......... 92% 261M 0s 1900K .......... .......... .......... .......... .......... 95% 591M 0s 1950K .......... .......... .......... .......... .......... 97% 392M 0s 2000K .......... .......... .......... .......... ........ 100% 574M=0.005s 2026-01-10 05:23:34 (403 MB/s) - 'largefile.zip' saved [2097152/2097152] --2026-01-10 05:23:34-- https://downloads.example.net/data.tar.gz Resolving downloads.example.net (downloads.example.net)... 203.0.113.50 Connecting to downloads.example.net (downloads.example.net)|203.0.113.50|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 1048576 (1.0M) [application/gzip] Saving to: 'data.tar.gz' 0K .......... .......... .......... .......... .......... 4% 515M 0s 50K .......... .......... .......... .......... .......... 9% 413M 0s 100K .......... .......... .......... .......... .......... 14% 290M 0s 150K .......... .......... .......... .......... .......... 19% 553M 0s 200K .......... .......... .......... .......... .......... 24% 394M 0s 250K .......... .......... .......... .......... .......... 29% 329M 0s 300K .......... .......... .......... .......... .......... 34% 262M 0s 350K .......... .......... .......... .......... .......... 39% 692M 0s 400K .......... .......... .......... .......... .......... 43% 532M 0s 450K .......... .......... .......... .......... .......... 48% 616M 0s 500K .......... .......... .......... .......... .......... 53% 517M 0s 550K .......... .......... .......... .......... .......... 58% 671M 0s 600K .......... .......... .......... .......... .......... 63% 654M 0s 650K .......... .......... .......... .......... .......... 68% 735M 0s 700K .......... .......... .......... .......... .......... 73% 606M 0s 750K .......... .......... .......... .......... .......... 78% 222M 0s 800K .......... .......... .......... .......... .......... 83% 449M 0s 850K .......... .......... .......... .......... .......... 87% 672M 0s 900K .......... .......... .......... .......... .......... 92% 753M 0s 950K .......... .......... .......... .......... .......... 97% 731M 0s 1000K .......... .......... .... 100% 809M=0.002s 2026-01-10 05:23:34 (472 MB/s) - 'data.tar.gz' saved [1048576/1048576] FINISHED --2026-01-10 05:23:34-- Total wall clock time: 0.04s Downloaded: 3 files, 3.0M in 0.007s (423 MB/s)The --input-file option instructs wget to read URLs from download-list.txt in order, handling redirects and HTTP status codes for each entry.
Re-running the same command without options such as --continue or --timestamping may download every object again, wasting bandwidth and potentially creating duplicate files with numeric suffixes like file1.jpg.1.
- Use additional wget options when the transfer must survive interruptions or respect strict bandwidth limits.
$ cd ~/batch-downloads && wget --input-file=download-list.txt --continue --limit-rate=200k --wait=2 --random-wait --2026-01-10 05:23:39-- https://www.example.com/data/image01.jpg Resolving www.example.com (www.example.com)... 203.0.113.50 Connecting to www.example.com (www.example.com)|203.0.113.50|:443... connected. HTTP request sent, awaiting response... 416 Requested Range Not Satisfiable The file is already fully retrieved; nothing to do. --2026-01-10 05:23:41-- https://www.example.com/files/largefile.zip Connecting to www.example.com (www.example.com)|203.0.113.50|:443... connected. HTTP request sent, awaiting response... 416 Requested Range Not Satisfiable The file is already fully retrieved; nothing to do. --2026-01-10 05:23:42-- https://downloads.example.net/data.tar.gz Resolving downloads.example.net (downloads.example.net)... 203.0.113.50 Connecting to downloads.example.net (downloads.example.net)|203.0.113.50|:443... connected. HTTP request sent, awaiting response... 416 Requested Range Not Satisfiable The file is already fully retrieved; nothing to do.Combining --continue with throttling options such as --limit-rate and --wait helps resume partial downloads safely while reducing the chance of triggering remote rate limits or abuse protections.
- List the contents of the directory to confirm that every expected file from download-list.txt has been stored successfully.
$ cd ~/batch-downloads && ls -lh total 3.1M -rw-r--r-- 1 user user 1.0M Jan 10 04:05 data.tar.gz -rw-r--r-- 1 user user 127 Jan 10 05:23 download-list.txt -rw-r--r-- 1 user user 14 Jan 10 04:05 image01.jpg -rw-r--r-- 1 user user 2.0M Jan 10 04:05 largefile.zip
Verifying that all filenames appear with nonzero sizes and without unexpected numeric suffixes indicates that the batch download completed cleanly.
Author: Mohd
Shakir Zakaria
Mohd Shakir Zakaria is a cloud architect with deep roots in software development and open-source advocacy. Certified in AWS, Red Hat, VMware, ITIL, and Linux, he specializes in designing and managing robust cloud and on-premises infrastructures.
Mohd Shakir Zakaria is a cloud architect with deep roots in software development and open-source advocacy. Certified in AWS, Red Hat, VMware, ITIL, and Linux, he specializes in designing and managing robust cloud and on-premises infrastructures.