How to download files from a list using wget

Batch downloads from long URL lists are easier to manage when a single tool automates every request. Using wget against a prepared list avoids repetitive manual pasting, reduces the chance of typing mistakes, and keeps large collections of files synchronized in a repeatable way.

Option --input-file makes wget read a plain‑text file line by line, treating each nonempty entry as a separate HTTP, HTTPS, or FTP URL to fetch. Combined with options such as --directory-prefix for controlling where files land and --continue for resuming interrupted transfers, this mechanism turns a short text file into a reusable batch‑download job.

Downloading dozens or hundreds of objects from remote servers can consume a lot of bandwidth and disk space and may stress rate‑limited services. Ensuring there is enough free space, verifying that every URL is trusted and allowed by provider terms, and enabling polite throttling with options like --limit-rate and --wait keeps bulk transfers safe, predictable, and friendly to the infrastructure being used.

1. Create a dedicated directory that will hold the URL list and downloaded files.

   $ mkdir -p ~/batch-downloads
   $ ls -d ~/batch-downloads
   /home/user/batch-downloads

   Keeping all inputs and outputs under a single directory simplifies later verification and cleanup.

2. Change into the new directory in a shell session.

   $ cd ~/batch-downloads
   $ pwd
   /home/user/batch-downloads

3. Create a text file named download-list.txt in the working directory, putting one complete URL per line.

   $ nano download-list.txt
   https://www.example.com/data/image01.jpg
   https://www.example.com/files/largefile.zip
   https://downloads.example.com/data.tar.gz

   wget ignores blank lines and any line beginning with #, which allows lightweight comments inside the list file.

4. Run wget with the prepared list to download every URL in sequence into the current directory.

   $ wget --input-file=download-list.txt
   --2025-12-28 11:59:19--  https://www.example.com/data/image01.jpg
   Resolving www.example.com (www.example.com)... 172.18.0.2
   Connecting to www.example.com (www.example.com)|172.18.0.2|:443... connected.
   HTTP request sent, awaiting response... 200 OK
   Length: 14 [image/jpeg]
   Saving to: 'image01.jpg'
   
        0K                                                       100%  906K=0s
   
   ##### snipped #####
   
   FINISHED --2025-12-28 11:59:19--
   Total wall clock time: 0.03s
   Downloaded: 3 files, 3.0M in 0.005s (587 MB/s)

   The --input-file option instructs wget to read URLs from download-list.txt in order, handling redirects and HTTP status codes for each entry.

   Re-running the same command without options such as --continue or --timestamping may download every object again, wasting bandwidth and potentially creating duplicate files with numeric suffixes like file1.jpg.1.

5. Use additional wget options when the transfer must survive interruptions or respect strict bandwidth limits.

   $ wget --input-file=download-list.txt --continue --limit-rate=200k --wait=2 --random-wait
   --2025-12-28 11:59:19--  https://www.example.com/data/image01.jpg
   Resolving www.example.com (www.example.com)... 172.18.0.2
   Connecting to www.example.com (www.example.com)|172.18.0.2|:443... connected.
   HTTP request sent, awaiting response... 200 OK
   
       The file is already fully retrieved; nothing to do.
   
   --2025-12-28 11:59:21--  https://www.example.com/files/largefile.zip
   Connecting to www.example.com (www.example.com)|172.18.0.2|:443... connected.
   HTTP request sent, awaiting response... 416 Requested Range Not Satisfiable
   
       The file is already fully retrieved; nothing to do.
   
   --2025-12-28 11:59:23--  https://downloads.example.com/data.tar.gz
   Resolving downloads.example.com (downloads.example.com)... 172.18.0.2
   Connecting to downloads.example.com (downloads.example.com)|172.18.0.2|:443... connected.
   HTTP request sent, awaiting response... 200 OK
   
       The file is already fully retrieved; nothing to do.

   Combining --continue with throttling options such as --limit-rate and --wait helps resume partial downloads safely while reducing the chance of triggering remote rate limits or abuse protections.

6. List the contents of the directory to confirm that every expected file from download-list.txt has been stored successfully.

   $ ls -lh
   total 3.1M
   -rw-r--r-- 1 user user 1.0M Dec 28 11:09 data.tar.gz
   -rw-r--r-- 1 user user  127 Dec 28 11:59 download-list.txt
   -rw-r--r-- 1 user user   14 Dec 28 11:05 image01.jpg
   -rw-r--r-- 1 user user 2.0M Dec 28 11:05 largefile.zip

   Verifying that all filenames appear with nonzero sizes and without unexpected numeric suffixes indicates that the batch download completed cleanly.