How to set a custom referer in wget

Some download endpoints check the HTTP Referer header before returning a file, especially when the URL is expected to be reached from a product page, portal view, or download landing page. In those cases the URL alone is not enough, and a direct wget request can fail even when the file path is otherwise correct.

GNU wget handles that requirement with --referer, which adds a Referer: header to the request while leaving the rest of the transfer logic alone. Redirects, HTTPS negotiation, resume support, and output handling still work normally, but the request now matches the server-side check that expects a specific source page.

Referer gates are often only part of a broader policy. Some endpoints also inspect cookies, User-Agent, or language headers, so a correct referer does not guarantee a 200 OK on its own. Use the smallest header set that satisfies the remote policy and avoid imitating an unrelated client unless the workflow genuinely requires it.

1. Confirm the exact Referer header value with a low-cost echo request before trying the real download URL.

   $ wget -qO- \
     --referer='https://portal.vendor.example/downloads/linux/' \
     https://diag.vendor.example/headers | jq -r '.headers.Referer'
   https://portal.vendor.example/downloads/linux/

   Match the exact scheme, host, path, and trailing slash pattern expected by the remote endpoint because small referer mismatches can still trigger a block.

2. Probe the target URL with --spider and the same referer before pulling the full file.

   $ wget --spider --server-response \
     --referer='https://portal.vendor.example/downloads/linux/' \
     https://downloads.vendor.example/releases/acme-agent-linux-amd64.sha256 2>&1 | sed -n '1,10p'
   Spider mode enabled. Check if remote file exists.
   --2026-03-29 09:17:05--  https://downloads.vendor.example/releases/acme-agent-linux-amd64.sha256
   Resolving downloads.vendor.example (downloads.vendor.example)... 198.51.100.24, 198.51.100.25
   Connecting to downloads.vendor.example (downloads.vendor.example)|198.51.100.24|:443... connected.
   HTTP request sent, awaiting response...
     HTTP/1.1 200 OK
     Content-Type: text/plain
     Content-Length: 98
   Length: 98 [text/plain]
   Remote file exists.

   Spider mode is a cheap validation pass for referer-gated URLs because it confirms acceptance without committing to the full download first.

3. Add more headers only when the endpoint checks more than referer alone.

   $ wget -qO- \
     --referer='https://portal.vendor.example/downloads/linux/' \
     --user-agent='Mozilla/5.0 (X11; Linux x86_64)' \
     --header='Accept-Language: en-US,en;q=0.9' \
     https://diag.vendor.example/headers | jq '{Referer: .headers.Referer, "User-Agent": .headers["User-Agent"], "Accept-Language": .headers["Accept-Language"]}'
   {
     "Referer": "https://portal.vendor.example/downloads/linux/",
     "User-Agent": "Mozilla/5.0 (X11; Linux x86_64)",
     "Accept-Language": "en-US,en;q=0.9"
   }

   Quote the referer when it includes query strings or other shell-significant characters.

   Related: How to change the user agent in wget
   Related: How to use custom cookies with wget

4. Download the file with the validated referer and confirm that the saved object matches the expected payload.

   $ wget -O acme-agent-linux-amd64.sha256 \
     --referer='https://portal.vendor.example/downloads/linux/' \
     https://downloads.vendor.example/releases/acme-agent-linux-amd64.sha256 2>&1 | sed -n '1,6p'
   --2026-03-29 09:17:05--  https://downloads.vendor.example/releases/acme-agent-linux-amd64.sha256
   Resolving downloads.vendor.example (downloads.vendor.example)... 198.51.100.24, 198.51.100.25
   Connecting to downloads.vendor.example (downloads.vendor.example)|198.51.100.24|:443... connected.
   HTTP request sent, awaiting response... 200 OK
   Length: 98 [text/plain]
   Saving to: 'acme-agent-linux-amd64.sha256'
   $ file acme-agent-linux-amd64.sha256
   acme-agent-linux-amd64.sha256: ASCII text

   Final verification should include both the HTTP status and a quick file-type or content check so a saved access-denied page is not mistaken for the real artifact.