How to change the user agent in wget

Changing the HTTP User-Agent header is useful when an endpoint varies behavior by client identity, rejects unknown automation, or needs to be tested against the same header pattern used by another tool. A controlled override makes those checks possible without editing application code or proxy rules first.

Wget normally sends its own version string in the User-Agent header, but --user-agent replaces that value for one command and the user_agent directive can persist a default in /$HOME/.wgetrc. The same header can also be injected with --header when a request already carries several custom fields.

User-agent changes should stay explicit and limited to the real test objective. Sending a misleading browser string can change analytics, trigger different content paths, or violate remote service policy, so it is safer to keep overrides temporary and easy to audit.

1. Check the current default header without local startup-file overrides influencing the result.

   $ wget --no-config --spider --debug https://httpbin.org/user-agent 2>&1 | sed -n '/request begin/,/request end/p'
   ---request begin---
   HEAD /user-agent HTTP/1.1
   Host: httpbin.org
   User-Agent: Wget/1.25.0
   Accept: */*
   Accept-Encoding: identity
   Connection: Keep-Alive
   ---request end---

   --no-config is the safest baseline when /$HOME/.wgetrc may already define user_agent.

2. Override the header for one request with --user-agent.

   $ wget --user-agent='Mozilla/5.0 (X11; Linux x86_64; rv:137.0) Gecko/20100101 Firefox/137.0' --spider --debug https://httpbin.org/user-agent 2>&1 | sed -n '/request begin/,/request end/p'
   ---request begin---
   HEAD /user-agent HTTP/1.1
   Host: httpbin.org
   User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:137.0) Gecko/20100101 Firefox/137.0
   Accept: */*
   Accept-Encoding: identity
   Connection: Keep-Alive
   ---request end---

   A per-command override is the cleanest choice for API tests, access-control checks, and other short-lived comparisons.

3. Use the header form when the command already includes several custom request fields.

   $ wget \
     --header='User-Agent: WgetOpsCheck/1.0' \
     --header='Accept: application/json' \
     --spider --debug https://httpbin.org/anything 2>&1 \
     | sed -n '/request begin/,/request end/p'
   ---request begin---
   HEAD /anything HTTP/1.1
   Host: httpbin.org
   User-Agent: WgetOpsCheck/1.0
   Accept: application/json
   Accept-Encoding: identity
   Connection: Keep-Alive
   ---request end---

   Repeated --header options keep multi-header requests readable when one-off testing already needs other custom fields.

4. Add or update one active user_agent line in /$HOME/.wgetrc for repeated automation under one account.

   $ vi ~/.wgetrc
   user_agent = WgetOpsDefaults/1.0

   Persistent defaults affect every future wget call for that account, so leave them in place only when the policy is acceptable for all scheduled jobs.

5. Verify the persistent value is active without supplying any extra header option.

   $ wget --spider --debug https://httpbin.org/user-agent 2>&1 | sed -n '/request begin/,/request end/p'
   ---request begin---
   HEAD /user-agent HTTP/1.1
   Host: httpbin.org
   User-Agent: WgetOpsDefaults/1.0
   Accept: */*
   Accept-Encoding: identity
   Connection: Keep-Alive
   ---request end---

   An exact match in the request block confirms that the startup file is applying the expected header value.