How to change user agent in Wget

The user agent string is a simple identifier sent by web browsers and other client applications to web servers. It provides details about the client, including the browser type and version. Servers use this information to determine how to respond, often tailoring content or blocking certain clients based on their user agent.

wget is a command-line tool that uses its own default user agent when making requests to a web server. This can sometimes lead to issues, as some websites either block wget or serve different content. Changing the user agent in wget can resolve these problems, allowing you to access restricted content or retrieve the same content that would be served to a different client, such as a popular web browser.

You can manually set a custom user agent in wget using the --user-agent option. This option allows wget to mimic different clients by sending a specified user agent string in its requests. For convenience, you can also configure wget to use a specific user agent persistently by editing its configuration file.

1. Open the terminal.
2. Use the --user-agent option followed by the custom user agent string to set a new user agent.

   $ wget --user-agent="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36" https://www.example.com/  
   --2024-09-16 10:10:10--  https://www.example.com/  
   Resolving www.example.com (www.example.com)... 93.184.216.34  
   Connecting to www.example.com (www.example.com)|93.184.216.34|:443... connected.  
   HTTP request sent, awaiting response... 200 OK  
   Length: 1270 [text/html]  
   Saving to: 'index.html'  
   index.html     100%[================>]  1.24K  --.-KB/s  in 0s 

   This example sets the user agent to mimic a Chrome browser on Windows 10.

3. To set a custom user agent for all future wget sessions, add the user agent string to the wget configuration file.

   $ echo 'user_agent = Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36' >> ~/.wgetrc 

   Be cautious about changing this permanently, especially if you're not the only user of the system.

4. Download content from a website to confirm the new user agent is applied.

   $ wget https://www.example.com/ 

5. Verify the user agent string being used by checking with an online tool.

   $ wget --user-agent="Custom User Agent String" https://www.whatsmyua.info/ 

   Remember, changing the user agent doesn't change the actual capabilities of wget. It only alters what wget reports to the server.