How to change user agent for Wget

The user agent string is a line of text that web browsers and other client applications send to web servers to identify themselves. By reading the user agent string, a web server can learn about the client's type, version, and other relevant attributes. This is often utilized to serve the most compatible content for that specific browser or client.

wget, a commonly used command-line utility for downloading content from the web, has its own default user agent string. This can sometimes be problematic. For instance, certain websites block or serve different content based on the user agent. In such cases, changing the user agent of wget can be beneficial, helping you access the desired content.

Thankfully, with a simple command-line flag, you can modify the user agent string used by wget.

1. Open the terminal.
2. Use the –user-agent flag followed by your desired user agent string to change wget's user agent.

   $ wget --user-agent="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36" https://www.example.com/

   This example sets the user agent to mimic a Chrome browser on Windows 10.

3. To persistently change the user agent for all future wget sessions, append the user agent string to your wget configuration file.

   echo "user_agent = Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36" >> ~/.wgetrc

   Be cautious about changing this permanently, especially if you're not the only user of the system.

4. Test downloading from a site to ensure the new user agent is in effect.

   $ wget https://www.example.com/

5. To verify the user agent being sent by wget, you can use online tools that echo back the received user agent string.

   $ wget --user-agent="Custom User Agent String" https://www.whatsmyua.info/

   Remember, changing the user agent doesn't change the actual capabilities of wget. It merely alters what wget reports to the server.