Some servers tailor responses based on the client’s user agent or block certain clients altogether. Changing the user agent in wget helps you access content that is restricted or formatted differently for specific browsers.
By mimicking common browser user agents, wget can retrieve the same data served to those browsers. This feature improves compatibility, aiding in scenarios where default wget identification might be flagged or treated differently.
Setting a custom user agent can also help testing, allowing you to simulate various client environments. Just ensure that using a misleading agent does not violate any terms or create confusion.
Steps to set a custom user agent in Wget:
- Use –user-agent followed by the desired agent string.
$ wget --user-agent="Mozilla/5.0 (Windows NT 10.0; Win64; x64)" https://www.example.com
- Permanently set a user agent in .wgetrc.
$ echo 'user_agent = Mozilla/5.0 (Windows NT 10.0; Win64; x64)' >> ~/.wgetrc
Be aware that changing the user agent globally affects all downloads.
- Verify the new user agent behavior by fetching pages that previously behaved differently.
$ wget https://www.whatsmyua.info/
Mohd Shakir Zakaria is a cloud architect with deep roots in software development and open-source advocacy. Certified in AWS, Red Hat, VMware, ITIL, and Linux, he specializes in designing and managing robust cloud and on-premises infrastructures.
Comment anonymously. Login not required.