How to block user agents in Apache

Blocking abusive or unwanted User-Agent strings in Apache helps cut noise from cheap scanners, reduce wasted work on expensive routes, and reject a predictable class of bot traffic before it reaches the application.

In Apache 2.4, a common pattern is to tag matching requests with SetEnvIfNoCase from mod_setenvif and then deny them with Require not env inside the relevant authorization context. That keeps the matching logic readable and lets the same environment variable be reused for a full site or a narrower path.

Because the User-Agent header is entirely client-supplied, blocking it is a traffic filter rather than a trust control. Use narrow match patterns, apply the rule in the right scope for the site or path you actually want to protect, and test the configuration before reload. Examples use the Debian or Ubuntu layout with /etc/apache2/conf-available and the apache2 service; on RHEL-family systems, the equivalent drop-ins usually live under /etc/httpd/conf.d/ and the unit name is httpd.

1. Decide whether the rule should apply to one site, one path, or the whole server.

   For a single virtual host, place the rule inside that site's <VirtualHost> block. For a global drop-in that should affect every site, use /etc/apache2/conf-available/ as shown below.

2. Create a dedicated configuration snippet for the block rule.

   $ sudoedit /etc/apache2/conf-available/block-user-agent.conf

   Default Ubuntu and Debian packages already load mod_setenvif. On custom builds, confirm it is available with sudo apachectl -M | grep setenvif before using SetEnvIfNoCase.

3. Add a match for the unwanted user agents and deny requests when the environment variable is set.

   SetEnvIfNoCase User-Agent "badbot|scanner|crawler" bad_ua=1
    
   <Location "/">
       <RequireAll>
           Require all granted
           Require not env bad_ua
       </RequireAll>
   </Location>

   Replace the example regex with the exact strings or patterns you actually want to block. Overly broad terms such as bot or curl can block legitimate clients, monitors, or integrations.

   Use a narrower <Location> such as /admin/ or place the directives inside the target <VirtualHost> when the block should not affect every site served by the host.

4. Enable the configuration snippet.

   $ sudo a2enconf block-user-agent
   Enabling conf block-user-agent.
   To activate the new configuration, you need to run:
     service apache2 reload

   On RHEL-family systems, place the file under /etc/httpd/conf.d/ instead of using a2enconf.

5. Validate the configuration before reload.

   $ sudo apachectl configtest
   Syntax OK

   Related: How to test Apache configuration

6. Reload Apache to apply the rule.

   $ sudo systemctl reload apache2

   If the server is not managed by systemd, use the service manager that owns the process, such as sudo service apache2 reload or the httpd unit on RHEL-family systems.

   Related: How to manage the Apache web server service

7. Send a request with a blocked User-Agent string and confirm that Apache returns HTTP 403.

   $ curl -I -A 'badbot' http://127.0.0.1/
   HTTP/1.1 403 Forbidden
   Date: Wed, 08 Apr 2026 04:07:31 GMT
   Server: Apache/2.4.58 (Ubuntu)
   Content-Type: text/html; charset=iso-8859-1

   If the rule is inside a name-based VirtualHost, add the expected host header with -H 'Host: site.example.com' so the request hits the right site.

8. Confirm normal traffic still works with a non-matching User-Agent.

   $ curl -I http://127.0.0.1/
   HTTP/1.1 200 OK
   Date: Wed, 08 Apr 2026 04:07:31 GMT
   Server: Apache/2.4.58 (Ubuntu)
   Last-Modified: Wed, 08 Apr 2026 04:07:29 GMT
   ETag: "29af-64eeb09d92bfd"
   Accept-Ranges: bytes
   Content-Length: 10671
   Vary: Accept-Encoding
   Content-Type: text/html

   Rollback: run sudo a2disconf block-user-agent and reload Apache again, or remove the directives from the target VirtualHost and reload it there.