How to analyze threats in Apache logs

Analyzing Apache logs is crucial for identifying potential security threats to your web server. These logs contain detailed information about every request, including IP addresses, request methods, and response codes. Proper analysis helps detect malicious activities such as SQL injection, cross-site scripting (XSS), and brute force attacks.

Monitoring Apache access logs can reveal patterns that indicate unauthorized access attempts or unusual behavior. By focusing on specific log entries, administrators can identify signs of hacking attempts, unauthorized access, or other suspicious activities. This proactive monitoring is key to securing your web infrastructure against evolving threats.

Due to the large volume of data in Apache logs, a systematic approach is necessary for effective analysis. Understanding the structure of these logs and using appropriate tools can help streamline the process, making it easier to detect and respond to security issues promptly.

1. Access the Apache log files.

   $ sudo cat /var/log/apache2/access.log

   The default location of the Apache access logs is typically in /var/log/apache2/access.log for Ubuntu and Debian-based systems or /var/log/httpd/access_log for Red Hat based systems.

   For systems with high traffic, logs can rotate quickly. Ensure you're analyzing the right log file by checking rotated logs like access.log.1 or archived logs.

2. Analyze request methods.

   $ awk '{print $6}' /var/log/apache2/access.log | sort | uniq -c | sort -nr

   A POST to a page that should not accept POST data could be malicious.

3. Search for unusual request methods.

   $ sudo grep -E "PUT|DELETE|TRACE" /var/log/apache2/access.log

   While GET and POST are common methods, other methods like PUT, DELETE, or TRACE might be suspicious.

4. Spot high-frequency requests from a single IP.

   $ sudo awk '{print $1}' /var/log/apache2/access.log | sort | uniq -c | sort -nr | head

   Rapid multiple requests from a single IP might indicate a DoS attack, scanning or web scraping.

5. Examine for common exploit URLs.

   $ grep -Ei "wp-login|xmlrpc|admin|setup|config" /var/log/apache2/access.log

   Cyber attackers often attempt to access known vulnerabilities.

   These endpoints might be legitimate, especially if you run a WordPress website. However, too many requests might indicate a brute-force attack.

6. Check for non-standard user-agents.

   $ awk -F" '{print $6}' /var/log/apache2/access.log | sort | uniq -c | sort -nr | less

   Attack scripts may not always send a standard browser's user-agent string.

7. Identify suspicious URL parameters.

   $ grep -Ei "base64_|eval|<script>|%3cscript%3e" /var/log/apache2/access.log

   Attackers might attempt to exploit vulnerabilities by sending malicious data in URL parameters.

   Spotting these patterns might indicate attempts to perform XSS attacks or code injection.

8. Inspect for SQL injection attempts.

   $ grep -Ei "union|select|drop|’|1=1" /var/log/apache2/access.log

   These may look like URLs containing SQL-related terms.

   If you see results from this command, it's critical to ensure your web applications are not vulnerable to SQL injection.

9. Look for unexpected HTTP response codes.

   $ awk '($9 ~ /500/)' /var/log/apache2/access.log | sort | uniq -c | sort -nr | head -10

   Large numbers of 500 responses could indicate successful exploitation attempts or software misconfigurations.

10. Identify potential command injection.

    $ grep -Ei "&|;|cmd=|exec" /var/log/apache2/access.log

    Searching for common command terminologies can indicate malicious activities.

    Always sanitize any inputs to prevent command injection.

11. Filter logs based on timestamps.

    $ awk '$4 ~ /01/Sep/2023/' /var/log/apache2/access.log | less

    Focus your analysis on a specific timeframe if you suspect an incident took place at a particular period.

12. Spot suspicious file upload attempts.

    $ grep -Ei ".php|.exe|.sh|.py" /var/log/apache2/access.log

    Malicious users might try to upload scripts or executables.

    Ensure your server configuration only allows necessary file types for uploads.

13. Examine logs for potential CSRF attacks.

    $ awk -F" '{print $4}' /var/log/apache2/access.log | sort | uniq -c | sort -nr | less

    Large numbers of unexpected referrers could be an indication.

14. Search for repeated 404 errors.

    $ awk '($9 ~ /404/)' /var/log/apache2/access.log | awk '{print $7}' | sort | uniq -c | sort -nr | head -20

    These might indicate someone trying to find vulnerabilities by guessing URL structures.

    This command will show the top 20 most-requested missing URLs.

15. Review requests to sensitive URIs.

    $ grep -Ei "/admin|/config|/setup" /var/log/apache2/access.log

    Detect unauthorized attempts to access admin pages or config files.

16. Check for large requests.

    $ awk '$10 > 5000' /var/log/apache2/access.log | less

    Large requests can be an attempt to buffer overflow attacks.

17. Spot requests for unexpected file types.

    $ grep -Ei ".exe|.sh|.tar|.zip" /var/log/apache2/access.log

    Attackers might try to access or upload non-standard file types.

18. Filter logs for requests from a single IP.

    $ grep '1.2.3.4' /var/log/apache2/access.log | less

19. Analyze referrer field.

    $ awk -F" '{print $4}' /var/log/apache2/access.log | sort | uniq -c | sort -nr | head -10

    Check for requests with no referrers or unexpected referrers.

20. Examine time patterns.

    $ cut -d[ -f2 /var/log/apache2/access.log | cut -d] -f1 | awk -F: '{print $2":"$3}' | sort | uniq -c

    Look for request patterns during off-hours, which might indicate non-human traffic or attacks.

21. Search for XSS attack patterns.

    $ sudo grep "<script" /var/log/apache2/access.log

    XSS attacks involve injecting malicious scripts into web pages viewed by other users. Spotting these patterns can help in early detection.

22. Examine common exploit URLs.

    $ grep -Ei "wp-login|xmlrpc|admin|setup|config" /var/log/apache2/access.log

    While these endpoints might be legitimate, especially for WordPress sites, excessive requests can indicate brute-force attacks.

23. Spot potential directory traversal attacks.

    $ grep -E "../../" /var/log/apache2/access.log

    These attacks involve navigating the file structure of a web server.

    Directory traversal attacks attempt to access files outside of the intended directory.

24. Identify potential command injection attempts.

    $ grep -Ei "&|;|cmd=|exec" /var/log/apache2/access.log

    Command injection attacks can be devastating. Always sanitize inputs to prevent such threats.

25. Filter logs based on timestamps.

    $ awk '$4 ~ /01/Sep/2023/' /var/log/apache2/access.log | less

    If you suspect an incident occurred at a specific time, filtering logs by timestamps can help focus your analysis.

26. Spot suspicious file upload attempts.

    $ grep -Ei ".php|.exe|.sh|.py" /var/log/apache2/access.log

    Ensure your server configuration only permits necessary file types for uploads to prevent malicious uploads.