Analyzing threats in web server logs is vital for maintaining the security and performance of any web application. Apache logs every request it receives, which can be a treasure trove of information about potential threats. This includes data about IP addresses, user agents, request types, referrers, and status codes.
By examining these logs, administrators can detect signs of hacking attempts, such as SQL injection, cross-site scripting (XSS), and brute force attacks. With the evolution of threats, it's becoming increasingly essential to analyze logs not just for post-incident investigations but for proactive threat hunting as well.
However, sifting through raw log data can be challenging due to its volume and complexity. Hence, a structured approach and sometimes additional tools can be beneficial.
$ sudo cat /var/log/apache2/access.log
The default location of the Apache access logs is typically in /var/log/apache2/access.log for Ubuntu and Debian-based systems or /var/log/httpd/access_log for Red Hat based systems.
For systems with high traffic, logs can rotate quickly. Ensure you're analyzing the right log file by checking rotated logs like access.log.1 or archived logs.
$ awk '{print $6}' /var/log/apache2/access.log | sort | uniq -c | sort -nr
A POST to a page that should not accept POST data could be malicious.
$ sudo grep -E "PUT|DELETE|TRACE" /var/log/apache2/access.log
While GET and POST are common methods, other methods like PUT, DELETE, or TRACE might be suspicious.
$ sudo awk '{print $1}' /var/log/apache2/access.log | sort | uniq -c | sort -nr | head
Rapid multiple requests from a single IP might indicate a DoS attack, scanning or web scraping.
$ grep -Ei "wp-login|xmlrpc|admin|setup|config" /var/log/apache2/access.log
Cyber attackers often attempt to access known vulnerabilities.
These endpoints might be legitimate, especially if you run a WordPress website. However, too many requests might indicate a brute-force attack.
$ awk -F" '{print $6}' /var/log/apache2/access.log | sort | uniq -c | sort -nr | less
Attack scripts may not always send a standard browser's user-agent string.
$ grep -Ei "base64_|eval|<script>|%3cscript%3e" /var/log/apache2/access.log
Attackers might attempt to exploit vulnerabilities by sending malicious data in URL parameters.
Spotting these patterns might indicate attempts to perform XSS attacks or code injection.
$ grep -Ei "union|select|drop|’|1=1" /var/log/apache2/access.log
These may look like URLs containing SQL-related terms.
If you see results from this command, it's critical to ensure your web applications are not vulnerable to SQL injection.
$ awk '($9 ~ /500/)' /var/log/apache2/access.log | sort | uniq -c | sort -nr | head -10
Large numbers of 500 responses could indicate successful exploitation attempts or software misconfigurations.
$ grep -Ei "&|;|cmd=|exec" /var/log/apache2/access.log
Searching for common command terminologies can indicate malicious activities.
Always sanitize any inputs to prevent command injection.
$ awk '$4 ~ /01/Sep/2023/' /var/log/apache2/access.log | less
Focus your analysis on a specific timeframe if you suspect an incident took place at a particular period.
$ grep -Ei ".php|.exe|.sh|.py" /var/log/apache2/access.log
Malicious users might try to upload scripts or executables.
Ensure your server configuration only allows necessary file types for uploads.
$ awk -F" '{print $4}' /var/log/apache2/access.log | sort | uniq -c | sort -nr | less
Large numbers of unexpected referrers could be an indication.
$ awk '($9 ~ /404/)' /var/log/apache2/access.log | awk '{print $7}' | sort | uniq -c | sort -nr | head -20
These might indicate someone trying to find vulnerabilities by guessing URL structures.
This command will show the top 20 most-requested missing URLs.
$ grep -Ei "/admin|/config|/setup" /var/log/apache2/access.log
Detect unauthorized attempts to access admin pages or config files.
$ awk '$10 > 5000' /var/log/apache2/access.log | less
Large requests can be an attempt to buffer overflow attacks.
$ grep -Ei ".exe|.sh|.tar|.zip" /var/log/apache2/access.log
Attackers might try to access or upload non-standard file types.
$ grep '1.2.3.4' /var/log/apache2/access.log | less
$ awk -F" '{print $4}' /var/log/apache2/access.log | sort | uniq -c | sort -nr | head -10
Check for requests with no referrers or unexpected referrers.
$ cut -d[ -f2 /var/log/apache2/access.log | cut -d] -f1 | awk -F: '{print $2":"$3}' | sort | uniq -c
Look for request patterns during off-hours, which might indicate non-human traffic or attacks.
$ sudo grep "<script" /var/log/apache2/access.log
XSS attacks involve injecting malicious scripts into web pages viewed by other users. Spotting these patterns can help in early detection.
$ grep -Ei "wp-login|xmlrpc|admin|setup|config" /var/log/apache2/access.log
While these endpoints might be legitimate, especially for WordPress sites, excessive requests can indicate brute-force attacks.
$ grep -E "../../" /var/log/apache2/access.log
These attacks involve navigating the file structure of a web server.
Directory traversal attacks attempt to access files outside of the intended directory.
$ grep -Ei "&|;|cmd=|exec" /var/log/apache2/access.log
Command injection attacks can be devastating. Always sanitize inputs to prevent such threats.
$ awk '$4 ~ /01/Sep/2023/' /var/log/apache2/access.log | less
If you suspect an incident occurred at a specific time, filtering logs by timestamps can help focus your analysis.
$ grep -Ei ".php|.exe|.sh|.py" /var/log/apache2/access.log
Ensure your server configuration only permits necessary file types for uploads to prevent malicious uploads.
Comment anonymously. Login not required.