Skip to content

Using fail2ban to block unfriendly web requests

Every time I peek into the webserver logfiles, I find quite a few 404 requests trying to figure out if certain exploits exist on this server. Now I get that these are automated attempts, and the number of requests coming from one IP show that they try several different exploits and path names. Nevertheless I thought that I don’t need this in my log, and on my webserver. fail2ban for the rescue.


The evidence

The first thing I need is to know the kind of errors I’m looking for. Go into the directory with the webserver logfiles and try the following:

grep -E '403|404|500' --exclude=error.log *.log | less

This should give you a rather long list of errors, some are valid (like from a real search engine), some are just scripts attempting to find something on your server. Pick a couple entries where you know these are invalid, but make sure you don’t accidentally pick something which will result in blocking legitimate clients. Any request accessing pictures or css files is probably a genuine error. Requests to “funny” path names like “defau1t.php” are worth investigating. Extract the path names from the requests, we need that later.


Install the fail2ban package. On Debian this package brings quite a few pre-configured rules, but leaves most of them deactivated. Only sshd is activated. Check out /etc/fail2ban/filter.d and /etc/fail2ban/jail.d.

New jail

In fail2ban terms, each group of service protection is named a “jail”. What we need is a new filter, and a new jail. Conveniently these can be dropped into filter.d and jail.d as new files, and I’m deploying this with Ansible.

The new filter is /etc/fail2ban/filter.d/apache-badurls.conf in my case:

# Fail2Ban configuration file
# Regexp to catch known spambots and software alike. Please verify
# that it is your intent to block IPs which were driven by
# above mentioned bots.


failregex = ^<HOST> -.*"(GET|POST|HEAD) \/shell.php.*HTTP.*
            ^<HOST> -.*"(GET|POST|HEAD) \/shells.php.*HTTP.*
            ^<HOST> -.*"(GET|POST|HEAD) \/web\/.env.*HTTP.*

ignoreregex =.*(robots.txt|favicon.ico|jpg|jpeg|png|gif|css)

datepattern = ^[^\[]*\[({DATE})

failregex can hold multiple lines, each a regex. That by itself is fine, but keep in mind that complicated regex increase the logfile parsing time. I added a few examples above, my actual file is larger and I expand it from time to time when I find new invalid requests in the logfile.

ignoreregex is used to exclude certain requests, in my case I don’t care if someone can’t find a css file, an image or a robots.txt.

The new jail configuration goes into /etc/fail2ban/jail.d/apache-badurls.conf:


port     = http,https
filter   = apache-badurls
logpath  = /var/log/apache2/*.log
maxretry = 1
findtime = 30m
bantime  = 48h
enabled  = true


  • The “port” line specifies which ports should be blocked, in this case 80 (http) and 443 (https)
  • The “filter” line specifies the filename in filter.d
  • logpath” is where your webserver logfile(s) live, you may have to change this
  • enabled” is important, because the default disables all jails
  • maxretry” specifies how often an entry can appear before the ban hammer hits. If you are unsure, increase this number
  • And finally “bantime” is the time fail2ban will block a certain IP address

Restart the service:

service fail2ban restart

And see if the new jail is working:

fail2ban-client status apache-badurls

In parallel you might want to check the webserver logfile, and once a bad request hits there you will see that fail2ban blocks the IP address.


fail2ban is a convenient way to keep webserver logfiles clean, block scripts and bots, and reduce the overall load on a server. It's quick to deploy and the configuration is easy to understand.


No Trackbacks


Display comments as Linear | Threaded

No comments

Add Comment

Enclosing asterisks marks text as bold (*word*), underscore are made via _word_.
E-Mail addresses will not be displayed and will only be used for E-Mail notifications.
To leave a comment you must approve it via e-mail, which will be sent to your address after submission.
Form options