Automated Intrusion Prevention: Fail2ban Configuration for SSH & Nginx Protection

1. Fail2ban Architecture & Log Parsing Engine

Fail2ban acts as an automated intrusion prevention framework for Linux systems. It operates by continuously monitoring system log files (such as /var/log/auth.log, /var/log/syslog, or systemd journald) for failed authentication attempts matching regular expression filters (failregex).

When an IP address exceeds the defined threshold of failed login attempts within a specified observation window (findtime), Fail2ban dynamically invokes netfilter firewall actions to block the attacking IP address for a duration defined by bantime.

Log parsing runs asynchronously via systemd journald or pyinotify file system watchers. This decoupled architecture guarantees minimal CPU overhead even on high-throughput web server instances processing millions of requests daily.

By automating firewall responses at the network boundary, Fail2ban prevents credential guessing, SSH dictionary attacks, and HTTP layer-7 scanning bots from compromising user accounts or consuming CPU resources.

# Install Fail2ban daemon on Debian/Ubuntu
sudo apt-get update && sudo apt-get install -y fail2ban

# Enable and start Fail2ban systemd daemon
systemctl enable --now fail2ban

# Verify active status and global jail count
fail2ban-client status

2. Deploying Production Jails via jail.local

Never modify the default /etc/fail2ban/jail.conf file directly. Package upgrades by system package managers will overwrite jail.conf, discarding custom security rules.

Instead, create /etc/fail2ban/jail.local to override default settings safely. Parameters specified in jail.local automatically take precedence over jail.conf settings.

In jail.local, configure global defaults under the [DEFAULT] section for parameters like bantime (ban duration), findtime (window period), maxretry (maximum allowed failures), and banaction (firewall action engine).

The following production jail.local template enforces a strict 1-hour ban for SSH brute-force attempts after 5 failures within a 10-minute window while ignoring trusted internal management subnets:

# /etc/fail2ban/jail.local
[DEFAULT]
bantime  = 1h
findtime = 10m
maxretry = 5
banaction = ufw
ignoreip = 127.0.0.1/8 ::1 10.0.0.0/8 192.168.1.0/24

[sshd]
enabled = true
port    = ssh
logpath = %(sshd_log)s
backend = systemd

3. Recidive Jails for Persistent Repeat Attackers

Automated botnets are designed to bypass standard short ban windows. Once a 1-hour ban expires, botnet clients resume password guessing attacks at calculated intervals to stay under rate limit thresholds.

The recidive jail solves this persistent threat by monitoring Fail2ban's own log file (/var/log/fail2ban.log). If an IP address is banned multiple times across any jail within a 1-day observation window, recidive triggers a long-term persistent ban (e.g., 7 days or 30 days).

Deploying a recidive jail dramatically reduces log volume, system load, and firewall table churn caused by aggressive repeat offender botnets.

# Append recidive jail section to /etc/fail2ban/jail.local
[recidive]
enabled  = true
logpath  = /var/log/fail2ban.log
banaction = ufw
findtime = 1d
maxretry = 2
bantime  = 7d

4. Mitigating Web Scanners via Nginx Log Filters

Automated web vulnerability scanners search public web servers for exposed environment configuration files (.env), database management panels (phpMyAdmin), and unpatched CMS logins, flooding Nginx access logs with HTTP 404, 403, and 400 error codes.

Create a custom Fail2ban filter in /etc/fail2ban/filter.d/nginx-noscript.conf to match scanner URI request patterns and automatically ban offensive IP addresses.

This prevents web scanners from consuming web server worker processes and backend database connection pools.

# Create custom filter /etc/fail2ban/filter.d/nginx-noscript.conf
[Definition]
failregex = ^ -.* "GET /.*(\..*|phpmyadmin|wp-login|\.env|\.git) HTTP/.*" (404|403|400)
ignoreregex =

# Configure jail entry in /etc/fail2ban/jail.local
[nginx-noscript]
enabled  = true
port     = http,https
logpath  = /var/log/nginx/access.log
maxretry = 6
findtime = 1m
bantime  = 24h

5. NFTables Backend & High-Performance Action Tuning

Legacy Fail2ban actions append individual iptables rules for every single banned IP address. Over time, long iptables rule chains increase packet processing latency for legitimate network traffic.

Configuring banaction = nftables-multiport instructs Fail2ban to insert banned IP addresses into native nftables sets. Nftables set lookups execute in constant time O(1), maintaining fast packet filtering throughput regardless of ban count.

# Configure high-performance nftables backend in /etc/fail2ban/jail.local
[DEFAULT]
banaction = nftables-multiport
banaction_allports = nftables-allports

6. Administration & Unban Command Reference

System administrators can inspect active jail health, query banned IP lists, and manually unban accidental administrator lockouts using the fail2ban-client CLI utility.

Always test configuration changes using fail2ban-client reload to ensure settings apply cleanly without dropping active bans.

# Inspect overall daemon status and active jails
fail2ban-client status

# Inspect detailed status and active banned IPs for SSH jail
fail2ban-client status sshd

# Unban an IP address locked out accidentally
fail2ban-client set sshd unbanip 192.168.1.100

# Reload Fail2ban configuration dynamically
fail2ban-client reload

Written by Zyekh Abdul Qadir Jailani

Digital Forensics & Incident Response (DFIR) Specialist & Security Researcher specializing in Linux kernel hardening, threat hunting, and system security research.

LinkedIn • GitHub • Discord • Email • PGP Key