Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/jrbrtsn/ban2fail
Simple & efficient log file scanning and iptable filtering
https://github.com/jrbrtsn/ban2fail
anti-bot attack-prevention attackers ban-hosts c fail2ban gplv3 intrusion-prevention iptables iptables-filtering iptables-rules linux-server loganalyzer logfile-analysis logs monitoring scanning security security-tools zero-trust
Last synced: about 1 month ago
JSON representation
Simple & efficient log file scanning and iptable filtering
- Host: GitHub
- URL: https://github.com/jrbrtsn/ban2fail
- Owner: jrbrtsn
- License: gpl-3.0
- Created: 2019-11-23T03:38:11.000Z (about 5 years ago)
- Default Branch: master
- Last Pushed: 2022-08-13T20:22:11.000Z (over 2 years ago)
- Last Synced: 2024-08-02T02:12:05.527Z (4 months ago)
- Topics: anti-bot, attack-prevention, attackers, ban-hosts, c, fail2ban, gplv3, intrusion-prevention, iptables, iptables-filtering, iptables-rules, linux-server, loganalyzer, logfile-analysis, logs, monitoring, scanning, security, security-tools, zero-trust
- Language: C
- Homepage:
- Size: 369 KB
- Stars: 27
- Watchers: 3
- Forks: 7
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- awesome-hacking-lists - jrbrtsn/ban2fail - Simple & efficient log file scanning and iptable filtering (C)
README
# ban2fail
(C) 2019 John D. Robertson
**ban2fail** is a simple and efficient tool to coordinate log file scanning,
reporting, and iptables filtering. As the name implies, *ban2fail* was
inspired by the popular *fail2ban* project (http://fail2ban.org). The main
technical advantages *ban2fail* provides over *fail2ban* are:
+ When installed as a systemd service, filtering response is well under 0.5
second.
+ By design all relevant logfiles on disk are scanned, not just the current
log files.
+ A unique and transparent caching scheme is employed to make this process at
least 100x as fast as doing the same thing with, say, *grep*.
+ Instantaneously and conveniently produces on command all offending logfile
entries which exist somewhere in the logfile history for given address(es).
+ Easily handles hundreds of thousands of blocked IP addresses.
+ Directly calls iptables, and handles filtering rules in one fell swoop using
fork() & execve() calls.
+ Provides integrated reporting with reverse and forward DNS information.
+ When reporting, DNS lookups are performed in parallel with 200 simultaneous
lookups.
+ Written in pure C, with less than 15,000 lines of source code. Your CPU has
better things to do than load the python runtime 10,000 times a day ;-)
+ Efficient enough to run every 0.4 seconds without monopolizing a CPU core on a
modest server.
+ In the case of a reboot or after iptables filters get flushed, blocking rules
will be instantaneously generated from entire log file histories (long memory).
*ban2fail* started with a few hours of frenzied C hacking after my mail server
was exploited to deliver spam for others who had cracked a user's SMTP send
password. After inspecting the log files I realized that crackers are now using
widely distributed attacks, and that I would need an extremely efficient tool
that could scan my entire log file history in a fraction of a second on my
rather modest Linode virtual server to have a chance of stopping them. Here are
the timing results for a typical scan on my server:
```
real 0m0.325s
user 0m0.186s
sys 0m0.150s
```
Currently I am running *ban2fail* from a *systemd* service file which triggers
*ban2fail* whenever a watched log file is modified. This gives attackers at
most a 0.4 second window to do their worst. I hope you find this code useful.
## Reports
There are 4 basic types of reporting provided:
+ summary
+ per-address
+ per-country
+ log file entries for specific address(es)
### Summary
The summary report is very brief, but useful for spotting sudden changes
in numbers of filtered addresses:
```
ban2fail -s
155 countries affected
7669 addresses currently blocked
```
### Per-Address
Per-address is the most useful report to a system administrator looking for
overly zealous address blocking. Records appear sorted with the most recent
offenders on top. Here is a truncated sample report with full DNS information:
```
ban2fail -a+
=============== ban2fail v0.13.10 =============
Scanning "/var/log/auth.log"... found 666 offenses (565 addresses)
Scanning "/var/log/auth.log.1"... found 1143 offenses (1075 addresses)
Scanning "/var/log/auth.log.2.gz"... found 217 offenses (174 addresses)
Scanning "/var/log/auth.log.3.gz"... found 966 offenses (429 addresses)
Scanning "/var/log/auth.log.4.gz"... found 267 offenses (168 addresses)
>>>> Found 3259 offenses (2399 addresses) for /var/log/auth.log*
Scanning "/var/log/exim4/mainlog"... found 92 offenses (90 addresses)
Scanning "/var/log/exim4/mainlog.1"... found 319 offenses (284 addresses)
Scanning "/var/log/exim4/mainlog.2.gz"... found 329 offenses (302 addresses)
Scanning "/var/log/exim4/mainlog.3.gz"... found 315 offenses (295 addresses)
Scanning "/var/log/exim4/mainlog.4.gz"... found 521 offenses (214 addresses)
Scanning "/var/log/exim4/mainlog.5.gz"... found 254 offenses (229 addresses)
Scanning "/var/log/exim4/mainlog.6.gz"... found 234 offenses (214 addresses)
Scanning "/var/log/exim4/mainlog.7.gz"... found 256 offenses (238 addresses)
Scanning "/var/log/exim4/mainlog.8.gz"... found 231 offenses (218 addresses)
Scanning "/var/log/exim4/mainlog.9.gz"... found 330 offenses (213 addresses)
Scanning "/var/log/exim4/mainlog.10.gz"... found 882 offenses (351 addresses)
Scanning "/var/log/exim4/mainlog.11.gz"... found 1795 offenses (504 addresses)
Scanning "/var/log/exim4/mainlog.12.gz"... found 2222 offenses (1201 addresses)
Scanning "/var/log/exim4/mainlog.13.gz"... found 732 offenses (349 addresses)
Scanning "/var/log/exim4/mainlog.14.gz"... found 736 offenses (370 addresses)
>>>> Found 9248 offenses (4997 addresses) for /var/log/exim4/mainlog*
Scanning "/var/log/apache2/access.log"... found 18 offenses (7 addresses)
Scanning "/var/log/apache2/access.log.1"... found 734 offenses (54 addresses)
Scanning "/var/log/apache2/access.log.2.gz"... found 649 offenses (23 addresses)
Scanning "/var/log/apache2/access.log.3.gz"... found 383 offenses (6 addresses)
Scanning "/var/log/apache2/access.log.4.gz"... found 235 offenses (4 addresses)
Scanning "/var/log/apache2/access.log.5.gz"... found 75 offenses (4 addresses)
Scanning "/var/log/apache2/access.log.6.gz"... found 156 offenses (2 addresses)
Scanning "/var/log/apache2/access.log.7.gz"... found 815 offenses (6 addresses)
Scanning "/var/log/apache2/access.log.8.gz"... found 582 offenses (27 addresses)
Scanning "/var/log/apache2/access.log.9.gz"... found 1001 offenses (16 addresses)
Scanning "/var/log/apache2/access.log.10.gz"... found 802 offenses (8 addresses)
Scanning "/var/log/apache2/access.log.11.gz"... found 386 offenses (5 addresses)
Scanning "/var/log/apache2/access.log.12.gz"... found 466 offenses (78 addresses)
Scanning "/var/log/apache2/access.log.13.gz"... found 524 offenses (25 addresses)
Scanning "/var/log/apache2/access.log.14.gz"... found 565 offenses (16 addresses)
>>>> Found 7391 offenses (255 addresses) for /var/log/apache2/access.log*
Scanning "/var/log/mail.log"... found 608 offenses (457 addresses)
Scanning "/var/log/mail.log.1"... found 964 offenses (894 addresses)
Scanning "/var/log/mail.log.2.gz"... found 170 offenses (5 addresses)
Scanning "/var/log/mail.log.3.gz"... found 0 offenses (0 addresses)
Scanning "/var/log/mail.log.4.gz"... found 0 offenses (0 addresses)
>>>> Found 1742 offenses (1351 addresses) for /var/log/mail.log*
===== Found 21640 total offenses (7660 addresses) =====
Performing DNS lookups for up to 60 seconds ...
==> Completed 7660 of 7660 lookups in 60.0 seconds
3 Dec 07 07:09 1/0 offenses US [BLK] 184.179.216.156 NXDOMAIN
0 Dec 07 07:08 1/0 offenses US [BLK] 23.254.228.40 client-23-254-228-40.hostwindsdns.com !!
2 Dec 07 07:04 1/0 offenses HK [BLK] 47.91.220.119 NXDOMAIN
3 Dec 07 06:57 2/0 offenses TR [BLK] 81.214.245.188 81.214.245.188.dynamic.ttnet.com.tr !!
0 Dec 07 06:51 1/0 offenses -- [BLK] 188.119.103.157 NXDOMAIN
0 Dec 07 06:50 1/0 offenses GB [BLK] 185.217.230.211 SERVFAIL
0 Dec 07 06:47 1/0 offenses RU [BLK] 92.38.176.115 journalagricult.casa !
0 Dec 07 06:47 1/0 offenses CO [BLK] 191.103.219.225 xdsl-191-103-219-225.edatel.net.co !!
4 Dec 07 06:46 2/0 offenses CN [BLK] 175.6.5.233 NXDOMAIN
0 Dec 07 06:44 1/0 offenses UA [BLK] 109.87.78.144 144.78.87.109.triolan.net !!
0 Dec 07 06:36 1/0 offenses BR [BLK] 200.209.34.34 NXDOMAIN
3 Dec 07 06:35 2/0 offenses US [BLK] 184.179.216.138 NXDOMAIN
3 Dec 07 06:31 2/0 offenses CN [BLK] 120.33.205.162 NXDOMAIN
0 Dec 07 06:30 1/0 offenses SG [BLK] 160.20.12.205 SERVFAIL
--- snipped ---
===============================================
7660 addresses currently blocked
```
### Per-Country
Per-country reports show how blocked address are distributed between the countries:
```
ban2fail -c
=============== ban2fail v0.13.10 =============
Scanning "/var/log/auth.log"... found 666 offenses (565 addresses)
Scanning "/var/log/auth.log.1"... found 1143 offenses (1075 addresses)
--- snipped ---
===== Found 21642 total offenses (7661 addresses) =====
US 1335 blocked addresses
CN 852 blocked addresses
VN 619 blocked addresses
RU 432 blocked addresses
BR 414 blocked addresses
IN 244 blocked addresses
GB 197 blocked addresses
-- 196 blocked addresses
AR 175 blocked addresses
TH 141 blocked addresses
FR 140 blocked addresses
ID 136 blocked addresses
--- snipped ---
===============================================
155 countries affected
```
### Log File Entries for Given Address(es)
Before unblocking any address it is usually necessary to investigate the
offending log file entries. Here is an example:
```
ban2fail 184.179.216.156 23.254.228.40
====== Report for 184.179.216.156 ======
------- /var/log/auth.log -------------
Dec 7 07:09:52 srv auth: pam_unix(dovecot:auth): authentication failure; logname= uid=0 euid=0 tty=dovecot [email protected] rhost=184.179.216.156
------- /var/log/mail.log -------------
Dec 7 07:12:29 srv dovecot: imap-login: Disconnected: Inactivity (auth failed, 1 attempts in 160 secs): user=, method=PLAIN, rip=184.179.216.156, lip=50.116.38.131, TLS, session=
====== Report for 23.254.228.40 ======
------- /var/log/exim4/mainlog -------------
2019-12-07 07:08:27 H=(green.medifeetz.icu) [23.254.228.40] F=<[email protected]> rejected RCPT : 23.254.228.40 is listed at zen.spamhaus.org (127.0.0.3: https://www.spamhaus.org/sbl/query/SBLCSS)
```
## Configuration
*ban2fail* works from a configuration file found at
"/etc/ban2fail/ban2fail.cfg". The overarching premise is that if any REGEX
appearing in a LOGTYPE clause matches a line in an associated log file, then by
default that IP will be blocked.
```
LOGTYPE auth {
# Where to find the log files
DIR= /var/log
PREFIX= auth.log
# How to read the timestamp
TIMESTAMP auth_ts {
# isolates the timestamp from a line matched by a TARGET
REGEX= ^(.*) srv
# Passed to strptime() to intrepret the timestamp string
STRPTIME= %b %d %T
# These stamps do not include the year, so it is implied.
FLAGS= GUESS_YEAR
}
TARGET imap {
# Pattern to search for, isolates the IP address
REGEX= imapd.*Login failed.*\[([0-9.a-f:]+)\]$
# Assign this as the severity of the offense.
SEVERITY= 3
}
TARGET ssh {
SEVERITY= 4
REGEX= sshd.*Failed password.*from ([0-9.a-f:]+) port [0-9]+ ssh2$
REGEX= sshd.*Invalid user.*from ([0-9.a-f:]+) port
}
TARGET negotiate_fail {
SEVERITY= 2
REGEX= Unable to negotiate with ([0-9.a-f:]+) port
}
TARGET dovecot {
SEVERITY= 3
REGEX= dovecot.*authentication failure.*rhost=([0-9.a-f:]+)
}
}
```
Syntax in the config file is pretty much the same as the nftables syntax. All
keywords must be in upper case. Any values in the key=value pairs have
whitespace stripped from the beginning and end of the line. Since there is
little escaping of characters going on, regular expressions are mostly WYSIWYG.
If you have a hash symbol '#' or a double quote '"' in your pattern (which are
special characters for the config file parser), you will need to escape
them like so:
```
# Nov 27 02:03:03 srv named[764]: client @0x7fe6a0053420 1.192.90.183#27388 (www.ipplus360.com): query (cache) 'www.ipplus360.com/A/IN' denied
REGEX= named.*client.* ([0-9.a-f:]+)\#.*denied$
```
Finding typos and so forth in the config file is easy; use the -v command flag
to print all unrecognized content (besides comments).
`ban2fail -v`
The only way to alter the default blocking behavior is with a MAX\_OFFENSES
clause. This clause allows you specify how many offenses are tolerated before an
IP is blocked. Offenses will naturally disappear as old logfiles are deleted by
*logrotate*.
```
# Whitelist ourself
MAX_OFFENSES -1 {
# Put your server's IP addresses here
# IP= 1.2.3.4
IP= 127.0.0.1
# IP= dead:beef::20::32a
IP= ::1
}
# Allegedly legit servers
MAX_OFFENSES 50 {
# Google Ireland
IP= 2a00:1450:4864:20::32a
IP= 2a00:1450:4864:20::336
# Google EU
# Attempted to break in
# IP= 35.205.240.168
# Google US
IP= 09.85.216.42
# Attempted to break in
# IP= 130.211.246.128
IP= 209.85.166.194
IP= 209.85.166.195
IP= 209.85.208.67
IP= 209.85.214.194
IP= 209.85.215.173
IP= 209.85.215.175
IP= 209.85.215.193
IP= 209.85.216.42
IP= 2607:f8b0:4864:20::1034
IP= 2607:f8b0:4864:20::a46
# Yahoo
IP= 106.10.244.139
# Outlook
IP= 40.92.4.30
IP= 40.107.73.61
IP= 40.107.74.48
IP= 40.107.74.72
IP= 40.107.76.74
IP= 40.107.79.52
IP= 40.107.79.59
IP= 40.107.80.40
IP= 40.107.80.53
IP= 40.107.80.78
IP= 40.107.82.75
IP= 52.101.129.30
IP= 52.101.132.108
IP= 52.101.136.79
IP= 52.101.140.230
}
# "trusted" addresses
MAX_OFFENSES 200 {
# me from home
# IP= 1.2.3.4/20
# Customer
# IP= 5.6.7.8/24
}
```
If you recieve a complaint about an address unjustly getting blocked, place it
in one of the MAX\_OFFENSES blocks, and the IP will be unblocked the next time
*ban2fail* runs in production mode.
## Working with *ban2fail*
There are two primary modes in which *ban2fail* is used:
* Production mode, where iptables rules are modified.
* Testing mode, where modifications to blocking rules are merely indicated.
### Production
In production mode it is expected that *ban2fail* is running non-interactively,
and no output is printed unless addresses are (un)blocked. It is also possible
to generate a listing of addresses, offense counts, and status with the -a
command flag. Likewise, a listing of countries and offense counts is available
with the *-c* flag. In order to get DNS information for the *-a* flag, follow
with a plus for all DNS info *-a+*, or a minus for only legit (backward &
forward match) info *-a-*. In the list, DNS issues are presented like so:
```
# DNS is good
0 Dec 06 08:31 1/0 offenses AR [BLK] 200.71.237.244 host244.200-71-237.telecom.net.ar
# Reverse lookup failed with DNS server
0 Dec 05 19:43 1/0 offenses GB [BLK] 185.217.230.146 SERVFAIL
# Reverse lookup is a non-existent domain
2 Dec 05 21:11 1/0 offenses US [BLK] 67.205.153.94 NXDOMAIN
# Forward lookup does not match reverse lookup
0 Dec 06 08:40 1/0 offenses LU [BLK] 92.38.132.54 ibocke43.monster !
# Forward DNS record does not exist
0 Dec 06 08:37 1/0 offenses US [BLK] 63.81.90.135 63-81-90-135.nca.lanset.com !!
# DNS is inconclusive due to lack of response from a DNS server
0 Dec 05 22:04 1/0 offenses RU [BLK] 77.221.144.107 news5.burningcoalsa.com ~
```
If you want to see the offending log lines for specific address(es), supply
them on the command line like so:
```
john@srv:~$ ban2fail 68.183.105.52
====== Report for 68.183.105.52 ======
------- /var/log/auth.log -------------
Dec 5 17:50:47 srv sshd[22326]: Invalid user cron from 68.183.105.52 port 41874
Dec 5 17:50:48 srv sshd[22326]: Failed password for invalid user cron from 68.183.105.52 port 41874 ssh2
```
### Testing
In test mode (-t flag) the presumption is that you are testing a modified
configuration which is not yet in place, and that you don't want to disturb the
production setup. This is how you might do that:
`ban2fail -t myNew.cfg -a`
No iptables rules will be modified. You will be shown in the listing which
addresses would be (un)blocked if the contents of "myNew.cfg" was in place, and
*ban2fail* was running in production mode.
When you are happy with the new configuration, copy it into place, and the the
iptable rule changes will be realized the next time *ban2fail* runs in
production mode.
## Building the Project
I've tested *ban2fail* only on Debian Buster, but it should compile on just
about any modern Linux distro. It uses the following libraries:
+ *libcrypto* from the libssl package, for md5 checksums
+ *libGeoIP* to identify the country of origin for IP addresses
+ *libz* to read compressed log files
+ *libpthread* for parallel DNS lookups (200 simultaneous)
+ *libdb* caching of offense location and size in log files
Build and install like so:
```
make release
make install
```
The make *install* target calls *install.sh*, which does a bunch of stuff
including setting up and enabling a systemd service, so you might want have a
look before pulling the trigger.
*ban2fail.service* points to *ban2fail.sh*, which can be tested from the command line for debugging. Remember to make sure the service is disabled:
```
systemctl stop ban2fail
```
In order to run *ban2fail* as non-root user, the user must belong to group
'adm'. This is so in order to run iptables, which is accomplished via setuid(0)
at the appropriate time.