{"id":15945779,"url":"https://github.com/ig3/blacklistd","last_synced_at":"2025-05-14T11:15:42.069Z","repository":{"id":77186268,"uuid":"345382978","full_name":"ig3/blacklistd","owner":"ig3","description":null,"archived":false,"fork":false,"pushed_at":"2021-03-08T18:54:56.000Z","size":124,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2024-12-25T20:24:35.206Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Perl","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ig3.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-03-07T15:33:40.000Z","updated_at":"2021-03-08T18:54:59.000Z","dependencies_parsed_at":null,"dependency_job_id":"29e2dd09-9b5f-4d49-9787-5c356926e6ef","html_url":"https://github.com/ig3/blacklistd","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ig3%2Fblacklistd","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ig3%2Fblacklistd/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ig3%2Fblacklistd/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ig3%2Fblacklistd/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ig3","download_url":"https://codeload.github.com/ig3/blacklistd/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":239225766,"owners_count":19603162,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-07T09:06:33.174Z","updated_at":"2025-02-17T02:41:42.229Z","avatar_url":"https://github.com/ig3.png","language":"Perl","funding_links":[],"categories":[],"sub_categories":[],"readme":"# blacklistd\n\nBlacklistd is a Perl script that listens for output\nfrom [logsurfer](https://github.com/k3/logsurfer)\nand maintains [nftables](https://www.nftables.org/)\nblacklist and whitelist.\n\nThis repository contains the blacklistd Perl script,\na slightly modified version of logsurfer (modifiecations\nare available from [ig3/logsufer](https://github.com/ig3/logsurfer)),\na few logsurfer configurations and systemd service files to run\nblacklistd and the logsurfer instances.\n\n## Prerequisites\n\n### logsurfer\nIf logsurfer isn't available as a package on your distribution, you can\neasily build it from [source](https://github.com/k3/logsurfer). For some\nminor enhancements you may find helpful, you can build from\n[this fork](https://github.com/ig3/logsurfer).\n\n### nftables\nSee [nftables from distributions](https://wiki.nftables.org/wiki-nftables/index.php/Nftables_from_distributions) \nif you want to install it from a package.\n\nSee [building and installing nftables from source](https://wiki.nftables.org/wiki-nftables/index.php/Building_and_installing_nftables_from_sources)\nif you want to build it from source.\n\n### Perl\nThe package from your distribution will suffice.\n\n#### Config::INI::Reader\n$ sudo cpan install Config::INI::Reader\n\n## Installation\n\nClone this repository and run the install script.\n\nThis will install most of the script and configuration to /usr/local/bin and\n/usr/local/etc. Systemd service files are installed to /etc/systemd/system.\n\nIn nftables.conf, add a set for the blacklist:\n\n```\n    set blacklist {\n        type ipv4_addr\n        # The interval flag allows network addresses (e.g. 192.168.0.0/24) in the set\n        flags interval\n    }\n```\n\nAt an appropriate point in you input chain, add the blacklist:\n\n```\n        ip saddr @blacklist counter name dropped drop\n```\n\nAnd include the blacklist itself:\n\n```\ninclude \"/usr/local/etc/nftables.d/*.nft\"\n```\n\nA rudimentary configuration might look like:\n\n```\n#!/usr/sbin/nft -f\n\nflush ruleset\n\ntable inet filter {\n    counter dropped {\n      packets 0 bytes 0\n    }\n    set blacklist {\n        type ipv4_addr\n        # The interval flag allows network addresses (e.g. 192.168.0.0/24) in the set\n        flags interval\n    }\n    set whitelist {\n        type ipv4_addr\n        # The interval flag allows network addresses (e.g. 192.168.0.0/24) in the set\n        flags interval\n    }\n    chain LOG_DROP {\n        log prefix \"iptables LOG_DROP: \" level debug\n        drop\n    }\n    chain LOG_ACCEPT {\n        log prefix \"iptables LOG_ACCEPT: \" level debug\n        accept\n    }\n    chain input {\n        type filter hook input priority 0;\n        ct state related,established accept\n        iifname \"lo\" accept\n        meta l4proto tcp ip saddr 192.168.1.0/24 tcp dport 22 accept\n        ip saddr @blacklist counter name dropped drop\n        meta l4proto tcp tcp dport 22 jump LOG_ACCEPT\n        meta l4proto tcp tcp dport 443 jump LOG_ACCEPT\n        limit rate 5/minute log prefix \"iptables denied: \" level debug\n        drop\n    }\n    chain forward {\n        type filter hook forward priority 0;\n    }\n    chain output {\n        type filter hook output priority 0;\n    }\n}\n\ninclude \"/usr/local/etc/nftables.d/*.nft\"\n```\n\nReview and modify the syslogd services as appropriate to your system:\n\n * logsurfer_auth.service\n * logsurfer_blacklistd.service\n * logsurfer_mail.service\n * logsurfer_nginx.service\n * nftables.service\n\nIn particular, you may prefer to continue with your system default nftables\nservice. The one here runs nftables with configuration in\n/usr/local/etc/nftables, and additional configurations (the blacklist) in\n/usr/local/etc/nftables.d/blacklist.nft (by way of an include in\n/usr/local/etc/nftables.conf).\n\nConfigurations for logsurfer are in /usr/local/etc/logsurfer.\n\nConfiguration for blacklistd is /usr/local/etc/blacklist/blacklistd.conf\n\nThe whitelist and FIFO are in /usr/local/data/blacklist.\n\n## Motivation\n\nYears ago, Internet connected servers I managed began to be subject to\nintrusion attempts on all their exposed services. I configured firewall with\nsome publicly available blacklists but still they were under constant\nattack. I wanted something that would add IP addresses to the blacklist\nbased on local logs.\n\nI reviewed open source packages available at the time and ultimately decided\nto implement this system based on logsurfer, which I had used for many\nyears, on many systems, for log scanning and alerting.\n\nI ran fail2ban for a while but was frustrated by limited documentation\nmaking it difficult to achieve the configurations I wanted. You should\nconsider it but for me, logsurfer was easier to work with.\n\n## Features\n\nIntrusion detection is done by logsurfer scanning log files.\n\nLogsurfer allows implementation of complex correlations but the provided\nconfigurations are simple. \n\nThe blacklist and whitelist are maintained by a simple Perl script that\nreads from a named pipe / FIFO for text inputs from logsurfer. These have\nthe form 'list address note' where list is one of blacklist or whitelist,\naddress is a single IP address and note is arbitrary text to be logged,\ndescribing why the address was added.\n\nBans are permanent and immediate. A failed login attempt, failed access to\nSMTP server or web server results in an immediate, permanent blacklist of\nthe IP address, unless the IP is already in the whitelist.\n\nA successful SSH authentication adds an address to the whitelist.\n\nAddresses may be added to the blacklist or whitelist manually. The blacklist\nis an nftables configuration file. The whitelist is a simple text file. Each\nline adds one IP address to the whitelist, with the form 'address comment',\nwhere address is the IP address and comment is arbitrary text.\n\nAddresses in the whitelist, local host address range (127.0/16) and common\nlocal network (192.168/16) will not be added to the blacklist. To change the\nlatter exclusions, you will have to edit the Perl script itself.\n\n## Description\n\nOne instance of logsurfer is run for each log file to be scanned. Each has\nits own configuration file, with patterns (regular expressions) to identify\naddresses to be added to the blacklist or a separately maintained whitelist.\nThese patterns are very simple - matching single lines, at the moment.\n\nThe whitelist isn't referenced in iptables but an address on the whitelist\nwill never be added to the blacklist. Addresses from trusted networks from\nwhich access is required should be added to whitelist. This will happen\nautomatically after successful ssh login or SENDING email.\n\nCurrent configuration / scanning rules are from a system running\nArmbian as a server hosting websites, git and npm repositories and providing\nemail service, accessible via ssh (headless server). There is configuration\nfor scanning auth.log, mail.log and nginx/access.log. It is just a start.\n\n\n## Why logsurfer\n\nIt is a familiar old tool that works well.\n\nAny log scanner could be used. All that is required is to write commands to\nthe blacklistd named pipe / FIFO with the form 'list address comment',\nwhere list is blacklist or whitelist and address is the IP address to be\nadded. The comment is logged but is otherwise ignored.\n\nLogsurfer is modified to work on modern Debian:\n\n * Added a missing function declaration\n * Add -D option to install, to create missing directories\n * Reopen logfile if it becomes smaller\n\nMy log syslog/log roller truncates log files rather than creating new files.\nLogsurfer did not detect this truncation, but it was easy to add detection\nof file size becoming smaller. \n\nOtherwise and in particular the rule processing is all standard logsurfer.\n\n## Why nftables\n\nI used iptables until I had a problem with it and on investigation found that\n[iptables is deprecated in debian](https://packages.debian.org/buster/iptables).\nSo, I took their advice and switched to using nftables. \n\n## Why Perl\n\nBecause it is stable, familiar and adequate to the task. The script is\nsimple. It is just text processing (a Perl forte). I might reimplement it in\nJavaScript on Node.\n\n\n## Logging\n\nThe blacklistd script logs to syslog facility local0. \n\n## TODO\n\nConfiguration needs to be refactored. Currently, it is in both\n/usr/local/etc and /etc, which is silly.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fig3%2Fblacklistd","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fig3%2Fblacklistd","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fig3%2Fblacklistd/lists"}