Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/firehol/blocklist-ipsets

ipsets dynamically updated with firehol's update-ipsets.sh script
https://github.com/firehol/blocklist-ipsets

abuses attacks blocklists command-and-control ipset ipset-lists malware

Last synced: about 14 hours ago
JSON representation

ipsets dynamically updated with firehol's update-ipsets.sh script

Awesome Lists containing this project

README 

        
> Due to the amount of data and the frequency of the updates on this repo,

> github has requested to limit the number of updates.

> The site [https://iplists.firehol.org](https://iplists.firehol.org) has direct links

> to all the files in this repo. **This repo is now updated once per day.**



---



### Contents



- [About this repo](#about-this-repo)



- [Using these ipsets](#using-these-ipsets)

 - [Which ones to use?](#which-ones-to-use)



 - [Why are open proxy lists included](#why-are-open-proxy-lists-included)

   

 - [Using them in FireHOL](#using-them-in-firehol)

    * [Adding the ipsets in your firehol.conf](#adding-the-ipsets-in-your-fireholconf)

    * [Updating the ipsets while the firewall is running](#updating-the-ipsets-while-the-firewall-is-running)

    

- [Dynamic List of ipsets included](#list-of-ipsets-included)



- [Comparison of ipsets](#comparison-of-ipsets)



---



# About this repo



This repository includes a list of ipsets dynamically updated with

[FireHOL](https://github.com/firehol/firehol)'s `update-ipsets.sh`

[documented in this wiki](https://github.com/firehol/blocklist-ipsets/wiki).



This repo is self maintained. It it updated automatically from the script via a cron job.



This repo has a site: [http://iplists.firehol.org](http://iplists.firehol.org).



## Why do we need blocklists?



As time passes and the internet matures in our life, cybercrime is becoming increasingly sophisticated.

Although there are many tools (detection of malware, viruses, intrusion detection and prevention systems,

etc) to help us isolate the bad guys, there are now a lot more than just such attacks.



What is more interesting is that the fraudsters or attackers in many cases are not going to do a direct

damage to you or your systems. They will use you and your systems to gain something else, possibly not

related or indirectly related to your business. Nowadays the attacks cannot be identified easily. They are

distributed and come to our systems from a vast amount of IPs around the world.



To get an idea, check for example the [XRumer](http://en.wikipedia.org/wiki/XRumer) software. This thing

mimics human behavior to post ads, it creates email accounts, responds to emails it receives, bypasses

captchas, it goes gently to stay unnoticed, etc.



To increase our effectiveness we need to complement our security solutions with our shared knowledge, our

shared experience in this fight.



Hopefully, there are many teams out there that do their best to identify the attacks and pinpoint the

attackers. These teams release blocklists. Blocklists of IPs (for use in firewalls), domains & URLs

(for use in proxies), etc.



What we are interested here is IPs.



Using IP blocklists at the internet side of your firewall is a key component of internet security. These

lists share key knowledge between us, allowing us to learn from each other and effectively isolate

fraudsters and attackers from our services.



I decided to upload these lists to a github repo because:



1. They are freely available on the internet. The intention of their creators is to help internet security.

 Keep in mind though that a few of these lists may have special licences attached. Before using them, please check their source site for any information regarding proper use.



2. Github provides (via `git pull`) a unified way of updating all the lists together.

 Pulling this repo regularly on your machines, you will update all the IP lists at once.



3. Github also provides a unified version control. Using it we can have a history of what each list has

done, which IPs or subnets were added and which were removed.



## DNSBLs



Check also another tool included in FireHOL v3+, called `dnsbl-ipset.sh`.



This tool is capable of creating an ipset based on your traffic by looking up information on DNSBLs and

scoring it according to your preferences.



More information [here](https://github.com/firehol/firehol/wiki/dnsbl-ipset.sh).



---



# Using these ipsets



Please be very careful what you choose to use and how you use it. If you blacklist traffic using these

lists you may end up blocking your users, your customers, even yourself (!) from accessing your services.



1. Go to to the site of each list and read how each list is maintained. You are going to trust these guys for doing their job right.



2. Most sites have either a donation system or commercial lists of higher quality. Try to support them. 



3. I have included the TOR network in these lists (`bm_tor`, `dm_tor`, `et_tor`). The TOR network is not necessarily bad and you should not block it if you want to allow your users be anonymous. I have included it because for certain cases, allowing an anonymity network might be a risky thing (such as eCommerce).



4. Apply any blacklist at the internet side of your firewall. Be very careful. The `bogons` and `fullbogons` lists contain private, unrouteable IPs that should not be routed on the internet. If you apply such a blocklist on your DMZ or LAN side, you will be blocked out of your firewall.



5. Always have a whitelist too, containing the IP addresses or subnets you trust. Try to build the rules in such a way that if an IP is in the whitelist, it should not be blocked by these blocklists.



## Which ones to use



### Level 1 - Basic



These are the ones I trust. **Level 1** provides basic security against the most well-known attackers, with the minimum of false positives.



1. **Abuse.ch** lists `feodo`, `palevo`, `sslbl`, `zeus`, `zeus_badips`

   

   These folks are doing a great job tracking crime ware. Their blocklists are very focused.

   Keep in mind `zeus` may include some false positives. You can use `zeus_badips` instead.



2. **DShield.org** list `dshield`



   It contains the top 20 attacking class C (/24) subnets, over the last three days.



3. **Spamhaus.org** lists `spamhaus_drop`, `spamhaus_edrop`

   

   DROP (Don't Route Or Peer) and EDROP are advisory "drop all traffic" lists, consisting of netblocks that are "hijacked" or leased by professional spam or cyber-crime operations (used for dissemination of malware, trojan downloaders, botnet controllers).

   According to Spamhaus.org:



   > When implemented at a network or ISP's 'core routers', DROP and EDROP will help protect the network's users from spamming, scanning, harvesting, DNS-hijacking and DDoS attacks originating on rogue netblocks.

   > 

   > Spamhaus strongly encourages the use of DROP and EDROP by tier-1s and backbones.



 Spamhaus is very responsive to adapt these lists when a network owner updates them that the issue has been solved (I had one such incident with one of my users).



4. **Team-Cymru.org** list `bogons` or `fullbogons`



   These are lists of IPs that should not be routed on the internet. No one should be using them.

   Be very careful to apply either of the two on the internet side of your network.



### Level 2 - Essentials



**Level 2** provide protection against current brute force attacks. This level may have a small percentage of false positives, mainly due to dynamic IPs being re-used by other users.



1. **OpenBL.org** lists `openbl*`

   

   The team of OpenBL tracks brute force attacks on their hosts. They have a very short list for hosts, under their own control, collecting this information, to eliminate false positives.

   They suggest to use the default blacklist which has a retention policy of 90 days (`openbl`), but they also provide lists with different retention policies (from 1 day to 1 year).

   Their goal is to report abuse to the responsible provider so that the infection is disabled.



2. **Blocklist.de** lists `blocklist_de*`

   

   Is a network of users reporting abuse mainly using `fail2ban`. They eliminate false positives using other lists available. Since they collect information from their users, their lists may be subject to poisoning, or false positives.

   I asked them about poisoning. [Here](https://forum.blocklist.de/viewtopic.php?f=4&t=244&sid=847d00d26b0735add3518ff515242cad) you can find their answer. In short, they track it down so that they have an ignorable rate of false positives.

   Also, they only include individual IPs (no subnets) which have attacked their users the last 48 hours and their list contains 20.000 to 40.000 IPs (which is small enough considering the size of the internet).

   Like `openbl`, their goal is to report abuse back, so that the infection is disabled.

   They also provide their blocklist per type of attack (mail, web, etc).



Of course, there are more lists included. You can check them and decide if they fit for your needs.



## Why are open proxy lists included



Of course, I haven't included them for you to use the open proxies. The port the proxy is listening, or the type of proxy, are not included (although most of them use the standard proxy ports and do serve web requests).



If you check the comparisons for the open proxy lists (`ri_connect_proxies`, `ri_web_proxies`, `xroxy`, `proxz`, `proxyrss`, etc)

you will find that they overlap to a great degree with other blocklists, like `blocklist_de`, `stopforumspam`, etc.



> This means the attackers also use open proxies to execute attacks.



So, if you are under attack, blocking the open proxies may help isolate a large part of the attack.



I don't suggest to permanently block IPs using the proxy lists. Their purpose of existence is questionable.

Their quality though may be acceptable, since lot of these sites advertise that they test open proxies before including them in their lists, so that there are no false positives, at least at the time they tested them.



---



## Using them in FireHOL



`update-ipsets.sh` itself does not alter your firewall. It can be used to update ipsets both on disk and in the kernel for any firewall solution you use.



The information below, shows you how to configure FireHOL to use the provides ipsets.



### Adding the ipsets in your firehol.conf



I use something like this:



```sh

	# our wan interface

	wan="dsl0"

	

	# our whitelist

	ipset4 create whitelist hash:net

	ipset4 add whitelist A.B.C.D/E # A.B.C.D/E is whitelisted

	

	# subnets - netsets

	for x in fullbogons dshield spamhaus_drop spamhaus_edrop

	do

		ipset4 create  ${x} hash:net

		ipset4 addfile ${x} ipsets/${x}.netset

		blacklist4 full inface "${wan}" log "BLACKLIST ${x^^}" ipset:${x} \

			except src ipset:whitelist

	done



	# individual IPs - ipsets

	for x in feodo palevo sslbl zeus openbl blocklist_de

	do

		ipset4 create  ${x} hash:ip

		ipset4 addfile ${x} ipsets/${x}.ipset

		blacklist4 full inface "${wan}" log "BLACKLIST ${x^^}" ipset:${x} \

			except src ipset:whitelist

	done



	... rest of firehol.conf ...

```



If you are concerned about iptables performance, change the `blacklist4` keyword `full` to `input`.

This will block only inbound NEW connections, i.e. only the first packet for every NEW inbound connection will be checked.

All other traffic passes through unchecked.



> Before adding these rules to your `firehol.conf` you should run `update-ipsets.sh` to enable them.



### Updating the ipsets while the firewall is running



Just use the `update-ipsets.sh` script from the firehol distribution.

This script will update each ipset and call firehol to update the ipset while the firewall is running.



> You can add `update-ipsets.sh` to cron, to run every 10 mins. `update-ipsets.sh` is smart enough to download

> a list only when it needs to.



---



# List of ipsets included