https://github.com/misp/misp-bloomfilter

A tool to create bloom filters from MISP records to share IOCs with others without breaking confidentiality.
https://github.com/misp/misp-bloomfilter

Last synced: 11 months ago
JSON representation

A tool to create bloom filters from MISP records to share IOCs with others without breaking confidentiality.

• Host: GitHub
• URL: https://github.com/misp/misp-bloomfilter
• Owner: MISP
• Created: 2013-05-03T17:58:12.000Z (about 13 years ago)
• Default Branch: master
• Last Pushed: 2013-07-24T12:08:49.000Z (almost 13 years ago)
• Last Synced: 2024-03-26T04:54:10.377Z (about 2 years ago)
• Language: Python
• Homepage:
• Size: 109 KB
• Stars: 6
• Watchers: 14
• Forks: 7
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

          
misp-bloomfilter

================

misp-bloomfilter is a tool creating a bloom filter from records in a MISP XML export.

The created bloom filter database can be then used to query logs files

without having to share the MISP database where the analysis

is performed. This can be used as an alternative way to lookup IOCs without

having to share these directly.

Requirements

------------

* Python

* pybloomfilter (https://github.com/axiak/pybloomfiltermmap/)

Security Considerations

-----------------------

Be aware that you can check against a bloom filter any finite

set of data. Especially IP addresses (2^32) can be easily enumerated

from a bloom filter. If you are using bloom filters to avoid sharing

the records from MISP, you should verify if the set cannot be easily

enumerated.

Usage

-----

        Usage: misp-bloomfilter.py url(s)

        Options:

          -h, --help            show this help message and exit

          -t RECORDTYPE, --type=RECORDTYPE

                                type of the record (default record is 'domain')

          -f FILENAME, --file=FILENAME

                                filename of the MISP XML file to read (default MISP

                                XML dump is 'misp.xml')

          -l LOOKUP, --lookup=LOOKUP

                                lookup a value in a bloomfilter

          -s, --streamlookup    lookup a set of value from stdin in a bloomfilter

          -d DBDIR, --dbdir=DBDIR

                                Bloom filters db directory (default is '.')

          -u URL, --url=URL     url to access MISP

          -a AUTHKEY, --authkey=AUTHKEY

                                authentication key to access MISP

Example

-------

Creating a bloomfilter database from the domain record type:

    python misp-bloomfilter.py -f ../in/misp.xml -d ../db/ -t domain

Creating a bloomfilter database from the ip-dst type using the REST MISP API:

    python misp-bloomfilter.py -u https://misp.server/ -a  -d ../db/ -t ip-dst

Testing the database for the existence of a record:

    python misp-bloomfilter.py -f ../in/misp.xml -d ../db/ -t domain -l foo.bar

    foo.bar True

License

-------

This software is licensed under GNU Affero General Public License version 3.

Copyright (c) 2012, 2013 Alexandre Dulaunoy (a AT foo be)