{"id":27976487,"url":"https://github.com/misp/misp-bloomfilter","last_synced_at":"2025-07-24T16:33:52.801Z","repository":{"id":142313533,"uuid":"9841557","full_name":"MISP/misp-bloomfilter","owner":"MISP","description":"A tool to create bloom filters from MISP records to share IOCs with others without breaking confidentiality.","archived":false,"fork":false,"pushed_at":"2013-07-24T12:08:49.000Z","size":112,"stargazers_count":6,"open_issues_count":0,"forks_count":7,"subscribers_count":14,"default_branch":"master","last_synced_at":"2024-03-26T04:54:10.377Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/MISP.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2013-05-03T17:58:12.000Z","updated_at":"2022-02-20T15:43:27.000Z","dependencies_parsed_at":"2023-03-13T06:56:01.795Z","dependency_job_id":null,"html_url":"https://github.com/MISP/misp-bloomfilter","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MISP%2Fmisp-bloomfilter","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MISP%2Fmisp-bloomfilter/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MISP%2Fmisp-bloomfilter/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MISP%2Fmisp-bloomfilter/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/MISP","download_url":"https://codeload.github.com/MISP/misp-bloomfilter/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252980630,"owners_count":21835276,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-05-08T01:27:18.368Z","updated_at":"2025-05-08T01:27:18.952Z","avatar_url":"https://github.com/MISP.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"misp-bloomfilter\n================\n\nmisp-bloomfilter is a tool creating a bloom filter from records in a MISP XML export.\nThe created bloom filter database can be then used to query logs files\nwithout having to share the MISP database where the analysis\nis performed. This can be used as an alternative way to lookup IOCs without\nhaving to share these directly.\n\nRequirements\n------------\n\n* Python\n* pybloomfilter (https://github.com/axiak/pybloomfiltermmap/)\n\nSecurity Considerations\n-----------------------\n\nBe aware that you can check against a bloom filter any finite\nset of data. Especially IP addresses (2^32) can be easily enumerated\nfrom a bloom filter. If you are using bloom filters to avoid sharing\nthe records from MISP, you should verify if the set cannot be easily\nenumerated.\n\nUsage\n-----\n\n        Usage: misp-bloomfilter.py url(s)\n\n        Options:\n          -h, --help            show this help message and exit\n          -t RECORDTYPE, --type=RECORDTYPE\n                                type of the record (default record is 'domain')\n          -f FILENAME, --file=FILENAME\n                                filename of the MISP XML file to read (default MISP\n                                XML dump is 'misp.xml')\n          -l LOOKUP, --lookup=LOOKUP\n                                lookup a value in a bloomfilter\n          -s, --streamlookup    lookup a set of value from stdin in a bloomfilter\n          -d DBDIR, --dbdir=DBDIR\n                                Bloom filters db directory (default is '.')\n          -u URL, --url=URL     url to access MISP\n          -a AUTHKEY, --authkey=AUTHKEY\n                                authentication key to access MISP\n\nExample\n-------\n\nCreating a bloomfilter database from the domain record type:\n\n    python misp-bloomfilter.py -f ../in/misp.xml -d ../db/ -t domain\n\nCreating a bloomfilter database from the ip-dst type using the REST MISP API:\n\n    python misp-bloomfilter.py -u https://misp.server/ -a \u003cyour auth key\u003e -d ../db/ -t ip-dst\n\nTesting the database for the existence of a record:\n\n    python misp-bloomfilter.py -f ../in/misp.xml -d ../db/ -t domain -l foo.bar\n    foo.bar True\n\n\nLicense\n-------\n\nThis software is licensed under GNU Affero General Public License version 3.\n\nCopyright (c) 2012, 2013 Alexandre Dulaunoy (a AT foo be)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmisp%2Fmisp-bloomfilter","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmisp%2Fmisp-bloomfilter","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmisp%2Fmisp-bloomfilter/lists"}