{"id":21814357,"url":"https://github.com/snawoot/bloom","last_synced_at":"2025-04-13T23:46:33.376Z","repository":{"id":23903850,"uuid":"27283810","full_name":"Snawoot/bloom","owner":"Snawoot","description":"An in-memory bloom filter with persistence and HTTP interface","archived":false,"fork":false,"pushed_at":"2020-08-09T22:03:56.000Z","size":93,"stargazers_count":34,"open_issues_count":0,"forks_count":7,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-03-27T13:51:23.593Z","etag":null,"topics":["bloom","bloom-filter","bloom-filters","bloom-persistent","bloom-server","bloomfilter","probabilistic","probabilistic-data-structures"],"latest_commit_sha":null,"homepage":"","language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Snawoot.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2014-11-28T22:13:37.000Z","updated_at":"2024-09-16T12:27:45.000Z","dependencies_parsed_at":"2022-08-07T11:00:48.980Z","dependency_job_id":null,"html_url":"https://github.com/Snawoot/bloom","commit_stats":null,"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Snawoot%2Fbloom","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Snawoot%2Fbloom/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Snawoot%2Fbloom/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Snawoot%2Fbloom/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Snawoot","download_url":"https://codeload.github.com/Snawoot/bloom/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248799697,"owners_count":21163398,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bloom","bloom-filter","bloom-filters","bloom-persistent","bloom-server","bloomfilter","probabilistic","probabilistic-data-structures"],"created_at":"2024-11-27T14:37:55.936Z","updated_at":"2025-04-13T23:46:33.352Z","avatar_url":"https://github.com/Snawoot.png","language":"C","funding_links":[],"categories":[],"sub_categories":[],"readme":"Bloom\n=====\n\n[![Build Status](https://travis-ci.org/Snawoot/bloom.svg?branch=master)](https://travis-ci.org/Snawoot/bloom) [![bloom](https://snapcraft.io//bloom/badge.svg)](https://snapcraft.io/bloom)\n\nBloom is a server, which contains [Bloom filter probabilistic data structure](https://en.wikipedia.org/wiki/Bloom_filter) in memory, provides access to it via HTTP and ensures data persistence on disk by mean of atomic consistent snapshots.\n\n---\n\n:heart: :heart: :heart:\n\nYou can say thanks to the author by donations to these wallets:\n\n- ETH: `0xB71250010e8beC90C5f9ddF408251eBA9dD7320e`\n- BTC:\n  - Legacy: `1N89PRvG1CSsUk9sxKwBwudN6TjTPQ1N8a`\n  - Segwit: `bc1qc0hcyxc000qf0ketv4r44ld7dlgmmu73rtlntw`\n\n---\n\n## Building\n\nConsider also [Prebuilt Docker image](#prebuilt-docker-image), binaries on [Releases](https://github.com/Snawoot/bloom/releases) page to use prebuilt ones and [installation from Snap Store](#from-snap-store).\n\n#### Debian/Ubuntu\n\nRun these commands in sources directory:\n\n```bash\nsudo apt-get install build-essential libevent-dev\nmake\n```\n\n#### RHEL/OEL/CentOS\n\nRun these commands in sources directory:\n\n```bash\nsudo yum install gcc libevent2-devel make\nmake\n```\n\nRun `make static` instead of `make` to build static binary.\n\n#### Mac OS X\n\nAssuming you are using [Homebrew](http://brew.sh/)\n\n```bash\nbrew install libevent\nmake\n```\n\nStatic build for Mac OS X is not available now.\n\n#### FreeBSD\n\nAccording to `siege` benchmarks, GCC compiler gains better performance for this application. If you want to use BSD `cc`, you may change CC variable in Makefile. Application can be built using both of them.\n\n```bash\npkg install gcc libevent2\nmake\n```\n\nRun `make static` instead of `make` to build static binary.\n\n#### Solaris\n\nYou have to build libevent2 before:\n\n```bash\nsudo pkg install gcc\nwget https://github.com/libevent/libevent/releases/download/release-2.0.22-stable/libevent-2.0.22-stable.tar.gz\ntar xf libevent-2.0.22-stable.tar.gz\ncd libevent-2.0.22-stable\n./configure\nmake\nsudo make install\n```\nYou may also need to add `/usr/local/lib` to library search path:\n\n```bash\nsudo crle\n# Settings output here. Check output and add /usr/local/lib at the end, delimiting it by colon\nsudo crle -l /lib:/usr/lib:/usr/local/lib\n```\n\nAfter that, run build of Bloom from its directory:\n\n```bash\nmake\n```\n\nStatic build for Solaris is not available now.\n\n## Installing\n\n### Prebuilt Docker image\n\nRun:\n\n```bash\ndocker volume bloom\ndocker run -dit \\\n    -v bloom:/var/lib/bloom \\\n    -p 8889:8889 \\\n    --restart unless-stopped \\\n    --name bloom \\\n    yarmak/bloom \\\n    /var/lib/bloom/bloom.dat\n```\n\nHelp:\n\n```bash\ndocker run -it \\\n    yarmak/bloom \\\n    -h\n```\n### From source built before\n\n```bash\nmake install\n```\nto install dynamic binary.\n\n### From Snap Store\n\n[![Get it from the Snap Store](https://snapcraft.io/static/images/badges/en/snap-store-black.svg)](https://snapcraft.io/bloom)\n\n```bash\nsudo snap install bloom\n```\n\n## Usage\n\n### Running daemon\n\n`bloom \u003cfilename_for_snapshot\u003e` or\n`./bloom.static \u003cfilename_for_snapshot\u003e` if you prefer statically linked version. \n\nCommand line options:\n\n```\n$ bloom -h\nUsage: bloom [options] SNAPSHOT_FILE\n\nOptions:\n\n-H BIND_ADDRESS\t\tHTTP interface bind address. Default: 0.0.0.0\n\n-P BIND_PORT\t\tHTTP interface bind port. Default: 8889\n\n-h\t\t\tPrint this help message\n\n-m M\t\t\tNumber of bits in bloom filter. Default: 2^33\n\n-k K\t\t\tNumber of hash functions. Default: 10\n\n-t SECONDS\t\tDump bloom filter snapshot to file every SECONDS\n\t\t\tseconds. You can set this value to 0 if you wish\n\t\t\tto disable this feature - snapshots are taken on USR1\n\t\t\tsignal and at exit in any case.\n```\n\nDefault settings is suitable for containing 500,000,000 elements with false positive probability 0.1%. See also [Utilities](https://github.com/Snawoot/bloom#utilities) for parameters calculator.\n\n### Querying\n\nTest whether an element is a member of a set:\n```\n$ curl http://127.0.0.1:8889/check?e=sdfdsafdsafsadf\nMISSING\n```\nAdd an element to set:\n```\n$ curl http://127.0.0.1:8889/add?e=sdfdsafdsafsadf\nADDED\n```\nCheck then add at once:\n```\n$ curl http://127.0.0.1:8889/checkthenadd?e=aaaaaabbb\nMISSING\n$ curl http://127.0.0.1:8889/checkthenadd?e=aaaaaabbb\nPRESENT\n```\n### Saving set\n\nServer saves data to snapshot file in following cases:\n* Server exit (received `SIGTERM` or `SIGINT`)\n* Timer event. By default server dumps snapshot to disk every 5 minutes. See also help for option `-t`.\n* On `SIGUSR1` signal. This way you may control dump process on your own by sending signal to daemon.\n\nSnapshot dumping process does not blocks serving request and uses copy-on-write method, so dumped data is always consistent.\n\n## Utilities\n\n* `utils/collision_meter.py` - Check structure occupancy by measuring false positive probability on completely random requests.\n* `utils/bf_calc.py` - Calculate parameters of bloom filter for given number of elements and false positives probability.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsnawoot%2Fbloom","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsnawoot%2Fbloom","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsnawoot%2Fbloom/lists"}