Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/irfansharif/cfilter
Cuckoo Filter implementation in Go, better than Bloom Filters (unmaintained)
https://github.com/irfansharif/cfilter
bloom-filter cuckoo-filter filter go
Last synced: about 2 months ago
JSON representation
Cuckoo Filter implementation in Go, better than Bloom Filters (unmaintained)
- Host: GitHub
- URL: https://github.com/irfansharif/cfilter
- Owner: irfansharif
- License: mit
- Created: 2016-07-22T06:54:14.000Z (over 8 years ago)
- Default Branch: master
- Last Pushed: 2017-07-05T17:34:18.000Z (over 7 years ago)
- Last Synced: 2024-08-03T23:27:14.336Z (5 months ago)
- Topics: bloom-filter, cuckoo-filter, filter, go
- Language: Go
- Homepage:
- Size: 20.5 KB
- Stars: 764
- Watchers: 21
- Forks: 37
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- awesome-golang-repositories - cfilter
README
# cfilter: Cuckoo Filter implementation in Go
[](https://godoc.org/github.com/irfansharif/cfilter)
[](https://travis-ci.org/irfansharif/cfilter)
[](https://coveralls.io/github/irfansharif/cfilter?branch=master)
[](https://goreportcard.com/report/github.com/irfansharif/cfilter)
Cuckoo filter is a Bloom filter replacement for approximated set-membership
queries. Cuckoo filters support adding and removing items dynamically while
achieving even higher performance than Bloom filters. For applications that
store many items and target moderately low false positive rates, cuckoo filters
have lower space overhead than space-optimized Bloom filters.
Some possible use-cases that depend on approximated set-membership queries
would be databases, caches, routers, and storage systems where it is used to
decide if a given item is in a (usually large) set, with some small false
positive probability. Alternatively, given it is designed to be a viable
replacement to Bloom filters, it can also be used to reduce the space required
in probabilistic routing tables, speed longest-prefix matching for IP
addresses, improve network state management and monitoring, and encode
multicast forwarding information in packets, among many other applications.
Cuckoo filters provide the flexibility to add and remove items dynamically. A
cuckoo filter is based on cuckoo hashing (and therefore named as cuckoo
filter). It is essentially a cuckoo hash table storing each key's fingerprint.
Cuckoo hash tables can be highly compact, thus a cuckoo filter could use less
space than conventional Bloom filters, for applications that require low false
positive rates (< 3%).
For details about the algorithm and citations please refer to the original
research paper, ["Cuckoo Filter: Better Than Bloom" by Bin Fan, Dave Andersen
and Michael Kaminsky](https://www.cs.cmu.edu/~dga/papers/cuckoo-conext2014.pdf).
## Interface
A cuckoo filter supports following operations:
* `Insert(item)`: insert an item to the filter
* `Lookup(item)`: return if item is already in the filter (may return false
positive results like Bloom filters)
* `Delete(item)`: delete the given item from the filter. Note that to use this
method, it must be ensured that this item is in the filter (e.g., based on
records on external storage); otherwise, a false item may be deleted.
* `Count()`: return the total number of items currently in the filter
## Example Usage
```go
import "github.com/irfansharif/cfilter"
cf := cfilter.New()
// inserts 'buongiorno' to the filter
cf.Insert([]byte("buongiorno"))
// looks up 'hola' in the filter, may return false positive
cf.Lookup([]byte("hola"))
// returns 1 (given only 'buongiorno' was added)
cf.Count()
// tries deleting 'bonjour' from filter, may delete another element
// this could occur when another byte slice with the same fingerprint
// as another is 'deleted'
cf.Delete([]byte("bonjour"))
```
This repository was featured on Hacker News, front page (discussion
[here](https://news.ycombinator.com/item?id=12241332)). Another implementation
in Go can be found at
[seiflotfy/cuckoofilter](https://github.com/seiflotfy/cuckoofilter) and is where I borrowed
the ideas for my tests, notably `TestMultipleInsertions`. The original
implementation in C++ by the authors of the research paper can be found at
[efficient/cuckoofilter](https://github.com/efficient/cuckoofilter).
## Author
Irfan Sharif: , [@irfansharifm](https://twitter.com/irfansharifm)
## License
cfilter source code is available under the MIT [License](/LICENSE).