https://github.com/seiflotfy/cuckoofilter

Cuckoo Filter: Practically Better Than Bloom
https://github.com/seiflotfy/cuckoofilter

Last synced: 2 days ago
JSON representation

Cuckoo Filter: Practically Better Than Bloom

• Host: GitHub
• URL: https://github.com/seiflotfy/cuckoofilter
• Owner: seiflotfy
• License: mit
• Created: 2015-06-28T23:22:09.000Z (over 9 years ago)
• Default Branch: master
• Last Pushed: 2024-07-15T13:13:51.000Z (5 months ago)
• Last Synced: 2024-12-03T02:04:18.821Z (9 days ago)
• Language: Go
• Homepage:
• Size: 51.8 KB
• Stars: 1,144
• Watchers: 22
• Forks: 111
• Open Issues: 15
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

• awesome-go - cuckoofilter - Cuckoo filter: a good alternative to a counting bloom filter implemented in Go. (Data Structures and Algorithms / Bloom and Cuckoo Filters)
• awesome-go - cuckoofilter - Cuckoo filter: a good alternative to a counting bloom filter implemented in Go. Stars:`1.1K`. (Data Structures and Algorithms / Bloom and Cuckoo Filters)
• awesome-go - cuckoofilter - Cuckoo filter: a good alternative to a counting bloom filter implemented in Go. (Data Structures and Algorithms / Bloom and Cuckoo Filters)
• awesome-go - cuckoofilter - Cuckoo Filter: Practically Better Than Bloom - ★ 387 (Data Structures)
• awesome-go-extra - cuckoofilter - 06-28T23:22:09Z|2022-04-11T07:59:57Z| (Generators / Bloom and Cuckoo Filters)
• awesome-ethereum-rollups - Go implementation that contains an explicit implementation of a fingerprint function

README

        
# Cuckoo Filter

[![GoDoc](https://godoc.org/github.com/seiflotfy/cuckoofilter?status.svg)](https://godoc.org/github.com/seiflotfy/cuckoofilter) [![CodeHunt.io](https://img.shields.io/badge/vote-codehunt.io-02AFD1.svg)](http://codehunt.io/sub/cuckoo-filter/?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge)

Cuckoo filter is a Bloom filter replacement for approximated set-membership queries. While Bloom filters are well-known space-efficient data structures to serve queries like "if item x is in a set?", they do not support deletion. Their variances to enable deletion (like counting Bloom filters) usually require much more space.

Cuckoo filters provide the flexibility to add and remove items dynamically. A cuckoo filter is based on cuckoo hashing (and therefore named as cuckoo filter). It is essentially a cuckoo hash table storing each key's fingerprint. Cuckoo hash tables can be highly compact, thus a cuckoo filter could use less space than conventional Bloom filters, for applications that require low false positive rates (< 3%).

For details about the algorithm and citations please use this article for now

["Cuckoo Filter: Better Than Bloom" by Bin Fan, Dave Andersen and Michael Kaminsky](https://www.cs.cmu.edu/~dga/papers/cuckoo-conext2014.pdf)

## Implementation details

The paper cited above leaves several parameters to choose. In this implementation

1. Every element has 2 possible bucket indices

2. Buckets have a static size of 4 fingerprints

3. Fingerprints have a static size of 8 bits

1 and 2 are suggested to be the optimum by the authors. The choice of 3 comes down to the desired false positive rate. Given a target false positive rate of `r` and a bucket size `b`, they suggest choosing the fingerprint size `f` using

    f >= log2(2b/r) bits

With the 8 bit fingerprint size in this repository, you can expect `r ~= 0.03`.

[Other implementations](https://github.com/panmari/cuckoofilter) use 16 bit, which correspond to a false positive rate of `r ~= 0.0001`.

## Example usage:

```go

package main

import "fmt"

import cuckoo "github.com/seiflotfy/cuckoofilter"

func main() {

  cf := cuckoo.NewFilter(1000)

  cf.InsertUnique([]byte("geeky ogre"))

  // Lookup a string (and it a miss) if it exists in the cuckoofilter

  cf.Lookup([]byte("hello"))

  count := cf.Count()

  fmt.Println(count) // count == 1

  // Delete a string (and it a miss)

  cf.Delete([]byte("hello"))

  count = cf.Count()

  fmt.Println(count) // count == 1

  // Delete a string (a hit)

  cf.Delete([]byte("geeky ogre"))

  count = cf.Count()

  fmt.Println(count) // count == 0

  

  cf.Reset()    // reset

}

```

## Documentation:

["Cuckoo Filter on GoDoc"](http://godoc.org/github.com/seiflotfy/cuckoofilter)