{"id":13817355,"url":"https://github.com/kevburnsjr/bloomfilter","last_synced_at":"2026-01-25T15:35:13.980Z","repository":{"id":57499560,"uuid":"152006876","full_name":"kevburnsjr/bloomfilter","owner":"kevburnsjr","description":"bloomfilter.js ported to Go","archived":false,"fork":false,"pushed_at":"2023-05-01T01:14:58.000Z","size":24,"stargazers_count":95,"open_issues_count":0,"forks_count":5,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-08-14T06:33:16.315Z","etag":null,"topics":["bloom-filters","go"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-2-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kevburnsjr.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2018-10-08T02:23:01.000Z","updated_at":"2023-04-27T21:36:13.000Z","dependencies_parsed_at":"2023-07-13T20:18:37.058Z","dependency_job_id":null,"html_url":"https://github.com/kevburnsjr/bloomfilter","commit_stats":null,"previous_names":["httpimp/bloomfilter"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/kevburnsjr/bloomfilter","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kevburnsjr%2Fbloomfilter","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kevburnsjr%2Fbloomfilter/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kevburnsjr%2Fbloomfilter/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kevburnsjr%2Fbloomfilter/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kevburnsjr","download_url":"https://codeload.github.com/kevburnsjr/bloomfilter/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kevburnsjr%2Fbloomfilter/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28754824,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-25T13:59:49.818Z","status":"ssl_error","status_checked_at":"2026-01-25T13:59:33.728Z","response_time":113,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bloom-filters","go"],"created_at":"2024-08-04T06:00:40.823Z","updated_at":"2026-01-25T15:35:13.952Z","avatar_url":"https://github.com/kevburnsjr.png","language":"Go","funding_links":[],"categories":["Misc"],"sub_categories":[],"readme":"Bloom Filter\n============\n\nThis bloom filter implementation is a Go port of https://github.com/jasondavies/bloomfilter.js\n\n[![Go Report Card](https://goreportcard.com/badge/github.com/httpimp/bloomfilter?1)](https://goreportcard.com/report/github.com/httpimp/bloomfilter)\n[![GoDoc](https://godoc.org/github.com/httpimp/bloomfilter?status.svg)](https://godoc.org/github.com/httpimp/bloomfilter)\n\nThe ability to build a bloom filter on a server in Go and evaluate that filter on a client in\nJavascript can have immense value for comparing application state in distributed single page\napplications with offline read/write capabilities and large data sets.\n\nThere are a lot of open source bloom filter implementations available on the internet. These\nimplementations are mostly incompatible. Every repo adds its own special sauce to its hashing\nalgorithms and hash derivation methods. After scouring the internet for a multi-language bloom\nfilter implemenation, none appeared that fit the requirements.\n\nSo, the most popular actively maintained javascript bloom filter on Github was selected and ported\nto Go.\n\nThe reference implementation uses a non-standard fnv1a hashing algorithm. It is also less than 120\nlines of javascript. This project proves that the reference implementation can be ported by a\nskilled developer to another language in a day.\n\n### Go Example\n\nbloomfilter is thread safe with 95% test coverage\n\n```go\npackage main\n\nimport (\n\t\"encoding/base64\"\n\t\"fmt\"\n\n\t\"github.com/httpimp/bloomfilter\"\n)\n\nfunc main() {\n\tm, k := bloomfilter.EstimateParameters(10, 1e-6)\n\tbf := bloomfilter.New(m, k)\n\tbf.Add([]byte(\"foo\"))\n\tbf.Add([]byte(\"bar\"))\n\tencoded := base64.StdEncoding.EncodeToString(bf.ToBytes())\n\tfmt.Println(m)\n\tfmt.Println(k)\n\tfmt.Println(string(encoded))\n}\n```\n\n\u003e 288  \n\u003e 20  \n\u003e iCCACAiAACAACIgAAAIIAqCAAIgogCAIAIgACAIAigiAAIqA\n\n### Javascript Example\n\nNow we can take that same base64 encoded byte array and evaluate it with bloomfilter.js in the\nbrowser\n\n```js\nvar bits = sjcl.codec.base64.toBits(\"iCCACAiAACAACIgAAAIIAqCAAIgogCAIAIgACAIAigiAAIqA\");\nvar bloom = new BloomFilter(bits, 20);\nconsole.log(bloom.test(\"foo\"));\nconsole.log(bloom.test(\"bar\"));\nconsole.log(bloom.test(\"baz\"));\nbloom.add(\"baz\");\nconsole.log(sjcl.codec.base64.fromBits(bloom.buckets));\n```\n\n\u003e true  \n\u003e true  \n\u003e false  \n\u003e iCCACAiAACAACIgAQAIIAqSIEKgowKEKAowIGBIgyomAAIqI\n\n### Go Example Again\n\nAfter deserializing the filter in javascript and altering it, we can send it back to the server\nagain to confirm that it now includes the additional element.\n\n```go\npackage main\n\nimport (\n\t\"encoding/base64\"\n\t\"log\"\n\n\t\"github.com/httpimp/bloomfilter\"\n)\n\nfunc main() {\n\tdecoded, err := base64.StdEncoding.DecodeString(\"iCCACAiAACAACIgAQAIIAqSIEKgowKEKAowIGBIgyomAAIqI\")\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\tbf := bloomfilter.NewFromBytes(decoded, 21)\n\tlog.Println(bf.Test([]byte(\"foo\")))\n\tlog.Println(bf.Test([]byte(\"bar\")))\n\tlog.Println(bf.Test([]byte(\"baz\")))\n\tlog.Println(bf.Test([]byte(\"bork\")))\n}\n\n```\n\n\u003e true  \n\u003e true  \n\u003e true  \n\u003e false\n\n## Performance\n\nThis is not the most efficient bloom filter available for Go. There are plenty of good options if\nyou don't need portability.\n\nThat said, it still does perform reasonably well.\n\n```\n\u003e go test -bench .\ngoos: linux\ngoarch: amd64\npkg: github.com/httpimp/bloomfilter\nBenchmarkSeparateTestAndAdd-2            1000000              1224 ns/op\nBenchmarkSeparateAdd-2                   2000000               702 ns/op\nPASS\nok      github.com/httpimp/bloomfilter  3.320s\n```\n\nWill Fitzgerald's `bloom` is an excellent bloom filter written in Go\n\nhttps://github.com/willf/bloom\n\n```\n\u003e go test -bench .\ngoos: linux\ngoarch: amd64\npkg: github.com/willf/bloom\nBenchmarkEstimated-2                    2000000000               0.10 ns/op\nBenchmarkSeparateTestAndAdd-2            2000000               761 ns/op\nBenchmarkCombinedTestAndAdd-2            3000000               757 ns/op\nBenchmarkAdd-2                           3000000               599 ns/op\nPASS\nok      github.com/willf/bloom  9.580s\n```\n\nThis project is about 15% slower for Add and 40% slower for Test and Add compared with bloom.\nThis performance penalty may be an acceptable trade-off in exchange for the portability this\nproject is designed to provide. Weigh your application's efficiency requirements for bloom filter\nuse and creation against the benefit of sharing filters with javascript applications before\nintegrating this package.\n\n### Standing on the shoulders of giants\n\nThanks to @jasondavies for creating the reference implementation, a functioning bloom filter\nin \u003c 120 lines of code. This port took about 6 hours.\n\nThanks to @willf for estimation.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkevburnsjr%2Fbloomfilter","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkevburnsjr%2Fbloomfilter","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkevburnsjr%2Fbloomfilter/lists"}