{"id":21672505,"url":"https://github.com/redis-developer/redisbloom-moby-dick","last_synced_at":"2025-04-12T03:52:53.936Z","repository":{"id":61662587,"uuid":"483569116","full_name":"redis-developer/redisbloom-moby-dick","owner":"redis-developer","description":"Experiments with RedisBloom and the text from Moby Dick","archived":false,"fork":false,"pushed_at":"2023-02-03T14:24:32.000Z","size":888,"stargazers_count":4,"open_issues_count":2,"forks_count":3,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-04-12T03:52:46.816Z","etag":null,"topics":["bloom-filter","hyperloglog","node-redis","nodejs","redis","redis-stack"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/redis-developer.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-04-20T08:24:30.000Z","updated_at":"2023-10-09T14:22:54.000Z","dependencies_parsed_at":"2023-02-18T07:45:19.038Z","dependency_job_id":null,"html_url":"https://github.com/redis-developer/redisbloom-moby-dick","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/redis-developer%2Fredisbloom-moby-dick","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/redis-developer%2Fredisbloom-moby-dick/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/redis-developer%2Fredisbloom-moby-dick/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/redis-developer%2Fredisbloom-moby-dick/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/redis-developer","download_url":"https://codeload.github.com/redis-developer/redisbloom-moby-dick/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248514209,"owners_count":21116899,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bloom-filter","hyperloglog","node-redis","nodejs","redis","redis-stack"],"created_at":"2024-11-25T13:29:43.797Z","updated_at":"2025-04-12T03:52:53.918Z","avatar_url":"https://github.com/redis-developer.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Experiments with Redis Bloom and the Text from \"Moby Dick\"\n\nThis repository contains a small example using the text from the book \"Moby Dick\" as a data source to compare and contrast the benefits and tradeoffs of using different Redis data structures.  We'll use the deterministic Set data structure, and four probabilistic data structures: Hyperloglog, Bloom Filter, Top-K and Count-Min Sketch.  These probabilistic data structures generally use hashing to be more memory efficient at the cost of some accuracy and the ability to recall data added to them.\n\nThe Set and Hyperloglog are built into core Redis, to use the Bloom Filter, Top K and Count-Min Sketch data structures, you'll need [Redis Stack](https://redis.io/docs/stack/) or the [RedisBloom module](https://github.com/RedisBloom/RedisBloom).\n\nCheck out our [video on YouTube](https://www.youtube.com/watch?v=FAJXq5Qqc0Y) where [Justin](https://github.com/justincastilla) and [Simon](https://simonprickett.dev) talk about counting things at scale with Redis Stack, using examples from this repository.\n\nWe also added some extra examples, which are currently on the [adds-other-examples](https://github.com/redis-developer/redisbloom-moby-dick/blob/adds-other-examples/README.md) branch of this repository.\n\n## Prerequisites\n\nTo try out this project yourself, you'll need:\n\n* [Redis Stack](https://redis.io/docs/stack/) (installed locally, available free [in the cloud](https://redis.com/try-free/), or use the supplied Docker Compose file).\n* [Docker Desktop](https://www.docker.com/products/docker-desktop/) (if using the Docker Compose file).\n* [git](https://git-scm.com/download) command line tools.\n* [Node.js](https://nodejs.org/) version 14.8 or higher (I've tested the code using Node.js version 16.4.2).\n* OR\n* [Python](https://python.org/) version 3.6 or higher (I've tested the code using Python version 3.10.6).\n## Get the Code\n\nTo get the code, clone the repo to your machine:\n\n```bash\n$ git clone https://github.com/redis-developer/redisbloom-moby-dick.git\n$ cd redisbloom-moby-dick/\n```\n\n## Application Setup\n\nThis project has two versions, one written in NodeJS and other in Python. For the NodeJS version, head to the `nodejs` directory by doing:\n\n```bash\n$ cd nodejs/\n```\nand follow the instructions given in the [README](/nodejs/README.md) file inside the directory.\n\nFor the Python version, head to the `python` directory by doing:\n\n```bash\n$ cd python/\n```\nand follow the instructions given in the [README](/python/README.md) file inside the directory.\n\n## Further Exercises\n\nTry using the following commands to check if a given word is in the Set, or possibly in the Bloom Filter:\n\n* [`SISMEMBER`](https://redis.io/commands/sismember/): Find out for sure whether a word is in the Set whose key is `mobydick:words:set`.  Try checking multiple words at once with [`SMISMEMER`](https://redis.io/commands/smismember/)\n* [`BF.EXISTS`](https://redis.io/commands/bf.exists/): Find out whether a word may be or is absolutely not in the Bloom Filter whose key is `mobydick:words:bloom`. Try checking multiple words at once with [`BF.MEXISTS`](https://redis.io/commands/bf.mexists/)\n\nWhy can't we say with absolute certainty that a word is in the Bloom Filter? \n\n## Licensing\n\nThe code is made available under the terms of the [MIT license](https://mit-license.org/).  I obtained the original text file of the book \"Moby Dick\" from [Project Gutenberg](https://www.gutenberg.org/policy/permission.html).  This text is in the public domain and available in its original form [here](https://www.gutenberg.org/files/2701/2701-0.txt).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fredis-developer%2Fredisbloom-moby-dick","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fredis-developer%2Fredisbloom-moby-dick","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fredis-developer%2Fredisbloom-moby-dick/lists"}