{"id":14155161,"url":"https://github.com/plabayo/venndb","last_synced_at":"2025-05-07T00:04:20.964Z","repository":{"id":148009732,"uuid":"618452402","full_name":"plabayo/venndb","owner":"plabayo","description":"in memory Rust database to query your data like a Venn diagram","archived":false,"fork":false,"pushed_at":"2024-04-30T21:50:19.000Z","size":5977,"stargazers_count":63,"open_issues_count":2,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-05-01T09:48:18.186Z","etag":null,"topics":["bitset","bitvec","database","db","memory","rust","sets","vector","venn-diagram"],"latest_commit_sha":null,"homepage":"https://venndb.rs","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/plabayo.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":".github/FUNDING.yml","license":"LICENSE-APACHE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":"plabayo"}},"created_at":"2023-03-24T13:55:31.000Z","updated_at":"2024-08-17T08:04:55.132Z","dependencies_parsed_at":"2023-12-21T08:11:28.149Z","dependency_job_id":"1144de59-5de8-4588-b343-2d16e94991d6","html_url":"https://github.com/plabayo/venndb","commit_stats":null,"previous_names":["plabayo/venndb","plabayo/flagdb"],"tags_count":7,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/plabayo%2Fvenndb","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/plabayo%2Fvenndb/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/plabayo%2Fvenndb/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/plabayo%2Fvenndb/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/plabayo","download_url":"https://codeload.github.com/plabayo/venndb/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252788514,"owners_count":21804284,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bitset","bitvec","database","db","memory","rust","sets","vector","venn-diagram"],"created_at":"2024-08-17T08:02:18.947Z","updated_at":"2025-05-07T00:04:20.913Z","avatar_url":"https://github.com/plabayo.png","language":"Rust","funding_links":["https://github.com/sponsors/plabayo","https://www.buymeacoffee.com/plabayo"],"categories":["rust","Libraries"],"sub_categories":["Database"],"readme":"# VennDB\n\nAn **append-only** in-memory database in Rust for rows queried using bit (flag) columns.\nThis database is designed for a very specific use case where you have mostly static data that you typically load at startup and have to query constantly using very simple filters. Datasets\nlike these can be large and should be both fast and compact.\n\nFor the limited usecases where `venndb` can be applied to,\nit has less dependencies and is faster then traditional choices,\nsuch as a naive implementation or a more heavy lifted dependency such as _Sqlite_.\n\n\u003e See [the benchmarks](#benchmarks) for more information on this topic.\n\nThis project was developed originally in function of [`rama`](https://ramaproxy.org),\nwhere you can see it being used for example to provide an in-memory (upstream) proxy database.\nDo let us know in case you use it as well in your project, such that we can assemble a showcase list.\n\n![venndb banner](https://raw.githubusercontent.com/plabayo/venndb/main/docs/img/banner.svg)\n\n[![Crates.io][crates-badge]][crates-url]\n[![Docs.rs][docs-badge]][docs-url]\n[![MIT License][license-mit-badge]][license-mit-url]\n[![Apache 2.0 License][license-apache-badge]][license-apache-url]\n[![rust version][rust-version-badge]][rust-version-url]\n[![Build Status][actions-badge]][actions-url]\n\n[![Discord][discord-badge]][discord-url]\n[![Buy Me A Coffee][bmac-badge]][bmac-url]\n[![GitHub Sponsors][ghs-badge]][ghs-url]\n\n[crates-badge]: https://img.shields.io/crates/v/venndb.svg\n[crates-url]: https://crates.io/crates/venndb\n[docs-badge]: https://img.shields.io/docsrs/venndb/latest\n[docs-url]: https://docs.rs/venndb/latest/venndb/index.html\n[license-mit-badge]: https://img.shields.io/badge/license-MIT-blue.svg\n[license-mit-url]: https://github.com/plabayo/venndb/blob/main/LICENSE-MIT\n[license-apache-badge]: https://img.shields.io/badge/license-APACHE-blue.svg\n[license-apache-url]: https://github.com/plabayo/venndb/blob/main/LICENSE-APACHE\n[rust-version-badge]: https://img.shields.io/badge/rustc-1.75+-blue?style=flat-square\u0026logo=rust\n[rust-version-url]: https://www.rust-lang.org\n[actions-badge]: https://github.com/plabayo/venndb/workflows/CI/badge.svg\n[actions-url]: https://github.com/plabayo/venndb/actions\n\n[discord-badge]: https://img.shields.io/badge/Discord-%235865F2.svg?style=for-the-badge\u0026logo=discord\u0026logoColor=white\n[discord-url]: https://discord.gg/29EetaSYCD\n[bmac-badge]: https://img.shields.io/badge/Buy%20Me%20a%20Coffee-ffdd00?style=for-the-badge\u0026logo=buy-me-a-coffee\u0026logoColor=black\n[bmac-url]: https://www.buymeacoffee.com/plabayo\n[ghs-badge]: https://img.shields.io/badge/sponsor-30363D?style=for-the-badge\u0026logo=GitHub-Sponsors\u0026logoColor=#EA4AAA\n[ghs-url]: https://github.com/sponsors/plabayo\n\n💬 Come join us at [Discord][discord-url] on the `#venndb` public channel. To ask questions, discuss ideas and ask how venndb may be useful for you.\n\n## Index\n\n`venndb` manual:\n\n- [Usage](#usage): quick introduction on how to use `venndb`;\n- [Benchmarks](#benchmarks): benchmark results to give you a rough idea how `venndb` peforms for the use case it is made for (write once, read constantly, using binary filters mostly);\n- [Q\u0026A](#qa): Frequently Asked Questions (FAQ);\n- [Example](#example): the full example (expanded version from [Usage](#usage)), tested and documented;\n- [Generated Code Summary](#generated-code-summary): a documented overview of the API that `venndb` will generate for you when using `#[derive(VennDB)]` on your _named field struct_;\n\ntechnical information:\n\n- [Safety](#--safety)\n- [Compatibility](#--compatibility)\n- [MSRV](#minimum-supported-rust-version) (older versions might work but we make no guarantees);\n- [Roadmap](#--roadmap)\n- [License](#--license): [MIT license][mit-license] and [Apache 2.0 License][apache-license]\n\nmisc:\n\n- [Contributing](#--contributing)\n- [Sponsors](#--sponsors)\n\n## Usage\n\nAdd `venndb` as a dependency:\n\n```sh\ncargo add venndb\n```\n\nand import the `derive` macro in the module where you want to use it:\n\n```rust,ignore\nuse venndb::VennDB\n\n#[derive(Debug, VennDB)]\npub struct Employee {\n    #[venndb(key)]\n    id: u32,\n    name: String,\n    is_manager: Option\u003cbool\u003e,\n    is_admin: bool,\n    #[venndb(skip)]\n    foo: bool,\n    #[venndb(filter, any)]\n    department: Department,\n    #[venndb(filter)]\n    country: Option\u003cString\u003e,\n}\n\nfn main() {\n    let db = EmployeeDB::from_iter(/* .. */);\n\n    let mut query = db.query();\n    let employee = query\n        .is_admin(true)\n        .is_manager(false)\n        .department(Department::Engineering)\n        .execute()\n        .expect(\"to have found at least one\")\n        .any();\n\n    println!(\"non-manager admin engineer: {:?}\", employee);\n}\n```\n\nSee [the full example](#example) or [the \"Generated Code Summary\" chapter](#generated-code-summary) below\nto learn how to use the `VennDB` and its generated code.\n\n## Benchmarks\n\nBenchmarks displayed here are taken on a dev machine with following specs:\n\n```text\nMacbook Pro — 16 inch (2023)\nChip: Apple M2 Pro\nMemory: 16 GB\nOS: Sonoma 14.2\n```\n\nThe benchmarks tests 3 different implementations of a proxy database\n\n- `venndb` version (very similar to [the example below](#example))\n- a `naive` version, which is just a `Vec\u003cProxy\u003e`, over which is iterated\n- an `sqlite` version (using [the `sqlite` crate (version: `0.34.0`)](https://docs.rs/sqlite/0.34.0/sqlite/))\n\nThe benchmarks are created by:\n\n1. running `just bench`;\n2. copying the output into [./scripts/plot_bench_charts](./scripts/plot_bench_charts.py) and running it.\n\nSnippet that is ran for each 3 implementations:\n\n```rust,ignore\nfn test_db(db: \u0026impl ProxyDB) {\n    let i = next_round();\n\n    let pool = POOLS[i % POOLS.len()];\n    let country = COUNTRIES[i % COUNTRIES.len()];\n\n    let result = db.get(i as u64);\n    divan::black_box(result);\n\n    let result = db.any_tcp(pool, country);\n    divan::black_box(result);\n\n    let result = db.any_socks5_isp(pool, country);\n    divan::black_box(result);\n}\n```\n\n### Benchmark Performance Results\n\nPerformance for Database with `100` records:\n\n| Proxy DB | Fastest (µs) | Median (µs) | Slowest (µs) |\n| --- | --- | --- | --- |\n| naive_proxy_db_100             | 6.50 | 8.00 | 18.04 |\n| sql_lite_proxy_db_100          | 32.58 | 37.37 | 302.00 |\n| venn_proxy_db_100              | 0.89 | 0.92 | 2.74 |\n\nPerformance for Database with `12_500` records:\n\n| Proxy DB | Fastest (µs) | Median (µs) | Slowest (µs) |\n| --- | --- | --- | --- |\n| naive_proxy_db_12_500          | 404.00 | 407.70 | 478.70 |\n| sql_lite_proxy_db_12_500       | 1061.00 | 1073.00 | 1727.00 |\n| venn_proxy_db_12_500           | 16.04 | 16.97 | 25.54 |\n\nPerformance for Database with `100_000` records:\n\n| Proxy DB | Fastest (µs) | Median (µs) | Slowest (µs) |\n| --- | --- | --- | --- |\n| naive_proxy_db_100_000         | 3790.00 | 3837.00 | 5731.00 |\n| sql_lite_proxy_db_100_000      | 8219.00 | 8298.00 | 9424.00 |\n| venn_proxy_db_100_000          | 124.20 | 129.20 | 156.30 |\n\nWe are not database nor hardware experts though. Please do open an issue if you think\nthese benchmarks are incorrect or if related improvements can be made.\nContributions in the form of Pull requests are welcomed as well.\n\nSee [the Contribution guidelines](#contribution) for more information.\n\n### Benchmark Allocations Results\n\nAllocations for Database with `100` records:\n\n| Proxy DB | Fastest (KB) | Median (KB) | Slowest (KB) |\n| --- | --- | --- | --- |\n| naive_proxy_db_100             | 0.33 | 0.33 | 0.33 |\n| sql_lite_proxy_db_100          | 4.04 | 4.04 | 4.04 |\n| venn_proxy_db_100              | 0.05 | 0.05 | 0.05 |\n\nAllocations for Database with `12_500` records:\n\n| Proxy DB | Fastest (KB) | Median (KB) | Slowest (KB) |\n| --- | --- | --- | --- |\n| naive_proxy_db_12_500          | 40.73 | 40.73 | 40.73 |\n| sql_lite_proxy_db_12_500       | 5.03 | 5.02 | 5.03 |\n| venn_proxy_db_12_500           | 3.15 | 3.15 | 3.15 |\n\nAllocations for Database with `100_000` records:\n\n| Proxy DB | Fastest (KB) | Median (KB) | Slowest (KB) |\n| --- | --- | --- | --- |\n| naive_proxy_db_100_000         | 323.30 | 323.30 | 323.70 |\n| sql_lite_proxy_db_100_000      | 5.02 | 5.02 | 5.01 |\n| venn_proxy_db_100_000          | 25.02 | 25.02 | 25.02 |\n\nWe are not database nor hardware experts though. Please do open an issue if you think\nthese benchmarks are incorrect or if related improvements can be made.\nContributions in the form of Pull requests are welcomed as well.\n\nSee [the Contribution guidelines](#contribution) for more information.\n\n## Q\u0026A\n\n\u003e ❓ Why use this over Database X?\n\n`venndb` is not a database, but is close enough for some specific purposes. It shines for long-lived read-only use cases where you need to filter on plenty of binary properties and get a rando matching result.\n\nDo not try to replace your usual database needs with it.\n\n\u003e ❓ Where can I propose a new feature X or some other improvement?\n\nPlease [open an issue](https://github.com/plabayo/venndb/issues) and also read [the Contribution guidelines](#contribution). We look forward to hear from you.\n\nAlternatively you can also [join our Discord][discord-url] and start a conversation / discussion over there.\n\n\u003e ❓ Can I use _whatever_ type for a `#[venndb(filter)]` property?\n\nYes, as long as it implements `PartialEq + Eq + Hash + Clone`.\nThat said, we do recommend that you use `enum` values if you can, or some other highly restricted form.\n\nUsing for example a `String` directly is a bad idea as that would mean that `bE` != `Be` != `BE` != `Belgium` != `Belgique` != `België`. Even though these are really referring all to the same country. In such cases a much better idea is to at the very least create a wrapper type such as `struct Country(String)`, to allow you to enforce sanitization/validation when creating the value and ensuring the hashes will be the same for those values that are conceptually the same.\n\n\u003e ❓ How do I make a filter optional?\n\nBoth filters (`bool` properties) and filter maps (`T != bool` properties with the `#[venndb(filter)]` attribute)\ncan be made optional by wrapping the types with `Option`, resulting in `Option\u003cbool\u003e` and `Option\u003cT\u003e`.\n\nRows that have the `Option::None` value for such an optional column cannot filter on that property,\nbut there is no other consequence beyond that.\n\n\u003e ❓ Why can do keys have to be unique and non-optional?\n\nWithin `venndb` keys are meant to be able to look up,\na row which was previously received via filters.\n\nAs such it makes no sense for such keys to be:\n\n- duplicate: it would mean: as that can result in multiple rows or the wrong row to be returned;\n- optional: as that would mean the row cannot be looked up when the key is not defined;\n\n\u003e ❓ How can I allow some rows to match for _any_ value of a certain (filter) column?\n\nFilter maps can allow to have a value to match all other values. It is up to you to declare the filter as such,\nand to also define for that type what the _one_ value to rule them all is.\n\nUsage:\n\n```rust,ignore\nuse venndb::{Any, VennDB};\n#[derive(Debug, PartialEq, Eq, Clone, Hash)]\npub enum Department {\n  Any,\n  Hr,\n  Engineering,\n}\n\nimpl Any for Department {\n  fn is_any(\u0026self) -\u003e bool {\n    self == Department::Any\n  }\n}\n\n#[derive(Debug, VennDB)]\npub struct Employee {\n  name: String,\n  #[venndb(filter, any)]\n  department: Department,\n}\n\nlet db = EmployeeDB::from_iter([\n  Employee { name: \"Jack\".to_owned(), department: Department::Any },\n  Employee { name: \"Derby\".to_owned(), department: Department::Hr },\n]);\nlet mut query = db.query();\n\n// will match Jack and Derby, as Jack is marked as Any, meaning it can work for w/e value\nlet hr_employees: Vec\u003c_\u003e = query.department(Department::Hr).execute().unwrap().iter().collect();\nassert_eq!(hr_employees.len(), 2);\n```\n\n\u003e ❓ How can I provide custom validation of rows prior to them getting appended?\n\nIs is possible to validate a row based on one or multiple of its properties? Validate in function of relationship\nbetween multiple properties? Is it possible to provide custom validation to prevent rows\nfrom getting appended that do not adhere to custom validation rules?\n\nYes to all of the above.\n\nExample:\n\n```rust,ignore\n#[derive(Debug, VennDB)]\n#[venndb(validator = my_validator_fn)]\npub struct Value {\n   pub foo: String,\n   pub bar: u32,\n}\n\nfn my_validator_fn(value: \u0026Value) -\u003e bool {\n    !value.foo.is_empty() \u0026\u0026 value.bar \u003e 0\n}\n\nlet mut db = ValueDB::default();\nassert!(db.append(Value {\n    foo: \"\".to_owned(),\n    bar: 42,\n}).is_err()); // fails because foo == empty\n```\n\n\u003e ❓ Why do `any` filter values only match rows that have an `any` value for that property?\n\nLet's say I have the following `struct`:\n\n```rust,ignore\nuse venndb::{Any, VennDB};\n\n#[derive(Debug, VennDB)]\npub struct Value {\n   #[venndb(filter, any)]\n   pub foo: MyString,\n   pub bar: u32,\n}\n\n#[derive(Debug)]\npub struct MyString(String);\n\nimpl Any for MyString {\n    fn is_any(\u0026self) -\u003e bool {\n        self.0 == \"*\"\n    }\n}\n\nlet db = ValueDB::from_iter([\n    Value {\n        foo: MyString(\"foo\".to_owned()),\n        bar: 8,\n    },\n    Value {\n        foo: MyString(\"*\".to_owned()),\n        bar: 16,\n    }\n].into_Iter()).unwrap();\n\nlet mut query = db.query();\nquery.foo(MyString(\"*\".to_owned()));\nlet value = query.execute().unwrap().any();\n// this will never match the row with bar == 8,\n// tiven foo != an any value\nassert_eq!(value.bar, 16);\n```\n\nWhy is this true? Because it is correct.\n\nAllowing it also to match the value `foo` would unfairly\ngive more chances for `foo` to be selected over the _any_ value.\nThis might not seem like a big difference, but it is. Because what if\nwe generate a random string for `Value`s with an _any value? If we\nwould allow all rows to be matched then that logic is now rigged,\nwith a value of `foo` being more likely then other strings.\n\nAs such the only correct answer when filtering for _any_ value,\nis to return rows that have _any_ value.\n\n\u003e ❓ How can I query on multiple variants of a \"filter map\" property?\n\nJust call the _query_ method multiple times. It will allow you to match\nrows that have either of these values.\n\nExample\n\n```rust,ignore\nuse venndb::{Any, VennDB};\n\n#[derive(Debug, VennDB)]\npub struct Value {\n   #[venndb(filter)]\n   pub foo: String,\n   pub bar: u32,\n}\n\nlet db = ValueDB::from_iter([\n    Value {\n        foo: \"a\".to_owned(),\n        bar: 8,\n    },\n    Value {\n        foo: \"b\".to_owned(),\n        bar: 12,\n    },\n    Value {\n        foo: \"c\".to_owned(),\n        bar: 16,\n    },\n].into_Iter()).unwrap();\n\nlet mut query = db.query();\nquery.foo(MyString(\"a\".to_owned()));\nquery.foo(MyString(\"c\".to_owned()));\nlet values: Vec\u003c_\u003e = query.execute().unwrap().iter().collect();\nassert_eq!(values.len(), 2);\nassert_eq!(values[0].bar, 8);\nassert_eq!(values[0].bar, 16);\n```\n\n## Example\n\nHere follows an example demonstrating all the features of `VennDB`.\n\nIf you prefer a summary of what is generated, or do not understand something from the example below,\nyou can also read [the \"Generated Code Summary\" chapter](#generated-code-summary) below.\n\n```rust\nuse itertools::Itertools;\nuse venndb::VennDB;\n\n#[derive(Debug, VennDB)]\n// These attributes are optional,\n// e.g. by default the database would be called `EmployeeDB` (name + 'DB').\n#[venndb(name = \"EmployeeInMemDB\", validator = employee_validator)]\npub struct Employee {\n    // you can use the `key` arg to be able to get an `Employee` instance\n    // directly by this key. It will effectively establishing a mapping from key to a reference\n    // of that Employee in the database. As such keys have to have unique values,\n    // or else you get an error while appending / creating the DB.\n    //\n    // NOTE: keys do not only have to be unique, they also have to implement `Clone`!!\n    //\n    // A property cannot be a filter and a key at the same time,\n    // trying to do so will result in a compile-team failure.\n    #[venndb(key)]\n    id: u32,\n    name: String,\n    is_manager: bool,\n    is_admin: bool,\n    // filter (booleans) can be made optional,\n    // meaning that the row will not be able to be filtered (found)\n    // on this column when the row has a `None` value for it\n    is_active: Option\u003cbool\u003e,\n    // booleans are automatically turned into (query) filters,\n    // use the `skip` arg to stop this. As such it is only really needed for\n    // bool properties :)\n    #[venndb(skip)]\n    foo: bool,\n    // non-bool values can also be turned into filters, turning them into 2D filters.\n    // For each uniquely inserted Department variant that is inserted,\n    // a new filter is kept track of. This allows you to apply a (query) filter\n    // based on department, a pretty useful thing to be able to do.\n    //\n    // NOTE: this does mean that such filter-map types have to also be:\n    // `PartialEq + Eq + Hash + Clone`!!\n    //\n    // A property cannot be a filter and a key at the same time,\n    // trying to do so will result in a compile-team failure.\n    #[venndb(filter)]\n    department: Department,\n    // similar to regular bool filters,\n    // filter maps can also be optional.\n    // When a filter map is optional and the row's property for that filter is None,\n    // it will not be registered and thus not be able to filtered (found) on that property\n    #[venndb(filter)]\n    country: Option\u003cString\u003e,\n}\n\nfn employee_validator(employee: \u0026Employee) -\u003e bool {\n    employee.id \u003e 0\n}\n\nfn main() {\n    let db = EmployeeInMemDB::from_iter([\n        RawCsvRow(\"1,John Doe,true,false,true,false,Engineering,USA\"),\n        RawCsvRow(\"2,Jane Doe,false,true,true,true,Sales,\"),\n        RawCsvRow(\"3,John Smith,false,false,,false,Marketing,\"),\n        RawCsvRow(\"4,Jane Smith,true,true,false,true,HR,\"),\n        RawCsvRow(\"5,John Johnson,true,true,true,true,Engineering,\"),\n        RawCsvRow(\"6,Jane Johnson,false,false,,false,Sales,BE\"),\n        RawCsvRow(\"7,John Brown,true,false,true,false,Marketing,BE\"),\n        RawCsvRow(\"8,Jane Brown,false,true,true,true,HR,BR\"),\n    ])\n    .expect(\"MemDB created without errors (e.g. no duplicate keys)\");\n\n    println!(\"\u003e\u003e\u003e Printing all employees...\");\n    let all_employees: Vec\u003c_\u003e = db.iter().collect();\n    assert_eq!(all_employees.len(), 8);\n    println!(\"All employees: {:#?}\", all_employees);\n\n    println!(\"\u003e\u003e\u003e You can lookup an employee by any registered key...\");\n    let employee = db\n        .get_by_id(\u00262)\n        .expect(\"to have found an employee with ID 2\");\n    assert_eq!(employee.name, \"Jane Doe\");\n\n    println!(\"\u003e\u003e\u003e Querying for all managers...\");\n    let mut query = db.query();\n    query.is_manager(true);\n    let managers: Vec\u003c_\u003e = query\n        .execute()\n        .expect(\"to have found at least one\")\n        .iter()\n        .collect();\n    assert_eq!(managers.len(), 4);\n    assert_eq!(\n        managers.iter().map(|e| e.id).sorted().collect::\u003cVec\u003c_\u003e\u003e(),\n        [1, 4, 5, 7]\n    );\n\n    println!(\"\u003e\u003e\u003e Querying for all managers with a last name of 'Johnson'...\");\n    let managers_result = query\n        .execute()\n        .expect(\"to have found at least one\")\n        .filter(|e| e.name.ends_with(\"Johnson\"))\n        .expect(\"to have found a manager with a last name of Johnson\");\n    let managers = managers_result.iter().collect::\u003cVec\u003c_\u003e\u003e();\n    assert_eq!(managers.len(), 1);\n    assert_eq!(managers.iter().map(|e| e.id).collect::\u003cVec\u003c_\u003e\u003e(), [5]);\n\n    println!(\"\u003e\u003e\u003e You can also just get the first result if that is all you care about...\");\n    let manager = managers_result.first();\n    assert_eq!(manager.id, 5);\n\n    println!(\"\u003e\u003e\u003e Querying for a random active manager in the Engineering department...\");\n    let manager = query\n        .reset()\n        .is_active(true)\n        .is_manager(true)\n        .department(Department::Engineering)\n        .execute()\n        .expect(\"to have found at least one\")\n        .any();\n    assert!(manager.id == 1 || manager.id == 5);\n\n    println!(\"\u003e\u003e\u003e Optional bool filters have three possible values, where None != false. An important distinction to make...\");\n    let mut query = db.query();\n    query.is_active(false);\n    let inactive_employees: Vec\u003c_\u003e = query\n        .execute()\n        .expect(\"to have found at least one\")\n        .iter()\n        .collect();\n    assert_eq!(inactive_employees.len(), 1);\n    assert_eq!(inactive_employees[0].id, 4);\n\n    println!(\"\u003e\u003e\u003e If you want you can also get the Employees back as a Vec, dropping the DB data all together...\");\n    let employees = db.into_rows();\n    assert_eq!(employees.len(), 8);\n    assert!(employees[1].foo);\n    println!(\"All employees: {:?}\", employees);\n\n    println!(\"\u003e\u003e\u003e You can also get the DB back from the Vec, if you want start to query again...\");\n    // of course better to just keep it as a DB to begin with, but let's pretend this is ok in this example\n    let mut db = EmployeeInMemDB::from_rows(employees).expect(\"DB created without errors\");\n    assert_eq!(db.iter().count(), 8);\n\n    println!(\"\u003e\u003e\u003e Querying for all active employees in the Sales department...\");\n    let mut query = db.query();\n    query.is_active(true);\n    query.department(Department::Sales);\n    let sales_employees: Vec\u003c_\u003e = query\n        .execute()\n        .expect(\"to have found at least one\")\n        .iter()\n        .collect();\n    assert_eq!(sales_employees.len(), 1);\n    assert_eq!(sales_employees[0].name, \"Jane Doe\");\n\n    println!(\"\u003e\u003e\u003e Filter maps that are optional work as well, e.g. you can query for all employees from USA...\");\n    query.reset().country(\"USA\".to_owned());\n    let usa_employees: Vec\u003c_\u003e = query\n        .execute()\n        .expect(\"to have found at least one\")\n        .iter()\n        .collect();\n    assert_eq!(usa_employees.len(), 1);\n    assert_eq!(usa_employees[0].id, 1);\n\n    println!(\"\u003e\u003e\u003e At any time you can also append new employees to the DB...\");\n    assert_eq!(EmployeeInMemDBErrorKind::DuplicateKey, db\n        .append(RawCsvRow(\"8,John Doe,true,false,true,false,Engineering,\"))\n        .unwrap_err().kind());\n    println!(\"\u003e\u003e\u003e This will fail however if a property is not correct (e.g. ID (key) is not unique in this case), let's try this again...\");\n    assert!(db\n        .append(RawCsvRow(\"9,John Doe,false,true,true,false,Engineering,\"))\n        .is_ok());\n    assert_eq!(db.len(), 9);\n\n    println!(\"\u003e\u003e\u003e Rows are also validated prior to appending in case a validator is defined...\");\n    println!(\"    The next insertion will fail due to the id being zero, a condition defined in the custom validator...\");\n    assert_eq!(EmployeeInMemDBErrorKind::InvalidRow, db\n        .append(RawCsvRow(\"0,John Doe,true,false,true,false,Engineering,\"))\n        .unwrap_err().kind());\n\n    println!(\"\u003e\u003e\u003e This new employee can now also be queried for...\");\n    let mut query = db.query();\n    query.department(Department::Engineering).is_manager(false);\n    let new_employee: Vec\u003c_\u003e = query\n        .execute()\n        .expect(\"to have found at least one\")\n        .iter()\n        .collect();\n    assert_eq!(new_employee.len(), 1);\n    assert_eq!(new_employee[0].id, 9);\n\n    println!(\"\u003e\u003e\u003e You can also extend it using an IntoIterator...\");\n    db.extend([\n        RawCsvRow(\"10,Glenn Doe,false,true,true,true,Engineering,\"),\n        RawCsvRow(\"11,Peter Miss,true,true,true,true,HR,USA\"),\n    ])\n    .unwrap();\n    let mut query = db.query();\n    query\n        .department(Department::HR)\n        .is_manager(true)\n        .is_active(true)\n        .is_admin(true);\n    let employees: Vec\u003c_\u003e = query\n        .execute()\n        .expect(\"to have found at least one\")\n        .iter()\n        .collect();\n    assert_eq!(employees.len(), 1);\n    assert_eq!(employees[0].id, 11);\n\n    println!(\"\u003e\u003e\u003e There are now 2 employees from USA...\");\n    query.reset().country(\"USA\".to_owned());\n    let employees: Vec\u003c_\u003e = query\n        .execute()\n        .expect(\"to have found at least one\")\n        .iter()\n        .collect();\n    assert_eq!(employees.len(), 2);\n    assert_eq!(\n        employees.iter().map(|e| e.id).sorted().collect::\u003cVec\u003c_\u003e\u003e(),\n        [1, 11]\n    );\n\n    println!(\"\u003e\u003e\u003e All previously data is still there as well of course...\");\n    query\n        .reset()\n        .is_active(true)\n        .is_manager(true)\n        .department(Department::Engineering);\n    let managers: Vec\u003c_\u003e = query\n        .execute()\n        .expect(\"to have found at least one\")\n        .iter()\n        .collect();\n    assert_eq!(managers.len(), 2);\n    assert_eq!(\n        managers.iter().map(|e| e.id).sorted().collect::\u003cVec\u003c_\u003e\u003e(),\n        [1, 5]\n    );\n}\n\n#[derive(Debug)]\nstruct RawCsvRow\u003cS\u003e(S);\n\nimpl\u003cS\u003e From\u003cRawCsvRow\u003cS\u003e\u003e for Employee\nwhere\n    S: AsRef\u003cstr\u003e,\n{\n    fn from(RawCsvRow(s): RawCsvRow\u003cS\u003e) -\u003e Employee {\n        let mut parts = s.as_ref().split(',');\n        let id = parts.next().unwrap().parse().unwrap();\n        let name = parts.next().unwrap().to_string();\n        let is_manager = parts.next().unwrap().parse().unwrap();\n        let is_admin = parts.next().unwrap().parse().unwrap();\n        let is_active = match parts.next().unwrap() {\n            \"\" =\u003e None,\n            s =\u003e Some(s.parse().unwrap()),\n        };\n        let foo = parts.next().unwrap().parse().unwrap();\n        let department = parts.next().unwrap().parse().unwrap();\n        let country = match parts.next().unwrap() {\n            \"\" =\u003e None,\n            s =\u003e Some(s.to_string()),\n        };\n        Employee {\n            id,\n            name,\n            is_manager,\n            is_admin,\n            is_active,\n            foo,\n            department,\n            country,\n        }\n    }\n}\n\n#[derive(Debug, PartialEq, Eq, Hash, Clone)]\npub enum Department {\n    Engineering,\n    Sales,\n    Marketing,\n    HR,\n}\n\nimpl std::str::FromStr for Department {\n    type Err = ();\n\n    fn from_str(s: \u0026str) -\u003e Result\u003cSelf, Self::Err\u003e {\n        match s {\n            \"Engineering\" =\u003e Ok(Department::Engineering),\n            \"Sales\" =\u003e Ok(Department::Sales),\n            \"Marketing\" =\u003e Ok(Department::Marketing),\n            \"HR\" =\u003e Ok(Department::HR),\n            _ =\u003e Err(()),\n        }\n    }\n}\n```\n\n### Generated Code Summary\n\nIn this chapter we'll list the API as generated by `VennDB` for the following example code from above:\n\n```rust,ignore\n#[derive(Debug, VennDB)]\n#[venndb(name = \"EmployeeInMemDB\", validator = employee_validator)]\npub struct Employee {\n    #[venndb(key)]\n    id: u32,\n    name: String,\n    is_manager: bool,\n    is_admin: bool,\n    is_active: Option\u003cbool\u003e,\n    #[venndb(skip)]\n    foo: bool,\n    #[venndb(filter)]\n    department: Department,\n    country: Option\u003cString\u003e,\n}\n```\n\nThe following public-API datastructures will be generated:\n\n- `struct EmployeeInMemDB`: the database, that can be used to query (by filters) or look up data (by keys);\n- `enum EmployeeInMemDBError`: the error type that is returned when mutating the DB and a property of the to be inserted row;\n- `enum EmployeeInMemDBErrorKind`: the kind of error that can happen as described for `EmployeeInMemDBError`;\n- `struct EmployeeInMemDBQuery`: the query builder that is used to build a query that can be `execute`d to query data from the db using filters;\n- `struct EmployeeInMemDBQueryResult`: the result when querying using `EmployeeInMemDBQuery` and at least one row was found that matched the defined filters;\n- `struct EmployeeInMemDBQueryResultIter`: the iterator type that is used when calling `EmployeeInMemDBQueryResult::iter`. It has no methods/api other then the fact that it is an `Iterator` and can be used as one;\n\nThe visual specifiers of these datastructures will be the same as the `struct` that the `VennDB` macro is applied to.\nE.g. in this example `Employee` has a specifier of `pub` so the above datastructures and their public-apy methods will also be `pub`.\n\nThere are also some other helper datastructures generated — all prefixed with the database name, e.g. `EmployeeInMemDB` in this example —\nbut we do not mention here as they should not be relied upon and given the prefix it should cause no conflict.\nIn case you do not want to expose these structures to the outside you can wrap your `struct` within its own `mod` (module).\n\n#### Generated Code Summary: Method API\n\nDatabase: (e.g. `EmployeeInMemDB`):\n\n| fn signature | description |\n| - | - |\n| `EmployeeInMemDB::new() -\u003e EmployeeInMemDB` | create a new database with zero capacity |\n| `EmployeeInMemDB::default() -\u003e EmployeeInMemDB` | same as `EmployeeInMemDB::new() -\u003e EmployeeInMemDB` |\n| `EmployeeInMemDB::capacity(capacity: usize) -\u003e EmployeeInMemDB` | create a new database with the given capacity, but no rows already inserted |\n| `EmployeeInMemDB::from_rows(rows: ::std::vec::Vec\u003cEmployee\u003e) -\u003e EmployeeInMemDB` or `EmployeeInMemDB::from_rows(rows: ::std::vec::Vec\u003cEmployee\u003e) -\u003e Result\u003cEmployeeInMemDB, EmployeeInMemDBError\u003c::std::vec::Vec\u003cEmployee\u003e\u003e\u003e` | constructor to create the database directly from a heap-allocated list of data instances. The second version is the one used if at least one `#[venndb(key)]` property is defined, otherwise it is the first one (without the `Result`). |\n| `EmployeeInMemDB::from_iter(iter: impl ::std::iter::IntoIterator\u003cItem = impl ::std::convert::Into\u003cEmployee\u003e\u003e) -\u003e EmployeeInMemDB` or `EmployeeInMemDB::from_rows(iter: impl ::std::iter::IntoIterator\u003cItem = impl ::std::convert::Into\u003cEmployee\u003e\u003e) -\u003e Result\u003cEmployeeInMemDB, EmployeeInMemDBError\u003c::std::vec::Vec\u003cEmployee\u003e\u003e\u003e` | Same as `from_rows` but using an iterator instead. The items do not have to be an `Employee` but can be anything that can be turned into one. E.g. in our example above we defined a struct `RawCsvRow` that was turned on the fly into an `Employee`. This happens all at once prior to inserting the database, which is why the version with a result does return a `Vec` and not an iterator. |\n| `EmployeeInMemDB::append(\u0026mut self, data: impl ::std::convert::Into\u003cEmployee\u003e)` or `EmployeeInMemDB::append(\u0026mut self, data: impl ::std::convert::Into\u003cEmployee\u003e) -\u003e Result\u003c(), EmployeeInMemDBError\u003cEmployee\u003e\u003e` | append a single row to the database. Depending on whether or not a `#[venndb(key)]` property is defined it will generate the `Result` version or not. Same as `from_rows` and `from_iter` |\n| `EmployeeInMemDB::extend\u003cI, Item\u003e(\u0026mut self, iter: I) where I: ::std::iter::IntoIterator\u003cItem = Item\u003e, Item: ::std::convert::Into\u003cEmployee\u003e` or `EmployeeInMemDB::extend\u003cI, Item\u003e(\u0026mut self, iter: I) -\u003e Result\u003c(), EmployeeInMemDBError\u003c(Employee, I::IntoIter)\u003e\u003e where I: ::std::iter::IntoIterator\u003cItem = Item\u003e, Item: ::std::convert::Into\u003cEmployee\u003e` | extend the database with the given iterator, once again returning a result in case such insertion can go wrong (e.g. because keys are used (duplication) or a row is invalid in case a validator is defined). Otherwise this function will return nothing. |\n| `EmployeeInMemDB::get_by_id\u003cQ\u003e(\u0026self, data: impl ::std::convert::Into\u003cEmployee\u003e) -\u003e Option\u003c\u0026Employee\u003e where Employee ::std::borrow::Borrow\u003cQ\u003e, Q: ::std::hash::Hash + ::std::cmp::Eq + ?::std::marker::Sized` | look up a row by the `id` key property. This method will be generated for each property marked with `#[venndb(key)`. e.g. if you have key property named `foo: MyType` property there will be also a `get_by_foo(\u0026self, ...)` method generated. |\n| `EmployeeInMemDB::query(\u0026self) -\u003e EmployeeInMemDBQuery` | create a `EmployeeInMemDBQuery` builder to compose a filter composition to query the database. The default builder will match all rows. See the method API for `EmployeeInMemDBQuery` for more information |\n\nQuery (e.g. `EmployeeInMemDBQuery`)\n\n| fn signature | description |\n| - | - |\n| `EmployeeInMemDBQuery::reset(\u0026mut self) -\u003e \u0026mut Self` | reset the query, bringing it back to the clean state it has on creation |\n| `EmployeeInMemDBQuery::execute(\u0026self) -\u003e Option\u003cEmployeeInMemDBQueryResult\u003c'a\u003e\u003e` | return the result of the query using the set filters. It will be `None` in case no rows matched the defined filters. Or put otherwise, the result will contain at least one row when `Some(_)` is returned. |\n| `EmployeeInMemDBQuery::is_manager(\u0026mut self, value: bool) -\u003e \u0026mut Self` | a filter setter for a `bool` filter. One such method per `bool` filter (that isn't `skip`ped) will be available. E.g. if you have ` foo` filter then there will be a `EmployeeInMemDBQuery:foo` method. For _bool_ filters that are optional (`Option\u003cbool\u003e`) this method is also generated just the same. |\n| `EmployeeInMemDBQuery::department(\u0026mut self, value: impl ::std::convert::Into\u003cDepartment\u003e) -\u003e \u0026mut Self` | a filter (map) setter for a non-`bool` filter. One such method per non-`bool` filter will be available. You can also `skip` these, but that's of course a bit pointless. The type will be equal to the actual field type. And the name will once again be equal to the original field name. Filter maps that have a `Option\u003cT\u003e` type have exactly the same signature. Duering query you can call this method multiple times in case you wish to allow multiple variants. |\n\nQuery Result (e.g. `EmployeeInMemDBQueryResult`)\n\n| fn signature | description |\n| - | - |\n| `EmployeeInMemDBQueryResult::first(\u0026self) -\u003e \u0026Employee` | return a reference to the first matched employee found. An implementation detail is that this will be the matched row that was first inserted, but for compatibility reasons you best not rely on this if you do not have to. |\n| `EmployeeInMemDBQueryResult::any(\u0026self) -\u003e \u0026Employee` | return a reference to a randomly selected matched employee. The randomness can be relied upon to be fair.  |\n| `EmployeeInMemDBQueryResult::iter(\u0026self) -\u003e `EmployeeInMemDBQueryResultIter` | return an iterator for the query result, which will allow you to iterate over all found results, and as such also collect them into an owned data structure should you wish. |\n| `EmployeeInMemDBQueryResult::filter\u003cF\u003e(\u0026self, predicate: F) -\u003e Option\u003c#EmployeeInMemDBQueryResult\u003e where F: Fn(\u0026#name) -\u003e bool` | return `Some(_)` `EmployeeInMemDBQueryResult` with the same reference data, but containing (and owning) only the indexes for which the linked row matches arcoding to the given `Fn` predicate |\n\n## ⛨ | Safety\n\nThis crate uses `#![forbid(unsafe_code)]` to ensure everything is implemented in 100% safe Rust.\n\n## 🦀 | Compatibility\n\nvenndb is developed mostly on MacOS M-Series machines and run in production\non a variety of Linux systems. Windows support is not officially guaranteed,\nbut is [tested using Github Actions](https://github.com/plabayo/venndb/blob/main/.github/workflows/CI.yml) with success.\n\n| platform | tested | test platform |\n|----------|--------|---------------|\n| MacOS    | ✅     | M2 (developer laptop) and macos-12 Intel ([GitHub Action](https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners/about-github-hosted-runners)) |\n| Windows  | ✅     | Windows 2022 ([GitHub Action](https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners/about-github-hosted-runners)) |\n| Linux    | ✅     | Ubuntu 22.04 ([GitHub Action](https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners/about-github-hosted-runners)) |\n\nPlease [open a ticket](https://github.com/plabayo/venndb/issues) in case you have compatibility issues for your setup/platform.\nOur goal is not to support all possible platformns in the world, but we do want to\nsupport as many as we reasonably can.\n\n### Minimum supported Rust version\n\nvenndb's MSRV is `1.75`.\n\n[Using GitHub Actions we also test](https://github.com/plabayo/venndb/blob/main/.github/workflows/CI.yml) if `venndb` on that version still works on\nthe stable and beta versions of _rust_ as well.\n\n## 🧭 | Roadmap\n\nPlease refer to \u003chttps://github.com/plabayo/venndb/milestones\u003e to know what's on the roadmap. Is there something not on the roadmap for the next version that you would really like? Please [create a feature request](https://github.com/plabayo/venndb/issues) to request it and [become a sponsor](#--sponsors) if you can.\n\n## 💼 | License\n\nThis project is dual-licensed under both the [MIT license][mit-license] and [Apache 2.0 License][apache-license].\n\n## 👋 | Contributing\n\n🎈 Thanks for your help improving the project! We are so happy to have\nyou! We have a [contributing guide][contributing] to help you get involved in the\n`venndb` project.\n\nContributions often come from people who already know what they want, be it a fix for a bug they encountered,\nor a feature that they are missing. Please do always make a ticket if one doesn't exist already.\n\nIt's possible however that you do not yet know what specifically to contribute, and yet want to help out.\nFor that we thank you. You can take a look at the open issues, and in particular:\n\n- [`good first issue`](https://github.com/plabayo/venndb/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22): issues that are good for those new to the `venndb` codebase;\n- [`easy`](https://github.com/plabayo/venndb/issues?q=is%3Aissue+is%3Aopen+label%3Aeasy): issues that are seen as easy;\n- [`mentor available`](https://github.com/plabayo/venndb/issues?q=is%3Aissue+is%3Aopen+label%3A%22mentor+available%22): issues for which we offer mentorship;\n- [`low prio`](https://github.com/plabayo/venndb/issues?q=is%3Aissue+is%3Aopen+label%3A%22low+prio%22): low prio issues that have no immediate pressure to be finished quick, great in case you want to help out but can only do with limited time to spare;\n\nIn general, any issue not assigned already is free to be picked up by anyone else. Please do communicate in the ticket\nif you are planning to pick it up, as to avoid multiple people trying to solve the same one.\n\nShould you want to contribure this project but you do not yet know how to program in Rust, you could start learning Rust with as goal to contribute as soon as possible to `venndb` by using \"[the Rust 101 Learning Guide](https://rust-lang.guide/)\" as your study companion. Glen can also be hired as a mentor or teacher to give you paid 1-on-1 lessons and other similar consultancy services. You can find his contact details at \u003chttps://www.glendc.com/\u003e.\n\n### Contribution\n\nUnless you explicitly state otherwise, any contribution intentionally submitted\nfor inclusion in `venndb` by you, shall be licensed as both [MIT][mit-license] and [Apache 2.0][apache-license],\nwithout any additional terms or conditions.\n\n[contributing]: https://github.com/plabayo/venndb/blob/main/CONTRIBUTING.md\n[mit-license]: https://github.com/plabayo/venndb/blob/main/LICENSE-MIT\n[apache-license]: https://github.com/plabayo/venndb/blob/main/LICENSE-APACHE\n\n### Acknowledgements\n\nSpecial thanks goes to all involved in developing, maintaining and supporting [the Rust programming language](https://www.rust-lang.org/). Also a big shoutout to the [\"Write Powerful Rust Macros\" book by _Sam Van Overmeire_](https://www.manning.com/books/write-powerful-rust-macros), which gave the courage to develop this crate.\n\nSome code was also copied/forked from [google/argh](https://github.com/google/argh), for which thank you,\nwe are big fans of that crate. Go use it if you want to create a CLI App.\n\n## 💖 | Sponsors\n\nvenndb is **completely free, open-source software** which needs lots of effort and time to develop and maintain.\n\nSupport this project by becoming a [sponsor][ghs-url]. One time payments are accepted [at GitHub][ghs-url] as well as at [\"Buy me a Coffee\"][bmac-url].\n\nSponsors help us continue to maintain and improve `venndb`, as well as other\nFree and Open Source (FOSS) technology. It also helps us to create\neducational content such as \u003chttps://github.com/plabayo/learn-rust-101\u003e,\nand other open source frameworks such as \u003chttps://github.com/plabayo/rama\u003e.\n\nSponsors receive perks and depending on your regular contribution it also\nallows you to rely on us for support and consulting.\n\nFinally, you can also support us by shopping Plabayo \u003c3 `VennDB` merchandise 🛍️ at \u003chttps://plabayo.threadless.com/\u003e.\n\n[![Plabayo's Store With VennDB Merchandise](https://raw.githubusercontent.com/plabayo/venndb/main/docs/img/plabayo_mech_store_venndb.png)](https://plabayo.threadless.com/)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fplabayo%2Fvenndb","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fplabayo%2Fvenndb","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fplabayo%2Fvenndb/lists"}