{"id":13632315,"url":"https://github.com/lun3x/multi_index_map","last_synced_at":"2025-12-12T13:50:33.674Z","repository":{"id":45584556,"uuid":"507418876","full_name":"lun3x/multi_index_map","owner":"lun3x","description":"Simple and flexible multi-index containers.","archived":false,"fork":false,"pushed_at":"2025-04-13T06:33:48.000Z","size":184,"stargazers_count":107,"open_issues_count":6,"forks_count":8,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-04-13T07:34:47.479Z","etag":null,"topics":["boost","container","data-structures","index","map","rust"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lun3x.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2022-06-25T21:18:13.000Z","updated_at":"2025-04-13T06:33:52.000Z","dependencies_parsed_at":"2022-08-21T10:10:21.881Z","dependency_job_id":"c96809ea-a1aa-4d4d-9630-4143675b5a50","html_url":"https://github.com/lun3x/multi_index_map","commit_stats":{"total_commits":69,"total_committers":2,"mean_commits":34.5,"dds":"0.17391304347826086","last_synced_commit":"a086ff85690d1e69bfc83bb96909f9e3ef3c03d2"},"previous_names":[],"tags_count":16,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lun3x%2Fmulti_index_map","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lun3x%2Fmulti_index_map/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lun3x%2Fmulti_index_map/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lun3x%2Fmulti_index_map/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lun3x","download_url":"https://codeload.github.com/lun3x/multi_index_map/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249414252,"owners_count":21267724,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["boost","container","data-structures","index","map","rust"],"created_at":"2024-08-01T22:02:59.642Z","updated_at":"2025-12-12T13:50:33.666Z","avatar_url":"https://github.com/lun3x.png","language":"Rust","funding_links":[],"categories":["Rust"],"sub_categories":[],"readme":"# MultiIndexMap [![Tests](https://github.com/lun3x/multi_index_map/actions/workflows/ci.yml/badge.svg?branch=master)](https://github.com/lun3x/multi_index_map/actions/workflows/ci.yml)\n\n[Also available on crates.io.](https://crates.io/crates/multi_index_map)\n\nRust library useful for storing structs that needs to be accessed through various different indexes of the fields of the struct. Inspired by [C++/Boost Multi-index Containers](https://www.boost.org/doc/libs/1_79_0/libs/multi_index/doc/index.html) but redesigned for a more idiomatic Rust API.\n\nCurrent implementation supports:\n* Hashed indexes using HashMap from [std::collections](https://doc.rust-lang.org/std/collections/struct.HashMap.html)\n* Sorted indexes using BTreeMap from [std::collections](https://doc.rust-lang.org/std/collections/struct.BTreeMap.html).\n* Unique and non-unique indexes.\n* Unindexed fields.\n* Iterators for each indexed field.\n* Iterators for the underlying backing storage.\n\n# Performance characteristics\n## Unique Indexes\n* Hashed index retrievals are constant-time. (HashMap + Slab).\n* Sorted indexes retrievals are logarithmic-time. (BTreeMap + Slab).\n* Iteration over hashed index is same as HashMap, plus a retrieval from the backing storage for each element.\n* Iteration over ordered index is same as BTreeMap, plus a retrieval from the backing storage for each element.\n* Iteration over the backing store is the same as Slab, so contiguous memory but with potentially vacant slots.\n* Insertion, removal, and modification complexity grows as the number of indexed fields grow. All indexes must be updated during these operations so these are slower.\n* Modification of unindexed fields through get_mut_by_ methods is the same as regular retrieval time.\n* Insertion such that uniqueness would be violated does not mutate the map, instead the element is returned to the user wrapped in an Err variant.\n\n## Non-Unique Indexes\n* Hashed index retrievals are still constant-time with the total number of elements, but linear-time with the number of matching elements. (HashMap + (Slab * num_matches)).\n* Sorted indexes retrievals are still logarithmic-time with total number of elements, but linear-time with the number of matching elements. (BTreeMap + (Slab * num_matches)).\n* Each equal range of any non-unique index is stored as a BTreeSet, which we must iterate through the length of when retrieving all matching elements, and also when iterating over the whole index.\n\n# Default Hasher\n* The feature `rustc-hash` is enabled by default. It will set the default hash as [`rustc-hash`](https://github.com/rust-lang/rustc-hash/).\n* The hash can always be changed by specifying a `BuildHasher` implementation in the `multi_index_hash` attribute, eg. `#[multi_index_hash(ahash::RandomState)]`.\n* With default features disabled the default hash will be the standard library default (currently `SipHash`). Default features can be disabled in `Cargo.toml` like so:\n\n```multi_index_map = { version = \"*\", default-features = false }```\n\n# How to use\n\n* This crate provides a derive macro `MultiIndexMap`, which when applied to the struct representing an element will generate a map to store and access these elements.\n* Annotations are used to specify which fields to index. Currently `hashed_unique`, `hashed_non_unique`, `ordered_unique`, and `ordered_non_unique` are supported.\n* The types of all indexed fields must implement `Clone`.\n* Optionally, `multi_index_derive` can be used to derive traits on the generated MultiIndexMap, eg. `#[multi_index_derive(Clone, Debug)]`\nSee `examples/main.rs` for more details.\n\n## Example\n\n```rust\nuse multi_index_map::MultiIndexMap;\n\n#[derive(MultiIndexMap, Debug)]\n#[multi_index_derive(Debug)]\n#[multi_index_hash(rustc_hash::FxBuildHasher)]\nstruct Order {\n    #[multi_index(hashed_unique)]\n    order_id: u32,\n    #[multi_index(ordered_unique)]\n    timestamp: u64,\n    #[multi_index(hashed_non_unique)]\n    trader_name: String,\n    filled: bool,\n    volume: u64,\n}\n\nfn main() {\n    let order1 = Order {\n        order_id: 1,\n        timestamp: 1656145181,\n        trader_name: \"JohnDoe\".into(),\n        filled: false,\n        volume: 100,\n    };\n\n    let order2 = Order {\n        order_id: 2,\n        timestamp: 1656145182,\n        trader_name: \"JohnDoe\".into(),\n        filled: false,\n        volume: 100,\n    };\n\n    let mut map = MultiIndexOrderMap::default();\n\n    map.try_insert(order1).unwrap();\n    map.insert(order2);\n\n    let orders = map.get_by_trader_name(\u0026\"JohnDoe\".to_string());\n    assert_eq!(orders.len(), 2);\n    println!(\"Found 2 orders for JohnDoe: [{orders:?}]\");\n\n    let order1_ref = map.get_by_order_id(\u00261).unwrap();\n    assert_eq!(order1_ref.timestamp, 1656145181);\n\n    let order2_ref = map\n        .modify_by_order_id(\u00262, |o| {\n            o.timestamp = 1656145183;\n            o.order_id = 42;\n        })\n        .unwrap();\n\n    assert_eq!(order2_ref.timestamp, 1656145183);\n    assert_eq!(order2_ref.order_id, 42);\n    assert_eq!(order2_ref.trader_name, \"JohnDoe\".to_string());\n\n    let order2_ref = map\n        .update_by_order_id(\u002642, |filled: \u0026mut bool, volume: \u0026mut u64| {\n            *filled = true;\n            *volume = 0;\n        })\n        .unwrap();\n    assert_eq!(order2_ref.filled, true);\n    assert_eq!(order2_ref.volume, 0);\n\n    let orders = map.get_by_trader_name(\u0026\"JohnDoe\".to_string());\n    assert_eq!(orders.len(), 2);\n    println!(\"Found 2 orders for JohnDoe: [{orders:?}]\");\n\n    let orders = map.remove_by_trader_name(\u0026\"JohnDoe\".to_string());\n    for (_idx, order) in map.iter() {\n        assert_eq!(order.trader_name, \"JohnDoe\");\n    }\n    assert_eq!(orders.len(), 2);\n\n    println!(\"{map:?}\");\n\n    // See examples and tests directories for more in depth usage.\n}\n```\n\n# Under the hood\n\nThe above example will generate the following MultiIndexMap and associated Iterators.\nThe `Order`s are stored in a `Slab`, in contiguous memory, which allows for fast lookup and quick iteration. \nA lookup table is created for each indexed field, which maps the index key to a index in the `Slab`.\nThe exact type used for these depends on the annotations.\nFor `hashed_unique` and `hashed_non_unique` a `HashMap` is used, for `ordered_unique` and `ordered_non_unique` a `BTreeMap` is used.\n* When inserting an element, we add it to the backing store, then add elements to each lookup table pointing to the index in the backing store.\n* When retrieving elements for a given key, we lookup the key in the lookup table, then retrieve the item at that index in the backing store.\n* When removing an element for a given key, we do the same, but we then must also remove keys from all the other lookup tables before returning the element.\n* When iterating over an index, we use the default iterators for the lookup table, then simply retrieve the element at the given index in the backing store.\n* When updating un-indexed fields, we lookup the element(s) through the given key, then apply the closure to modify just the unindexed fields in-place.\nWe then return a reference to the modified element(s).\nIf the key doesn't match, the closure won't be applied, and Option::None will be returned.\n* When modifying indexed fields of an element, we do the same process, but the closure takes a mutable reference to the whole element.\nAny fields, indexed and un-indexed can be modified.\nWe must then update all the lookup tables to account for any changes to indexed fields, so this is slower than an un-indexed update.\n\n\n```rust\nstruct MultiIndexOrderMap {\n    _store: slab::Slab\u003cOrder\u003e,\n    _order_id_index: HashMap\u003cu32, usize, rustc_hash::FxBuildHasher\u003e,\n    _timestamp_index: BTreeMap\u003cu64, usize\u003e,\n    _trader_name_index: HashMap\u003cString, BTreeSet\u003cusize\u003e, rustc_hash::FxBuildHasher\u003e,\n}\n\nstruct MultiIndexOrderMapOrderIdIter\u003c'a\u003e {\n    ...\n}\n\nstruct MultiIndexOrderMapTimestampIter\u003c'a\u003e {\n    ...\n}\n\nstruct MultiIndexOrderMapTraderNameIter\u003c'a\u003e {\n    ...\n}\n\nstruct OrderMutIter\u003c'a\u003e {\n    ...\n}\n\nimpl MultiIndexOrderMap {\n    fn try_insert(\u0026mut self, elem: Order) -\u003e Result\u003c\u0026Order, MultiIndexMapError\u003cOrder\u003e\u003e;\n    fn insert(\u0026mut self, elem: Order) -\u003e \u0026Order;\n    \n    fn len(\u0026self) -\u003e usize;\n    fn is_empty(\u0026self) -\u003e bool;\n    fn clear(\u0026mut self);\n    \n    fn get_by_order_id(\u0026self, key: \u0026u32) -\u003e Option\u003c\u0026Order\u003e;\n    fn get_by_timestamp(\u0026self, key: \u0026u64) -\u003e Option\u003c\u0026Order\u003e;\n    fn get_by_trader_name(\u0026self, key: \u0026String) -\u003e Vec\u003c\u0026Order\u003e;\n\n    fn get_mut_by_order_id(\u0026mut self, key: \u0026u32) -\u003e Option\u003c(\u0026mut bool, \u0026mut u64)\u003e;\n    fn get_mut_by_timestamp(\u0026mut self, key: \u0026u64) -\u003e Option\u003c(\u0026mut bool, \u0026mut u64)\u003e;\n    fn get_mut_by_trader_name(\u0026mut self, key: \u0026String) -\u003e Vec\u003c(\u0026mut bool, \u0026mut u64)\u003e;\n\n    fn update_by_order_id(\u0026mut self, key: \u0026u32, f: impl FnOnce(\u0026mut bool, \u0026mut u64)) -\u003e Option\u003c\u0026Order\u003e;\n    fn update_by_timestamp(\u0026mut self, key: \u0026u64, f: impl FnOnce(\u0026mut bool, \u0026mut u64)) -\u003e Option\u003c\u0026Order\u003e;\n    fn update_by_trader_name(\u0026mut self, key: \u0026String, f: impl FnMut(\u0026mut bool, \u0026mut u64)) -\u003e Vec\u003c\u0026Order\u003e;\n    \n    fn modify_by_order_id(\u0026mut self, key: \u0026u32, f: impl FnOnce(\u0026mut Order)) -\u003e Option\u003c\u0026Order\u003e;\n    fn modify_by_timestamp(\u0026mut self, key: \u0026u64, f: impl FnOnce(\u0026mut Order)) -\u003e Option\u003c\u0026Order\u003e;\n    fn modify_by_trader_name(\u0026mut self, key: \u0026String, f: impl FnMut(\u0026mut Order)) -\u003e Vec\u003c\u0026Order\u003e;\n    \n    fn remove_by_order_id(\u0026mut self, key: \u0026u32) -\u003e Option\u003cOrder\u003e;\n    fn remove_by_timestamp(\u0026mut self, key: \u0026u64) -\u003e Option\u003cOrder\u003e;\n    fn remove_by_trader_name(\u0026mut self, key: \u0026String) -\u003e Vec\u003cOrder\u003e;\n    \n    fn iter(\u0026self) -\u003e slab::Iter\u003cOrder\u003e;\n    fn iter_mut(\u0026mut self) -\u003e OrderMutIter;\n    \n    fn iter_by_order_id(\u0026self) -\u003e MultiIndexOrderMapOrderIdIter;\n    fn iter_by_timestamp(\u0026self) -\u003e MultiIndexOrderMapTimestampIter;\n    fn iter_by_trader_name(\u0026self) -\u003e MultiIndexOrderMapTraderNameIter;\n}\n\nimpl\u003c'a\u003e Iterator for OrderMutIter\u003c'a\u003e {\n    type Item = (\u0026mut bool, \u0026mut u64);\n\n    fn next(\u0026mut self) -\u003e Option\u003cSelf::Item\u003e {\n        ...\n    }\n}\n```\n\n# Dependencies\nSee [Cargo.toml](Cargo.toml) for information on each dependency.\n\n# Future work\n* Potentially a vector-map style lookup table would be very quick for small tables with integer indexes.\n* Allow overwriting behaviour upon inserting a duplicate unique index, returning a Vec of the overwritten elements.\n* Implement [clever tricks](https://www.boost.org/doc/libs/1_36_0/libs/multi_index/doc/performance.html) used in boost::multi_index_containers to improve performance.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flun3x%2Fmulti_index_map","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flun3x%2Fmulti_index_map","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flun3x%2Fmulti_index_map/lists"}