{"id":20704955,"url":"https://github.com/kixiron/lasso","last_synced_at":"2025-04-12T21:24:32.395Z","repository":{"id":40235222,"uuid":"250698776","full_name":"Kixiron/lasso","owner":"Kixiron","description":"A fast, concurrent string interner","archived":false,"fork":false,"pushed_at":"2024-08-19T23:01:55.000Z","size":517,"stargazers_count":149,"open_issues_count":10,"forks_count":21,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-04-04T00:52:26.219Z","etag":null,"topics":["compiler-backend","interning-strings","langdev","rust","string-interning"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Kixiron.png","metadata":{"files":{"readme":"README.md","changelog":"Changelog.md","contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE-APACHE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":["Kixiron"],"custom":["https://www.buymeacoffee.com/kixiron"]}},"created_at":"2020-03-28T02:40:54.000Z","updated_at":"2025-04-03T02:17:51.000Z","dependencies_parsed_at":"2024-06-19T05:32:09.308Z","dependency_job_id":"df23ca24-9e45-4e87-af5a-1745eb20a565","html_url":"https://github.com/Kixiron/lasso","commit_stats":{"total_commits":228,"total_committers":13,"mean_commits":17.53846153846154,"dds":0.4078947368421053,"last_synced_commit":"30a541360f5c4b0eeaea7f94d02a85651278f4a6"},"previous_names":[],"tags_count":11,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Kixiron%2Flasso","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Kixiron%2Flasso/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Kixiron%2Flasso/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Kixiron%2Flasso/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Kixiron","download_url":"https://codeload.github.com/Kixiron/lasso/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248633161,"owners_count":21136816,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["compiler-backend","interning-strings","langdev","rust","string-interning"],"created_at":"2024-11-17T01:15:23.364Z","updated_at":"2025-04-12T21:24:32.372Z","avatar_url":"https://github.com/Kixiron.png","language":"Rust","funding_links":["https://github.com/sponsors/Kixiron","https://www.buymeacoffee.com/kixiron"],"categories":[],"sub_categories":[],"readme":"[![CI][1]][0]\n[![Security Audit][2]][0]\n[![Coverage][3]][4]\n[![Docs.rs][6]][7]\n[![Crates.io][8]][9]\n\nA multithreaded and single threaded [string interner](https://en.wikipedia.org/wiki/String_interning) that allows strings to be cached with a minimal memory footprint,\nassociating them with a unique [key] that can be used to retrieve them at any time. A [`Rodeo`] allows `O(1)`\ninternment and resolution and can be turned into a [`RodeoReader`] to allow for contention-free resolutions\nwith both key to str and str to key operations. It can also be turned into a [`RodeoResolver`] with only\nkey to str operations for the lowest possible memory usage.\n\n## Which interner do I use?\n\nFor single-threaded workloads [`Rodeo`] is encouraged, while multi-threaded applications should use [`ThreadedRodeo`].\nBoth of these are the only way to intern strings, but most applications will hit a stage where they are done interning\nstrings, and at that point is where the choice between [`RodeoReader`] and [`RodeoResolver`]. If the user needs to get\nkeys for strings still, then they must use the [`RodeoReader`] (although they can still transfer into a  [`RodeoResolver`])\nat this point. For users who just need key to string resolution, the [`RodeoResolver`] gives contention-free access at the\nminimum possible memory usage. Note that to gain access to [`ThreadedRodeo`] the `multi-threaded` feature is required.\n\n| Interner          | Thread-safe | Intern String | str to key | key to str | Contention Free | Memory Usage |\n| ----------------- | :---------: | :-----------: | :--------: | :--------: | :-------------: | :----------: |\n| [`Rodeo`]         |      ❌      |       ✅       |     ✅      |     ✅      |       N/A       |    Medium    |\n| [`ThreadedRodeo`] |      ✅      |       ✅       |     ✅      |     ✅      |        ❌        |     Most     |\n| [`RodeoReader`]   |      ✅      |       ❌       |     ✅      |     ✅      |        ✅        |    Medium    |\n| [`RodeoResolver`] |      ✅      |       ❌       |     ❌      |     ✅      |        ✅        |    Least     |\n\n## Cargo Features\n\nBy default `lasso` has one dependency, `hashbrown`, and only [`Rodeo`] is exposed. Hashbrown is used since the [`raw_entry` api] is currently unstable in the standard library's hashmap.\nThe raw hashmap API is used for custom hashing within the hashmaps, which works to dramatically reduce memory usage\nTo make use of [`ThreadedRodeo`], you must enable the `multi-threaded` feature.\n\n* `multi-threaded` - Enables [`ThreadedRodeo`], the interner for multi-threaded tasks\n* `ahasher` - Use [`ahash`]'s `RandomState` as the default hasher\n* `no-std` - Enables `no_std` + `alloc` support for [`Rodeo`] and [`ThreadedRodeo`]\n  * Automatically enables the following required features:\n    * `ahasher` - `no_std` hashing function\n* `serialize` - Implements `Serialize` and `Deserialize` for all `Spur` types and all interners\n* `inline-more` - Annotate external apis with `#[inline]`\n\n## Example: Using Rodeo\n\n```rust\nuse lasso::Rodeo;\n\nlet mut rodeo = Rodeo::default();\nlet key = rodeo.get_or_intern(\"Hello, world!\");\n\n// Easily retrieve the value of a key and find the key for values\nassert_eq!(\"Hello, world!\", rodeo.resolve(\u0026key));\nassert_eq!(Some(key), rodeo.get(\"Hello, world!\"));\n\n// Interning the same string again will yield the same key\nlet key2 = rodeo.get_or_intern(\"Hello, world!\");\n\nassert_eq!(key, key2);\n```\n\n## Example: Using ThreadedRodeo\n\n```rust\nuse lasso::ThreadedRodeo;\nuse std::{thread, sync::Arc};\n\nlet rodeo = Arc::new(ThreadedRodeo::default());\nlet key = rodeo.get_or_intern(\"Hello, world!\");\n\n// Easily retrieve the value of a key and find the key for values\nassert_eq!(\"Hello, world!\", rodeo.resolve(\u0026key));\nassert_eq!(Some(key), rodeo.get(\"Hello, world!\"));\n\n// Interning the same string again will yield the same key\nlet key2 = rodeo.get_or_intern(\"Hello, world!\");\n\nassert_eq!(key, key2);\n\n// ThreadedRodeo can be shared across threads\nlet moved = Arc::clone(\u0026rodeo);\nlet hello = thread::spawn(move || {\n    assert_eq!(\"Hello, world!\", moved.resolve(\u0026key));\n    moved.get_or_intern(\"Hello from the thread!\")\n})\n.join()\n.unwrap();\n\nassert_eq!(\"Hello, world!\", rodeo.resolve(\u0026key));\nassert_eq!(\"Hello from the thread!\", rodeo.resolve(\u0026hello));\n```\n\n## Example: Creating a RodeoReader\n\n```rust\nuse lasso::Rodeo;\n\n// Rodeo and ThreadedRodeo are interchangeable here\nlet mut rodeo = Rodeo::default();\n\nlet key = rodeo.get_or_intern(\"Hello, world!\");\nassert_eq!(\"Hello, world!\", rodeo.resolve(\u0026key));\n\nlet reader = rodeo.into_reader();\n\n// Reader keeps all the strings from the parent\nassert_eq!(\"Hello, world!\", reader.resolve(\u0026key));\nassert_eq!(Some(key), reader.get(\"Hello, world!\"));\n\n// The Reader can now be shared across threads, no matter what kind of Rodeo created it\n```\n\n## Example: Creating a RodeoResolver\n\n```rust\nuse lasso::Rodeo;\n\n// Rodeo and ThreadedRodeo are interchangeable here\nlet mut rodeo = Rodeo::default();\n\nlet key = rodeo.get_or_intern(\"Hello, world!\");\nassert_eq!(\"Hello, world!\", rodeo.resolve(\u0026key));\n\nlet resolver = rodeo.into_resolver();\n\n// Resolver keeps all the strings from the parent\nassert_eq!(\"Hello, world!\", resolver.resolve(\u0026key));\n\n// The Resolver can now be shared across threads, no matter what kind of Rodeo created it\n```\n\n## Example: Making a custom-ranged key\n\nSometimes you want your keys to only inhabit (or *not* inhabit) a certain range of values so that you can have custom [niches].\nThis allows you to pack more data into what would otherwise be unused space, which can be critical for memory-sensitive applications.\n\n```rust\nuse lasso::{Key, Rodeo};\n\n// First make our key type, this will be what we use as handles into our interner\n#[derive(Copy, Clone, PartialEq, Eq)]\nstruct NicheKey(u32);\n\n// This will reserve the upper 255 values for us to use as niches\nconst NICHE: usize = 0xFF000000;\n\n// Implementing `Key` is unsafe and requires that anything given to `try_from_usize` must produce the\n// same `usize` when `into_usize` is later called\nunsafe impl Key for NicheKey {\n    fn into_usize(self) -\u003e usize {\n        self.0 as usize\n    }\n\n    fn try_from_usize(int: usize) -\u003e Option\u003cSelf\u003e {\n        if int \u003c NICHE {\n            // The value isn't in our niche range, so we're good to go\n            Some(Self(int as u32))\n        } else {\n            // The value interferes with our niche, so we return `None`\n            None\n        }\n    }\n}\n\n// To make sure we're upholding `Key`'s safety contract, let's make two small tests\n#[test]\nfn value_in_range() {\n    let key = NicheKey::try_from_usize(0).unwrap();\n    assert_eq!(key.into_usize(), 0);\n\n    let key = NicheKey::try_from_usize(NICHE - 1).unwrap();\n    assert_eq!(key.into_usize(), NICHE - 1);\n}\n\n#[test]\nfn value_out_of_range() {\n    let key = NicheKey::try_from_usize(NICHE);\n    assert!(key.is_none());\n\n    let key = NicheKey::try_from_usize(u32::max_value() as usize);\n    assert!(key.is_none());\n}\n\n// And now we're done and can make `Rodeo`s or `ThreadedRodeo`s that use our custom key!\nlet mut rodeo: Rodeo\u003cNicheKey\u003e = Rodeo::new();\nlet key = rodeo.get_or_intern(\"It works!\");\nassert_eq!(rodeo.resolve(\u0026key), \"It works!\");\n```\n\n## Example: Creation using `FromIterator`\n\n```rust\nuse lasso::Rodeo;\nuse core::iter::FromIterator;\n\n// Works for both `Rodeo` and `ThreadedRodeo`\nlet rodeo = Rodeo::from_iter(vec![\n    \"one string\",\n    \"two string\",\n    \"red string\",\n    \"blue string\",\n]);\n\nassert!(rodeo.contains(\"one string\"));\nassert!(rodeo.contains(\"two string\"));\nassert!(rodeo.contains(\"red string\"));\nassert!(rodeo.contains(\"blue string\"));\n```\n\n```rust\nuse lasso::Rodeo;\nuse core::iter::FromIterator;\n\n// Works for both `Rodeo` and `ThreadedRodeo`\nlet rodeo: Rodeo = vec![\"one string\", \"two string\", \"red string\", \"blue string\"]\n    .into_iter()\n    .collect();\n\nassert!(rodeo.contains(\"one string\"));\nassert!(rodeo.contains(\"two string\"));\nassert!(rodeo.contains(\"red string\"));\nassert!(rodeo.contains(\"blue string\"));\n```\n\n## Benchmarks\n\nBenchmarks were gathered with [Criterion.rs](https://github.com/bheisler/criterion.rs)  \nOS: Windows 10  \nCPU: Ryzen 9 3900X at 3800Mhz  \nRAM: 3200Mhz  \nRustc: Stable 1.44.1  \n\n### Rodeo\n\n#### STD RandomState\n\n| Method                       |   Time    |  Throughput  |\n| :--------------------------- | :-------: | :----------: |\n| `resolve`                    | 1.9251 μs | 13.285 GiB/s |\n| `try_resolve`                | 1.9214 μs | 13.311 GiB/s |\n| `resolve_unchecked`          | 1.4356 μs | 17.816 GiB/s |\n| `get_or_intern` (empty)      | 60.350 μs | 433.96 MiB/s |\n| `get_or_intern` (filled)     | 57.415 μs | 456.15 MiB/s |\n| `try_get_or_intern` (empty)  | 58.978 μs | 444.06 MiB/s |\n| `try_get_or_intern (filled)` | 57.421 μs | 456.10 MiB/s |\n| `get` (empty)                | 37.288 μs | 702.37 MiB/s |\n| `get` (filled)               | 55.095 μs | 475.36 MiB/s |\n\n#### AHash\n\n| Method                       |   Time    |  Throughput  |\n| :--------------------------- | :-------: | :----------: |\n| `try_resolve`                | 1.9282 μs | 13.264 GiB/s |\n| `resolve`                    | 1.9404 μs | 13.181 GiB/s |\n| `resolve_unchecked`          | 1.4328 μs | 17.851 GiB/s |\n| `get_or_intern` (empty)      | 38.029 μs | 688.68 MiB/s |\n| `get_or_intern` (filled)     | 33.650 μs | 778.30 MiB/s |\n| `try_get_or_intern` (empty)  | 39.392 μs | 664.84 MiB/s |\n| `try_get_or_intern (filled)` | 33.435 μs | 783.31 MiB/s |\n| `get` (empty)                | 12.565 μs | 2.0356 GiB/s |\n| `get` (filled)               | 26.545 μs | 986.61 MiB/s |\n\n#### FXHash\n\n| Method                       |   Time    |  Throughput  |\n| :--------------------------- | :-------: | :----------: |\n| `resolve`                    | 1.9014 μs | 13.451 GiB/s |\n| `try_resolve`                | 1.9278 μs | 13.267 GiB/s |\n| `resolve_unchecked`          | 1.4449 μs | 17.701 GiB/s |\n| `get_or_intern` (empty)      | 32.523 μs | 805.27 MiB/s |\n| `get_or_intern` (filled)     | 30.281 μs | 864.88 MiB/s |\n| `try_get_or_intern` (empty)  | 31.630 μs | 828.00 MiB/s |\n| `try_get_or_intern (filled)` | 31.002 μs | 844.78 MiB/s |\n| `get` (empty)                | 12.699 μs | 2.0141 GiB/s |\n| `get` (filled)               | 29.220 μs | 896.28 MiB/s |\n\n\n### ThreadedRodeo\n\n#### STD RandomState\n\n| Method                       | Time (1 Thread) | Throughput (1 Thread) | Time (24 Threads) | Throughput (24 Threads) |\n| :--------------------------- | :-------------: | :-------------------: | :---------------: | :---------------------: |\n| `resolve`                    |    54.336 μs    |     482.00 MiB/s      |     364.27 μs     |      71.897 MiB/s       |\n| `try_resolve`                |    54.582 μs    |     479.82 MiB/s      |     352.67 μs     |      74.261 MiB/s       |\n| `get_or_intern` (empty)      |    266.03 μs    |     98.447 MiB/s      |        N\\A        |           N\\A           |\n| `get_or_intern` (filled)     |    103.04 μs    |     254.17 MiB/s      |     441.42 μs     |      59.331 MiB/s       |\n| `try_get_or_intern` (empty)  |    261.80 μs    |     100.04 MiB/s      |        N\\A        |           N\\A           |\n| `try_get_or_intern` (filled) |    102.61 μs    |     255.25 MiB/s      |     447.42 μs     |      58.535 MiB/s       |\n| `get` (empty)                |    80.346 μs    |     325.96 MiB/s      |        N\\A        |           N\\A           |\n| `get` (filled)               |    92.669 μs    |     282.62 MiB/s      |     439.24 μs     |      59.626 MiB/s       |\n\n#### AHash\n\n| Method                       | Time (1 Thread) | Throughput (1 Thread) | Time (24 Threads) | Throughput (24 Threads) |\n| :--------------------------- | :-------------: | :-------------------: | :---------------: | :---------------------: |\n| `resolve`                    |    22.261 μs    |     1.1489 GiB/s      |     265.46 μs     |      98.658 MiB/s       |\n| `try_resolve`                |    22.378 μs    |     1.1429 GiB/s      |     268.58 μs     |      97.513 MiB/s       |\n| `get_or_intern` (empty)      |    157.86 μs    |     165.91 MiB/s      |        N\\A        |           N\\A           |\n| `get_or_intern` (filled)     |    56.320 μs    |     465.02 MiB/s      |     357.13 μs     |      73.335 MiB/s       |\n| `try_get_or_intern` (empty)  |    161.46 μs    |     162.21 MiB/s      |        N\\A        |           N\\A           |\n| `try_get_or_intern` (filled) |    55.874 μs    |     468.73 MiB/s      |     360.25 μs     |      72.698 MiB/s       |\n| `get` (empty)                |    43.520 μs    |     601.79 MiB/s      |        N\\A        |           N\\A           |\n| `get` (filled)               |    53.720 μs    |     487.52 MiB/s      |     360.66 μs     |      72.616 MiB/s       |\n\n#### FXHash\n\n| Method                       | Time (1 Thread) | Throughput (1 Thread) | Time (24 Threads) | Throughput (24 Threads) |\n| :--------------------------- | :-------------: | :-------------------: | :---------------: | :---------------------: |\n| `try_resolve`                |    17.289 μs    |     1.4794 GiB/s      |     238.29 μs     |      109.91 MiB/s       |\n| `resolve`                    |    19.833 μs    |     1.2896 GiB/s      |     237.05 μs     |      110.48 MiB/s       |\n| `get_or_intern` (empty)      |    130.97 μs    |     199.97 MiB/s      |        N\\A        |           N\\A           |\n| `get_or_intern` (filled)     |    42.630 μs    |     614.35 MiB/s      |     301.60 μs     |      86.837 MiB/s       |\n| `try_get_or_intern` (empty)  |    129.30 μs    |     202.55 MiB/s      |        N\\A        |           N\\A           |\n| `try_get_or_intern` (filled) |    42.508 μs    |     616.12 MiB/s      |     337.29 μs     |      77.648 MiB/s       |\n| `get` (empty)                |    28.001 μs    |     935.30 MiB/s      |        N\\A        |           N\\A           |\n| `get` (filled)               |    37.700 μs    |     694.68 MiB/s      |     292.15 μs     |      89.645 MiB/s       |\n\n### RodeoReader\n\n#### STD RandomState\n\n| Method              | Time (1 Thread) | Throughput (1 Thread) | Time (24 Threads) | Throughput  (24 Threads) |\n| :------------------ | :-------------: | :-------------------: | :---------------: | :----------------------: |\n| `resolve`           |    1.9398 μs    |     13.185 GiB/s      |     4.3153 μs     |       5.9269 GiB/s       |\n| `try_resolve`       |    1.9315 μs    |     13.242 GiB/s      |     4.1956 μs     |       6.0959 GiB/s       |\n| `resolve_unchecked` |    1.4416 μs    |     17.741 GiB/s      |     3.1204 μs     |       8.1964 GiB/s       |\n| `get` (empty)       |    38.886 μs    |     673.50 MiB/s      |        N\\A        |           N\\A            |\n| `get` (filled)      |    56.271 μs    |     465.42 MiB/s      |     105.12 μs     |       249.14 MiB/s       |\n\n#### AHash\n\n| Method              | Time (1 Thread) | Throughput (1 Thread) | Time (24 Threads) | Throughput (24 Threads) |\n| :------------------ | :-------------: | :-------------------: | :---------------: | :---------------------: |\n| `resolve`           |    1.9404 μs    |     13.181 GiB/s      |     4.1881 μs     |      6.1069 GiB/s       |\n| `try_resolve`       |    1.8932 μs    |     13.509 GiB/s      |     4.2410 μs     |      6.0306 GiB/s       |\n| `resolve_unchecked` |    1.4128 μs    |     18.103 GiB/s      |     3.1691 μs     |      8.0703 GiB/s       |\n| `get` (empty)       |    11.952 μs    |     2.1399 GiB/s      |        N\\A        |           N\\A           |\n| `get` (filled)      |    27.093 μs    |     966.65 MiB/s      |     56.269 μs     |      465.44 MiB/s       |\n\n#### FXHash\n\n| Method              | Time (1 Thread) | Throughput (1 Thread) | Time (24 Threads) | Throughput (24 Threads) |\n| :------------------ | :-------------: | :-------------------: | :---------------: | :---------------------: |\n| `resolve`           |    1.8987 μs    |     13.471 GiB/s      |     4.2117 μs     |      6.0727 GiB/s       |\n| `try_resolve`       |    1.9103 μs    |     13.389 GiB/s      |     4.2254 μs     |      6.0529 GiB/s       |\n| `resolve_unchecked` |    1.4469 μs    |     17.677 GiB/s      |     3.0923 μs     |      8.2709 GiB/s       |\n| `get` (empty)       |    12.994 μs    |     1.9682 GiB/s      |        N\\A        |           N\\A           |\n| `get` (filled)      |    29.745 μs    |     880.49 MiB/s      |     52.387 μs     |      499.93 MiB/s       |\n\n### RodeoResolver\n\n| Method              | Time (1 Thread) | Throughput (1 Thread) | Time (24 Threads) | Throughput (24 Threads) |\n| :------------------ | :-------------: | :-------------------: | :---------------: | :---------------------: |\n| `resolve`           |    1.9416 μs    |     13.172 GiB/s      |     3.9114 μs     |      6.5387 GiB/s       |\n| `try_resolve`       |    1.9264 μs    |     13.277 GiB/s      |     3.9289 μs     |      6.5097 GiB/s       |\n| `resolve_unchecked` |    1.6638 μs    |     15.372 GiB/s      |     3.1741 μs     |      8.0578 GiB/s       |\n\n[0]: https://github.com/Kixiron/lasso\n[1]: https://github.com/Kixiron/lasso/workflows/CI/badge.svg\n[2]: https://github.com/Kixiron/lasso/workflows/Security%20Audit/badge.svg\n[3]: https://coveralls.io/repos/github/Kixiron/lasso/badge.svg?branch=master\n[4]: https://coveralls.io/github/Kixiron/lasso?branch=master\n[6]: https://docs.rs/lasso/badge.svg\n[7]: https://docs.rs/lasso\n[8]: https://img.shields.io/crates/v/lasso.svg\n[9]: https://crates.io/crates/lasso\n[key]: crate::Key\n[`Rodeo`]: crate::Rodeo\n[`ThreadedRodeo`]: crate::ThreadedRodeo\n[`RodeoResolver`]: crate::RodeoResolver\n[`RodeoReader`]: crate::RodeoReader\n[`hashbrown`]: https://crates.io/crates/hashbrown\n[`ahash`]: https://crates.io/crates/ahash\n[`raw_entry` api]: https://github.com/rust-lang/rust/issues/56167\n[niches]: https://rust-lang.github.io/unsafe-code-guidelines/glossary.html#niche\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkixiron%2Flasso","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkixiron%2Flasso","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkixiron%2Flasso/lists"}