{"id":18823602,"url":"https://github.com/yvt/tokenlock","last_synced_at":"2025-04-14T01:26:17.432Z","repository":{"id":47287603,"uuid":"107561169","full_name":"yvt/tokenlock","owner":"yvt","description":"Provides cell types that decouple permissions from data.","archived":false,"fork":false,"pushed_at":"2023-06-27T16:58:45.000Z","size":91,"stargazers_count":9,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-11T16:16:12.546Z","etag":null,"topics":["memory-safety","multithreading","rust"],"latest_commit_sha":null,"homepage":"https://crates.io/crates/tokenlock","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/yvt.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE-APACHE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-10-19T14:59:27.000Z","updated_at":"2024-08-15T20:28:09.000Z","dependencies_parsed_at":"2022-09-26T20:40:44.245Z","dependency_job_id":null,"html_url":"https://github.com/yvt/tokenlock","commit_stats":null,"previous_names":[],"tags_count":11,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yvt%2Ftokenlock","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yvt%2Ftokenlock/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yvt%2Ftokenlock/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yvt%2Ftokenlock/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/yvt","download_url":"https://codeload.github.com/yvt/tokenlock/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248806101,"owners_count":21164451,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["memory-safety","multithreading","rust"],"created_at":"2024-11-08T00:54:09.367Z","updated_at":"2025-04-14T01:26:17.400Z","avatar_url":"https://github.com/yvt.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# tokenlock\n\n[\u003cimg src=\"https://docs.rs/tokenlock/badge.svg\" alt=\"docs.rs\"\u003e](https://docs.rs/tokenlock/)\n\nThis crate provides a cell type, `TokenLock`, which can only be borrowed\nby presenting the correct unforgeable token, thus decoupling permissions\nfrom data.\n\n## Examples\n\n### Basics\n\n```rust\n// Create a token\nlet mut token = IcToken::new();\n\n// Create a keyhole by `token.id()` and use this to create a `TokenLock`.\nlet lock: IcTokenLock\u003ci32\u003e = TokenLock::new(token.id(), 1);\nassert_eq!(*lock.read(\u0026token), 1);\n\n// Unlock the `TokenLock` using the matching token\nlet mut guard = lock.write(\u0026mut token);\nassert_eq!(*guard, 1);\n*guard = 2;\n```\n\nOnly the matching `Token`'s owner can access its contents. `Token`\ncannot be cloned:\n\n```rust\nlet lock = Arc::new(TokenLock::new(token.id(), 1));\n\nlet lock_1 = Arc::clone(\u0026lock);\nstd::thread::spawn(move || {\n    let lock_1 = lock_1;\n    let mut token_1 = token;\n\n    // I have `Token` so I can get a mutable reference to the contents\n    lock_1.write(\u0026mut token_1);\n});\n\n// can't access the contents; I no longer have `Token`\n// lock.write(\u0026mut token);\n```\n\n### Zero-sized tokens\n\nSome token types, such as `BrandedToken` and `SingletonToken`, rely\nsolely on type safety and compile-time checks to guarantee uniqueness and\ndon't use runtime data for identification. As such, the keyholes for such\ntokens can be default-constructed. `TokenLock::wrap` lets you construct a\n`TokenLock` with a default-constructed keyhole.\nOn the other hand, creating such tokens usually has specific requirements.\nSee the following example that uses `with_branded_token`:\n\n```rust\nwith_branded_token(|mut token| {\n    // The lifetime of `token: BrandedToken\u003c'brand\u003e` is bound to\n    // this closure.\n\n    // lock: BrandedTokenLock\u003c'brand, i32\u003e\n    let lock = BrandedTokenLock::wrap(42);\n\n    lock.set(\u0026mut token, 56);\n    assert_eq!(lock.get(\u0026token), 56);\n});\n```\n\n### Lifetimes\n\nThe lifetime of the returned reference is limited by both of the `TokenLock`\nand `Token`.\n\n```rust\nlet mut token = IcToken::new();\nlet lock = TokenLock::new(token.id(), 1);\nlet guard = lock.write(\u0026mut token);\ndrop(lock); // compile error: `guard` cannot outlive `TokenLock`\ndrop(guard);\n```\n\n```rust\ndrop(token); // compile error: `guard` cannot outlive `Token`\ndrop(guard);\n```\n\nIt also prevents from forming a reference to the contained value when\nthere already is a mutable reference to it:\n\n```rust\nlet write_guard = lock.write(\u0026mut token);\nlet read_guard = lock.read(\u0026token); // compile error\ndrop(write_guard);\n```\n\nWhile allowing multiple immutable references:\n\n```rust\nlet read_guard1 = lock.read(\u0026token);\nlet read_guard2 = lock.read(\u0026token);\n```\n\n### Use case: Linked lists\n\nAn operating system kernel often needs to store the global state in a global\nvariable. Linked lists are a common data structure used in a kernel, but\nRust's ownership does not allow forming `'static` references into values\nprotected by a mutex. Common work-arounds, such as smart pointers and index\nreferences, take a heavy toll on a small microcontroller with a single-issue\nin-order pipeline and no hardware multiplier.\n\n```rust\nstruct Process {\n    prev: Option\u003c\u0026 /* what lifetime? */ Process\u003e,\n    next: Option\u003c\u0026 /* what lifetime? */ Process\u003e,\n    state: u8,\n    /* ... */\n}\nstruct SystemState {\n    first_process: Option\u003c\u0026 /* what lifetime? */ Process\u003e,\n    process_pool: [Process; 64],\n}\nstatic STATE: Mutex\u003cSystemState\u003e = todo!();\n```\n\n`tokenlock` makes the `'static` reference approach possible by detaching the\nlock granularity from the protected data's granularity.\n\n```rust\nuse tokenlock::*;\nuse std::cell::Cell;\nstruct Tag;\nimpl_singleton_token_factory!(Tag);\n\ntype KLock\u003cT\u003e = UnsyncSingletonTokenLock\u003cT, Tag\u003e;\ntype KLockToken = UnsyncSingletonToken\u003cTag\u003e;\ntype KLockTokenId = SingletonTokenId\u003cTag\u003e;\n\nstruct Process {\n    prev: KLock\u003cOption\u003c\u0026'static Process\u003e\u003e,\n    next: KLock\u003cOption\u003c\u0026'static Process\u003e\u003e,\n    state: KLock\u003cu8\u003e,\n    /* ... */\n}\nstruct SystemState {\n    first_process: KLock\u003cOption\u003c\u0026'static Process\u003e\u003e,\n    process_pool: [Process; 1],\n}\nstatic STATE: SystemState = SystemState {\n    first_process: KLock::new(KLockTokenId::new(), None),\n    process_pool: [\n        Process {\n            prev: KLock::new(KLockTokenId::new(), None),\n            next: KLock::new(KLockTokenId::new(), None),\n            state: KLock::new(KLockTokenId::new(), 0),\n        }\n    ],\n};\n```\n\n## Cell types\n\nThe `TokenLock` type family is comprised of the following types:\n\n|            | `Sync` tokens    | `!Sync` tokens²        |\n| ---------- | ---------------- | ---------------------- |\n| Unpinned   | `TokenLock`      | `UnsyncTokenLock`      |\n| Pinned¹    | `PinTokenLock`   | `UnsyncPinTokenLock`   |\n\n\u003csub\u003e¹That is, these types respect `T` being `!Unpin` and prevent the\nexposure of `\u0026mut T` through `\u0026Self` or `Pin\u003c\u0026mut Self\u003e`.\u003c/sub\u003e\n\n\u003csub\u003e²`Unsync*TokenLock` require that tokens are `!Sync` (not sharable\nacross threads). In exchange, such cells can be `Sync` even if the contained\ndata is not `Sync`, just like `std::sync::Mutex`.\u003c/sub\u003e\n\n## Token types\n\nThis crate provides the following types implementing `Token`.\n\n(**`std` only**) `IcToken` uses a global counter (with thread-local pools)\nto generate unique 128-bit tokens.\n\n(**`alloc` only**) `RcToken` and `ArcToken` ensure their uniqueness by\nreference-counted memory allocations.\n\n`SingletonToken\u003cTag\u003e` is a singleton token, meaning only one of such\ninstance can exist at any point of time during the program's execution.\n`impl_singleton_token_factory!` instantiates a `static` flag to indicate\n`SingletonToken`'s liveness and allows you to construct it safely by\n`SingletonToken::new`. Alternatively, you can use\n`SingletonToken::new_unchecked`, but this is unsafe if misused.\n\n`BrandedToken\u003c'brand\u003e` implements an extension of [`GhostCell`][1]. It's\ncreated by `with_branded_token` or `with_branded_token_async`, which\nmakes the created token available only within the provided closure or the\ncreated `Future`. This token incurs no runtime cost.\n\n[1]: http://plv.mpi-sws.org/rustbelt/ghostcell/\n\n| Token ID (keyhole)             | Token (key)                       |\n| ------------------------------ | --------------------------------- |\n| `IcTokenId`                    | `IcToken` + `u128` comparison     |\n| `RcTokenId`                    | `RcToken` + `usize` comparison    |\n| `ArcTokenId`                   | `ArcToken` + `usize` comparison   |\n| `SingletonTokenId\u003cTag\u003e`        | `SingletonToken\u003cTag\u003e`             |\n| `BrandedTokenId\u003c'brand\u003e`       | `BrandedToken\u003c'brand\u003e`            |\n\n## `!Sync` tokens\n\n`UnsyncTokenLock` is similar to `TokenLock` but designed for non-`Sync`\ntokens and has relaxed requirements on the inner type for thread safety.\nSpecifically, it can be `Sync` even if the inner type is not `Sync`. This\nallows for storing non-`Sync` cells such as `Cell` and reading and\nwriting them using shared references (all of which must be on the same\nthread because the token is `!Sync`) to the token.\n\n```rust\nuse std::cell::Cell;\nlet mut token = ArcToken::new();\nlet lock = Arc::new(UnsyncTokenLock::new(token.id(), Cell::new(1)));\n\nlet lock_1 = Arc::clone(\u0026lock);\nstd::thread::spawn(move || {\n    // \"Lock\" the token to the current thread using\n    // `ArcToken::borrow_as_unsync`\n    let token = token.borrow_as_unsync();\n\n    // Shared references can alias\n    let (token_1, token_2) = (\u0026token, \u0026token);\n\n    lock_1.read(token_1).set(2);\n    lock_1.read(token_2).set(4);\n});\n```\n\n`!Sync` tokens, of course, cannot be shared between threads:\n\n```rust\nlet mut token = ArcToken::new();\nlet token = token.borrow_as_unsync();\nlet (token_1, token_2) = (\u0026token, \u0026token);\n\n// compile error: `\u0026ArcTokenUnsyncRef` is not `Send` because\n//                `ArcTokenUnsyncRef` is not `Sync`\nstd::thread::spawn(move || {\n    let _ = token_2;\n});\n\nlet _ = token_1;\n```\n\n## Cargo Features\n\n - **`std`** enables the items that depend on `std` or `alloc`.\n - **`alloc`** enables the items that depend on `alloc`.\n - **`unstable`** enables experimental items that are not subject to the\n   semver guarantees.\n - **`const-default_1`** enables the implementation of `ConstDefault` from\n   [`const-default ^1`][].\n\n[`const-default ^1`]: https://crates.io/crates/const-default/1.0.0\n\n## Related Work\n\n - [`ghost-cell`][1] is the official implementation of [`GhostCell`][2] and\n   has been formally proven to be sound. It provides an equivalent of\n   `BrandedTokenLock` with a simpler, more focused interface.\n\n - `SCell` from [`singleton-cell`][3] is a more generalized version of\n   `GhostCell` and accepts any singleton token types, and thus it's more\n   closer to our `TokenLock`. It provides equivalents of our\n   `BrandedToken` and `SingletonToken` out-of-box. It trades away\n   non-ZST token types for an advantage: `SCell\u003cKey, [T]\u003e` can be transposed\n   to `[SCell\u003cKey, T\u003e]`. It uses the [`singleton-trait`][5] crate (which did\n   not exist when `tokenlock::SingletonToken` was added) to mark singleton\n   token types.\n\n - [`qcell`][4] provides multiple cell types with different check\n   mechanisms. `QCell` uses a 32-bit integer as a token identifier, `TCell`\n   and `TLCell` use a marker type, and `LCell` uses lifetime branding.\n\n - `TokenCell` from [`token-cell`][6] is related to our `SingletonToken`,\n   but like `SCell` (but differing slightly), it supports transposition\n   from `\u0026TokenCell\u003cToken, \u0026[T]\u003e` to `\u0026[TokenCell\u003cToken, T\u003e]`. It uses a\n   custom trait to mark singleton token types.\n\n[1]: https://crates.io/crates/ghost-cell\n[2]: http://plv.mpi-sws.org/rustbelt/ghostcell/\n[3]: https://crates.io/crates/singleton-cell\n[4]: https://crates.io/crates/qcell\n[5]: https://crates.io/crates/singleton-trait\n[6]: https://crates.io/crates/token-cell\n\nLicense: MIT/Apache-2.0\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyvt%2Ftokenlock","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fyvt%2Ftokenlock","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyvt%2Ftokenlock/lists"}