{"id":21802098,"url":"https://github.com/tickbh/algorithm-rs","last_synced_at":"2025-03-21T07:12:40.160Z","repository":{"id":240625341,"uuid":"803088985","full_name":"tickbh/algorithm-rs","owner":"tickbh","description":" about algorithm data structure, now has lru/lru-k/lfu/slab/rbtree, 关于算法常用的数据结构 ","archived":false,"fork":false,"pushed_at":"2024-05-30T08:55:10.000Z","size":107,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2024-05-30T09:04:39.255Z","etag":null,"topics":["lfu","lru","lru-cache","lru-k","lrucache","rbtree","slab","slab-allocator"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tickbh.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-05-20T03:37:36.000Z","updated_at":"2024-05-30T09:04:40.812Z","dependencies_parsed_at":"2024-05-30T09:14:44.197Z","dependency_job_id":null,"html_url":"https://github.com/tickbh/algorithm-rs","commit_stats":null,"previous_names":["tickbh/algorithm","tickbh/algorithm-rs"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tickbh%2Falgorithm-rs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tickbh%2Falgorithm-rs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tickbh%2Falgorithm-rs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tickbh%2Falgorithm-rs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tickbh","download_url":"https://codeload.github.com/tickbh/algorithm-rs/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244752355,"owners_count":20504255,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["lfu","lru","lru-cache","lru-k","lrucache","rbtree","slab","slab-allocator"],"created_at":"2024-11-27T11:22:54.504Z","updated_at":"2025-03-21T07:12:40.140Z","avatar_url":"https://github.com/tickbh.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"## algorithm/ 算法结构相关\n[![crates.io](https://img.shields.io/crates/v/algorithm.svg)](https://crates.io/crates/algorithm)\n[![rustc 1.70.0](https://img.shields.io/badge/rust-1.70%2B-orange.svg)](https://img.shields.io/badge/rust-1.70%2B-orange.svg)\n[![Released API docs](https://docs.rs/algorithm/badge.svg)](https://docs.rs/algorithm)\n\n将提供一些常用的数据结构以供使用。目前提供的数据结构\n* **LruCache** 最近未使用缓存，可用feature启用ttl\n* **LruKCache** 最近未使用缓存, K次分类列表，可用feature启用ttl\n* **LfuCache** 按缓存访问次数做排序,优先淘汰访问最少次数的，可用feature启用ttl\n* **ArcCache** Adaptive Replacement Cache，自适应缓存替换算法，可用feature启用ttl\n* **Slab** 仿linux中的Slab结构,对大对象做到初始化缓存使用\n* **BitMap** 位图, 按位做标记的图\n* **RoaringBitMap** 位图, 因为位图占用的内存太大, 对于稀疏位图会更小内存\n* **TimerWheel** 计时器轮, 模仿时钟的高效定时器组件\n* **CircularBuffer** 环形Buffer组件, 适用于内存限定较严格的, 设置不超过缓存值的环形结构\n* **RBTree** 红黑村, 高效的排序树, 可用于做定时器组件\n* **FixedVec** 模拟指针的可变长数组\n\n# lru 全称是Least Recently Used，即最近最久未使用的意思。\n每次元素访问将其更新到列表的最前，时间复杂度为O(1)。当达到容量限制时将淘汰双向列表中的链尾数据\n```rust\nuse algorithm::LruCache;\nfn main() {\n    let mut lru = LruCache::new(3);\n    lru.insert(\"now\", \"ok\");\n    lru.insert(\"hello\", \"algorithm\");\n    lru.insert(\"this\", \"lru\");\n    lru.insert(\"auth\", \"tickbh\");\n    assert!(lru.len() == 3);\n    assert_eq!(lru.get(\"hello\"), Some(\u0026\"algorithm\"));\n    assert_eq!(lru.get(\"this\"), Some(\u0026\"lru\"));\n    assert_eq!(lru.get(\"now\"), None);\n}\n```\n# lru-k\n将访问次数达到k的目标值放进到优先队列，lru-k的主要目的是为了解决LRU算法“缓存污染”的问题，其核心思想是将“最近使用过1次”的判断标准扩展为“最近使用过K次”。\n相比LRU，LRU-K需要多维护一个队列，用于记录所有缓存数据被访问的历史。只有当数据的访问次数达到K次的时候，才将数据放入缓存。当需要淘汰数据时，LRU-K会淘汰第K次访问时间距当前时间最大的数据。\n\n```rust\nuse algorithm::LruKCache;\nfn main() {\n    let mut lru = LruKCache::with_times(3, 3);\n    lru.insert(\"this\", \"lru\");\n    for _ in 0..3 {\n        let _ = lru.get(\"this\");\n    }\n    lru.insert(\"hello\", \"algorithm\");\n    lru.insert(\"auth\", \"tickbh\");\n    assert!(lru.len() == 3);\n    lru.insert(\"auth1\", \"tickbh\");\n    assert_eq!(lru.get(\"this\"), Some(\u0026\"lru\"));\n    assert_eq!(lru.get(\"hello\"), None);\n    assert!(lru.len() == 3);\n}\n```\n\n# lfu (least frequently used)最近频次使用\n每个元素在被访问或者更新的时候将其访问次数+1，当元素满时将优先淘汰掉访问次数最少的数据。\n```rust\n\nuse algorithm::LfuCache;\nfn main() {\n    let mut lru = LfuCache::new(3);\n    lru.insert(\"hello\", \"algorithm\");\n    lru.insert(\"this\", \"lru\");\n    lru.set_reduce_count(100);\n    assert!(lru.get_visit(\u0026\"hello\") == Some(5));\n    assert!(lru.get_visit(\u0026\"this\") == Some(5));\n    for _ in 0..98 {\n        let _ = lru.get(\"this\");\n    }\n    assert!(lru.get_visit(\u0026\"this\") == Some(51));\n    assert!(lru.get_visit(\u0026\"hello\") == Some(2));\n    let mut keys = lru.keys();\n    assert!(keys.next()==Some(\u0026\"this\"));\n    assert!(keys.next()==Some(\u0026\"hello\"));\n    assert!(keys.next() == None);\n}\n```\n\n# slab 缓存块组，linux中缓存对象的分配器\n缓存对象需实现Default，将会使对象缓存起来，避免频繁的重复申请释放带来的开销\n\n以下我们以简单的测试来进行对比，algorithm::Slab与slab::Slab与普通的alloc\n\n以下测试场景相对简单，可能对`slab::Slab`较为不公平\n\n```rust\nuse std::{ptr, time::Instant};\n\nuse algorithm::{Reinit, Slab};\n\nconst ARRAY_SIZE: usize = 10240;\nconst NUM: usize = usize::MAX - 99999;\nconst ZERO_ARRAY: [usize; ARRAY_SIZE] = [NUM; ARRAY_SIZE];\nstruct TestStruct {\n    array: [usize; ARRAY_SIZE],\n    size: usize,\n    val: String,\n}\n\nimpl Default for TestStruct {\n    fn default() -\u003e Self {\n        Self { array: [NUM; ARRAY_SIZE], size: 0, val:  \"slab\".to_string(), }\n    }\n}\n\nimpl Reinit for TestStruct {\n    #[inline(always)]\n    fn reinit(\u0026mut self) {\n        self.size = 0;\n        self.val.clear();\n        self.val.push_str(\"slab\");\n        unsafe {\n            ptr::copy_nonoverlapping(\u0026ZERO_ARRAY[0], \u0026mut self.array[0], ARRAY_SIZE);\n        }\n    }\n}\n\nfn main() {\n    let times = 100000;\n    let now = Instant::now();\n    let mut slab = Slab::\u003cTestStruct\u003e::new();\n    let mut sum: usize = 0;\n    for i in 0..times {\n        let (next, test) = slab.get_reinit_next_val();\n        test.array[i % 20] = test.array[i % 20].wrapping_add(i % 1024);\n        sum = sum.wrapping_add(test.array[10] + test.size + test.val.len());\n        slab.remove(next);\n    }\n    println!(\"algorithm: all cost times {}ms, sum = {}\", now.elapsed().as_millis(), sum);\n\n\n    let now = Instant::now();\n    let mut slab = slab::Slab::\u003cTestStruct\u003e::new();\n    let mut sum: usize = 0;\n    for i in 0..times {\n        let next = slab.insert(TestStruct::default());\n        let test = \u0026mut slab[next];\n        test.array[i % 20] = test.array[i % 20].wrapping_add(i % 1024);\n        sum = sum.wrapping_add(test.array[10] + test.size + test.val.len());\n        slab.remove(next);\n    }\n    println!(\"tokio::slab: all cost times {}ms, sum = {}\", now.elapsed().as_millis(), sum);\n\n    let now = Instant::now();\n    let mut sum: usize = 0;\n    for i in 0..times {\n        let mut test = TestStruct::default();\n        test.array[i % 20] = test.array[i % 20].wrapping_add(i % 1024);\n        sum = sum.wrapping_add(test.array[10] + test.size + test.val.len());\n        drop(test);\n    }\n    println!(\"normal alloc: all cost times {}ms, sum = {}\", now.elapsed().as_millis(), sum);\n}\n```\n最终用release命令进行输出测试，结果均为一致\n\n但是耗时algorithm避免了申请创建的开销，耗时相对较短，做的仅仅将对象重新reinit\n\n在此场景中tokio::slab即进行了申请又开销了插入及删除，反而耗时最长\n```console\nalgorithm: all cost times 132ms, sum = 18446744063712505088\ntokio::slab: all cost times 477ms, sum = 18446744063712505088\nnormal alloc: all cost times 337ms, sum = 18446744063712505088\n```\n\n# 计时器轮（TimerWheel），模拟时钟格式组成的高效计时器\n\n1. **环形数据结构**：TimerWheel，即时间轮，是一个环形的数据结构，类似于时钟的面，被等分为多个格子或槽位（slot）。\n\n2. **槽位时间间隔**：每个槽位代表一个固定的时间间隔，例如1毫秒、1秒等。这个时间间隔决定了定时器的精度。\n\n3. **初始化**：在算法开始时，需要初始化时间轮，包括设定时间轮的大小（即槽位的数量）和每个槽位代表的时间间隔。即当插入数据后即不允许修改时轮信息。\n\n```rust\nuse algorithm::TimerWheel;\n\nfn main() {\n    let mut timer = TimerWheel::new();\n    timer.append_timer_wheel(12, 60 * 60, \"HourWheel\");\n    timer.append_timer_wheel(60, 60, \"MinuteWheel\");\n    timer.append_timer_wheel(60, 1, \"SecondWheel\");\n\n    timer.add_timer(30);\n    assert_eq!(timer.get_delay_id(), 30);\n    timer.add_timer(149);\n    assert_eq!(timer.get_delay_id(), 30);\n    let t = timer.add_timer(600);\n    assert_eq!(timer.get_delay_id(), 30);\n    timer.add_timer(1);\n    assert_eq!(timer.get_delay_id(), 1);\n    timer.del_timer(t);\n    timer.add_timer(150);\n    assert_eq!(timer.get_delay_id(), 1);\n\n    let val = timer.update_deltatime(30).unwrap();\n    assert_eq!(val, vec![1, 30]);\n\n    timer.add_timer(2);\n\n    let val = timer.update_deltatime(119).unwrap();\n    assert_eq!(val, vec![2, 149]);\n\n    let val = timer.update_deltatime(1).unwrap();\n    assert_eq!(val, vec![150]);\n    \n    assert!(timer.is_empty());\n}\n```\n\n\n# 添加宏支持, 可快速的缓存函数的结果\n\n\n```rust\nuse algorithm::LruCache;\nuse algorithm_macro::cache;\n\n#[cache(LruCache : LruCache::new(20))]\n#[cache_cfg(ignore_args = call_count)]\n#[cache_cfg(thread)]\nfn fib(x: u64, call_count: \u0026mut u32) -\u003e u64 {\n    *call_count += 1;\n    if x \u003c= 1 {\n        1\n    } else {\n        fib(x - 1, call_count) + fib(x - 2, call_count)\n    }\n}\n```\n如此就可以快速将函数的执行结果进行缓存加速.\n\n## Star History\n\n[![Star History Chart](https://api.star-history.com/svg?repos=tickbh/algorithm-rs\u0026type=Date)](https://star-history.com/#tickbh/algorithm-rs\u0026Date)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftickbh%2Falgorithm-rs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftickbh%2Falgorithm-rs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftickbh%2Falgorithm-rs/lists"}