{"id":20467235,"url":"https://github.com/zhangyunhao116/skipset","last_synced_at":"2025-04-09T09:11:11.233Z","repository":{"id":39964196,"uuid":"239768370","full_name":"zhangyunhao116/skipset","owner":"zhangyunhao116","description":"skipset is a high-performance, scalable concurrent sorted set based on skip-list. Up to 15x faster than sync.Map in the typical pattern.","archived":false,"fork":false,"pushed_at":"2023-04-02T04:27:02.000Z","size":200,"stargazers_count":255,"open_issues_count":1,"forks_count":19,"subscribers_count":8,"default_branch":"master","last_synced_at":"2025-04-02T07:11:16.650Z","etag":null,"topics":["concurrent-data-structure","go","set","skiplist","sorted-sets","threadsafe","zset"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/zhangyunhao116.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-02-11T13:28:01.000Z","updated_at":"2025-03-13T13:51:07.000Z","dependencies_parsed_at":"2024-06-18T16:35:54.319Z","dependency_job_id":null,"html_url":"https://github.com/zhangyunhao116/skipset","commit_stats":null,"previous_names":["zyunh/skipset"],"tags_count":24,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zhangyunhao116%2Fskipset","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zhangyunhao116%2Fskipset/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zhangyunhao116%2Fskipset/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zhangyunhao116%2Fskipset/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/zhangyunhao116","download_url":"https://codeload.github.com/zhangyunhao116/skipset/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248008630,"owners_count":21032556,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["concurrent-data-structure","go","set","skiplist","sorted-sets","threadsafe","zset"],"created_at":"2024-11-15T13:27:46.330Z","updated_at":"2025-04-09T09:11:11.198Z","avatar_url":"https://github.com/zhangyunhao116.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://raw.githubusercontent.com/zhangyunhao116/public-data/master/skipset-logo2.png\"/\u003e\n\u003c/p\u003e\n\n## Introduction\n\n\u003e From v0.12.0, the skipset requires Go version \u003e= 1.18, if your Go version is older, use v0.11.0 instead.\n\nSkipset is a high-performance, scalable, concurrent-safe set based on the skip-list. In the typical pattern (100000 operations, 90%CONTAINS 9%ADD 1%REMOVE, 8C16T), the skipset is up to 15x faster than the built-in `sync.Map`.\n\nThe main idea behind the skipset is [A Simple Optimistic Skiplist Algorithm](\u003chttps://people.csail.mit.edu/shanir/publications/LazySkipList.pdf\u003e).\n\nDifferent from the `sync.Map`, the items in the skipset are always sorted, and the `Contains` and `Range` operations are wait-free (A goroutine is guaranteed to complete an operation as long as it keeps taking steps, regardless of the activity of other goroutines).\n\nThe skipset is a set instead of a map, if you need a high-performance full replacement of `sync.Map`, see [skipmap](\u003chttps://github.com/zhangyunhao116/skipmap\u003e).\n\n\n\n## Features\n\n- Scalable, high-performance, concurrent-safe.\n- Wait-free `Contains` and `Range` operations (wait-free algorithms have stronger guarantees than lock-free).\n- Sorted items.\n\n\n\n## When should you use skipset\n\nIn most cases, `skipset` is better than `sync.Map`, especially in these situations:\n\n- **Concurrent calls of multiple operations**. Such as using both `Range` and `Add` at the same time. In this situation, using skipset can greatly improve the performance.\n- **Memory intensive**. The skipset saves at least 50% of the memory in the benchmark.\n\nIf only one goroutine accesses the set for most of the time, such to insert a batch of elements and then uses only `Contains` or `Range`, using built-in map is better.\n\n\n\n## QuickStart\n\nSee [Go doc](https://godoc.org/github.com/zhangyunhao116/skipset) for more information.\n\n```go\npackage main\n\nimport (\n\t\"fmt\"\n\n\t\"github.com/zhangyunhao116/skipset\"\n)\n\nfunc main() {\n\tl := skipset.NewInt()\n\n\tfor _, v := range []int{10, 12, 15} {\n\t\tif l.Add(v) {\n\t\t\tfmt.Println(\"skipset add\", v)\n\t\t}\n\t}\n\n\tif l.Contains(10) {\n\t\tfmt.Println(\"skipset contains 10\")\n\t}\n\n\tl.Range(func(value int) bool {\n\t\tfmt.Println(\"skipset range found \", value)\n\t\treturn true\n\t})\n\n\tl.Remove(15)\n\tfmt.Printf(\"skipset contains %d items\\r\\n\", l.Len())\n}\n\n```\n\n\n\nFrom `v0.12.0` on, you can use a generic version of APIs.\n\n**Note that the generic APIs are always slower than typed APIs, but are more suitable for some scenarios such as functional programming.**\n\n\u003e e.g. `New[int]` is \\~2x slower than `NewInt`, and `NewFunc(func(a, b int) bool { return a \u003c b })` is 1\\~2x slower than `New[int]`.\n\u003e\n\u003e Performance ranking: NewInt \u003e New[Int] \u003e NewFunc(func(a, b int) bool { return a \u003c b })\n\n```go\npackage main\n\nimport (\n\t\"math\"\n\n\t\"github.com/zhangyunhao116/skipset\"\n)\n\nfunc main() {\n\tx1 := skipset.New[int]()\n\tfor _, v := range []int{2, 1, 3} {\n\t\tx1.Add(v)\n\t}\n\tx1.Range(func(value int) bool {\n\t\tprintln(value)\n\t\treturn true\n\t})\n\tx2 := skipset.NewFunc(func(a, b float64) bool {\n\t\treturn a \u003c b || (math.IsNaN(a) \u0026\u0026 !math.IsNaN(b))\n\t})\n\tfor _, v := range []float64{math.NaN(), 3, 1, math.NaN(), 2} {\n\t\tx2.Add(v)\n\t}\n\tx2.Range(func(value float64) bool {\n\t\tprintln(value)\n\t\treturn true\n\t})\n}\n\n```\n\n## Benchmark\n\n\u003e Based on typed APIs.\n\nGo version: go1.16.2 linux/amd64\n\nCPU: AMD 3700x(8C16T), running at 3.6GHz\n\nOS: ubuntu 18.04\n\nMEMORY: 16G x 2 (3200MHz)\n\n![benchmark](https://raw.githubusercontent.com/zhangyunhao116/public-data/master/skipset-benchmark.png)\n\n```\nname                                              time/op\nInt64/Add/skipset-16                              86.6ns ±11%\nInt64/Add/sync.Map-16                              674ns ± 6%\nInt64/Contains50Hits/skipset-16                   9.85ns ±12%\nInt64/Contains50Hits/sync.Map-16                  14.7ns ±30%\nInt64/30Add70Contains/skipset-16                  38.8ns ±18%\nInt64/30Add70Contains/sync.Map-16                  586ns ± 5%\nInt64/1Remove9Add90Contains/skipset-16            24.9ns ±17%\nInt64/1Remove9Add90Contains/sync.Map-16            493ns ± 5%\nInt64/1Range9Remove90Add900Contains/skipset-16    25.9ns ±16%\nInt64/1Range9Remove90Add900Contains/sync.Map-16   1.00µs ±12%\nString/Add/skipset-16                              130ns ±14%\nString/Add/sync.Map-16                             878ns ± 4%\nString/Contains50Hits/skipset-16                  18.3ns ± 9%\nString/Contains50Hits/sync.Map-16                 19.2ns ±18%\nString/30Add70Contains/skipset-16                 61.0ns ±15%\nString/30Add70Contains/sync.Map-16                 756ns ± 7%\nString/1Remove9Add90Contains/skipset-16           31.3ns ±13%\nString/1Remove9Add90Contains/sync.Map-16           614ns ± 6%\nString/1Range9Remove90Add900Contains/skipset-16   36.2ns ±18%\nString/1Range9Remove90Add900Contains/sync.Map-16  1.20µs ±17%\n\nname                                              alloc/op\nInt64/Add/skipset-16                               65.0B ± 0%\nInt64/Add/sync.Map-16                               128B ± 1%\nInt64/Contains50Hits/skipset-16                    0.00B     \nInt64/Contains50Hits/sync.Map-16                   0.00B     \nInt64/30Add70Contains/skipset-16                   19.0B ± 0%\nInt64/30Add70Contains/sync.Map-16                  77.7B ±16%\nInt64/1Remove9Add90Contains/skipset-16             5.00B ± 0%\nInt64/1Remove9Add90Contains/sync.Map-16            57.5B ± 4%\nInt64/1Range9Remove90Add900Contains/skipset-16     5.00B ± 0%\nInt64/1Range9Remove90Add900Contains/sync.Map-16     255B ±22%\nString/Add/skipset-16                              97.0B ± 0%\nString/Add/sync.Map-16                              152B ± 0%\nString/Contains50Hits/skipset-16                   15.0B ± 0%\nString/Contains50Hits/sync.Map-16                  15.0B ± 0%\nString/30Add70Contains/skipset-16                  40.0B ± 0%\nString/30Add70Contains/sync.Map-16                 98.2B ±11%\nString/1Remove9Add90Contains/skipset-16            23.0B ± 0%\nString/1Remove9Add90Contains/sync.Map-16           73.9B ± 4%\nString/1Range9Remove90Add900Contains/skipset-16    23.0B ± 0%\nString/1Range9Remove90Add900Contains/sync.Map-16    261B ±18%\n\nname                                              allocs/op\nInt64/Add/skipset-16                                1.00 ± 0%\nInt64/Add/sync.Map-16                               4.00 ± 0%\nInt64/Contains50Hits/skipset-16                     0.00     \nInt64/Contains50Hits/sync.Map-16                    0.00     \nInt64/30Add70Contains/skipset-16                    0.00     \nInt64/30Add70Contains/sync.Map-16                   1.00 ± 0%\nInt64/1Remove9Add90Contains/skipset-16              0.00     \nInt64/1Remove9Add90Contains/sync.Map-16             0.00     \nInt64/1Range9Remove90Add900Contains/skipset-16      0.00     \nInt64/1Range9Remove90Add900Contains/sync.Map-16     0.00     \nString/Add/skipset-16                               2.00 ± 0%\nString/Add/sync.Map-16                              5.00 ± 0%\nString/Contains50Hits/skipset-16                    1.00 ± 0%\nString/Contains50Hits/sync.Map-16                   1.00 ± 0%\nString/30Add70Contains/skipset-16                   1.00 ± 0%\nString/30Add70Contains/sync.Map-16                  2.00 ± 0%\nString/1Remove9Add90Contains/skipset-16             1.00 ± 0%\nString/1Remove9Add90Contains/sync.Map-16            1.00 ± 0%\nString/1Range9Remove90Add900Contains/skipset-16     1.00 ± 0%\nString/1Range9Remove90Add900Contains/sync.Map-16    1.00 ± 0%\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzhangyunhao116%2Fskipset","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzhangyunhao116%2Fskipset","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzhangyunhao116%2Fskipset/lists"}