{"id":13741603,"url":"https://github.com/judofyr/ish","last_synced_at":"2025-10-30T05:10:15.318Z","repository":{"id":52810710,"uuid":"439628461","full_name":"judofyr/ish","owner":"judofyr","description":"Sketches for Zig","archived":false,"fork":false,"pushed_at":"2024-05-19T13:32:49.000Z","size":43,"stargazers_count":26,"open_issues_count":0,"forks_count":2,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-04-04T08:11:10.614Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Zig","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/judofyr.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-12-18T13:56:13.000Z","updated_at":"2025-03-18T17:42:14.000Z","dependencies_parsed_at":"2022-08-23T06:11:16.679Z","dependency_job_id":"330e0388-470e-4398-97c8-1e7048e2f447","html_url":"https://github.com/judofyr/ish","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/judofyr/ish","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/judofyr%2Fish","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/judofyr%2Fish/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/judofyr%2Fish/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/judofyr%2Fish/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/judofyr","download_url":"https://codeload.github.com/judofyr/ish/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/judofyr%2Fish/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":281748716,"owners_count":26554822,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-30T02:00:06.501Z","response_time":61,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-03T04:01:00.814Z","updated_at":"2025-10-30T05:10:15.289Z","avatar_url":"https://github.com/judofyr.png","language":"Zig","funding_links":[],"categories":["Libraries"],"sub_categories":[],"readme":"# ish - Sketches for Zig\n\n**ish** is a library written in [Zig](https://ziglang.org/) for _sketches_.\nA sketch is a data structure which stores a summary of a data set using much less memory than the full data set.\nThis summary is only an approximation, but sometimes they can be surprisingly effective.\n\n## Features\n\n* **Tug of War**: Implementation of [\"The Space Complexity of Approximating the Frequency Moments\"](https://www.sciencedirect.com/science/article/pii/S0022000097915452) used for frequency moments (F^2), multiplicity queries, self-join size, and set difference.\n* **MiniSketch**: Zig wrapper around [MiniSketch](https://github.com/sipa/minisketch) used for compact set reconciliation.\n* **PBS**: Implementation of [\"Space- and Computationally-Efficient Set Reconciliation via Parity Bitmap Sketch\"](https://arxiv.org/pdf/2007.14569.pdf) used for compact set reconciliation.\n  Scales better than MiniSketch for large number of differences (according to the paper).\n\n## Examples\n\n### Tug of War: Set difference\n\nHere's a program which uses 256 bytes to summarize two sets (which actually have thousands of items).\nThen these two sketches are used to estimate how many items are distinct between those two sets.\nThe estimated answer ends up with an error of ~2% which isn't too bad for merely 256 bytes!\n\n```zig\n// Run this with:\n// zig run examples/est-set-diff.zig --main-pkg-path .\n\nvar tow1 = try AutoTugOfWar(i32, i64).init(allocator, 64);\ndefer tow1.deinit(allocator);\n\nvar tow2 = try AutoTugOfWar(i32, i64).init(allocator, 64);\ndefer tow2.deinit(allocator);\n\n// Store [1000, 7000] in the first\nvar size1: usize = 0;\nvar i: i64 = 1000;\nwhile (i \u003c 7000) : (i += 1) {\n\ttow1.addOne(i);\n\tsize1 += @sizeOf(@TypeOf(i));\n}\n\n// Store [3000, 10000] in the second\nvar size2: usize = 0;\ni = 3000;\nwhile (i \u003c 10000) : (i += 1) {\n\ttow2.addOne(i);\n\tsize2 += @sizeOf(@TypeOf(i));\n}\n\n// This means that:\n// - 1000-2999 are only in the first.\n// - 3000-6999 are in both.\n// - 7000-9999 are only in the second.\n// Thus there are unique 6000 items.\n\nstd.debug.print(\"First set would take {} bytes\\n\", .{size1});\nstd.debug.print(\"Second set would take {} bytes\\n\", .{size2});\n\nconst bytes_used = @sizeOf(i32) * tow2.counters.len;\nstd.debug.print(\"Using {} bytes per sketch\\n\", .{bytes_used});\n\n// Calculate the number of different items.\ntow1.removeSketch(\u0026tow2);\nconst diff = tow1.meanOfSquares();\nstd.debug.print(\"Estimated number of different items: {}\\n\", .{diff});\nstd.debug.print(\"Exact number of different items: 6000\\n\", .{});\n```\n\nOutput:\n\n```\nFirst set would take 48000 bytes\nSecond set would take 56000 bytes\nUsing 256 bytes per sketch\nEstimated number of different items: 7316\nExact number of different items: 6000\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjudofyr%2Fish","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjudofyr%2Fish","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjudofyr%2Fish/lists"}