{"id":28393044,"url":"https://github.com/harfbuzz/packtab","last_synced_at":"2026-03-01T21:16:02.560Z","repository":{"id":50553497,"uuid":"182556927","full_name":"harfbuzz/packtab","owner":"harfbuzz","description":"Unicode (and other integer) table packer","archived":false,"fork":false,"pushed_at":"2026-02-18T03:49:44.000Z","size":226,"stargazers_count":10,"open_issues_count":0,"forks_count":2,"subscribers_count":7,"default_branch":"master","last_synced_at":"2026-02-18T08:53:41.804Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/harfbuzz.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2019-04-21T16:29:17.000Z","updated_at":"2026-02-18T03:49:45.000Z","dependencies_parsed_at":"2025-04-19T06:34:09.607Z","dependency_job_id":"40b13689-3370-47c4-b85e-b8e7945dc25e","html_url":"https://github.com/harfbuzz/packtab","commit_stats":null,"previous_names":[],"tags_count":6,"template":false,"template_full_name":null,"purl":"pkg:github/harfbuzz/packtab","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/harfbuzz%2Fpacktab","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/harfbuzz%2Fpacktab/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/harfbuzz%2Fpacktab/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/harfbuzz%2Fpacktab/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/harfbuzz","download_url":"https://codeload.github.com/harfbuzz/packtab/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/harfbuzz%2Fpacktab/sbom","scorecard":{"id":455742,"data":{"date":"2025-08-11","repo":{"name":"github.com/harfbuzz/packtab","commit":"9cd71648a75ab7a3aff97c2d4c2def045446c4db"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":3.2,"checks":[{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Code-Review","score":0,"reason":"Found 0/30 approved changesets -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Maintained","score":1,"reason":"2 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 1","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Pinned-Dependencies","score":-1,"reason":"no dependencies found","details":null,"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"Token-Permissions","score":-1,"reason":"No tokens found","details":null,"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"SAST","score":0,"reason":"no SAST tool detected","details":["Warn: no pull requests merged into dev branch"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}},{"name":"Dangerous-Workflow","score":-1,"reason":"no workflows found","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"Vulnerabilities","score":10,"reason":"0 existing vulnerabilities detected","details":null,"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: Apache License 2.0: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'master'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}}]},"last_synced_at":"2025-08-19T09:35:07.126Z","repository_id":50553497,"created_at":"2025-08-19T09:35:07.126Z","updated_at":"2025-08-19T09:35:07.126Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29984715,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-01T21:06:37.093Z","status":"ssl_error","status_checked_at":"2026-03-01T21:05:45.052Z","response_time":124,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-05-31T15:10:33.322Z","updated_at":"2026-03-01T21:16:02.542Z","avatar_url":"https://github.com/harfbuzz.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# packTab\n\nPack static integer tables into compact multi-level lookup tables\nto save space.  Generates C or Rust code.\n\n## Installation\n\n```\npip install packtab\n```\n\n## Usage\n\n### Command line\n\n```bash\n# Generate C lookup code\npython -m packTab 1 2 3 4\n\n# Generate Rust lookup code\npython -m packTab --rust 1 2 3 4\n\n# Generate Rust with unsafe array access\npython -m packTab --rust --unsafe 1 2 3 4\n\n# Analyze compression without generating code\npython -m packTab --analyze 1 2 3 4\n\n# Read data from stdin\nseq 0 255 | python -m packTab --rust\n\n# Tune compression (higher = smaller, slower)\necho \"1 2 3 4\" | python -m packTab --compression 5\n```\n\n### As a library\n\n```python\nfrom packTab import pack_table, Code, languages\n\ndata = [0, 1, 2, 3, 0, 1, 2, 3]\nsolution = pack_table(data, default=0, compression=1)\n\ncode = Code(\"mytable\")\nsolution.genCode(code, \"lookup\", language=\"c\", private=False)\ncode.print_code(language=\"c\")\n```\n\nThe `pack_table` function accepts:\n- A list of integers, or a dict mapping integer keys to values\n- `default`: value for missing keys (default `0`)\n- `compression`: tunes the size-vs-speed tradeoff (default `1`)\n- `mapping`: optional mapping between string values and integers\n\n### Rust with unsafe access\n\n```python\nfrom packTab import pack_table, Code, languageClasses\n\ndata = list(range(256)) * 4\nsolution = pack_table(data, default=0)\n\nlang = languageClasses[\"rust\"](unsafe_array_access=True)\ncode = Code(\"mytable\")\nsolution.genCode(code, \"lookup\", language=lang, private=False)\ncode.print_code(language=lang)\n```\n\n## Examples\n\n### Simple linear data\n\nFor data that's already sequential, the identity optimization kicks in:\n\n```bash\n$ python -m packTab --analyze $(seq 0 255)\nOriginal data: 256 values, range [0..255]\nOriginal storage: 8 bits/value, 256 bytes total\n\nFound 1 Pareto-optimal solutions:\n  0 lookups, 5 extra ops, 0 bytes\n  Compression ratio: ∞ (computed inline, no storage)\n```\n\nGenerated code just returns the input: `return u \u003c 256 ? u : 0`\n\n### Sparse data\n\nFor sparse lookup tables with many repeated values:\n\n```python\nfrom packTab import pack_table, Code\n\n# Sparse Unicode-like table: mostly 0, some special values\ndata = [0] * 100\ndata[10] = 5\ndata[20] = 10\ndata[50] = 15\ndata[80] = 20\n\nsolution = pack_table(data, default=0)\ncode = Code(\"sparse\")\nsolution.genCode(code, \"lookup\", language=\"c\")\ncode.print_code(language=\"c\")\n```\n\nThe packer will use multi-level tables and sub-byte packing to minimize storage.\n\n### Generated code structure\n\nFor small datasets, values are inlined as bit-packed constants:\n\n```c\n// Input: [1, 2, 3, 4]\nextern inline uint8_t data_get (unsigned u)\n{\n  return u\u003c4 ? (uint8_t)(u)+(uint8_t)(((15u\u003e\u003e(u))\u00261)) : 0;\n}\n// Uses identity optimization: data[i] = i + 1, stored as 0b1111\n```\n\nFor larger datasets, generates lookup tables:\n\n```rust\n// Input: 256 values with pattern\nstatic data_u8: [u8; 256] = [ ... ];\n\n#[inline]\npub(crate) fn data_get (u: usize) -\u003e u8\n{\n  if u\u003c256 { data_u8[u] as u8 } else { 0 }\n}\n```\n\n## How it works\n\nThe algorithm builds multi-level lookup tables using dynamic programming\nto find optimal split points.  Values that fit in fewer bits get packed\ninto sub-byte storage (1, 2, or 4 bits per item).  An outer layer applies\narithmetic reductions (GCD factoring, bias subtraction) before splitting.\n\nThe solver produces a set of Pareto-optimal solutions trading off table\nsize against lookup speed, and `pick_solution` selects the best one based\non the `compression` parameter.\n\n## Testing\n\n```bash\npytest\n```\n\n## History\n\nI first wrote something like this back in 2001 when I needed it in FriBidi:\n\n  https://github.com/fribidi/fribidi/blob/master/gen.tab/packtab.c\n\nIn 2019 I wanted to use that to produce more compact Unicode data tables\nfor HarfBuzz, but for convenience I wanted to use it from Python.  While\nI considered wrapping the C code in a module, it occurred to me that I\ncan rewrite it in pure Python in a much cleaner way.  That code remains\na stain on my resume in terms of readability (or lack thereof!). :D\n\nThis Python version builds on the same ideas, but is different from the\nC version in two major ways:\n\n1. Whereas the C version uses backtracking to find best split opportunities,\n   I found that the same can be achieved using dynamic-programming.  So the\n   Python version implements the DP approach, which is much faster.\n\n2. The C version does not try packing multiple items into a single byte.\n   The Python version does.  Ie. if items fit, they might get packed into\n   1, 2, or 4 bits per item.\n\nThere's also a bunch of other optimizations, which make (eventually, when\ncomplete) the Python version more generic and usable for a wider variety\nof data tables.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fharfbuzz%2Fpacktab","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fharfbuzz%2Fpacktab","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fharfbuzz%2Fpacktab/lists"}