{"id":48512954,"url":"https://github.com/iro96/turboquant-h","last_synced_at":"2026-04-07T18:01:11.281Z","repository":{"id":349651137,"uuid":"1202926909","full_name":"Iro96/TurboQuant-H","owner":"Iro96","description":"A more deep research about TurboQuant algorithms","archived":false,"fork":false,"pushed_at":"2026-04-06T23:05:36.000Z","size":41,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-04-07T01:10:11.323Z","etag":null,"topics":["algorithms","llm","llm-quantization","machine-learning","turboquant"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Iro96.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-04-06T14:44:52.000Z","updated_at":"2026-04-06T23:05:40.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/Iro96/TurboQuant-H","commit_stats":null,"previous_names":["iro96/turboquant-h"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/Iro96/TurboQuant-H","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Iro96%2FTurboQuant-H","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Iro96%2FTurboQuant-H/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Iro96%2FTurboQuant-H/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Iro96%2FTurboQuant-H/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Iro96","download_url":"https://codeload.github.com/Iro96/TurboQuant-H/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Iro96%2FTurboQuant-H/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31522574,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-07T16:28:08.000Z","status":"ssl_error","status_checked_at":"2026-04-07T16:28:06.951Z","response_time":105,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["algorithms","llm","llm-quantization","machine-learning","turboquant"],"created_at":"2026-04-07T18:01:04.066Z","updated_at":"2026-04-07T18:01:11.275Z","avatar_url":"https://github.com/Iro96.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# TurboQuant-H\n\nTurboQuant-H is a research-oriented benchmark harness for experimenting with hierarchical KV-cache compression. A stronger hybrid design that keeps the mathematical core of TurboQuant. This algorithm uses data-oblivious, two-stage vector quantization scheme for large language model key–value (KV) cache compression, combining random rotation plus scalar quantization with a 1-bit Quantized Johnson–Lindenstrauss (QJL) residual to preserve both mean-squared error (MSE) and inner-product fidelity.\n\n## New Contributor Docs\n\nStart with the guides in [`docs/`](./docs/README.md):\n\n- [`docs/project-map.md`](./docs/project-map.md)\n- [`docs/runtime-flow.md`](./docs/runtime-flow.md)\n- [`docs/compression-guide.md`](./docs/compression-guide.md)\n\n## Project Layout\n\n```text\nTurboQuant-H/\n|-- README.md\n|-- pyproject.toml\n|-- turboquant_h_smollm_benchmark.py\n|-- docs/\n|   |-- compression-guide.md\n|   |-- project-map.md\n|   |-- runtime-flow.md\n|   |-- TurboQuant-H_paper_draft.md\n|   `-- README.md\n|-- src/\n|   `-- turboquant_h/\n|       |-- __init__.py\n|       |-- benchmark.py\n|       |-- cli.py\n|       |-- config.py\n|       |-- reporting.py\n|       `-- compression/\n|           |-- __init__.py\n|           |-- attention.py\n|           |-- cache.py\n|           |-- common.py\n|           |-- correction.py\n|           |-- packing.py\n|           |-- quantization.py\n|           `-- rotation.py\n`-- tests/\n    |-- __init__.py\n    |-- test_cache.py\n    |-- test_benchmark.py\n    |-- test_config.py\n    `-- test_packing.py\n```\n\n## Quick Start\n\nRun the original script name:\n\n```bash\npython turboquant_h_smollm_benchmark.py --prompt \"Explain KV cache compression.\"\n```\n\nOr use the package CLI with `src` on `PYTHONPATH`:\n\n```bash\n$env:PYTHONPATH = \"src\"\npython -m turboquant_h.cli --prompt \"Explain KV cache compression.\"\n```\n\nIf you install the project:\n\n```bash\npip install -e .\nturboquant-h-benchmark --prompt \"Explain KV cache compression.\"\n```\n\n## Example Options\n\n```bash\npython turboquant_h_smollm_benchmark.py \\\n  --model HuggingFaceTB/SmolLM-135M-Instruct \\\n  --quant_bits_old 2 \\\n  --rotation_mode hadamard \\\n  --correction_type low_rank \\\n  --prompt \"Explain why KV cache compression matters in one paragraph.\"\n```\n\n## Module Guide\n\n- `config.py`: benchmark configuration, validation, and result dataclasses.\n- `benchmark.py`: model loading, generation loop, and end-to-end benchmark execution.\n- `reporting.py`: terminal-friendly result formatting.\n- `compression/packing.py`: bit-packing and unpacking utilities.\n- `compression/rotation.py`: Hadamard-based rotation helpers.\n- `compression/quantization.py`: tensor quantization and dequantization.\n- `compression/correction.py`: low-rank and QJL-style residual correction logic.\n- `compression/cache.py`: compressed KV-cache block, segment, and cache classes.\n- `compression/attention.py`: compressed attention execution and model patching.\n\n## Validation\n\nThe included tests are lightweight and focus on deterministic utilities:\n\n```bash\npython -m unittest discover -s tests\n```\n\n## Notes\n\n- This is still a research harness, not a production CUDA implementation.\n- The benchmark currently targets transformer attention modules with Llama-style projections.\n- `TurboQuant-H_paper_draft.md` remains as supporting design context.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Firo96%2Fturboquant-h","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Firo96%2Fturboquant-h","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Firo96%2Fturboquant-h/lists"}