{"id":31392159,"url":"https://github.com/StargazerX0/ScaleKV","last_synced_at":"2025-09-29T03:02:10.986Z","repository":{"id":295718450,"uuid":"990442862","full_name":"StargazerX0/ScaleKV","owner":"StargazerX0","description":"ScaleKV: Memory-Efficient Visual Autoregressive Modeling with Scale-Aware KV Cache Compression","archived":false,"fork":false,"pushed_at":"2025-05-27T02:23:28.000Z","size":3649,"stargazers_count":15,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-05-27T03:30:04.527Z","etag":null,"topics":["auto-regressive-model","efficient-image-generation","model-acceleration","transformers"],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/2505.19602","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/StargazerX0.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-05-26T06:01:22.000Z","updated_at":"2025-05-27T02:23:31.000Z","dependencies_parsed_at":"2025-05-27T03:40:57.468Z","dependency_job_id":null,"html_url":"https://github.com/StargazerX0/ScaleKV","commit_stats":null,"previous_names":["stargazerx0/scalekv"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/StargazerX0/ScaleKV","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/StargazerX0%2FScaleKV","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/StargazerX0%2FScaleKV/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/StargazerX0%2FScaleKV/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/StargazerX0%2FScaleKV/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/StargazerX0","download_url":"https://codeload.github.com/StargazerX0/ScaleKV/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/StargazerX0%2FScaleKV/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":277458690,"owners_count":25821322,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-29T02:00:09.175Z","response_time":84,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["auto-regressive-model","efficient-image-generation","model-acceleration","transformers"],"created_at":"2025-09-29T03:01:46.001Z","updated_at":"2025-09-29T03:02:10.981Z","avatar_url":"https://github.com/StargazerX0.png","language":"Python","funding_links":[],"categories":["VAR model"],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n\u003ch1\u003eScaleKV: Memory-Efficient Visual Autoregressive Modeling with Scale-Aware KV Cache Compression\u003c/h1\u003e\n\n  \u003cdiv align=\"center\"\u003e\n  \u003ca href=\"https://opensource.org/license/mit-0\"\u003e\n    \u003cimg alt=\"MIT\" src=\"https://img.shields.io/badge/License-MIT-4E94CE.svg\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://arxiv.org/abs/2505.19602\"\u003e\n    \u003cimg src=\"https://img.shields.io/badge/Paper-Arxiv-darkred.svg\" alt=\"Paper\"\u003e\n  \u003c/a\u003e\n\u003c/div\u003e\n\u003c/div\u003e\n\n\u003e **Memory-Efficient Visual Autoregressive Modeling with Scale-Aware KV Cache Compression**   \n\u003e [Kunjun Li](https://kunjun-li.github.io/), [Zigeng Chen](https://github.com/czg1225), [Cheng-Yen Yang](https://yangchris11.github.io/), [Jenq-Neng Hwang](https://people.ece.uw.edu/hwang/)   \n\u003e [University of Washington](https://www.washington.edu/)，[National University of Singapore](https://nus.edu.sg/)\n\n\u003c!-- ![figure](assets/intro.png) --\u003e\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"assets/teaser.png\" width=\"100%\" \u003e\u003c/img\u003e\n  \u003cbr\u003e\n\u003c/div\u003e\n\u003cbr\u003e\n\n\n## 💡 Introduction\nWe propose Scale-Aware KV Cache (ScaleKV), a novel KV Cache compression framework tailored for VAR’s next-scale prediction paradigm. ScaleKV leverages on two critical observations: varying cache demands across transformer layers and distinct attention patterns at different scales. Based on these insights, we categorizes transformer layers into two functional groups termed drafters and refiners, implementing adaptive cache management strategies based on these roles and optimize multi-scale inference by identifying each layer's function at every scale, enabling adaptive cache allocation that aligns with specific computational demands of each layer. On Infinity-8B, it achieves 10x memory reduction from 85 GB to 8.5 GB with negligible quality degradation (GenEval score remains at 0.79 and DPG score marginally decreases from 86.61 to 86.49).\n\n\u003c!-- ![figure](assets/intro.png) --\u003e\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"assets/overview.png\" width=\"100%\" \u003e\u003c/img\u003e\n  \u003cimg src=\"assets/method.png\" width=\"100%\" \u003e\u003c/img\u003e\n  \u003cbr\u003e\n\u003c/div\u003e\n\u003cbr\u003e\n\n\n## 🔥Updates\n* 🔥 **May 26, 2025**: Our paper is available now!\n* 🔥 **May 25, 2025**: Code repo is released! Arxiv paper will come soon!\n\n## 🔧  Installation:\n### Reequirements\n```bash\npip install -r requirements.txt\n```\n\n### Model Checkpoints\nDownload google flan-t5-xl:\n```bash\npip install -U huggingface_hub\nhuggingface-cli download google/flan-t5-xl --local-dir ./weights/flan-t5-xl\n```\n\nDownload Infinity-2B:\n```bash\nhuggingface-cli download FoundationVision/Infinity --include \"infinity_2b_reg.pth\" --local-dir ./weights/\nhuggingface-cli download FoundationVision/Infinity --include \"infinity_vae_d32reg.pth\" --local-dir ./weights/\n```\n\nDownload Infinity-8B:\n```bash\nhuggingface-cli download FoundationVision/Infinity --include \"infinity_8b_weights/**\" --local-dir ./weights/infinity_8b_weights\nhuggingface-cli download FoundationVision/Infinity --include \"infinity_vae_d56_f8_14_patchify.pth\" --local-dir ./weights/\n\n```\n\n## ⚡ Quick Start:\n\nSample images with ScaleKV-Compressed Infinity-8B (10% KV Cache):\n```python\npython infer_8B.py\n```\n\nSample images with ScaleKV-Compressed Infinity-2B (10% KV Cache):\n```python\npython infer_2B.py\n```\n\n## ⚡ Sample \u0026 Evaluations\n### Sampling 5000 images from COCO-2017 captions with Infinity-8B.\n\n```python\ntorchrun --nproc_per_node=$N_GPUS scripts/sample_8b.py\n```\n\nSample images with ScaleKV compressed Infinity-8B (10% KV Cache):\n```python\ntorchrun --nproc_per_node=$N_GPUS scripts/sample_kv_8b.py\n```\n\nAfter you sample all the images, you can calculate PSNR, LPIPS and FID with:\n```python\npython scripts/compute_metrics.py --input_root0 samples/gt_8b --input_root1 samples/scalekv_8b\n```\n\n### Sampling 5000 images from COCO captions with Infinity-2B.\n```python\ntorchrun --nproc_per_node=$N_GPUS scripts/sample_2b.py\n```\n\n```python\ntorchrun --nproc_per_node=$N_GPUS scripts/sample_kv_2b.py\n```\n\n```python\npython scripts/compute_metrics.py --input_root0 samples/gt_2b --input_root1 samples/scalekv_2b\n```\n\n## 📚 Key Results\n\u003cdiv align=\"center\"\u003e\n\u003cimg src=\"assets/picture.png\" width=\"100%\"\u003e\n\u003c/div\u003e\n\n\u003cdiv align=\"center\"\u003e\n\u003cimg src=\"assets/exp.png\" width=\"100%\"\u003e\n\u003c/div\u003e\n\n\n\u003cdiv align=\"center\"\u003e\n\u003cimg src=\"assets/mem.png\" width=\"100%\"\u003e\n\u003c/div\u003e\n\n## Acknowlegdement\nThanks to [Infinity](https://github.com/FoundationVision/Infinity) for their wonderful work and codebase!\n\n\n## Citation\nIf our research assists your work, please give us a star ⭐ or cite us using:\n```\n@article{li2025scalekv,\n  title={Memory-Efficient Visual Autoregressive Modeling with Scale-Aware KV Cache Compression},\n  author={Li, Kunjun and Chen, Zigeng and Yang, Cheng-Yen and Hwang, Jenq-Neng},\n  journal={arXiv preprint arXiv:2505.19602},\n  year={2025}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FStargazerX0%2FScaleKV","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FStargazerX0%2FScaleKV","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FStargazerX0%2FScaleKV/lists"}