{"id":13458116,"url":"https://github.com/bitsandbytes-foundation/bitsandbytes","last_synced_at":"2026-04-15T19:00:38.915Z","repository":{"id":44988942,"uuid":"373674258","full_name":"bitsandbytes-foundation/bitsandbytes","owner":"bitsandbytes-foundation","description":"Accessible large language models via k-bit quantization for PyTorch.","archived":false,"fork":false,"pushed_at":"2026-04-14T03:03:46.000Z","size":5672,"stargazers_count":8113,"open_issues_count":40,"forks_count":840,"subscribers_count":51,"default_branch":"main","last_synced_at":"2026-04-14T04:27:04.562Z","etag":null,"topics":["llm","machine-learning","pytorch","qlora","quantization"],"latest_commit_sha":null,"homepage":"https://huggingface.co/docs/bitsandbytes/main/en/index","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bitsandbytes-foundation.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":"NOTICE.md","maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null},"funding":{"open_collective":"bitsandbytes"}},"created_at":"2021-06-04T00:10:34.000Z","updated_at":"2026-04-14T02:55:27.000Z","dependencies_parsed_at":"2026-02-16T21:00:54.460Z","dependency_job_id":null,"html_url":"https://github.com/bitsandbytes-foundation/bitsandbytes","commit_stats":{"total_commits":668,"total_committers":92,"mean_commits":7.260869565217392,"dds":0.6586826347305389,"last_synced_commit":"e4674531dd54874c0abbc786ad5635c92c34dc3e"},"previous_names":["bitsandbytes-foundation/bitsandbytes","timdettmers/bitsandbytes"],"tags_count":37,"template":false,"template_full_name":null,"purl":"pkg:github/bitsandbytes-foundation/bitsandbytes","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bitsandbytes-foundation%2Fbitsandbytes","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bitsandbytes-foundation%2Fbitsandbytes/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bitsandbytes-foundation%2Fbitsandbytes/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bitsandbytes-foundation%2Fbitsandbytes/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bitsandbytes-foundation","download_url":"https://codeload.github.com/bitsandbytes-foundation/bitsandbytes/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bitsandbytes-foundation%2Fbitsandbytes/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31855432,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-15T15:24:51.572Z","status":"ssl_error","status_checked_at":"2026-04-15T15:24:39.138Z","response_time":63,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["llm","machine-learning","pytorch","qlora","quantization"],"created_at":"2024-07-31T09:00:44.953Z","updated_at":"2026-04-15T19:00:38.908Z","avatar_url":"https://github.com/bitsandbytes-foundation.png","language":"Python","funding_links":["https://opencollective.com/bitsandbytes"],"categories":["Core Model/Training Techniques","Python","Optimizations and fine-tuning","Performance","Computation and Communication Optimisation","📋 Contents"],"sub_categories":["ML Compiler","⚡ 3. Inference Engines \u0026 Serving"],"readme":"\u003cp align=\"center\"\u003e\u003cimg src=\"https://avatars.githubusercontent.com/u/175231607?s=200\u0026v=4\" alt=\"\"\u003e\u003c/p\u003e\n\u003ch1 align=\"center\"\u003ebitsandbytes\u003c/h1\u003e\n\u003cp align=\"center\"\u003e\n    \u003ca href=\"https://github.com/bitsandbytes-foundation/bitsandbytes/main/LICENSE\"\u003e\u003cimg alt=\"License\" src=\"https://img.shields.io/github/license/bitsandbytes-foundation/bitsandbytes.svg?color=blue\"\u003e\u003c/a\u003e\n    \u003ca href=\"https://pepy.tech/project/bitsandbytes\"\u003e\u003cimg alt=\"Downloads\" src=\"https://static.pepy.tech/badge/bitsandbytes/month\"\u003e\u003c/a\u003e\n    \u003ca href=\"https://github.com/bitsandbytes-foundation/bitsandbytes/actions/workflows/tests-nightly.yml\"\u003e\u003cimg alt=\"Nightly Unit Tests\" src=\"https://img.shields.io/github/actions/workflow/status/bitsandbytes-foundation/bitsandbytes/tests-nightly.yml?logo=github\u0026label=Nightly%20Tests\"\u003e\u003c/a\u003e\n    \u003ca href=\"https://github.com/bitsandbytes-foundation/bitsandbytes/releases\"\u003e\u003cimg alt=\"GitHub Release\" src=\"https://img.shields.io/github/v/release/bitsandbytes-foundation/bitsandbytes\"\u003e\u003c/a\u003e\n    \u003ca href=\"https://pypi.org/project/bitsandbytes/\"\u003e\u003cimg alt=\"PyPI - Python Version\" src=\"https://img.shields.io/pypi/pyversions/bitsandbytes\"\u003e\u003c/a\u003e\n\u003c/p\u003e\n\n`bitsandbytes` enables accessible large language models via k-bit quantization for PyTorch. We provide three main features for dramatically reducing memory consumption for inference and training:\n\n* 8-bit optimizers uses block-wise quantization to maintain 32-bit performance at a small fraction of the memory cost.\n* LLM.int8() or 8-bit quantization enables large language model inference with only half the required memory and without any performance degradation. This method is based on vector-wise quantization to quantize most features to 8-bits and separately treating outliers with 16-bit matrix multiplication.\n* QLoRA or 4-bit quantization enables large language model training with several memory-saving techniques that don't compromise performance. This method quantizes a model to 4-bits and inserts a small set of trainable low-rank adaptation (LoRA) weights to allow training.\n\nThe library includes quantization primitives for 8-bit \u0026 4-bit operations, through `bitsandbytes.nn.Linear8bitLt` and `bitsandbytes.nn.Linear4bit` and 8-bit optimizers through `bitsandbytes.optim` module.\n\n## System Requirements\nbitsandbytes has the following minimum requirements for all platforms:\n\n* Python 3.10+\n* [PyTorch](https://pytorch.org/get-started/locally/) 2.3+\n  * _Note: While we aim to provide wide backwards compatibility, we recommend using the latest version of PyTorch for the best experience._\n\n#### Accelerator support:\n\n\u003csmall\u003eNote: this table reflects the status of the current development branch. For the latest stable release, see the\n[document in the 0.49.2 tag](https://github.com/bitsandbytes-foundation/bitsandbytes/blob/0.49.2/README.md#accelerator-support).\n\u003c/small\u003e\n\n##### Legend:\n🚧 = In Development,\n〰️ = Partially Supported,\n✅ = Supported,\n🐢 = Slow Implementation Supported,\n❌ = Not Supported\n\n\u003ctable\u003e\n  \u003cthead\u003e\n    \u003ctr\u003e\n      \u003cth\u003ePlatform\u003c/th\u003e\n      \u003cth\u003eAccelerator\u003c/th\u003e\n      \u003cth\u003eHardware Requirements\u003c/th\u003e\n      \u003cth\u003eLLM.int8()\u003c/th\u003e\n      \u003cth\u003eQLoRA 4-bit\u003c/th\u003e\n      \u003cth\u003e8-bit Optimizers\u003c/th\u003e\n    \u003c/tr\u003e\n  \u003c/thead\u003e\n  \u003ctbody\u003e\n    \u003ctr\u003e\n      \u003ctd colspan=\"6\"\u003e🐧 \u003cstrong\u003eLinux, glibc \u003e= 2.24\u003c/strong\u003e\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003ctd align=\"right\"\u003ex86-64\u003c/td\u003e\n      \u003ctd\u003e◻️ CPU\u003c/td\u003e\n      \u003ctd\u003eMinimum: AVX2\u003cbr\u003eOptimized: AVX512F, AVX512BF16\u003c/td\u003e\n      \u003ctd\u003e✅\u003c/td\u003e\n      \u003ctd\u003e✅\u003c/td\u003e\n      \u003ctd\u003e✅\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003ctd\u003e\u003c/td\u003e\n      \u003ctd\u003e🟩 NVIDIA GPU \u003cbr\u003e\u003ccode\u003ecuda\u003c/code\u003e\u003c/td\u003e\n      \u003ctd\u003eSM60+ minimum\u003cbr\u003eSM75+ recommended\u003c/td\u003e\n      \u003ctd\u003e✅\u003c/td\u003e\n      \u003ctd\u003e✅\u003c/td\u003e\n      \u003ctd\u003e✅\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003ctd\u003e\u003c/td\u003e\n      \u003ctd\u003e🟥 AMD GPU \u003cbr\u003e\u003ccode\u003ecuda\u003c/code\u003e\u003c/td\u003e\n      \u003ctd\u003e\n        CDNA: gfx90a, gfx942, gfx950\u003cbr\u003e\n        RDNA: gfx1100, gfx1101, gfx1102, gfx1103, gfx1150, gfx1151, gfx1152, gfx1153, gfx1200, gfx1201\n      \u003c/td\u003e\n      \u003ctd\u003e✅\u003c/td\u003e\n      \u003ctd\u003e✅\u003c/td\u003e\n      \u003ctd\u003e✅\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003ctd\u003e\u003c/td\u003e\n      \u003ctd\u003e🟦 Intel GPU \u003cbr\u003e\u003ccode\u003expu\u003c/code\u003e\u003c/td\u003e\n      \u003ctd\u003e\n        Data Center GPU Max Series\u003cbr\u003e\n        Arc A-Series (Alchemist)\u003cbr\u003e\n        Arc B-Series (Battlemage)\n      \u003c/td\u003e\n      \u003ctd\u003e✅\u003c/td\u003e\n      \u003ctd\u003e✅\u003c/td\u003e\n      \u003ctd\u003e✅\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003ctd\u003e\u003c/td\u003e\n      \u003ctd\u003e🟪 Intel Gaudi \u003cbr\u003e\u003ccode\u003ehpu\u003c/code\u003e\u003c/td\u003e\n      \u003ctd\u003eGaudi2, Gaudi3\u003c/td\u003e\n      \u003ctd\u003e✅\u003c/td\u003e\n      \u003ctd\u003e〰️\u003c/td\u003e\n      \u003ctd\u003e❌\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003ctd align=\"right\"\u003eaarch64\u003c/td\u003e\n      \u003ctd\u003e◻️ CPU\u003c/td\u003e\n      \u003ctd\u003e\u003c/td\u003e\n      \u003ctd\u003e✅\u003c/td\u003e\n      \u003ctd\u003e✅\u003c/td\u003e\n      \u003ctd\u003e❌\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003ctd\u003e\u003c/td\u003e\n      \u003ctd\u003e🟩 NVIDIA GPU \u003cbr\u003e\u003ccode\u003ecuda\u003c/code\u003e\u003c/td\u003e\n      \u003ctd\u003eSM75+\u003c/td\u003e\n      \u003ctd\u003e✅\u003c/td\u003e\n      \u003ctd\u003e✅\u003c/td\u003e\n      \u003ctd\u003e✅\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003ctd colspan=\"6\"\u003e🪟 \u003cstrong\u003eWindows 11 / Windows Server 2022+\u003c/strong\u003e\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003ctd align=\"right\"\u003ex86-64\u003c/td\u003e\n      \u003ctd\u003e◻️ CPU\u003c/td\u003e\n      \u003ctd\u003eAVX2\u003c/td\u003e\n      \u003ctd\u003e✅\u003c/td\u003e\n      \u003ctd\u003e✅\u003c/td\u003e\n      \u003ctd\u003e✅\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003ctd\u003e\u003c/td\u003e\n      \u003ctd\u003e🟩 NVIDIA GPU \u003cbr\u003e\u003ccode\u003ecuda\u003c/code\u003e\u003c/td\u003e\n      \u003ctd\u003eSM60+ minimum\u003cbr\u003eSM75+ recommended\u003c/td\u003e\n      \u003ctd\u003e✅\u003c/td\u003e\n      \u003ctd\u003e✅\u003c/td\u003e\n      \u003ctd\u003e✅\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003ctd\u003e\u003c/td\u003e\n      \u003ctd\u003e🟥 AMD GPU \u003cbr\u003e\u003ccode\u003ecuda\u003c/code\u003e\u003c/td\u003e\n      \u003ctd\u003e\n        RDNA: gfx1100, gfx1101, gfx1102,\u003cbr\u003e\n        gfx1150, gfx1151,\u003cbr\u003e\n        gfx1200, gfx1201\n      \u003c/td\u003e\n      \u003ctd\u003e✅\u003c/td\u003e\n      \u003ctd\u003e✅\u003c/td\u003e\n      \u003ctd\u003e✅\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003ctd\u003e\u003c/td\u003e\n      \u003ctd\u003e🟦 Intel GPU \u003cbr\u003e\u003ccode\u003expu\u003c/code\u003e\u003c/td\u003e\n      \u003ctd\u003e\n        Arc A-Series (Alchemist) \u003cbr\u003e\n        Arc B-Series (Battlemage)\n      \u003c/td\u003e\n      \u003ctd\u003e✅\u003c/td\u003e\n      \u003ctd\u003e✅\u003c/td\u003e\n      \u003ctd\u003e✅\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003ctd colspan=\"6\"\u003e🍎 \u003cstrong\u003emacOS 14+\u003c/strong\u003e\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003ctd align=\"right\"\u003earm64\u003c/td\u003e\n      \u003ctd\u003e◻️ CPU\u003c/td\u003e\n      \u003ctd\u003eApple M1+\u003c/td\u003e\n      \u003ctd\u003e✅\u003c/td\u003e\n      \u003ctd\u003e✅\u003c/td\u003e\n      \u003ctd\u003e❌\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003ctd\u003e\u003c/td\u003e\n      \u003ctd\u003e⬜ Metal \u003cbr\u003e\u003ccode\u003emps\u003c/code\u003e\u003c/td\u003e\n      \u003ctd\u003eApple M1+\u003c/td\u003e\n      \u003ctd\u003e🐢\u003c/td\u003e\n      \u003ctd\u003e🐢\u003c/td\u003e\n      \u003ctd\u003e❌\u003c/td\u003e\n  \u003c/tbody\u003e\n\u003c/table\u003e\n\n## :book: Documentation\n* [Official Documentation](https://huggingface.co/docs/bitsandbytes/main)\n* 🤗 [Transformers](https://huggingface.co/docs/transformers/quantization/bitsandbytes)\n* 🤗 [Diffusers](https://huggingface.co/docs/diffusers/quantization/bitsandbytes)\n* 🤗 [PEFT](https://huggingface.co/docs/peft/developer_guides/quantization#quantize-a-model)\n\n## :heart: Sponsors\nThe continued maintenance and development of `bitsandbytes` is made possible thanks to the generous support of our sponsors. Their contributions help ensure that we can keep improving the project and delivering valuable updates to the community.\n\n\u003ckbd\u003e\u003ca href=\"https://hf.co\" target=\"_blank\"\u003e\u003cimg width=\"100\" src=\"https://huggingface.co/datasets/huggingface/brand-assets/resolve/main/hf-logo.svg\" alt=\"Hugging Face\"\u003e\u003c/a\u003e\u003c/kbd\u003e\n\u0026nbsp;\n\u003ckbd\u003e\u003ca href=\"https://intel.com\" target=\"_blank\"\u003e\u003cimg width=\"100\" src=\"https://avatars.githubusercontent.com/u/17888862?s=100\u0026v=4\" alt=\"Intel\"\u003e\u003c/a\u003e\u003c/kbd\u003e\n\n## License\n`bitsandbytes` is MIT licensed.\n\n## How to cite us\nIf you found this library useful, please consider citing our work:\n\n### QLoRA\n\n```bibtex\n@article{dettmers2023qlora,\n  title={Qlora: Efficient finetuning of quantized llms},\n  author={Dettmers, Tim and Pagnoni, Artidoro and Holtzman, Ari and Zettlemoyer, Luke},\n  journal={arXiv preprint arXiv:2305.14314},\n  year={2023}\n}\n```\n\n### LLM.int8()\n\n```bibtex\n@article{dettmers2022llmint8,\n  title={LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale},\n  author={Dettmers, Tim and Lewis, Mike and Belkada, Younes and Zettlemoyer, Luke},\n  journal={arXiv preprint arXiv:2208.07339},\n  year={2022}\n}\n```\n\n### 8-bit Optimizers\n\n```bibtex\n@article{dettmers2022optimizers,\n  title={8-bit Optimizers via Block-wise Quantization},\n  author={Dettmers, Tim and Lewis, Mike and Shleifer, Sam and Zettlemoyer, Luke},\n  journal={9th International Conference on Learning Representations, ICLR},\n  year={2022}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbitsandbytes-foundation%2Fbitsandbytes","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbitsandbytes-foundation%2Fbitsandbytes","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbitsandbytes-foundation%2Fbitsandbytes/lists"}