https://github.com/itanmaysharma/bitnet-1.58b
This project showcases the cutting-edge 1-bit LLM architecture as outlined in recent whitepapers, emphasizing efficient training, inference, and fostering open-source collaboration.
https://github.com/itanmaysharma/bitnet-1.58b
1-bit-quantization deep-learning large-language-models llm nlp-machine-learning pytorch
Last synced: 2 months ago
JSON representation
This project showcases the cutting-edge 1-bit LLM architecture as outlined in recent whitepapers, emphasizing efficient training, inference, and fostering open-source collaboration.
- Host: GitHub
- URL: https://github.com/itanmaysharma/bitnet-1.58b
- Owner: itanmaysharma
- Created: 2025-01-03T23:22:50.000Z (5 months ago)
- Default Branch: main
- Last Pushed: 2025-01-03T23:42:32.000Z (5 months ago)
- Last Synced: 2025-01-22T04:15:51.790Z (4 months ago)
- Topics: 1-bit-quantization, deep-learning, large-language-models, llm, nlp-machine-learning, pytorch
- Language: Python
- Homepage:
- Size: 1000 KB
- Stars: 0
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Bitnet 1.58 Implementation
## Original Paper
* [BitNet: Scaling 1-bit Transformers for Large Language Models](https://github.com/puneetkakkar/bitnet-1.58/blob/main/original-whitepaper/2310.11453.pdf)
* [The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits](https://github.com/puneetkakkar/bitnet-1.58/blob/main/original-whitepaper/2402.17764v1.pdf)
* [The Era of 1-bit LLMs: Training Tips, Code and FAQ](https://github.com/puneetkakkar/bitnet-1.58/blob/main/original-whitepaper/The-Era-of-1-bit-LLMs__Training_Tips_Code_FAQ.pdf)
## Collaborators
- [Puneet Kakkar](https://github.com/puneetkakkar)
- [Arush Kumar](https://github.com/arushkumar05)
- [Tanmay Sharma](https://github.com/itanmaysharma)
## Acknowledgements
* [BitNet](https://github.com/kyegomez/BitNet)
* [BitNet Transformers](https://github.com/Beomi/BitNet-Transformers)