https://github.com/jakubschwenkbeck/transformer
Rust implementation of the Transformer architecture, designed as a foundation for large language models (LLMs)
https://github.com/jakubschwenkbeck/transformer
Last synced: 2 months ago
JSON representation
Rust implementation of the Transformer architecture, designed as a foundation for large language models (LLMs)
- Host: GitHub
- URL: https://github.com/jakubschwenkbeck/transformer
- Owner: JakubSchwenkbeck
- License: mit
- Created: 2024-12-02T19:15:52.000Z (6 months ago)
- Default Branch: main
- Last Pushed: 2025-02-25T12:55:03.000Z (3 months ago)
- Last Synced: 2025-02-25T13:44:44.659Z (3 months ago)
- Language: Rust
- Homepage:
- Size: 186 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Transfomer
writing a Transformer (base for LLMs) in Rust
## References
- **Ashish Vaswani et al.** (2017). *Attention Is All You Need*. [Read the Paper (PDF)](https://arxiv.org/pdf/1706.03762)
- **Alec Radford et al.** *Language Models are Unsupervised Multitask Learners*. [Read the Paper (PDF)](https://cdn.openai.com/better-language-models/language_models_are_unsupervised_multitask_learners.pdf)
- **Harvard NLP Group.** *The Annotated Transformer*. [Read the Article](https://nlp.seas.harvard.edu/annotated-transformer/)
- **3Blue1Brown** *Deep-Learning Series*. [Introduction to Transformers](https://www.youtube.com/watch?v=wjZofJX0v4M) , [Attention in Transformers](https://www.youtube.com/watch?v=eMlx5fFNoYc)
---
## **License**
This project is licensed under the MIT License. See the [LICENSE](LICENSE) file for details.