https://github.com/draganaleksic99/llm
A ChatGPT-like LLM implementation in PyTorch from scratch.
https://github.com/draganaleksic99/llm
llm python pytorch
Last synced: about 2 months ago
JSON representation
A ChatGPT-like LLM implementation in PyTorch from scratch.
- Host: GitHub
- URL: https://github.com/draganaleksic99/llm
- Owner: DraganAleksic99
- Created: 2024-06-17T23:43:42.000Z (over 1 year ago)
- Default Branch: master
- Last Pushed: 2024-07-21T19:02:59.000Z (about 1 year ago)
- Last Synced: 2025-08-15T20:49:47.333Z (about 2 months ago)
- Topics: llm, python, pytorch
- Language: Python
- Homepage:
- Size: 32.2 KB
- Stars: 1
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# LLM
This repository contains a from-scratch implementation of a GPT-like Large Language Model (LLM) using PyTorch.
The implementation is based on the book [Build a Large Language Model (From Scratch)](https://www.manning.com/books/build-a-large-language-model-from-scratch) by Sebastian Raschka.
For learning purposes only.
## Overview
This project demonstrates the step-by-step process of building a transformer-based language model similar to GPT
(Generative Pre-trained Transformer). It covers key concepts including:
-Transformer achitecture
-Self-attention mechanisms
-Token embedding
-Positional encodings
## Installation
Run the following command to clone the repo:
```
git clone https://github.com/DraganAleksic99/llm.git
```
Install dependencies:
```
cd llm
pip install -r requirements.txt
```