https://github.com/betarixm/llm-from-scratch

https://github.com/betarixm/llm-from-scratch

Last synced: about 15 hours ago
JSON representation

• Host: GitHub
• URL: https://github.com/betarixm/llm-from-scratch
• Owner: betarixm
• Created: 2024-07-04T09:10:41.000Z (5 months ago)
• Default Branch: main
• Last Pushed: 2024-08-08T23:58:58.000Z (3 months ago)
• Last Synced: 2024-08-10T01:42:44.462Z (3 months ago)
• Language: Jupyter Notebook
• Size: 200 KB
• Stars: 3
• Watchers: 1
• Forks: 1
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# LLM from Scratch

> Originally from the "Build a Large Language Model (From Scratch)"

This repository contains the code and exercises for building a Large Language Model (LLM) from scratch. Follow along to understand the intricacies of language models and implement your own.

## Requirements

- Python 3.11+

## Environment Setup

### (Optional) Create a Virtual Environment

It is recommended to create a virtual environment to manage dependencies and avoid conflicts with other projects.

```bash

python -m venv .venv

source .venv/bin/activate

```

### Install Dependencies

Install the required dependencies by running:

```bash

pip install -e .

```

Ensure that you are in the correct environment before proceeding.

## Note

Each chapter includes a `tests.py` file which could provide answers and hints for the respective `exercise.ipynb`. Refer to this file if you need guidance or want to verify your solutions.

## Chapters

The project is divided into several chapters, each focusing on different aspects of building a Large Language Model.

### Chapter 2: Working with Text Data

In this chapter, you'll learn how to preprocess and work with text data, a crucial step in building any language model.

[Exercise Link](./llm_from_scratch/chapter_02/exercise.ipynb)