https://github.com/yas-sim/micro-llama

https://github.com/yas-sim/micro-llama

deep-learning jupyter-notebook large-language-models llama llm python text-generation tutorial

Last synced: 3 days ago
JSON representation

• Host: GitHub
• URL: https://github.com/yas-sim/micro-llama
• Owner: yas-sim
• Created: 2024-01-29T23:35:01.000Z (10 months ago)
• Default Branch: main
• Last Pushed: 2024-01-31T00:58:23.000Z (10 months ago)
• Last Synced: 2024-11-08T21:04:12.794Z (11 days ago)
• Topics: deep-learning, jupyter-notebook, large-language-models, llama, llm, python, text-generation, tutorial
• Language: Jupyter Notebook
• Homepage:
• Size: 466 KB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# Micro llama

Working Jupyter notebook and stand alone Python codes from following tech blog article.

----

## Building a Million-Parameter LLM from Scratch Using Python

A Step-by-Step Guide to Replicating LLaMA Architecture

https://levelup.gitconnected.com/building-a-million-parameter-llm-from-scratch-using-python-f612398f06c2

----

## Description

The Llama model in this blog article is tiny and easy to read and understand.

The model uses a character-tokenization strategy, not sub-word-tokenization, to keep the model small enough for education.

Due to the limitations above, the model accuracy is not good. It generates English-like text. However, the accuracy of the model is not the goal of the model and blog article.

I appreciate the well-organized and detailed description of the original blog article.

![micro-llama](./resources/micro-llama.gif)