{"id":50398581,"url":"https://github.com/dsnsgithub/mango-llm","last_synced_at":"2026-05-30T22:01:42.625Z","repository":{"id":356852176,"uuid":"1216415426","full_name":"dsnsgithub/mango-llm","owner":"dsnsgithub","description":"Goal: Create a LLM that can produce English text using training (initially from stories, later web text).","archived":false,"fork":false,"pushed_at":"2026-05-17T06:36:34.000Z","size":148585,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-17T06:43:30.074Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dsnsgithub.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-04-20T22:07:58.000Z","updated_at":"2026-05-17T06:36:37.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/dsnsgithub/mango-llm","commit_stats":null,"previous_names":["dsnsgithub/mango-llm"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/dsnsgithub/mango-llm","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dsnsgithub%2Fmango-llm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dsnsgithub%2Fmango-llm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dsnsgithub%2Fmango-llm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dsnsgithub%2Fmango-llm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dsnsgithub","download_url":"https://codeload.github.com/dsnsgithub/mango-llm/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dsnsgithub%2Fmango-llm/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33711018,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-05-30T02:00:06.278Z","response_time":92,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-05-30T22:01:41.834Z","updated_at":"2026-05-30T22:01:42.620Z","avatar_url":"https://github.com/dsnsgithub.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 🥭 Mango LLM\n\u003e **M**y **A**nswers **N**eed **G**enuine **O**versight =\u003e **MANGO**\n\n### **Work in progress.**\n\n\nGoal: Create a LLM that can produce coherent English text (initially trained on stories, later web text), in the process learning machine learning concepts.\n\nAs many layers/parts of the LLM as possible are built from basic components and individual matrix multiplications (instead of using pre-made components).\n\nAI was used to assist/help me understand LLM concepts, but almost all of the code in this repo was handwritten and loosely based off of GPT-2 and the original Attention is All You Need paper.\n\n## Requirements\n- [uv](https://docs.astral.sh/uv/)\n\nuv can automatically install the required Python version (even if you don't have Python installed), along with any required packages.\n\nOn Linux and Windows, this LLM uses the CUDA accelerator. On macOS, it uses the default MPS (Metal Performance Shaders) accelerator for Apple Silicon if possible.\n\n## Run\n\nTo train, download the required datasets from Kaggle:\n\nDownload `train.csv` and `validation.csv` and create/place the files in `./dataset/TinyStories`.\n\nLink: https://www.kaggle.com/datasets/thedevastator/tinystories-narrative-classification/data\n\n---\n\nWith [uv](https://docs.astral.sh/uv/) (recommended given the project config):\n\n```bash\nuv sync\nuv run src/train.py\nuv run src/run.py\n```\n\n## Layout\n\n| Path | Role |\n|------|------|\n| `src/` | LLM code, main files being `src/run.py` and `src/train.py` |\n| `dist/` | Contains trained LLM that can be run with `src/run.py` |\n| `old/` | Contains original LLM, useful for beginners trying to understand the basics |\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdsnsgithub%2Fmango-llm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdsnsgithub%2Fmango-llm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdsnsgithub%2Fmango-llm/lists"}