{"id":39159212,"url":"https://github.com/toy-gpt/train-300-context-2","last_synced_at":"2026-04-03T05:01:22.543Z","repository":{"id":333035891,"uuid":"1135916166","full_name":"toy-gpt/train-300-context-2","owner":"toy-gpt","description":"Demonstrates, at very small scale, how a language model is trained (two-context model).","archived":false,"fork":false,"pushed_at":"2026-01-17T01:17:03.000Z","size":723,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-01-18T06:47:59.868Z","etag":null,"topics":["computer-science-education","dual-context","educational","inspectable-models","language-model","machine-learning-education","mkdocs","next-token-prediction","probabilistic-model","pyright","python","reproducibility","softmax-regression","src-layout","teaching","toy-gpt","toy-model","two-context-model","uv"],"latest_commit_sha":null,"homepage":"https://toy-gpt.github.io/train-300-context-2/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/toy-gpt.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-01-16T19:21:29.000Z","updated_at":"2026-01-17T01:19:18.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/toy-gpt/train-300-context-2","commit_stats":null,"previous_names":["toy-gpt/train-300-context-2"],"tags_count":4,"template":false,"template_full_name":null,"purl":"pkg:github/toy-gpt/train-300-context-2","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/toy-gpt%2Ftrain-300-context-2","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/toy-gpt%2Ftrain-300-context-2/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/toy-gpt%2Ftrain-300-context-2/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/toy-gpt%2Ftrain-300-context-2/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/toy-gpt","download_url":"https://codeload.github.com/toy-gpt/train-300-context-2/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/toy-gpt%2Ftrain-300-context-2/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31335168,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-03T04:42:29.251Z","status":"ssl_error","status_checked_at":"2026-04-03T04:42:12.667Z","response_time":107,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["computer-science-education","dual-context","educational","inspectable-models","language-model","machine-learning-education","mkdocs","next-token-prediction","probabilistic-model","pyright","python","reproducibility","softmax-regression","src-layout","teaching","toy-gpt","toy-model","two-context-model","uv"],"created_at":"2026-01-17T22:01:33.007Z","updated_at":"2026-04-03T05:01:22.529Z","avatar_url":"https://github.com/toy-gpt.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Toy-GPT: train-300-context-2\n\n[![PyPI version](https://img.shields.io/pypi/v/toy-gpt-train-300-context-2)](https://pypi.org/project/toy-gpt-train-300-context-2/)\n[![Latest Release](https://img.shields.io/github/v/release/toy-gpt/train-300-context-2)](https://github.com/toy-gpt/train-300-context-2/releases)\n[![Docs](https://img.shields.io/badge/docs-live-blue)](https://toy-gpt.github.io/train-300-context-2/)\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/license/MIT)\n[![CI](https://github.com/toy-gpt/train-300-context-2/actions/workflows/ci-python-mkdocs-shared.yml/badge.svg?branch=main)](https://github.com/toy-gpt/train-300-context-2/actions/workflows/ci-python-mkdocs-shared.yml)\n[![Deploy-Docs](https://github.com/toy-gpt/train-300-context-2/actions/workflows/deploy-mkdocs-shared.yml/badge.svg?branch=main)](https://github.com/toy-gpt/train-300-context-2/actions/workflows/deploy-mkdocs-shared.yml)\n[![Check Links](https://github.com/toy-gpt/train-300-context-2/actions/workflows/links.yml/badge.svg)](https://github.com/toy-gpt/train-300-context-2/actions/workflows/links.yml)\n[![Dependabot](https://img.shields.io/badge/Dependabot-enabled-brightgreen.svg)](https://github.com/toy-gpt/train-300-context-2/security)\n\n\u003e Demonstrates, at very small scale, how a language model is trained.\n\nThis repository is part of a series of toy training repositories plus a companion client repository:\n\n- [**Training repositories**](https://github.com/toy-gpt) produce pretrained artifacts (vocabulary, weights, metadata).\n- A [**web app**](https://toy-gpt.github.io/toy-gpt-chat/) loads the artifacts and provides an interactive prompt.\n\n## Contents\n\n- a small, declared text corpus\n- a tokenizer and vocabulary builder\n- a simple next-token prediction model\n- a repeatable training loop\n- committed, inspectable artifacts for downstream use\n\n## Scope\n\nThis is:\n\n- an intentionally inspectable training pipeline\n- a next-token predictor trained on an explicit corpus\n\nThis is not:\n\n- a production system\n- a full Transformer implementation\n- a chat interface\n- a claim of semantic understanding\n\n## Outputs\n\nThis repository produces and commits pretrained artifacts under `artifacts/`.\n\nTraining logs and evidence are written under `outputs/`\n(for example, `outputs/train_log.csv`).\n\n## Quick start\n\nSee `SETUP.md` for full setup and workflow instructions.\n\nRun the full training script:\n\n```shell\nuv run python src/toy_gpt_train/d_train.py\n```\n\nRun individually:\n\n- a/b/c are demos (can be run alone if desired)\n- d_train produces artifacts\n- e_infer consumes artifacts\n\n```shell\nuv run python src/toy_gpt_train/a_tokenizer.py\nuv run python src/toy_gpt_train/b_vocab.py\nuv run python src/toy_gpt_train/c_model.py\nuv run python src/toy_gpt_train/d_train.py\nuv run python src/toy_gpt_train/e_infer.py\n```\n\n## Provenance and Purpose\n\nThe primary corpus used for training is declared in `SE_MANIFEST.toml`.\n\nThis repository commits pretrained artifacts so the client can run\nwithout retraining.\n\n## Annotations\n\n[ANNOTATIONS.md](./ANNOTATIONS.md) - REQ/WHY/OBS annotations used\n\n## Citation\n\n[CITATION.cff](./CITATION.cff)\n\n## License\n\n[MIT](./LICENSE)\n\n## SE Manifest\n\n[SE_MANIFEST.toml](./SE_MANIFEST.toml) - project intent, scope, and role\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftoy-gpt%2Ftrain-300-context-2","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftoy-gpt%2Ftrain-300-context-2","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftoy-gpt%2Ftrain-300-context-2/lists"}