{"id":32747699,"url":"https://github.com/lin826/nanogpt-demo","last_synced_at":"2026-05-05T01:39:31.147Z","repository":{"id":322172785,"uuid":"1088431963","full_name":"lin826/nanoGPT-demo","owner":"lin826","description":"Training and finetuning local GPTs.","archived":false,"fork":false,"pushed_at":"2025-11-03T00:28:36.000Z","size":4,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-11-03T02:25:57.819Z","etag":null,"topics":["gpt","nanogpt","numpy","pytorch","tqdm","transformers"],"latest_commit_sha":null,"homepage":"https://github.com/karpathy/nanoGPT","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lin826.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-11-03T00:08:00.000Z","updated_at":"2025-11-03T00:28:40.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/lin826/nanoGPT-demo","commit_stats":null,"previous_names":["lin826/nanogpt-demo"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/lin826/nanoGPT-demo","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lin826%2FnanoGPT-demo","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lin826%2FnanoGPT-demo/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lin826%2FnanoGPT-demo/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lin826%2FnanoGPT-demo/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lin826","download_url":"https://codeload.github.com/lin826/nanoGPT-demo/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lin826%2FnanoGPT-demo/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":282521096,"owners_count":26683141,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-11-03T02:00:05.676Z","response_time":108,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["gpt","nanogpt","numpy","pytorch","tqdm","transformers"],"created_at":"2025-11-03T20:00:58.881Z","updated_at":"2026-05-05T01:39:31.140Z","avatar_url":"https://github.com/lin826.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# nanoGPT Reimplement\n\nReimplement nanoGPT, a local trainable and finetunable agent, into object-oriented style with typing checks.\n\nInstead of using the example `karpathy/chr_rnn/tinyshakespeare/input.txt`, this repo support multiple .txt files from the folder `input/` to be trained with.\n\n## Transformer - model architecture\n\n\u003e Deviation: Unlike the `Add \u0026 Norm` after each layer of the original paper, this repo implements pre-norm formulation.\n\n![Figure from the attention research paper](data/images/attention_research_figure.png)\n\n## Getting Started\n\nCreate virtual environment and install pyhton libraries.\n\n\u003e Hint: On [the official webpage](https://www.anaconda.com/download), scroll down to find Miniconda Installers section!\n\n### Conda for System Dependencies\n\n```sh\n% conda env create -f environment.yml\n```\n\n### UV for no-sys-dependent packages\n\n```sh\n% uv venv\n% uv add -r requirements.txt\n```\n\n### [Optional] Compile as a module\n\n```sh\n% pip install -e .\n```\n\n## Train and Validate\n\n```sh\n% pyhon src/main.py\n```\n\n\u003c!--\n[TODO] Add arg parsing \n\n```sh\nnanogpt train\n    --device=cpu\n    --eval_iters=20\n    --log_interval=1\n    --block_size=64\n    --batch_size=12\n    --n_layer=4\n    --n_head=4\n    --n_embd=128\n    --max_iters=2000\n    --lr_decay_iters=2000\n    --dropout=0.0\n```\n--\u003e\n\n### After optional local compile\n\n```sh\n% nanogpt\n```\n\n## Sample\n\n## Finetune\n\n\u003c!-- TODO: Extract finetuning hyperparameters --\u003e\n\n## Testing\n\n```sh\npytest .\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flin826%2Fnanogpt-demo","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flin826%2Fnanogpt-demo","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flin826%2Fnanogpt-demo/lists"}