{"id":42789569,"url":"https://github.com/vinhowe/piston","last_synced_at":"2026-01-29T23:34:29.119Z","repository":{"id":272730278,"uuid":"917419838","full_name":"vinhowe/piston","owner":"vinhowe","description":"Train small sequence models in your browser with WebGPU.","archived":false,"fork":false,"pushed_at":"2025-11-29T03:01:42.000Z","size":26057,"stargazers_count":15,"open_issues_count":4,"forks_count":1,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-11-30T11:56:26.150Z","etag":null,"topics":["deep-learning","transformer","webgpu"],"latest_commit_sha":null,"homepage":"https://sequence.toys","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/vinhowe.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-01-16T00:17:15.000Z","updated_at":"2025-11-27T21:01:25.000Z","dependencies_parsed_at":"2025-03-08T01:19:01.277Z","dependency_job_id":"06a0e446-9046-4624-9afd-c60675e55ca5","html_url":"https://github.com/vinhowe/piston","commit_stats":null,"previous_names":["vinhowe/ratchet-backward","vinhowe/toy-transformer","vinhowe/piston"],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/vinhowe/piston","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vinhowe%2Fpiston","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vinhowe%2Fpiston/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vinhowe%2Fpiston/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vinhowe%2Fpiston/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/vinhowe","download_url":"https://codeload.github.com/vinhowe/piston/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vinhowe%2Fpiston/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28890412,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-29T21:06:44.224Z","status":"ssl_error","status_checked_at":"2026-01-29T21:06:42.160Z","response_time":59,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","transformer","webgpu"],"created_at":"2026-01-29T23:34:28.249Z","updated_at":"2026-01-29T23:34:29.112Z","avatar_url":"https://github.com/vinhowe.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n\u003ch1\u003eSequence Toy + Piston\u003c/h1\u003e\n\u003ca href=\"https://sequence.toys\"\u003e\n\u003cimg width=\"700px\" src=\".github/media/sequence-toy-hero.webp\"\u003e\n\u003c/a\u003e\n\u003cp\u003e\u003ca href=\"https://sequence.toys\"\u003eweb playground\u003c/a\u003e \u0026bull; \u003ca href=\"https://x.com/vinhowe/status/1991859939799912561\"\u003etweet thread\u003c/a\u003e \u0026bull; \u003ca href=\"https://vin.how/blog/train-a-language-model-in-your-browser\"\u003eblog post\u003c/a\u003e \u003c/p\u003e\n\u003cp align=\"center\"\u003e\nTrain language models in your browser with WebGPU-powered autodiff.\n\u003c/p\u003e\n\u003cbr\u003e\n\u003c/div\u003e\n\nSequence Toy is a web playground for training sequence models (Transformers, LSTMs, GRUs, and vanilla RNNs) using Piston, a proof-of-concept WebGPU automatic differentiation library. This repository houses these related projects.\n\n## Attribution\n\n- Piston is a fork of [Ratchet](https://github.com/huggingface/ratchet), hacked and butchered to add automatic differentiation. I picked Ratchet because it is simple enough to reason about, but it thoughtfully supports WebGPU via [wgpu](https://github.com/gfx-rs/wgpu).\n- My implementation of backprop borrows heavily from [Candle](https://github.com/huggingface/candle).\n- The lazy execution model is an implementation of [LazyTensor](https://arxiv.org/abs/2102.13267), and borrows from [torch/csrc/lazy](https://github.com/pytorch/pytorch/tree/main/torch/csrc/lazy).\n- I used [Keller Jordan](https://x.com/kellerjordan0)'s [Modded-NanoGPT](https://github.com/KellerJordan/modded-nanogpt) as a reference for his Muon optimizer.\n- My BPE tokenizer implementation is simplified from [transformers.js](https://github.com/huggingface/transformers.js), which in turn mirrors [transformers](https://github.com/huggingface/transformers).\n- My GPT-2 model implementation was originally based on [minGPT](https://github.com/karpathy/minGPT).\n- I adapted dataset preprocessing code from [llm.c](https://github.com/karpathy/llm.c).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvinhowe%2Fpiston","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvinhowe%2Fpiston","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvinhowe%2Fpiston/lists"}