{"id":50910362,"url":"https://github.com/roerohan/slate","last_synced_at":"2026-06-16T09:32:51.932Z","repository":{"id":348348528,"uuid":"1194578860","full_name":"roerohan/slate","owner":"roerohan","description":"Training a VLM to generate HTML (Design2Code)","archived":false,"fork":false,"pushed_at":"2026-04-09T07:08:17.000Z","size":80,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-04-09T09:11:57.354Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/roerohan.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":"AGENTS.md","dco":null,"cla":null}},"created_at":"2026-03-28T14:49:49.000Z","updated_at":"2026-04-09T07:08:21.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/roerohan/slate","commit_stats":null,"previous_names":["roerohan/slate"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/roerohan/slate","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/roerohan%2Fslate","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/roerohan%2Fslate/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/roerohan%2Fslate/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/roerohan%2Fslate/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/roerohan","download_url":"https://codeload.github.com/roerohan/slate/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/roerohan%2Fslate/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34400451,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-16T02:00:06.860Z","response_time":126,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-06-16T09:32:50.724Z","updated_at":"2026-06-16T09:32:51.927Z","avatar_url":"https://github.com/roerohan.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Slate\n\nSlate is my first attempt at training a vision-language model (VLM) to generate HTML from a screenshot of a webpage. You give it a picture of a site, it writes the code.\n\n## How it works\n\n1. **Dataset**: ~500 screenshot/HTML pairs from HuggingFace's WebSight dataset. All Tailwind CSS, rendered at 1280x720.\n\n2. **Reward function**: Renders the model's HTML in a browser, compares it to the original screenshot using SSIM (visual similarity), text matching, and color palette comparison. Score from -1 to +1.\n\n3. **Evaluation**: Sends screenshots to the model via [tinker](https://tinker-docs.thinkingmachines.ai/), scores the generated HTML, saves artifacts (generated screenshots, diff images, per-example scores).\n\n4. **Training**: GRPO (Group Relative Policy Optimization). For each screenshot, the model writes multiple HTML attempts. The ones that score above average get reinforced, the ones below average get discouraged. Uses LoRA so we only train a small adapter, not the full model.\n\n## Setup\n\n```bash\nnpm install\npip install -r requirements.txt\nplaywright install chromium\n```\n\nCopy `.env.example` to `.env` and fill in your Cloudflare and Tinker API keys.\n\n## Usage\n\n```bash\n# Generate the dataset\npnpm run prepare-data # or use npm\n\n# Evaluate the base model\nsource venv/bin/activate\npython -m rl.eval\n\n# Train\npython -m rl.training.single_shot\n\n# Evaluate the trained model\npython -m rl.eval --model-path tinker://...\n```\n\n## Stack\n\n- TypeScript (data pipeline)\n- Python (training, eval, reward)\n- Tinker SDK (model inference + LoRA training)\n- Playwright (HTML rendering for reward scoring)\n- Qwen 3.5-4B (base VLM, you can change it!)\n\n## License\n\nMIT\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Froerohan%2Fslate","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Froerohan%2Fslate","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Froerohan%2Fslate/lists"}