{"id":21135242,"url":"https://github.com/cloneofsimo/min-max-in-dit","last_synced_at":"2025-07-09T03:32:51.133Z","repository":{"id":236448102,"uuid":"792631865","full_name":"cloneofsimo/min-max-in-dit","owner":"cloneofsimo","description":null,"archived":false,"fork":false,"pushed_at":"2024-05-03T04:56:19.000Z","size":101,"stargazers_count":19,"open_issues_count":1,"forks_count":0,"subscribers_count":6,"default_branch":"main","last_synced_at":"2024-05-03T10:52:52.081Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cloneofsimo.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-04-27T05:49:01.000Z","updated_at":"2024-05-03T04:56:22.000Z","dependencies_parsed_at":"2024-04-28T06:45:42.087Z","dependency_job_id":null,"html_url":"https://github.com/cloneofsimo/min-max-in-dit","commit_stats":null,"previous_names":["cloneofsimo/min-max-in-dit"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloneofsimo%2Fmin-max-in-dit","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloneofsimo%2Fmin-max-in-dit/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloneofsimo%2Fmin-max-in-dit/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloneofsimo%2Fmin-max-in-dit/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cloneofsimo","download_url":"https://codeload.github.com/cloneofsimo/min-max-in-dit/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":225482022,"owners_count":17481179,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-20T06:46:52.114Z","updated_at":"2024-11-20T06:46:52.787Z","avatar_url":"https://github.com/cloneofsimo.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Min-Max-Imagenet DiT\n\nIn a similar spirit to the Keller Jordan's [Fastest CIFAR-10 training](https://github.com/KellerJordan/cifar10-airbench), I want to be the fastest diffusion trainer in the east. I'll keep the progress here. Currently very much WIP.\n\nFeaturing:\n\n* DeepSpeed training of [Diffusion Transformer](https://arxiv.org/abs/2212.09748). Supports Zero-1,2,3.\n* CPU-offloaded, skipped EMA trick for [Karras' Post-hoc EMA analysis](https://arxiv.org/abs/2312.02696v1), where you EMA once in every `N` steps instead. You have to adjust `beta_1` and `beta_2` so they are properly accounting for the fact you skipped last `N-1` steps. Of course, saving codes are there.\n* Featuring Streaming Dataset, specially my quantized [imagenet.int8](https://github.com/cloneofsimo/imagenet.int8) for insanely lightweight imagenet training.\n\n## Dataset\n\nSince this dataset is so small, you don't need to setup massive remote data setup stuff, just point to the `local_dir`, set `remote_dir` to `None`. \n\n## Running\n\nFor single-node setup, just\n\n```bash\nrun.sh\n```\n\n## Whats the goal here?\n\nMy goal is to get FID score of 30 under 20 hours of training. I'll keep updating this README as I make progress.\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcloneofsimo%2Fmin-max-in-dit","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcloneofsimo%2Fmin-max-in-dit","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcloneofsimo%2Fmin-max-in-dit/lists"}