{"id":20157236,"url":"https://github.com/robinka/discrete-flow-matching-pytorch","last_synced_at":"2025-10-26T18:16:43.671Z","repository":{"id":261331515,"uuid":"883990331","full_name":"RobinKa/discrete-flow-matching-pytorch","owner":"RobinKa","description":"Discrete Flow Matching implemented in PyTorch","archived":false,"fork":false,"pushed_at":"2025-03-23T11:18:10.000Z","size":400,"stargazers_count":22,"open_issues_count":0,"forks_count":2,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-09-20T01:09:54.793Z","etag":null,"topics":["ai","compile","diffusion","discrete","flops","flow-matching","generation","generative-ai","lightning","machine-learning","mfu","python","pytorch","text"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/RobinKa.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-11-05T23:55:43.000Z","updated_at":"2025-09-02T08:54:36.000Z","dependencies_parsed_at":"2024-11-06T01:17:59.541Z","dependency_job_id":"b9b7db1e-ba72-4d76-9e3c-3527df832e7a","html_url":"https://github.com/RobinKa/discrete-flow-matching-pytorch","commit_stats":null,"previous_names":["robinka/discrete-flow-matching-pytorch"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/RobinKa/discrete-flow-matching-pytorch","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RobinKa%2Fdiscrete-flow-matching-pytorch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RobinKa%2Fdiscrete-flow-matching-pytorch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RobinKa%2Fdiscrete-flow-matching-pytorch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RobinKa%2Fdiscrete-flow-matching-pytorch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/RobinKa","download_url":"https://codeload.github.com/RobinKa/discrete-flow-matching-pytorch/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RobinKa%2Fdiscrete-flow-matching-pytorch/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":276229042,"owners_count":25606942,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-21T02:00:07.055Z","response_time":72,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","compile","diffusion","discrete","flops","flow-matching","generation","generative-ai","lightning","machine-learning","mfu","python","pytorch","text"],"created_at":"2024-11-13T23:44:44.716Z","updated_at":"2025-09-21T10:31:57.949Z","avatar_url":"https://github.com/RobinKa.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Discrete Flow Matching implemented in PyTorch\n\nImplementation of Discrete Flow Matching [[1]](https://arxiv.org/abs/2402.04997)[[2]](https://arxiv.org/abs/2407.15595), which is a generative model for generating discrete things such as text with flow matching. The code is implemented in PyTorch.\n\n| Step 0 of 128 (input)      | Step 64 of 128               | Step 128 of 128 (output)       |\n| -------------------------- | ---------------------------- | ------------------------------ |\n| ![Step 0](media/step0.png) | ![Step 64](media/step64.png) | ![Step 128](media/step128.png) |\n\n## How to run\n\n### Environment setup\n\n1. Install [uv](https://github.com/astral-sh/uv) for package management, e.g. `pip install uv`\n2. Make sure Python 3.12 is installed: `uv python install 3.12`\n3. Install the dependencies: `uv sync --group jupyter`\n\nRun `python -m discrete_flow_matching_pytorch.train --config configs/conv-8.yaml` to start training a text generation model logging to wandb.\n\nThe [sample notebook](notebooks/sample.ipynb) demonstrates the sampling process.\n\n**Note**: Instead of using uv, it is also possible to install the dependencies in [pyproject.toml](pyproject.toml) with pip.\n\n## Summary of discrete flow matching compared to continuous flow matching\n\n- During training, we mask out text tokens according to the timestep\n- The model is trained to predict the original unmasked tokens with cross entropy loss\n- In sampling, we unmask text gradually with the sampled tokens\n\n## References\n\n- [[1] Generative Flows on Discrete State-Spaces: Enabling Multimodal Flows with Applications to Protein Co-Design](https://arxiv.org/abs/2402.04997) ([YouTube presentation](https://www.youtube.com/watch?v=yzc29vhM2Aw)): Combines discrete and continuous flow matching. Originally introduced Discrete Flow Matching. Appendix F was very useful for the implementation\n- [[2] Discrete Flow Matching](https://arxiv.org/abs/2407.15595): Builds on Multiflow's Discrete Flow Matching\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frobinka%2Fdiscrete-flow-matching-pytorch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frobinka%2Fdiscrete-flow-matching-pytorch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frobinka%2Fdiscrete-flow-matching-pytorch/lists"}