{"id":22230428,"url":"https://barquerogerman.github.io/FlowMDM/","last_synced_at":"2025-07-27T19:33:13.334Z","repository":{"id":224437662,"uuid":"742013413","full_name":"BarqueroGerman/FlowMDM","owner":"BarqueroGerman","description":"[CVPR 2024] Official Implementation of \"Seamless Human Motion Composition with Blended Positional Encodings\".","archived":false,"fork":false,"pushed_at":"2024-05-22T08:52:05.000Z","size":36644,"stargazers_count":158,"open_issues_count":3,"forks_count":7,"subscribers_count":11,"default_branch":"main","last_synced_at":"2024-05-22T09:55:38.060Z","etag":null,"topics":["cvpr","cvpr2024","diffusion","generative-model","human-motion","human-motion-composition","human-motion-extrapolation","motion-generation"],"latest_commit_sha":null,"homepage":"https://barquerogerman.github.io/FlowMDM/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/BarqueroGerman.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-01-11T15:27:10.000Z","updated_at":"2024-05-22T08:52:08.000Z","dependencies_parsed_at":"2024-03-18T14:46:07.699Z","dependency_job_id":null,"html_url":"https://github.com/BarqueroGerman/FlowMDM","commit_stats":null,"previous_names":["barquerogerman/flowmdm"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BarqueroGerman%2FFlowMDM","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BarqueroGerman%2FFlowMDM/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BarqueroGerman%2FFlowMDM/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BarqueroGerman%2FFlowMDM/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/BarqueroGerman","download_url":"https://codeload.github.com/BarqueroGerman/FlowMDM/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":227831082,"owners_count":17826155,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cvpr","cvpr2024","diffusion","generative-model","human-motion","human-motion-composition","human-motion-extrapolation","motion-generation"],"created_at":"2024-12-03T01:14:59.508Z","updated_at":"2024-12-03T01:15:14.547Z","avatar_url":"https://github.com/BarqueroGerman.png","language":"Python","funding_links":[],"categories":["Uncategorized"],"sub_categories":["Uncategorized"],"readme":"\u003ch1 align=\"center\"\u003eFlowMDM\u003c/h3\u003e\n\n\u003ch3 align=\"center\"\u003eSeamless Human Motion Composition with Blended Positional Encodings (CVPR'24)\u003c/h3\u003e\n\n  \u003cp align=\"center\"\u003e\n    \u003ca href=\"https://barquerogerman.github.io/FlowMDM/\"\u003e\u003cimg alt=\"Project\" src=\"https://img.shields.io/badge/-Project%20Page-lightgrey?logo=Google%20Chrome\u0026color=informational\u0026logoColor=white\"\u003e\u003c/a\u003e\n    \u003ca href=\"https://arxiv.org/abs/2402.15509\"\u003e\u003cimg alt=\"arXiv\" src=\"https://img.shields.io/badge/arXiv-2402.15509-b31b1b.svg\"\u003e\u003c/a\u003e \n    \u003cimg alt=\"visits\" src=\"https://hits.seeyoufarm.com/api/count/incr/badge.svg?url=https%3A%2F%2Fgithub.com%2FBarqueroGerman%2FFlowMDM\u0026count_bg=%2320AF15\u0026title_bg=%23555555\u0026icon=\u0026icon_color=%23E7E7E7\u0026title=visits\u0026edge_flat=false\"\u003e\n  \u003c/p\u003e\n\n\u003cbr\u003e\n\nHuman Motion Composition             |  Human Motion Extrapolation\n:-------------------------:|:-------------------------:\n![](assets/example_composition.gif)  |  ![](assets/example_extrapolation.gif)\n\n\n## 🔎 About\n\u003cdiv style=\"text-align: center;\"\u003e\n    \u003cimg src=\"assets/main_figure.png\" align=\"center\" width=100% \u003e\n\u003c/div\u003e\n\u003c/br\u003e\nConditional human motion generation is an important topic with many applications in virtual reality, gaming, and robotics. \nWhile prior works have focused on generating motion guided by text, music, or scenes, these typically result in isolated motions confined to short durations. \nInstead, we address the generation of long, continuous sequences guided by a series of varying textual descriptions. In this context, we introduce FlowMDM, the first diffusion-based model that generates seamless Human Motion Compositions (HMC) without any postprocessing or redundant denoising steps. For this, we introduce the Blended Positional Encodings, a technique that leverages both absolute and relative positional encodings in the denoising chain. More specifically, global motion coherence is recovered at the absolute stage, whereas smooth and realistic transitions are built at the relative stage. As a result, we achieve state-of-the-art results in terms of accuracy, realism, and smoothness on the Babel and HumanML3D datasets. FlowMDM excels when trained with only a single description per motion sequence thanks to its Pose-Centric Cross-ATtention, which makes it robust against varying text descriptions at inference time. Finally, to address the limitations of existing HMC metrics, we propose two new metrics: the Peak Jerk and the Area Under the Jerk, to detect abrupt transitions.\n\n\u003c!--\n## Running instructions\n--\u003e\n\n## 📌 News\n- [2024-05-13] Eval/Gen instructions updated (wrong value in --bpe_denoising_step fixed)\n- [2024-03-18] Code + model weights released!\n- [2024-02-27] FlowMDM is now accepted at CVPR 2024!\n- [2024-02-26] Our paper is available in [Arxiv](https://arxiv.org/abs/2402.15509).\n\n## 📝 TODO List\n- [x] Release pretrained models.\n- [x] Release generation (skeletons + blender support for meshes) + evaluation + training code.\n- [ ] Release generation code for demo-style visualizations.\n\n## 👩🏻‍🏫 Getting started\n\nThis code was tested on Ubuntu 20.04.6 LTS + Python 3.8 + PyTorch 1.13.0 While other versions might work as well, we recommend using this conda environment to avoid any issues.\n\n\n1. Install ffmpeg (if not already installed):\n\n```shell\nsudo apt update\nsudo apt install ffmpeg\n```\nFor windows use [this](https://www.geeksforgeeks.org/how-to-install-ffmpeg-on-windows/) instead.\n\n2. Setup conda env:\n```shell\nconda env create -f environment.yml\nconda activate FlowMDM\npython -m spacy download en_core_web_sm\npip install git+https://github.com/openai/CLIP.git\npip install git+https://github.com/GuyTevet/smplx.git\nconda install ffmpeg -y\n```\n\nThis [README file](https://github.com/BarqueroGerman/FlowMDM/blob/main/runners/README.md) contains instructions on how to visualize, evaluate, and train the model.\n\n\u003e [!NOTE]\n\u003e This repository inherits a lot of work from the original MDM and Guided-Diffusion repositories. Most of FlowMDM's contribution can be found in the `model/FlowMDM.py` and  `diffusion/diffusion_wrappers.py` files, and the `model/x_transformers` folder.\n\n## 📚 Citation\n\nIf you find our work helpful, please cite:\n\n```bibtex\n@inproceedings{barquero2024seamless,\n  title={Seamless Human Motion Composition with Blended Positional Encodings},\n  author={Barquero, German and Escalera, Sergio and Palmero, Cristina},\n  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},\n  year={2024}\n}\n```\n\n## 🤝🏼 Acknowledgements\n- [TEMOS](https://github.com/Mathux/TEMOS): We inherit a lot of the code from TEMOS.\n- [TEACH](https://github.com/athn-nik/teach): We use TEACH in our work, and inherit part of the code from them.\n- [MDM](https://guytevet.github.io/mdm-page/): We use MDM in our work, and inherit as well part of the code.\n- [PriorMDM](https://github.com/priorMDM/priorMDM): We use PriorMDM in our work, and inherit as well part of the code.\n- [x-transformers](https://github.com/lucidrains/x-transformers): BPEs are built on their transformers library.\n\n## ⭐ Star History\n\n\n\u003cp align=\"center\"\u003e\n    \u003ca href=\"https://star-history.com/#BarqueroGerman/FlowMDM\u0026Date\" target=\"_blank\"\u003e\n        \u003cimg width=\"500\" src=\"https://api.star-history.com/svg?repos=BarqueroGerman/FlowMDM\u0026type=Date\" alt=\"Star History Chart\"\u003e\n    \u003c/a\u003e\n\u003cp\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/barquerogerman.github.io%2FFlowMDM%2F","html_url":"https://awesome.ecosyste.ms/projects/barquerogerman.github.io%2FFlowMDM%2F","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/barquerogerman.github.io%2FFlowMDM%2F/lists"}