{"id":14961268,"url":"https://github.com/linghaochan/unimocap","last_synced_at":"2025-10-24T20:31:33.446Z","repository":{"id":200468309,"uuid":"701127656","full_name":"LinghaoChan/UniMoCap","owner":"LinghaoChan","description":"[Open-source Project] UniMoCap: community implementation to unify the text-motion datasets (HumanML3D, KIT-ML, and BABEL) and whole-body motion dataset (Motion-X).","archived":false,"fork":false,"pushed_at":"2024-01-04T06:45:23.000Z","size":2506,"stargazers_count":165,"open_issues_count":1,"forks_count":4,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-01-31T04:16:27.256Z","etag":null,"topics":["amass","babel","humanml3d","kit-ml","motion","motion-capture","motion-generation","motion-x","text-to-motion","text2motion","unifer","unifier"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/LinghaoChan.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2023-10-06T01:29:11.000Z","updated_at":"2025-01-14T10:36:21.000Z","dependencies_parsed_at":"2024-01-04T07:42:46.947Z","dependency_job_id":null,"html_url":"https://github.com/LinghaoChan/UniMoCap","commit_stats":null,"previous_names":["linghaochan/unimocap"],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LinghaoChan%2FUniMoCap","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LinghaoChan%2FUniMoCap/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LinghaoChan%2FUniMoCap/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LinghaoChan%2FUniMoCap/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/LinghaoChan","download_url":"https://codeload.github.com/LinghaoChan/UniMoCap/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":238035385,"owners_count":19405682,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["amass","babel","humanml3d","kit-ml","motion","motion-capture","motion-generation","motion-x","text-to-motion","text2motion","unifer","unifier"],"created_at":"2024-09-24T13:24:18.395Z","updated_at":"2025-10-24T20:31:25.353Z","avatar_url":"https://github.com/LinghaoChan.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"![unimocap logo.png](./resource/imgs/logo.png)\n\n[中文文档](./resource/docs/cn-README.md) ｜ [Tutorial Video (coming soon)](https://www.youtube.com/channel/UC5P1ZtWm47L78t-Snn2xhhQ) \n\n# ❓ What is UniMoCap?\n\nUniMoCap is a community implementation to unify the text-motion mocap datasets. In this repository, we unify the AMASS-based text-motion datasets (HumanML3D, BABEL, and KIT-ML). We support to process the AMASS data to both :\n\n- [x] body-only H3D-format (263-dim, 24 joints)\n- [x] whole-body SMPL-X-format (322-dim SMPL-X parameters). \n\n***We believe this repository will be useful for training models on larger mocap text-motion data. We will support more T-M mocap datasets in near feature.***\n\nWe make the data processing as simple as possible. For those who are not familiar with the datasets, we will provide a video tutorial to tell you how to do it in the following weeks. This is a community implementation to support text-motion datasets. For the Chinese community, we provide a Chinese document ([中文文档](./resource/docs/cn-README.md)) for users. \n\n\n# 🏃🏼 TODO List\n\n- [ ] ***Support SMPL-X motion representation (including position, velocity, and rotations for body and hands) calculation (expected in 1-2 week).***\n- [ ] Support `seg` only and `seq` only BABEL unifier.\n- [ ] Provide a tutorial video.\n- [ ] Support more language documents.\n- [ ] Support more T-M datasets (e.g.: [FLAG3D](https://andytang15.github.io/FLAG3D), [STDM](https://druthrie.github.io/sequential-texts-to-motion)). Welcome to support your own dataset here! \n- [ ] Provide trained models based on UniMoCap.\n\n# 🛠️ Installation\n\n```bash\npip install -r requirements.txt\n```\n\n# 🚀 How to use?\n\n## 1. Data Preparing\n\n\n\u003cdetails\u003e\n\u003csummary\u003eDownload SMPL+H and DMPLs.\u003c/summary\u003e\n\nDownload SMPL+H mode from [SMPL+H](https://mano.is.tue.mpg.de/download.php) (choose Extended SMPL+H model used in AMASS project), DMPL model from [DMPL](https://smpl.is.tue.mpg.de/download.php) (choose DMPLs compatible with SMPL), and SMPL-X model from [SMPL-X](https://smpl-x.is.tue.mpg.de/download.php). Then place all the models under `./body_model/`. The `./body_model/` folder tree should be:\n\n```bash\n./body_models\n├── dmpls\n│   ├── female\n│   │   └── model.npz\n│   ├── male\n│   │   └── model.npz\n│   └── neutral\n│       └── model.npz\n├── smplh\n│   ├── female\n│   │   └── model.npz\n│   ├── info.txt\n│   ├── male\n│   │   └── model.npz\n│   └── neutral\n│       └── model.npz\n├── smplx\n│   ├── female\n│   │   ├── model.npz\n│   │   └── model.pkl\n│   ├── male\n│   │   ├── model.npz\n│   │   └── model.pkl\n│   └── neutral\n│       ├── model.npz\n└───────└── model.pkl\n```\n\n\u003c/details\u003e\n\n\n\u003cdetails\u003e\n\u003csummary\u003eDownload AMASS motions.\u003c/summary\u003e\n    \n  - Download [AMASS](https://amass.is.tue.mpg.de/download.php) motions. \n  - If you are using the SMPL (in HumanML3D, BABEL, and KIT-ML), please download the AMASS data with `SMPL-H G` into `./datasets/amass_data/`.\n  - If you are using the SMPL-X (in Motion-X), please download the AMASS data with `SMPL-X G`. If you use the SMPL-X data, please save them at `./datasets/amass_data-x/`.\n  \n  The `datasets/amass_data/` folder tree should be:\n  \n  ```bash\n  ./datasets/amass_data/\n  ├── ACCAD\n  ├── BioMotionLab_NTroje\n  ├── BMLhandball\n  ├── BMLmovi\n  ├── CMU\n  ├── DanceDB\n  ├── DFaust_67\n  ├── EKUT\n  ├── Eyes_Japan_Dataset\n  ├── GRAB\n  ├── HUMAN4D\n  ├── HumanEva\n  ├── KIT\n  ├── MPI_HDM05\n  ├── MPI_Limits\n  ├── MPI_mosh\n  ├── SFU\n  ├── SOMA\n  ├── SSM_synced\n  ├── TCD_handMocap\n  ├── TotalCapture\n  └── Transitions_mocap\n  ```\n\u003c/details\u003e    \n\n\n\u003cdetails\u003e\n\u003csummary\u003eHumanML3D Dataset\u003c/summary\u003e\n    \nClone the [HumanML3D](https://github.com/EricGuo5513/HumanML3D) repo to `datasets/HumanML3D/`, unzip the `humanact12.zip` to `datasets/pose_data` and unzip the `texts.zip` file.\n\n```bash\nmkdir datasets\ncd datasets\ngit clone https://github.com/EricGuo5513/HumanML3D/tree/main\nmkdir pose_data\nunzip HumanML3D/pose_data/humanact12.zip -d pose_data/\nmv pose_data/humanact12/humanact12/*.npy pose_data/humanact12/\nrm -rf pose_data/humanact12/humanact12\ncd HumanML3D/HumanML3D\nunzip texts.zip\ncd ../../..\n```\n\u003c/details\u003e    \n\n\n\u003cdetails\u003e\n\u003csummary\u003eKIT-ML Dataset\u003c/summary\u003e\n    \nDownload [KIT-ML](https://drive.google.com/file/d/14XX0PRDRtsEywpBZ87mKfV1sMbpTu_Ct/view?usp=sharing) motions, and unzip in the folder `datasets/kit-mocap/`.\n\u003c/details\u003e  \n    \n\u003cdetails\u003e\n\u003csummary\u003eBABEL Dataset\u003c/summary\u003e\n    \nDownload the [BABEL](https://teach.is.tue.mpg.de/download.php) annotations from TEACH into `datasets/babel-teach/`.\n\u003c/details\u003e \n    \n\n## 2. Generate mapping files and text files\n\nIn this step, we will get mapping files (`.csv`) and text files (`./{dataset}_new_text`). \n\n\u003cdetails\u003e\n\u003csummary\u003eHumanML3D Dataset\u003c/summary\u003e\n\nDue to the HumanML3D dataset is under the MIT License, I have preprocessed the `.json` (`./outputs-json/humanml3d.json`) file and `.csv` file (`h3d_h3dformat.csv`). Besides, the `.csv` file can be generated by the following command. \n\n```bash\npython h3d_to_h3d.py\n```\n\u003c/details\u003e \n    \n\u003cdetails\u003e\n\u003csummary\u003eBABEL Dataset\u003c/summary\u003e\n    \nWe provide the code to generate unified BABEL annotation. Both `.json` (`./outputs-json/babel{_mode}.json`) file and `.csv` file (`babel{mode}_h3dformat.csv`) are generated. You can generate related files with the following command. The `.json` file is only an intermediate generated file and will not be used in subsequent processing.\n\nFor BABEL, `babel_seg` and `babel_seq` denote the segmentation level and whole-sequence level annotation respectively. The `babel` denotes the both level annotation. \n\n```bash\npython babel.py\n```\n\u003c/details\u003e \n    \n\u003cdetails\u003e\n\u003csummary\u003eKIT-ML Dataset\u003c/summary\u003e\n    \nWe provide the code to generate unified KIT-ML annotation. Both `.json` (`./outputs-json/kitml.json`) file and `.csv` file (`kitml_h3dformat.csv`) are generated. You can generate related files with the following command. The `.json` file is only an intermediate generated file and will not be used in subsequent processing.\n\n```bash\npython kitml.py\n```\n\u003c/details\u003e \n\n## 3. Extract and Process Data\n\nIn this step, we follow the method in [HumanML3D](https://github.com/EricGuo5513/HumanML3D) to extract motion feature.  \n\nNow, you are at the root position of `./UniMoCap`. To generated body-only motion feature, we create a folder and copy some tools provided by HumanML3D first:\n\n```bash\nmkdir body-only-unimocap\ncp -r ./datasets/HumanML3D/common ./\ncp -r ./datasets/HumanML3D/human_body_prior ./\ncp -r ./datasets/HumanML3D/paramUtil.py ./\n```\n\nIf you would like to get body-only motions, please refer to [Body-only data preprocessing](./resource/docs/en-bodyonly.md).\n\nIf you would like to get whole-body motions, please refer to [Whole-body motion processing](./resource/docs/en-wholebody.md).\n\n\n## 💖 Opening for Community Contributions\n\nWe sincerely wish the community to support more text-motion mocap datasets. \n\n## 🌹 Acknowledgement\n\nOur code is modified on the basis of [TMR](https://github.com/Mathux/TMR), [AMASS-Annotation-Unifier](https://github.com/Mathux/AMASS-Annotation-Unifier), and [HumanML3D](https://github.com/EricGuo5513/HumanML3D), thanks to all contributors!\n\n## Star History\n\n\u003cp align=\"center\"\u003e\n    \u003ca href=\"https://star-history.com/#LinghaoChan/UniMoCap\u0026Date\" target=\"_blank\"\u003e\n        \u003cimg width=\"500\" src=\"https://api.star-history.com/svg?repos=LinghaoChan/UniMoCap\u0026type=Date\" alt=\"Star History Chart\"\u003e\n    \u003c/a\u003e\n\u003cp\u003e\n\n\n# 🤝🏼 Citation\n\nIf you use this repository for research, you need to cite: \n```bash\n@article{chen2023unimocap,\n  title={UniMocap: Unifier for BABEL, HumanML3D, and KIT},\n  author={Chen, Ling-Hao and UniMocap, Contributors},\n  journal={https://github.com/LinghaoChan/UniMoCap},\n  year={2023}\n}\n```\nAs some components of UniMoCap are borrowed from [AMASS-Annotation-Unifier](https://github.com/Mathux/AMASS-Annotation-Unifier) and [HumanML3D](https://github.com/EricGuo5513/HumanML3D). You need to cite them accordingly.\n\n```bash\n@inproceedings{petrovich23tmr,\n    title     = {{TMR}: Text-to-Motion Retrieval Using Contrastive {3D} Human Motion Synthesis},\n    author    = {Petrovich, Mathis and Black, Michael J. and Varol, G{\\\"u}l},\n    booktitle = {International Conference on Computer Vision ({ICCV})},\n    year      = {2023}\n}\n```\n\n```bash\n@InProceedings{Guo_2022_CVPR,\n    author    = {Guo, Chuan and Zou, Shihao and Zuo, Xinxin and Wang, Sen and Ji, Wei and Li, Xingyu and Cheng, Li},\n    title     = {Generating Diverse and Natural 3D Human Motions From Text},\n    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},\n    month     = {June},\n    year      = {2022},\n    pages     = {5152-5161}\n}\n```\n\nIf you use the dataset, you need to cite subset KIT-ML and AMASS.\n\n```bash\n@article{Plappert2016,\n    author = {Matthias Plappert and Christian Mandery and Tamim Asfour},\n    title = {The {KIT} Motion-Language Dataset},\n    journal = {Big Data}\n    publisher = {Mary Ann Liebert Inc},\n    year = {2016},\n    month = {dec},\n    volume = {4},\n    number = {4},\n    pages = {236--252}\n}\n```\n\n```bash\n@conference{AMASS2019,\n  title = {AMASS: Archive of Motion Capture as Surface Shapes},\n  author = {Mahmood, Naureen and Ghorbani, Nima and Troje, Nikolaus F. and Pons-Moll, Gerard and Black, Michael J.},\n  booktitle = {International Conference on Computer Vision},\n  pages = {5442--5451},\n  month = oct,\n  year = {2019},\n  month_numeric = {10}\n}\n```\n\nIf you use the Motion-X dataset, please cite it accordingly.\n```bash\n@article{lin2023motionx,\n  title={Motion-X: A Large-scale 3D Expressive Whole-body Human Motion Dataset},\n  author={Lin, Jing and Zeng, Ailing and Lu, Shunlin and Cai, Yuanhao and Zhang, Ruimao and Wang, Haoqian and Zhang, Lei},\n  journal={Advances in Neural Information Processing Systems},\n  year={2023}\n}\n```\nIf you have any question, please contact Ling-Hao CHEN (thu [DOT] lhchen [AT] gmail [DOT] com).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flinghaochan%2Funimocap","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flinghaochan%2Funimocap","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flinghaochan%2Funimocap/lists"}