{"id":15012654,"url":"https://github.com/microsoft/mocapact","last_synced_at":"2025-04-04T13:10:09.012Z","repository":{"id":41280366,"uuid":"500801299","full_name":"microsoft/MoCapAct","owner":"microsoft","description":"A Multi-Task Dataset for Simulated Humanoid Control","archived":false,"fork":false,"pushed_at":"2025-03-27T05:16:48.000Z","size":559,"stargazers_count":183,"open_issues_count":2,"forks_count":23,"subscribers_count":11,"default_branch":"main","last_synced_at":"2025-04-04T12:58:29.417Z","etag":null,"topics":["deepmind-control-suite","dm-control","humanoid-control","imitation-learning","locomotion","mocap","motion-capture","motion-imitation","mujoco","reinforcement-learning","reinforcement-learning-datasets"],"latest_commit_sha":null,"homepage":"https://microsoft.github.io/MoCapAct","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/microsoft.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-06-07T10:55:25.000Z","updated_at":"2025-04-04T01:58:20.000Z","dependencies_parsed_at":"2024-08-12T13:34:58.316Z","dependency_job_id":"7fcb2623-697b-4c02-891d-36dc51ce1fa7","html_url":"https://github.com/microsoft/MoCapAct","commit_stats":{"total_commits":39,"total_committers":5,"mean_commits":7.8,"dds":0.1282051282051282,"last_synced_commit":"52a65cf45799a7b910b72e6035774d811f131e2c"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/microsoft%2FMoCapAct","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/microsoft%2FMoCapAct/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/microsoft%2FMoCapAct/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/microsoft%2FMoCapAct/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/microsoft","download_url":"https://codeload.github.com/microsoft/MoCapAct/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247182399,"owners_count":20897381,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deepmind-control-suite","dm-control","humanoid-control","imitation-learning","locomotion","mocap","motion-capture","motion-imitation","mujoco","reinforcement-learning","reinforcement-learning-datasets"],"created_at":"2024-09-24T19:42:59.019Z","updated_at":"2025-04-04T13:10:08.982Z","avatar_url":"https://github.com/microsoft.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# MoCapAct\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://raw.githubusercontent.com/mhauskn/mocapact.github.io/master/assets/MoCapAct.gif\" alt=\"montage\" width=\"70%\"\u003e\n\u003c/p\u003e\n\n[![Code License](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT) \u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp; [\u003cimg src=\"https://cdla.dev/wp-content/uploads/sites/52/2017/10/cdla_logo.png\" alt=\"Dataset License\" width=\"150\"/\u003e](https://cdla.dev/permissive-2-0/)\n\n\u003cb\u003ePaper: [MoCapAct: A Multi-Task Dataset for Simulated Humanoid Control](https://arxiv.org/abs/2208.07363)\u003c/b\u003e\n\nThis is the codebase for the MoCapAct project, which uses motion capture (MoCap) clips to learn low-level motor skills for the \"CMU Humanoid\" from the \u003ctt\u003edm_control\u003c/tt\u003e package.\nThis repo contains all code to:\n- train the clip snippet experts,\n- collect expert rollouts into a dataset,\n- download our experts and rollouts from the command line,\n- perform policy distillation,\n- perform reinforcement learning on downstream tasks, and\n- perform motion completion.\n\nFor more information on the project and to download the entire dataset, please visit the [project website](https://microsoft.github.io/MoCapAct/).\n\nFor users interested in development, we recommend reading the following documentation on \u003ctt\u003edm_control\u003c/tt\u003e:\n- [The \u003ctt\u003edm_control\u003c/tt\u003e whitepaper](https://arxiv.org/abs/2006.12983)\n- [The \u003ctt\u003edm_control\u003c/tt\u003e README](https://github.com/deepmind/dm_control/blob/main/README.md)\n- [The README for \u003ctt\u003edm_control\u003c/tt\u003e's locomotion task library](https://github.com/deepmind/dm_control/blob/main/dm_control/locomotion/README.md)\n\n## Setup\nMoCapAct requires Python 3.7+.\nWe recommend that you use a virtual environment.\nFor example, using conda:\n```bash\nconda create -n MoCapAct pip python==3.8\nconda activate MoCapAct\n```\n\nTo install the package, we recommend cloning the repo and installing the local copy:\n```bash\ngit clone https://github.com/microsoft/MoCapAct.git\ncd MoCapAct\npip install -e .\n```\n\n## Dataset\nThe MoCapAct dataset consists of clip experts trained on the MoCap snippets and the rollouts from those experts.\n\nWe provide the dataset and models via the [MoCapAct collection on Hugging Face](https://huggingface.co/collections/microsoft/mocapact-66c030d8579a37e69ae3cb26). This collection consists of two pages:\n- A [model zoo](https://huggingface.co/microsoft/mocapact-models) which contains the clip snippet experts, multiclip policies, RL-trained policies for the transfer tasks, and the GPT policy.\n- A [dataset page](https://huggingface.co/datasets/microsoft/mocapact-data) which contains the small rollout dataset and large rollout dataset.\n\n### Description\n\u003cdetails\u003e\n\u003csummary\u003eClip snippet experts\u003c/summary\u003e\nWe signify a clip snippet expert by the snippet it is tracking.\nTaking \u003ctt\u003eCMU_009_12-165-363\u003c/tt\u003e as an example expert, the file hierarchy for the snippet expert is:\n\n```\nCMU_009_12-165-363\n├── clip_info.json         # Contains clip ID, start step, and end step\n└── eval_rsi/model\n    ├── best_model.zip     # Contains policy parameters and hyperparameters\n    └── vecnormalize.pkl   # Used to get normalizer for observation and reward\n```\n\nThe expert policy can be loaded using our repository:\n```python\nfrom mocapact import observables\nfrom mocapact.sb3 import utils\nexpert_path = \"data/experts/CMU_009_12-165-363/eval_rsi/model\"\nexpert = utils.load_policy(expert_path, observables.TIME_INDEX_OBSERVABLES)\n\nfrom mocapact.envs import tracking\nfrom dm_control.locomotion.tasks.reference_pose import types\ndataset = types.ClipCollection(ids=['CMU_009_12'], start_steps=[165], end_steps=[363])\nenv = tracking.MocapTrackingGymEnv(dataset)\nobs, done = env.reset(), False\nwhile not done:\n    action, _ = expert.predict(obs, deterministic=True)\n    obs, rew, done, _ = env.step(action)\n    print(rew)\n```\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eExpert rollouts\u003c/summary\u003e\nThe expert rollouts consist of a collection of HDF5 files, one per clip.\nAn HDF5 file contains expert rollouts for each constituent snippet as well as miscellaneous information and statistics.\nTo facilitate efficient loading of the observations, we concatenate all the proprioceptive observations (joint angles, joint velocities, actuator activations, etc.) from an episode into a single numerical array and provide indices for the constituent observations in the \u003ctt\u003eobservable_indices\u003c/tt\u003e group.\n\nTaking \u003ctt\u003eCMU_009_12.hdf5\u003c/tt\u003e (which contains three snippets) as an example, we have the following HDF5 hierarchy:\n```\nCMU_009_12.hdf5\n├── n_rsi_rollouts                # R, number of rollouts from random time steps in snippet\n├── n_start_rollouts              # S, number of rollouts from start of snippet\n├── ref_steps                     # Indices of MoCap reference relative to current time step. Here, (1, 2, 3, 4, 5).\n├── observable_indices\n│   └── walker\n│       ├── actuator_activation   # (0, 1, ..., 54, 55)\n│       ├── appendages_pos        # (56, 57, ..., 69, 70)\n│       ├── body_height           # (71)\n│       ├── ...\n│       └── world_zaxis           # (2865, 2866, 2867)\n│\n├── stats                         # Statistics computed over the entire dataset\n│   ├── act_mean                  # Mean of the experts' sampled actions\n│   ├── act_var                   # Variance of the experts' sampled actions\n│   ├── mean_act_mean             # Mean of the experts' mean actions\n│   ├── mean_act_var              # Variance of the experts' mean actions\n│   ├── proprio_mean              # Mean of the proprioceptive observations\n│   ├── proprio_var               # Variance of the proprioceptive observations\n│   └── count                     # Number of observations in dataset\n│\n├── CMU_009_12-0-198              # Rollouts for the snippet CMU_009_12-0-198\n├── CMU_009_12-165-363            # Rollouts for the snippet CMU_009_12-165-363\n└── CMU_009_12-330-529            # Rollouts for the snippet CMU_009_12-330-529\n```\n\nEach snippet group contains $R+S$ snippets.\nThe first $S$ episodes correspond to episodes initialized from the start of the snippet and the last $R$ episodes to episodes initialized at random points in the snippet.\nWe now uncollapse the \u003ctt\u003eCMU_009_12-165-363\u003c/tt\u003e group within the HDF5 file to reveal the rollout structure:\n```\nCMU_009_12-165-363\n├── early_termination                  # (R+S)-boolean array indicating which episodes terminated early\n├── rsi_metrics                        # Metrics for episodes that initialize at random points in snippet\n│   ├── episode_returns                # R-array of episode returns\n│   ├── episode_lengths                # R-array of episode lengths\n│   ├── norm_episode_returns           # R-array of normalized episode rewards\n│   └── norm_episode_lengths           # R-array of normalized episode lengths\n├── start_metrics                      # Metrics for episodes that initialize at start in snippet\n│\n├── 0                                  # First episode, of length T\n│   ├── observations\n│   │   ├── proprioceptive             # (T+1)-array of proprioceptive observations\n│   │   ├── walker/body_camera         # (T+1)-array of images from body camera **(not included)**\n│   │   └── walker/egocentric_camera   # (T+1)-array of images from egocentric camera **(not included)**\n│   ├── actions                        # T-array of sampled actions executed in environment\n│   ├── mean_actions                   # T-array of corresponding mean actions\n│   ├── rewards                        # T-array of rewards from environment\n│   ├── values                         # T-array computed using the policy's value network\n│   └── advantages                     # T-array computed using generalized advantage estimation\n│\n├── 1                                  # Second episode\n├── ...\n└── R+S-1                              # (R+S)th episode\n```\nTo keep the dataset size manageable, we do *not* include image observations in the dataset.\nThe camera images can be logged by providing the flags `--log_all_proprios --log_cameras` to the `mocapact/distillation/rollout_experts.py` script.\n\nThe HDF5 rollouts can be read and utilized in Python:\n```python\nimport h5py\ndset = h5py.File(\"data/small_dataset/CMU_009_12.hdf5\", \"r\")\nprint(\"Expert actions from first rollout episode of second snippet:\")\nprint(dset[\"CMU_009_12-165-363/0/actions\"][...])\n```\n\nWe provide a \"large\" dataset where $R = S = 100$ (with size 600 GB) and a \"small\" dataset where $R = S = 10$ (with size 50 GB).\n\u003c/details\u003e\n\n## Examples\n\nBelow are Python commands we used for our paper.\n\n\u003cdetails\u003e\n\u003csummary\u003eClip snippet experts\u003c/summary\u003e\n\nTraining a clip snippet expert:\n```bash\npython -m mocapact.clip_expert.train \\\n  --clip_id [CLIP_ID] `# e.g., CMU_016_22` \\\n  --start_step [START_STEP] `# e.g., 0` \\\n  --max_steps [MAX_STEPS] `# e.g., 210 (can be larger than clip length)` \\\n  --n_workers [N_CPU] `# e.g., 8` \\\n  --log_root experts \\\n  $(cat cfg/clip_expert/train.txt)\n```\n\nEvaluating a clip snippet expert (numerical evaluation and visual evaluation):\n```bash\npython -m mocapact.clip_expert.evaluate \\\n  --policy_root [POLICY_ROOT] `# e.g., experts/CMU_016-22-0-82/0/eval_rsi/model` \\\n  --n_workers [N_CPU] `# e.g., 8` \\\n  --n_eval_episodes 1000 `# set to 0 to just run the visualizer` \\\n  $(cat cfg/clip_expert/evaluate.txt)\n```\n\nWe can also load the experts in Python:\n```python\nfrom mocapact import observables\nfrom mocapact.sb3 import utils\nexpert_path = \"experts/CMU_016_22-0-82/0/eval_rsi/model\" # example path\nexpert = utils.load_policy(expert_path, observables.TIME_INDEX_OBSERVABLES)\n\nfrom mocapact.envs import tracking\nfrom dm_control.locomotion.tasks.reference_pose import types\ndataset = types.ClipCollection(ids=['CMU_016_22'])\nenv = tracking.MocapTrackingGymEnv(dataset)\nobs, done = env.reset(), False\nwhile not done:\n    action, _ = expert.predict(obs, deterministic=True)\n    obs, rew, done, _ = env.step(action)\n    print(rew)\n```\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eCreating rollout dataset\u003c/summary\u003e\n\nRolling out a collection of experts and collecting into a dataset:\n```bash\npython -m mocapact.distillation.rollout_experts \\\n  --input_dirs [EXPERT_ROOT] `# e.g., experts` \\\n  --n_workers [N_CPU] `# e.g., 8` \\\n  --device [DEVICE] `# e.g., cuda` \\\n  --output_path dataset/file_name_ignored.hdf5 \\\n  $(cat cfg/rollout.txt)\n```\n\nThis will result in a collection of HDF5 files (one per clip), which can be read and utilized in Python:\n```python\nimport h5py\ndset = h5py.File(\"dataset/CMU_016_22.hdf5\", \"r\")\nprint(\"Expert actions from first rollout episode:\")\nprint(dset[\"CMU_016_22-0-82/0/actions\"][...])\n```\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eMulti-clip policy\u003c/summary\u003e\n\nTraining a multi-clip policy on the entire MoCapAct dataset:\n```bash\nsource scripts/get_all_clips.sh [PATH_TO_DATASET]\npython -m mocapact.distillation.train \\\n  --train_dataset_paths $train \\\n  --val_dataset_paths $val \\\n  --dataset_metrics_path $metrics \\\n  --extra_clips $clips \\\n  --output_root multi_clip/all \\\n  --gpus 0 `# indices of GPUs` \\\n  $(cat cfg/multi_clip/train.txt) \\\n  $(cat cfg/multi_clip/rwr.txt) `# rwr can be replaced with awr, cwr, or bc` \\\n  --model.config.embed_size 60 \\\n  --eval.n_workers [N_CPU] `# e.g., 16`\n```\n\nTraining a multi-clip policy on the locomotion subset of the MoCapAct dataset:\n```bash\nsource scripts/get_locomotion_clips.sh [PATH_TO_DATASET]\npython -m mocapact.distillation.train \\\n  --train_dataset_paths $train \\\n  --dataset_metrics_path $metrics \\\n  --extra_clips $clips \\\n  --output_root multi_clip/locomotion \\\n  --gpus 0 `# indices of GPUs` \\\n  $(cat cfg/multi_clip/train.txt) \\\n  $(cat cfg/multi_clip/rwr.txt) `# rwr can be replaced with awr, cwr, or bc` \\\n  --model.config.embed_size 20 \\\n  --eval.n_workers [N_CPU] `# e.g., 16`\n```\n\nEvaluating a multi-clip policy on all the snippets within the MoCapAct dataset (numerical evaluation and visual evaluation):\n```bash\nsource scripts/get_all_clips.sh [PATH_TO_DATASET]\npython -m mocapact.distillation.evaluate \\\n  --policy_path [POLICY_PATH] `# e.g., multi_clip/all/eval/train_rsi/best_model.ckpt` \\\n  --clip_snippets $snippets \\\n  --n_workers [N_CPU] `# e.g., 8` \\\n  --device [DEVICE] `# e.g., cuda` \\\n  --n_eval_episodes 1000 `# set to 0 to just run the visualizer` \\\n  $(cat cfg/multi_clip/evaluate.txt)\n```\n\nThe multi-clip policy can be loaded using PyTorch Lightning's functionality to interact with the environment:\n```python\nfrom mocapact.distillation import model\nmodel_path = \"multi_clip/all/eval/train_rsi/best_model.ckpt\"\npolicy = model.NpmpPolicy.load_from_checkpoint(model_path, map_location=\"cpu\")\n\nfrom mocapact.envs import tracking\nfrom dm_control.locomotion.tasks.reference_pose import cmu_subsets\ndataset = cmu_subsets.ALL\nref_steps = (1, 2, 3, 4, 5)\nenv = tracking.MocapTrackingGymEnv(dataset, ref_steps)\nobs, done = env.reset(), False\nembed = policy.initial_state(deterministic=False)\nwhile not done:\n    action, embed = expert.predict(obs, state=embed, deterministic=False)\n    obs, rew, done, _ = env.step(action)\n    print(rew)\n```\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eRL transfer tasks\u003c/summary\u003e\n\nTraining a task policy (here, with a pre-defined low-level policy):\n```bash\npython -m mocapact.transfer.train \\\n  --log_root [LOG_ROOT] `# e.g., transfer/go_to_target` \\\n  $(cat cfg/transfer/train.txt) \\\n  $(cat cfg/transfer/go_to_target.txt) `# set to cfg/transfer/velocity_control.txt for velocity control` \\\n  $(cat cfg/transfer/with_low_level.txt) `# set to cfg/transfer/no_low_level.txt for no low-level policy`\n```\n\nEvaluating a task policy:\n```bash\npython -m mocapact.transfer.evaluate \\\n  --model_root [MODEL_ROOT] `# e.g., transfer/go_to_target/0/eval/model` \\\n  --task [TASK] `# e.g., mocapact/transfer/config.py:go_to_target or velocity_control`\n```\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eMotion completion\u003c/summary\u003e\n\nTraining a GPT policy on the entire MoCapAct dataset:\n```bash\nsource scripts/get_all_clips.sh [PATH_TO_DATASET]\npython -m mocapact.distillation.train \\\n  --train_dataset_paths $train \\\n  --val_dataset_paths $val \\\n  --dataset_metrics_path $metrics \\\n  --output_root motion_completion \\\n  $(cat cfg/motion_completion/train.txt)\n```\n\nPerforming motion completion with a trained GPT policy:\n```bash\npython -m mocapact.distillation.motion_completion \\\n  --policy_path [POLICY_PATH] `# e.g., motion_completion/model/last.ckpt` \\\n  --expert_root [EXPERT_ROOT] `# e.g., experts` \\\n  --clip_snippet [CLIP_SNIPPET] `# e.g., CMU_016_22` \\\n  --n_workers [N_CPU] `# e.g., 8` \\\n  --device [DEVICE] `# e.g., cuda` \\\n  --n_eval_episodes 100 `# Set to 0 to just run the visualizer` \\\n  $(cat cfg/motion_completion/evaluate.txt)\n```\nTo generate a prompt, we also input a path to the directory of snippet experts.\nAlternatively, you can pass a path to a multi-clip policy through `--distillation_path`, though it will likely produce lower-quality prompts than the snippet experts.\n\u003c/details\u003e\n\n## Dataset Interface\nWe provide two datasets in this repo.\nThe [`ExpertDataset`](https://github.com/microsoft/MoCapAct/blob/main/mocapact/distillation/dataset.py) is used to perform imitation learning, e.g., to train a multi-clip tracking policy or a GPT policy for motion completion.\nThe [`D4RLDataset`](https://github.com/microsoft/MoCapAct/blob/main/mocapact/offline_rl/d4rl_dataset.py) is used for offline reinforcement learning. \nFor small enough instantiations of the datasets that fit into memory, the user can use `D4RLDataset.get_in_memory_rollouts` to load a batch of transitions into memory.\nFor instantiations that do not fit into memory (e.g., the entire MoCapAct dataset), the user can use the dataset as a PyTorch `Dataset` by using an iterator over the transitions obtained by using `__getitem__`.\n\n## Example Projects That Use MoCapAct\n- H-GAP: Humanoid Control with a Generalist Planner, by Zhengyao Jiang, Yingchen Xu, et al. ([Paper](https://arxiv.org/abs/2312.02682), [Website](https://yingchenxu.com/hgap), [Code](https://github.com/facebookresearch/hgap))\n- Leveraging Demonstrations with Latent Space Priors, by Jonas Gehring et al. ([Paper](https://arxiv.org/abs/2210.14685), [Website](https://facebookresearch.github.io/latent-space-priors/), [Code](https://github.com/facebookresearch/latent-space-priors))\n- Hierarchical World Models as Visual Whole-Body Humanoid Controllers, by Nicklas Hansen et al. ([Paper](https://arxiv.org/abs/2405.18418), [Website](https://www.nicklashansen.com/rlpuppeteer/), [Code](https://github.com/nicklashansen/puppeteer))\n- Body Transformer: Leveraging Robot Embodiment for Policy Learning, by Carmelo Sferrazza et al. ([Paper](https://arxiv.org/abs/2408.06316), [Website](https://sferrazza.cc/bot_site/), [Code](https://github.com/carlosferrazza/BodyTransformer))\n\n## Citation\nIf you reference or use MoCapAct in your research, please cite:\n\n```bibtex\n@inproceedings{wagener2022mocapact,\n  title={{MoCapAct}: A Multi-Task Dataset for Simulated Humanoid Control},\n  author={Wagener, Nolan and Kolobov, Andrey and Frujeri, Felipe Vieira and Loynd, Ricky and Cheng, Ching-An and Hausknecht, Matthew},\n  booktitle={Advances in Neural Information Processing Systems},\n  volume={35},\n  pages={35418--35431},\n  year={2022}\n}\n```\n\n## Licenses\nThe code is licensed under the [MIT License](https://opensource.org/licenses/MIT).\nThe dataset is licensed under the [CDLA Permissive v2 License](https://cdla.dev/permissive-2-0/).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmicrosoft%2Fmocapact","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmicrosoft%2Fmocapact","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmicrosoft%2Fmocapact/lists"}