{"id":20095849,"url":"https://github.com/markfzp/humanplus","last_synced_at":"2025-04-04T10:02:54.861Z","repository":{"id":244275324,"uuid":"814396494","full_name":"MarkFzp/humanplus","owner":"MarkFzp","description":"[CoRL 2024] HumanPlus: Humanoid Shadowing and Imitation from Humans","archived":false,"fork":false,"pushed_at":"2024-07-01T22:28:54.000Z","size":20805,"stargazers_count":688,"open_issues_count":0,"forks_count":108,"subscribers_count":17,"default_branch":"main","last_synced_at":"2025-03-28T09:06:06.944Z","etag":null,"topics":["humanoids","imitation-learning","reinforcement-learning","robotics"],"latest_commit_sha":null,"homepage":"https://humanoid-ai.github.io/","language":"Python","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/MarkFzp.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-06-12T23:51:27.000Z","updated_at":"2025-03-28T08:00:58.000Z","dependencies_parsed_at":"2024-06-13T20:04:54.700Z","dependency_job_id":"64cd7dc2-6a97-4b1e-86ab-9ebc2e396f0e","html_url":"https://github.com/MarkFzp/humanplus","commit_stats":null,"previous_names":["markfzp/humanplus"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MarkFzp%2Fhumanplus","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MarkFzp%2Fhumanplus/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MarkFzp%2Fhumanplus/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MarkFzp%2Fhumanplus/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/MarkFzp","download_url":"https://codeload.github.com/MarkFzp/humanplus/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247157079,"owners_count":20893202,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["humanoids","imitation-learning","reinforcement-learning","robotics"],"created_at":"2024-11-13T16:56:39.468Z","updated_at":"2025-04-04T10:02:54.835Z","avatar_url":"https://github.com/MarkFzp.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# HumanPlus: Humanoid Shadowing and Imitation from Humans\n\n\n#### Project Website: https://humanoid-ai.github.io/\n\nThis repository contains the updating implementation for the Humanoid Shadowing Transformer (HST) and the Humanoid Imitation Transformer (HIT), along with instructions for whole-body pose estimation and the associated hardware codebase.\n\n\n## Humanoid Shadowing Transformer (HST)\nReinforcement learning in simulation is based on [legged_gym](https://github.com/leggedrobotics/legged_gym) and [rsl_rl](https://github.com/leggedrobotics/rsl_rl).\n#### Installation\nInstall IsaacGym v4 first from the [official source](https://developer.nvidia.com/isaac-gym). Place the isaacgym fold inside the HST folder.\n\n    cd HST/rsl_rl \u0026\u0026 pip install -e . \n    cd HST/legged_gym \u0026\u0026 pip install -e .\n\n#### Example Usages\nTo train HST:\n\n    python legged_gym/scripts/train.py --run_name 0001_test --headless --sim_device cuda:0 --rl_device cuda:0\n\nTo play a trained policy:\n\n    python legged_gym/scripts/play.py --run_name 0001_test --checkpoint -1 --headless --sim_device cuda:0 --rl_device cuda:0\n\n\n## Humanoid Imitation Transformer (HIT)\nImitation learning in the real world is based on [ACT repo](https://github.com/tonyzhaozh/act) and [Mobile ALOHA repo](https://github.com/MarkFzp/act-plus-plus).\n#### Installation\n    conda create -n HIT python=3.8.10\n    conda activate HIT\n    pip install torchvision\n    pip install torch\n    pip install pyquaternion\n    pip install pyyaml\n    pip install rospkg\n    pip install pexpect\n    pip install mujoco==2.3.7\n    pip install dm_control==1.0.14\n    pip install opencv-python\n    pip install matplotlib\n    pip install einops\n    pip install packaging\n    pip install h5py\n    pip install ipython\n    pip install getkey\n    pip install wandb\n    pip install chardet\n    pip install h5py_cache\n    cd HIT/detr \u0026\u0026 pip install -e .\n#### Example Usages\nCollect your own data or download our dataset from [here](https://drive.google.com/drive/folders/1i3eGTd9Nl_tSieoE0grxuKqUAumBr2EV?usp=drive_link) and place it in the HIT folder.\n\nTo set up a new terminal, run:\n\n    conda activate HIT\n    cd HIT\n\nTo train HIT:\n\n    # Fold Clothes task\n    python imitate_episodes_h1_train.py --task_name data_fold_clothes --ckpt_dir fold_clothes/ --policy_class HIT --chunk_size 50 --hidden_dim 512 --batch_size 48 --dim_feedforward 512 --lr 1e-5 --seed 0 --num_steps 100000 --eval_every 100000 --validate_every 1000 --save_every 10000 --no_encoder --backbone resnet18 --same_backbones --use_pos_embd_image 1 --use_pos_embd_action 1 --dec_layers 6 --gpu_id 0 --feature_loss_weight 0.005 --use_mask --data_aug --wandb\n\n## Hardware Codebase\nHardware codebase is based on [unitree_ros2](https://github.com/unitreerobotics/unitree_ros2).\n\n#### Installation\n\ninstall [unitree_sdk](https://github.com/unitreerobotics/unitree_sdk2)\n\ninstall [unitree_ros2](https://support.unitree.com/home/en/developer/ROS2_service)\n\n    conda create -n lowlevel python=3.8\n    conda activate lowlevel\n\ninstall [nvidia-jetpack](https://docs.nvidia.com/jetson/archives/jetpack-archived/jetpack-461/install-jetpack/index.html)\n\ninstall torch==1.11.0 and torchvision==0.12.0:  \nplease refer to the following links:   \nhttps://forums.developer.nvidia.com/t/pytorch-for-jetson/72048\nhttps://docs.nvidia.com/deeplearning/frameworks/install-pytorch-jetson-platform/index.html\n\n#### Example Usages\nPut your trained policy in the `hardware-script/ckpt` folder and rename it to `policy.pt`\n\n    conda activate lowlevel\n    cd hardware-script\n    python hardware_whole_body.py --task_name stand\n\n\n## Pose Estimation\nFor body pose estimation, please refer to [WHAM](https://github.com/yohanshin/WHAM). \nFor hand pose estimation, please refer to [HaMeR](https://github.com/geopavlakos/hamer). \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmarkfzp%2Fhumanplus","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmarkfzp%2Fhumanplus","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmarkfzp%2Fhumanplus/lists"}