{"id":18303574,"url":"https://github.com/ericguo5513/text-to-motion","last_synced_at":"2025-04-05T02:12:36.641Z","repository":{"id":41237039,"uuid":"469479246","full_name":"EricGuo5513/text-to-motion","owner":"EricGuo5513","description":"Official implementation for \"Generating Diverse and Natural 3D Human Motions from Texts (CVPR2022).\"","archived":false,"fork":false,"pushed_at":"2024-08-18T00:21:07.000Z","size":19180,"stargazers_count":465,"open_issues_count":20,"forks_count":40,"subscribers_count":12,"default_branch":"main","last_synced_at":"2024-10-14T18:02:42.298Z","etag":null,"topics":["motion-generation","pytorch-implementation"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/EricGuo5513.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-03-13T19:53:59.000Z","updated_at":"2024-10-13T17:51:39.000Z","dependencies_parsed_at":"2024-08-18T01:37:51.482Z","dependency_job_id":null,"html_url":"https://github.com/EricGuo5513/text-to-motion","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EricGuo5513%2Ftext-to-motion","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EricGuo5513%2Ftext-to-motion/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EricGuo5513%2Ftext-to-motion/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EricGuo5513%2Ftext-to-motion/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/EricGuo5513","download_url":"https://codeload.github.com/EricGuo5513/text-to-motion/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247276189,"owners_count":20912288,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["motion-generation","pytorch-implementation"],"created_at":"2024-11-05T15:26:06.669Z","updated_at":"2025-04-05T02:12:36.626Z","avatar_url":"https://github.com/EricGuo5513.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Generating Diverse and Natural 3D Human Motions from Text (CVPR 2022)\n## [[Project Page]](https://ericguo5513.github.io/text-to-motion) [[Paper]](https://openaccess.thecvf.com/content/CVPR2022/papers/Guo_Generating_Diverse_and_Natural_3D_Human_Motions_From_Text_CVPR_2022_paper.pdf)\n\n![teaser_image](https://github.com/EricGuo5513/text-to-motion/blob/main/docs/teaser_image.png)\n  \n  Given a textual description for example, *\"the figure rises from a lying position and walks in a counterclockwise circle, and then lays back down the ground\"*, our approach generates a diverse set of 3d human motions that are faithful to the provided text.\n  \n## Python Virtual Environment\n\nAnaconda is recommended to create this virtual environment.\n  \n  ```sh\n  conda create -f environment.yaml\n  conda activate text2motion_pub\n  ```\n  \nIf you cannot successfully create the environment, here is a list of required libraries:\n  ```\n  Python = 3.7.9   # Other version may also work but are not tested.\n  PyTorch = 1.6.0 (conda install pytorch==1.6.0 torchvision==0.7.0 -c pytorch)  #Other version may also work but are not tested.\n  scipy\n  numpy\n  tensorflow       # For use of tensorboard only\n  spacy\n  tqdm\n  ffmpeg = 4.3.1   # Other version may also work but are not tested.\n  matplotlib = 3.3.1\n  ```\n  \n  After all, if you want to generate 3D motions from customized raw texts, you still need to install the language model for spacy. \n  ```sh\n  python -m spacy download en_core_web_sm\n  ```\n  \n  ## Download Data \u0026 Pre-trained Models\n  \n  **If you just want to play our pre-trained models, you don't need to download datasets.**\n  ### Datasets\n  We are using two 3D human motion-language dataset: HumanML3D and KIT-ML. For both datasets, you could find the details as well as download link [[here]](https://github.com/EricGuo5513/HumanML3D).   \n  Please note you don't need to clone that git repository, since all related codes have already been included in current git project.\n  \n  Download and unzip the dataset files -\u003e Create a dataset folder -\u003e Place related data files in dataset folder:\n  ```sh\n  mkdir ./dataset/\n  ```\n  Take HumanML3D for an example, the file directory should look like this:  \n  ```\n  ./dataset/\n  ./dataset/HumanML3D/\n  ./dataset/HumanML3D/new_joint_vecs/\n  ./dataset/HumanML3D/texts/\n  ./dataset/HumanML3D/Mean.mpy\n  ./dataset/HumanML3D/Std.npy\n  ./dataset/HumanML3D/test.txt\n  ./dataset/HumanML3D/train.txt\n  ./dataset/HumanML3D/train_val.txt\n  ./dataset/HumanML3D/val.txt  \n  ./dataset/HumanML3D/all.txt \n  ```\n ### Pre-trained Models\n  Create a checkpoint folder to place pre-traine models:\n  ```sh\n  mkdir ./checkpoints\n  ```\n    \n #### Download models for HumanML3D from [[here]](https://drive.google.com/file/d/1IgrFCnxeg4olBtURUHimzS03ZI0df_6W/view?usp=sharing). Unzip and place them under checkpoint directory, which should be like\n```\n./checkpoints/t2m/\n./checkpoints/t2m/Comp_v6_KLD01/           # Text-to-motion generation model\n./checkpoints/t2m/Decomp_SP001_SM001_H512/ # Motion autoencoder\n./checkpoints/t2m/length_est_bigru/        # Text-to-length sampling model\n./checkpoints/t2m/text_mot_match/          # Motion \u0026 Text feature extractors for evaluation\n ```\n #### Download models for KIT-ML [[here]](https://drive.google.com/file/d/12liZW5iyvoybXD8eOw4VanTgsMtynCuU/view?usp=drive_link). Unzip and place them under checkpoint directory.\n    \n ## Training Models\n \n All intermediate meta files/animations/models will be saved to checkpoint directory under the folder specified by argument \"--name\".\n ### Training motion autoencoder\n #### HumanML3D\n```sh\npython train_decomp_v3.py --name Decomp_SP001_SM001_H512 --gpu_id 0 --window_size 24 --dataset_name t2m\n```\n#### KIT-ML\n```sh\npython train_decomp_v3.py --name Decomp_SP001_SM001_H512 --gpu_id 0 --window_size 24 --dataset_name kit\n```\n\n### Train text2length model:\n#### HumanML3D\n```sh\npython train_length_est.py --name length_est_bigru --gpu_id 0 --dataset_name t2m\n```\n#### KIT-ML\n```sh\npython train_length_est.py --name length_est_bigru --gpu_id 0 --dataset_name kit\n```\n### Training text2motion model:\n#### HumanML3D\n```sh\npython train_comp_v6.py --name Comp_v6_KLD01 --gpu_id 0 --lambda_kld 0.01 --dataset_name t2m\n```\n#### KIT-ML\n```sh\npython train_comp_v6.py --name Comp_v6_KLD005 --gpu_id 0 --lambda_kld 0.005 --dataset_name kit\n```\n### Training motion \u0026 text feature extractors:\n#### HumanML3D\n```sh\npython train_tex_mot_match.py --name text_mot_match --gpu_id 1 --batch_size 8 --dataset_name t2m\n```\n#### KIT-ML\n```sh\npython train_tex_mot_match.py --name text_mot_match --gpu_id 1 --batch_size 8 --dataset_name kit\n```\n    \n## Generating and Animating 3D Motions (HumanML3D)\n#### Sampling results from test sets\n```sh\npython eval_comp_v6.py --name Comp_v6_KLD01 --est_length --repeat_time 3 --num_results 10 --ext default --gpu_id 1\n```\nwhere *--est_length* asks the model to use sampled motion lengths for generation, *--repeat_time* gives how many sampling rounds are carried out for each description. This script will results in 3x10 animations under directory *./eval_results/t2m/Comp_v6_KLD01/default/*.\n\n#### Sampling results from customized descriptions\n```sh\npython gen_motion_script.py --name Comp_v6_KLD01 --text_file input.txt --repeat_time 3 --ext customized --gpu_id 1\n```\nThis will generate 3 animated motions for each description given in text_file *./input.txt*.\n\nIf you find problem with installing ffmpeg, you may not be able to animate 3d results in mp4. Try gif instead.\n\n## Quantitative Evaluations\n```sh\npython final_evaluation.py \n```\nThis will evaluate the model performance on HumanML3D dataset by default. You could also run on KIT-ML dataset by uncommenting certain lines in *./final_evaluation.py*. The statistical results will saved to *./t2m_evaluation.log*.\n\n### Misc\n Contact Chuan Guo at cguo2@ualberta.ca for any questions or comments.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fericguo5513%2Ftext-to-motion","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fericguo5513%2Ftext-to-motion","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fericguo5513%2Ftext-to-motion/lists"}