{"id":13656112,"url":"https://github.com/google-research/mint","last_synced_at":"2025-04-05T10:08:25.392Z","repository":{"id":43031269,"uuid":"368958866","full_name":"google-research/mint","owner":"google-research","description":"Multi-modal Content Creation Model Training Infrastructure including the FACT model (AI Choreographer) implementation. ","archived":false,"fork":false,"pushed_at":"2022-03-02T09:06:52.000Z","size":191,"stargazers_count":529,"open_issues_count":50,"forks_count":89,"subscribers_count":13,"default_branch":"main","last_synced_at":"2025-03-29T09:10:04.970Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/google-research.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-05-19T18:03:08.000Z","updated_at":"2025-03-25T14:42:03.000Z","dependencies_parsed_at":"2022-07-08T03:03:51.640Z","dependency_job_id":null,"html_url":"https://github.com/google-research/mint","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google-research%2Fmint","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google-research%2Fmint/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google-research%2Fmint/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google-research%2Fmint/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/google-research","download_url":"https://codeload.github.com/google-research/mint/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247318744,"owners_count":20919484,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-02T04:00:51.231Z","updated_at":"2025-04-05T10:08:25.369Z","avatar_url":"https://github.com/google-research.png","language":"Python","funding_links":[],"categories":["Papers","其他_机器视觉"],"sub_categories":["Music-Driven motion generation","网络服务_其他"],"readme":"# AI Choreographer: Music Conditioned 3D Dance Generation with AIST++ [ICCV-2021].\n\n## Overview\n\nThis package contains the model implementation and training infrastructure of\nour AI Choreographer. \n\n## Get started\n\n#### Pull the code\n```\ngit clone https://github.com/liruilong940607/mint --recursive\n```\nNote here `--recursive` is important as it will automatically clone the submodule ([orbit](https://github.com/tensorflow/models/tree/master/orbit)) as well.\n\n#### Install dependencies\n```\nconda create -n mint python=3.7\nconda activate mint\nconda install protobuf numpy\npip install tensorflow absl-py tensorflow-datasets librosa\n\nsudo apt-get install libopenexr-dev\npip install --upgrade OpenEXR\npip install tensorflow-graphics tensorflow-graphics-gpu\n\ngit clone https://github.com/arogozhnikov/einops /tmp/einops\ncd /tmp/einops/ \u0026\u0026 pip install . -U\n\ngit clone https://github.com/google/aistplusplus_api /tmp/aistplusplus_api\ncd /tmp/aistplusplus_api \u0026\u0026 pip install -r requirements.txt \u0026\u0026 pip install . -U\n```\nNote if you meet environment conflicts about numpy, you can try with `pip install numpy==1.20`. \n\n#### Get the data\nSee the [website](https://google.github.io/aistplusplus_dataset/)\n\n#### Get the checkpoint\nDownload from google drive [here](https://drive.google.com/drive/folders/17GHwKRZbQfyC9-7oEpzCG8pp_rAI0cOm?usp=sharing), and put them to the folder `./checkpoints/`\n\n#### Run the code\n\n1. complie protocols\n```\nprotoc ./mint/protos/*.proto\n```\n\n2. preprocess dataset into tfrecord\n```\npython tools/preprocessing.py \\\n    --anno_dir=\"/mnt/data/aist_plusplus_final/\" \\\n    --audio_dir=\"/mnt/data/AIST/music/\" \\\n    --split=train\npython tools/preprocessing.py \\\n    --anno_dir=\"/mnt/data/aist_plusplus_final/\" \\\n    --audio_dir=\"/mnt/data/AIST/music/\" \\\n    --split=testval\n```\n\n3. run training\n```\npython trainer.py --config_path ./configs/fact_v5_deeper_t10_cm12.config --model_dir ./checkpoints\n```\nNote you might want to change the `batch_size` in the config file if you meet OUT-OF-MEMORY issue.\n\n4. run testing and evaluation\n```\n# caching the generated motions (seed included) to `./outputs`\npython evaluator.py --config_path ./configs/fact_v5_deeper_t10_cm12.config --model_dir ./checkpoints\n# calculate FIDs\npython tools/calculate_scores.py\n```\n\n\n## Citation\n\n```bibtex\n@inproceedings{li2021dance,\n  title={AI Choreographer: Music Conditioned 3D Dance Generation with AIST++},\n  author={Ruilong Li and Shan Yang and David A. Ross and Angjoo Kanazawa},\n  booktitle = {The IEEE International Conference on Computer Vision (ICCV)},\n  year = {2021}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgoogle-research%2Fmint","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgoogle-research%2Fmint","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgoogle-research%2Fmint/lists"}