{"id":13405591,"url":"https://github.com/facebookresearch/AnimatedDrawings","last_synced_at":"2025-03-14T10:31:01.495Z","repository":{"id":66082551,"uuid":"572668924","full_name":"facebookresearch/AnimatedDrawings","owner":"facebookresearch","description":"Code to accompany \"A Method for Animating Children's Drawings of the Human Figure\"","archived":false,"fork":false,"pushed_at":"2024-08-09T16:40:00.000Z","size":113074,"stargazers_count":10726,"open_issues_count":15,"forks_count":951,"subscribers_count":90,"default_branch":"main","last_synced_at":"2024-12-17T01:37:07.304Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/facebookresearch.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-11-30T19:16:21.000Z","updated_at":"2024-12-17T01:31:36.000Z","dependencies_parsed_at":"2023-07-18T12:00:28.477Z","dependency_job_id":"f1c529d8-a72b-4b74-863f-3e3749b953cf","html_url":"https://github.com/facebookresearch/AnimatedDrawings","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/facebookresearch%2FAnimatedDrawings","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/facebookresearch%2FAnimatedDrawings/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/facebookresearch%2FAnimatedDrawings/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/facebookresearch%2FAnimatedDrawings/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/facebookresearch","download_url":"https://codeload.github.com/facebookresearch/AnimatedDrawings/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243561924,"owners_count":20311200,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-30T19:02:06.122Z","updated_at":"2025-03-14T10:31:01.489Z","avatar_url":"https://github.com/facebookresearch.png","language":"Python","funding_links":[],"categories":["Python","其他_机器视觉","Repos","others","🤖 AI \u0026 LLM (人工智能与大模型)"],"sub_categories":["网络服务_其他"],"readme":"# Animated Drawings\n\n![Sequence 02](https://user-images.githubusercontent.com/6675724/219223438-2c93f9cb-d4b5-45e9-a433-149ed76affa6.gif)\n\n\nThis repo contains an implementation of the algorithm described in the paper, [A Method for Animating Children's Drawings of the Human Figure](https://dl.acm.org/doi/10.1145/3592788).\n\nIn addition, this repo aims to be a useful creative tool in its own right, allowing you to flexibly create animations starring your own drawn characters. If you do create something fun with this, let us know! Use hashtag **#FAIRAnimatedDrawings**, or tag me on twitter: [@hjessmith](https://twitter.com/hjessmith/).\n\nProject website: [http://www.fairanimateddrawings.com](http://www.fairanimateddrawings.com)\n\nVideo overview of [Animated Drawings OS Project](https://www.youtube.com/watch?v=WsMUKQLVsOI)\n\n\n## Installation\n*This project has been tested with macOS Ventura 13.2.1 and Ubuntu 18.04. If you're installing on another operating system, you may encounter issues.*\n\nWe *strongly* recommend activating a Python virtual environment prior to installing Animated Drawings.\nConda's Miniconda is a great choice. Follow [these steps](https://conda.io/projects/conda/en/stable/user-guide/install/index.html) to download and install it. Then run the following commands:\n\n````bash\n    # create and activate the virtual environment\n    conda create --name animated_drawings python=3.8.13\n    conda activate animated_drawings\n\n    # clone AnimatedDrawings and use pip to install\n    git clone https://github.com/facebookresearch/AnimatedDrawings.git\n    cd AnimatedDrawings\n    pip install -e .\n````\n\nMac M1/M2 users: if you get architecture errors, make sure your `~/.condarc` does not have `osx-64`, but only `osx-arm64` and `noarch` in its subdirs listing. You can see that it's going to go sideways as early as `conda create` because it will show `osx-64` instead of `osx-arm64` versions of libraries under \"The following NEW packages will be INSTALLED\".\n\n## Using Animated Drawings\n\n### Quick Start\nNow that everything's set up, let's animate some drawings! To get started, follow these steps:\n1. Open a terminal and activate the animated_drawings conda environment:\n````bash\n~ % conda activate animated_drawings\n````\n\n2. Ensure you're in the root directory of AnimatedDrawings:\n````bash\n(animated_drawings) ~ % cd {location of AnimatedDrawings on your computer}\n````\n\n3. Start up a Python interpreter:\n````bash\n(animated_drawings) AnimatedDrawings % python\n````\n\n4. Copy and paste the follow two lines into the interpreter:\n````python\nfrom animated_drawings import render\nrender.start('./examples/config/mvc/interactive_window_example.yaml')\n````\n\nIf everything is installed correctly, an interactive window should appear on your screen.\n(Use spacebar to pause/unpause the scene, arrow keys to move back and forth in time, and q to close the screen.)\n\n\u003cimg src='./media/interactive_window_example.gif' width=\"256\" height=\"256\" /\u003e \u003c/br\u003e\u003c/br\u003e\u003c/br\u003e\n\nThere's a lot happening behind the scenes here. Characters, motions, scenes, and more are all controlled by configuration files, such as `interactive_window_example.yaml`. Below, we show how different effects can be achieved by varying the config files. You can learn more about the [config files here](examples/config/README.md).\n\n### Export MP4 video\n\nSuppose you'd like to save the animation as a video file instead of viewing it directly in a window. Specify a different example config by copying these lines into the Python interpreter:\n\n````python\nfrom animated_drawings import render\nrender.start('./examples/config/mvc/export_mp4_example.yaml')\n````\n\nInstead of an interactive window, the animation was saved to a file, video.mp4, located in the same directory as your script.\n\n\u003cimg src='./media/mp4_export_video.gif' width=\"256\" height=\"256\" /\u003e \u003c/br\u003e\u003c/br\u003e\u003c/br\u003e\n\n### Export transparent .gif\n\nPerhaps you'd like a transparent .gif instead of an .mp4? Copy these lines in the Python interpreter instead:\n\n````python\nfrom animated_drawings import render\nrender.start('./examples/config/mvc/export_gif_example.yaml')\n````\n\nInstead of an interactive window, the animation was saved to a file, video.gif, located in the same directory as your script.\n\n\u003cimg src='./media/gif_export_video.gif' width=\"256\" height=\"256\" /\u003e \u003c/br\u003e\u003c/br\u003e\u003c/br\u003e\n\n### Headless Rendering\n\nIf you'd like to generate a video headlessly (e.g. on a remote server accessed via ssh), you'll need to specify `USE_MESA: True` within the `view` section of the config file.\n\n````yaml\n    view:\n      USE_MESA: True\n````\n\n### Animating Your Own Drawing\n\nAll of the examples above use drawings with pre-existing annotations.\nTo understand what we mean by *annotations* here, look at one of the 'pre-rigged' character's [annotation files](examples/characters/char1/).\nYou can use whatever process you'd like to create those annotations files and, as long as they are valid, AnimatedDrawings will give you an animation.\n\nSo you'd like to animate your own drawn character.\nI wouldn't want you to create those annotation files manually. That would be tedious.\nTo make it fast and easy, we've trained a drawn humanoid figure detector and pose estimator and provided scripts to automatically generate annotation files from the model predictions.\nThere are currently two options for setting this up.\n\n#### Option 1: Docker\nTo get it working, you'll need to set up a Docker container that runs TorchServe.\nThis allows us to quickly show your image to our machine learning models and receive their predictions.\n\nTo set up the container, follow these steps:\n\n1. [Install Docker Desktop](https://docs.docker.com/get-docker/)\n2. Ensure Docker Desktop is running.\n3. Run the following commands, starting from the Animated Drawings root directory:\n\n````bash\n    (animated_drawings) AnimatedDrawings % cd torchserve\n\n    # build the docker image... this takes a while (~5-7 minutes on Macbook Pro 2021)\n    (animated_drawings) torchserve % docker build -t docker_torchserve .\n\n    # start the docker container and expose the necessary ports\n    (animated_drawings) torchserve % docker run -d --name docker_torchserve -p 8080:8080 -p 8081:8081 docker_torchserve\n````\n\nWait ~10 seconds, then ensure Docker and TorchServe are working by pinging the server:\n\n````bash\n    (animated_drawings) torchserve % curl http://localhost:8080/ping\n\n    # should return:\n    # {\n    #   \"status\": \"Healthy\"\n    # }\n````\n\nIf, after waiting, the response is `curl: (52) Empty reply from server`, one of two things is likely happening.\n1. Torchserve hasn't finished initializing yet, so wait another 10 seconds and try again.\n2. Torchserve is failing because it doesn't have enough RAM.  Try [increasing the amount of memory available to your Docker containers](https://docs.docker.com/desktop/settings/mac/#advanced) to 16GB by modifying Docker Desktop's settings.\n\nWith that set up, you can now go directly from image -\u003e animation with a single command:\n\n````bash\n    (animated_drawings) torchserve % cd ../examples\n    (animated_drawings) examples % python image_to_animation.py drawings/garlic.png garlic_out\n````\n\nAs you waited, the image located at `drawings/garlic.png` was analyzed, the character detected, segmented, and rigged, and it was animated using BVH motion data from a human actor.\nThe resulting animation was saved as `./garlic_out/video.gif`.\n\n\u003cimg src='./examples/drawings/garlic.png' height=\"256\" /\u003e\u003cimg src='./media/garlic.gif' width=\"256\" height=\"256\" /\u003e\u003c/br\u003e\u003c/br\u003e\u003c/br\u003e\n\n#### Option 2: Running locally on macOS\n\nGetting Docker working can be complicated, and it's unnecessary if you just want to play around with this locally.\nContributer @Gravityrail kindly submitted a script that sets up Torchserve locally on MacOS, no Docker required.\n\n```bash\ncd torchserve\n./setup_macos.sh\ntorchserve --start --ts-config config.local.properties --foreground\n```\n\nWith torchserve running locally like this, you can use the same command as before to make the garlic dance:\n\n```bash \npython image_to_animation.py drawings/garlic.png garlic_out\n```\n### Fixing bad predictions\nYou may notice that, when you ran `python image_to_animation.py drawings/garlic.png garlic_out`, there were additional non-video files within `garlic_out`.\n`mask.png`, `texture.png`, and `char_cfg.yaml` contain annotation results of the image character analysis step. These annotations were created from our model predictions.\nIf the mask predictions are incorrect, you can edit the mask with an image editing program like Paint or Photoshop.\nIf the joint predictions are incorrect, you can run `python fix_annotations.py` to launch a web interface to visualize, correct, and update the annotations. Pass it the location of the folder containing incorrect joint predictions (here we use `garlic_out/` as an example):\n\n````bash\n    (animated_drawings) examples % python fix_annotations.py garlic_out/\n    ...\n     * Running on http://127.0.0.1:5050\n    Press CTRL+C to quit\n````\n\nNavigate to `http://127.0.0.1:5050` in your browser to access the web interface. Drag the joints into the appropriate positions, and hit `Submit` to save your edits.\n\nOnce you've modified the annotations, you can render an animation using them like so:\n\n````bash\n    # specify the folder where the fixed annoations are located\n    (animated_drawings) examples % python annotations_to_animation.py garlic_out\n````\n\n### Adding multiple characters to scene\nMultiple characters can be added to a video by specifying multiple entries within the config scene's 'ANIMATED_CHARACTERS' list.\nTo see for yourself, run the following commands from a Python interpreter within the AnimatedDrawings root directory:\n\n````python\nfrom animated_drawings import render\nrender.start('./examples/config/mvc/multiple_characters_example.yaml')\n````\n\u003cimg src='./examples/characters/char1/texture.png' height=\"256\" /\u003e \u003cimg src='./examples/characters/char2/texture.png' height=\"256\" /\u003e \u003cimg src='./media/multiple_characters_example.gif' height=\"256\" /\u003e\n\n### Adding a background image\nSuppose you'd like to add a background to the animation. You can do so by specifying the image path within the config.\nRun the following commands from a Python interpreter within the AnimatedDrawings root directory:\n\n````python\nfrom animated_drawings import render\nrender.start('./examples/config/mvc/background_example.yaml')\n````\n\n\u003cimg src='./examples/characters/char4/texture.png' height=\"256\" /\u003e \u003cimg src='./examples/characters/char4/background.png' height=\"256\" /\u003e \u003cimg src='./media/background_example.gif' height=\"256\" /\u003e\n\n### Using BVH Files with Different Skeletons\nYou can use any motion clip you'd like, as long as it is in BVH format.\n\nIf the BVH's skeleton differs from the examples used in this project, you'll need to create a new motion config file and retarget config file.\nOnce you've done that, you should be good to go.\nThe following code and resulting clip uses a BVH with completely different skeleton.\nRun the following commands from a Python interpreter within the AnimatedDrawings root directory:\n\n````python\nfrom animated_drawings import render\nrender.start('./examples/config/mvc/different_bvh_skeleton_example.yaml')\n````\n\n\u003cimg src='./media/different_bvh_skeleton_example.gif' height=\"256\" /\u003e\n\n### Creating Your Own BVH Files\nYou may be wondering how you can create BVH files of your own.\nYou used to need a motion capture studio.\nBut now, thankfully, there are simple and accessible options for getting 3D motion data from a single RGB video.\nFor example, I created this Readme's banner animation by:\n1. Recording myself doing a silly dance with my phone's camera.\n2. Using [Rokoko](https://www.rokoko.com/) to export a BVH from my video.\n3. Creating a new [motion config file](examples/config/README.md#motion) and [retarget config file](examples/config/README.md#retarget) to fit the skeleton exported by Rokoko.\n4. Using AnimatedDrawings to animate the characters and export a transparent animated gif.\n5. Combining the animated gif, original video, and original drawings in Adobe Premiere.\n\u003cimg src='https://user-images.githubusercontent.com/6675724/219223438-2c93f9cb-d4b5-45e9-a433-149ed76affa6.gif' height=\"256\" /\u003e\n\nHere is an example of the configs I used apply my motion to a character. To use these config files, ensure that the Rokoko exports the BVH with the Mixamo skeleton preset:\n\n ````python\nfrom animated_drawings import render\nrender.start('./examples/config/mvc/rokoko_motion_example.yaml')\n ````\n\nIt will show this in a new window:\n\n![Sequence 01](https://user-images.githubusercontent.com/6675724/233157474-1506d219-c085-49f9-a537-43d6c1bae93a.gif)\n\n\n\n\n### Adding Addition Character Skeletons\nAll of the example animations above depict \"human-like\" characters; they have two arms and two legs.\nOur method is primarily designed with these human-like characters in mind, and the provided pose estimation model assumes a human-like skeleton is present.\nBut you can manually specify a different skeletons within the `character config` and modify the specified `retarget config` to support it.\nIf you're interested, look at the configuration files specified in the two examples below.\n\n\n````python\nfrom animated_drawings import render\nrender.start('./examples/config/mvc/six_arms_example.yaml')\n````\n\n\u003cimg src='https://user-images.githubusercontent.com/6675724/223584962-925ee5aa-11de-47e5-ace2-a6d5940b34ae.png' height=\"256\" /\u003e\u003cimg src='https://user-images.githubusercontent.com/6675724/223585000-dc8acf4e-974d-4cae-998b-94543f5f42c8.gif' width=\"256\" height=\"256\" /\u003e\u003c/br\u003e\u003c/br\u003e\u003c/br\u003e\n\n````python\nfrom animated_drawings import render\nrender.start('./examples/config/mvc/four_legs_example.yaml')\n````\n\n\u003cimg src='https://user-images.githubusercontent.com/6675724/223585033-f11e4e66-0443-405a-80e5-09b6aa0e335d.png' height=\"256\" /\u003e\u003cimg src='https://user-images.githubusercontent.com/6675724/223585043-7ce9eac0-bb4c-4547-b038-c63ca2852ef2.gif' width=\"256\" height=\"256\" /\u003e\u003c/br\u003e\u003c/br\u003e\u003c/br\u003e\n\n### Creating Your Own Config Files\nIf you want to create your own config files, see the [configuration file documentation](examples/config/README.md).\n\n## Browser-Based Demo\n\nIf you'd like to animate a drawing of your own, but don't want to deal with downloading code and using the command line, check out our browser-based demo:\n\n[www.sketch.metademolab.com](https://sketch.metademolab.com/)\n\n## Paper \u0026 Citation\n If you find the resources in this repo helpful, please consider citing the accompanying paper, [A Method for Animating Children's Drawings of The Human Figure](https://dl.acm.org/doi/10.1145/3592788)).\n\nCitation:\n\n```\n@article{10.1145/3592788,\nauthor = {Smith, Harrison Jesse and Zheng, Qingyuan and Li, Yifei and Jain, Somya and Hodgins, Jessica K.},\ntitle = {A Method for Animating Children’s Drawings of the Human Figure},\nyear = {2023},\nissue_date = {June 2023},\npublisher = {Association for Computing Machinery},\naddress = {New York, NY, USA},\nvolume = {42},\nnumber = {3},\nissn = {0730-0301},\nurl = {https://doi.org/10.1145/3592788},\ndoi = {10.1145/3592788},\nabstract = {Children’s drawings have a wonderful inventiveness, creativity, and variety to them. We present a system that automatically animates children’s drawings of the human figure, is robust to the variance inherent in these depictions, and is simple and straightforward enough for anyone to use. We demonstrate the value and broad appeal of our approach by building and releasing the Animated Drawings Demo, a freely available public website that has been used by millions of people around the world. We present a set of experiments exploring the amount of training data needed for fine-tuning, as well as a perceptual study demonstrating the appeal of a novel twisted perspective retargeting technique. Finally, we introduce the Amateur Drawings Dataset, a first-of-its-kind annotated dataset, collected via the public demo, containing over 178,000 amateur drawings and corresponding user-accepted character bounding boxes, segmentation masks, and joint location annotations.},\njournal = {ACM Trans. Graph.},\nmonth = {jun},\narticleno = {32},\nnumpages = {15},\nkeywords = {2D animation, motion retargeting, motion stylization, Skeletal animation}\n}\n```\n\n## Amateur Drawings Dataset\n\nTo obtain the Amateur Drawings Dataset, run the following two commands from the command line:\n\n````bash\n# download annotations (~275Mb)\nwget https://dl.fbaipublicfiles.com/amateur_drawings/amateur_drawings_annotations.json\n\n# download images (~50Gb)\nwget https://dl.fbaipublicfiles.com/amateur_drawings/amateur_drawings.tar\n````\n\nIf you have feedback about the dataset, please fill out [this form](https://forms.gle/kE66yskh9uhtLbFz9).\n\n## Trained Model Weights\n\nTrained model weights for human-like figure detection and pose estimation are included in the [repo releases](https://github.com/facebookresearch/AnimatedDrawings/releases). Model weights are released under [MIT license](https://github.com/facebookresearch/AnimatedDrawings/blob/main/LICENSE). The .mar files were generated using the OpenMMLab framework ([OpenMMDet Apache 2.0 License](https://github.com/open-mmlab/mmdetection/blob/main/LICENSE), [OpenMMPose Apache 2.0 License](https://github.com/open-mmlab/mmpose/blob/main/LICENSE))\n\n## As-Rigid-As-Possible Shape Manipulation\n\nThese characters are deformed using [As-Rigid-As-Possible (ARAP) shape manipulation](https://www-ui.is.s.u-tokyo.ac.jp/~takeo/papers/takeo_jgt09_arapFlattening.pdf).\nWe have a Python implementation of the algorithm, located [here](https://github.com/fairinternal/AnimatedDrawings/blob/main/animated_drawings/model/arap.py), that might be of use to other developers.\n\n## License\nAnimated Drawings is released under the [MIT license](https://github.com/fairinternal/AnimatedDrawings/blob/main/LICENSE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffacebookresearch%2FAnimatedDrawings","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffacebookresearch%2FAnimatedDrawings","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffacebookresearch%2FAnimatedDrawings/lists"}