{"id":17614314,"url":"https://github.com/sumith1896/motion2prog_release","last_synced_at":"2025-07-12T04:34:19.637Z","repository":{"id":41154136,"uuid":"379401181","full_name":"sumith1896/motion2prog_release","owner":"sumith1896","description":"This repository contains the code for the paper \"Hierarchical Motion Understanding via Motion Programs\"","archived":false,"fork":false,"pushed_at":"2021-07-16T20:19:55.000Z","size":28,"stargazers_count":44,"open_issues_count":2,"forks_count":2,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-04-30T17:03:17.313Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://sumith1896.github.io/motion2prog/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sumith1896.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-06-22T21:08:32.000Z","updated_at":"2024-01-29T10:14:27.000Z","dependencies_parsed_at":"2022-09-05T14:01:27.870Z","dependency_job_id":null,"html_url":"https://github.com/sumith1896/motion2prog_release","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/sumith1896/motion2prog_release","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sumith1896%2Fmotion2prog_release","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sumith1896%2Fmotion2prog_release/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sumith1896%2Fmotion2prog_release/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sumith1896%2Fmotion2prog_release/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sumith1896","download_url":"https://codeload.github.com/sumith1896/motion2prog_release/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sumith1896%2Fmotion2prog_release/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":264939597,"owners_count":23686208,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-22T18:43:26.868Z","updated_at":"2025-07-12T04:34:19.354Z","avatar_url":"https://github.com/sumith1896.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Hierarchical Motion Understanding via Motion Programs (CVPR 2021)\n\n[![Paper](https://img.shields.io/badge/arXiv-Paper-b31b1b.svg)](https://arxiv.org/abs/2104.11216)\n\nThis repository contains the official implementation of:\n\n**Hierarchical Motion Understanding via Motion Programs**\n\n[full paper](https://arxiv.org/abs/2104.11216) | [short talk](https://www.youtube.com/watch?v=EKP2BIRlaXQ) | [long talk](https://www.youtube.com/watch?v=OpyY-s0LKAs) | [project webpage](https://sumith1896.github.io/motion2prog/) \n\n![Motion Programs example](https://sumith1896.github.io/motion2prog/static/primf.png)\n\n## Running motion2prog\n\n**0. We start with video file and first prepare the input data**\n\n```sh\n$ ffmpeg -i ${video_dir}/video.mp4 ${video_dir}/frames/%05d.jpg\n$ python ${alphapose_dir}/scripts/demo_inference.py \\\n    --cfg ${alphapose_dir}/pretrained_models/256x192_res50_lr1e-3_1x.yaml \\\n    --checkpoint ${alphapose_dir}/pretrained_models/halpe26_fast_res50_256x192.pth \\\n    --indir ${video_dir}/frames --outdir ${video_dir}/pose_mpii_track \\\n    --pose_track --showbox --flip --qsize 256\n$ mv ${video_dir}/pose_mpii_track/alphapose-results.json \\\n    ${video_dir}/alphapose-results-halpe26-posetrack.json\n```\n\nWe packaged a demo video with necessary inputs for quickly testing our code\n\n```sh\n$ wget https://sumith1896.github.io/motion2prog/static/demo.zip\n$ mv demo.zip data/  \u0026\u0026 cd data/ \u0026\u0026 unzip demo.zip \u0026\u0026 cd ..\n```\n\n- We need 2D pose detection results \u0026 extracted frames of video (for visualization)\n\n- We support loading from different pose detector formats in the `load` function in `lkeypoints.py`.\n\n- We used `AlphaPose` with the above commands for all pose detection results.\n\n### Run motion program synthesis pipeline\n**1. With the data prepared, you can run the synthesis with the following command:**\n\n```sh\n$ python fit.py -d data/demo/276_reg -k coco -a -x -c -p 1 -w 20 --no-acc \\\n--stat-thres 5 --span-thres 5 --cores 9 -r 1600 -o ./visualization/static/data/demo\n```\n\n- The various options and their descriptions are explained in the `fit.py` file.\n\n- The results can be found under `./visualization/static/data/demo`.\n\n### Visualizing the synthesized programs\n**2. We package a visualization server for visualizing the generated programs**\n```sh\n$ cd visualization/\n$ bash deploy.sh p\n```\n\n- Open the directed the webpage and browse the results interactively.\n\n\n## Citations\nIf you find our code or paper useful to your research, please consider citing:\n\n```bibtex\n@inproceedings{motion2prog2021,\n    Author = {Sumith Kulal and Jiayuan Mao and Alex Aiken and Jiajun Wu},\n    Title = {Hierarchical Motion Understanding via Motion Programs},\n    booktitle={CVPR},\n    year={2021},\n}\n```\n\n## Checklist\nPlease open a GitHub issue or contact [sumith@cs.stanford.edu](sumith@cs.stanford.edu) for any issues or questions!\n\n- [ ] Upload pre-processed data used in paper.\n\n## Acknowledgements\nWe thank Karan Chadha, Shivam Garg and Shubham Goel for helpful discussions. This work is in part supported by Magic Grant from the Brown Institute for Media Innovation, the Samsung Global Research Outreach (GRO) Program, Autodesk, Amazon Web Services, and Stanford HAI for AWS Cloud Credits.\n\nParts of this repo use materials from [SCANimate](https://github.com/shunsukesaito/SCANimate) and [fit](https://github.com/trehansiddharth/fit).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsumith1896%2Fmotion2prog_release","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsumith1896%2Fmotion2prog_release","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsumith1896%2Fmotion2prog_release/lists"}