{"id":31211519,"url":"https://github.com/cyberagentailab/sprite-decompose","last_synced_at":"2025-09-21T05:30:27.155Z","repository":{"id":252016754,"uuid":"837185920","full_name":"CyberAgentAILab/sprite-decompose","owner":"CyberAgentAILab","description":"Fast Sprite Decomposition from Animated Graphics [ECCV2024]","archived":false,"fork":false,"pushed_at":"2024-09-26T08:30:06.000Z","size":57740,"stargazers_count":31,"open_issues_count":0,"forks_count":3,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-09-10T07:42:50.625Z","etag":null,"topics":["eccv2024","pytorch","segmentation"],"latest_commit_sha":null,"homepage":"https://cyberagentailab.github.io/sprite-decompose","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/CyberAgentAILab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-08-02T11:54:26.000Z","updated_at":"2025-09-04T11:48:37.000Z","dependencies_parsed_at":"2024-08-26T05:41:06.531Z","dependency_job_id":null,"html_url":"https://github.com/CyberAgentAILab/sprite-decompose","commit_stats":null,"previous_names":["cyberagentailab/sprite-decompose"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/CyberAgentAILab/sprite-decompose","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CyberAgentAILab%2Fsprite-decompose","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CyberAgentAILab%2Fsprite-decompose/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CyberAgentAILab%2Fsprite-decompose/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CyberAgentAILab%2Fsprite-decompose/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/CyberAgentAILab","download_url":"https://codeload.github.com/CyberAgentAILab/sprite-decompose/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CyberAgentAILab%2Fsprite-decompose/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":276195627,"owners_count":25601152,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-21T02:00:07.055Z","response_time":72,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["eccv2024","pytorch","segmentation"],"created_at":"2025-09-21T05:30:25.799Z","updated_at":"2025-09-21T05:30:27.137Z","avatar_url":"https://github.com/CyberAgentAILab.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Fast Sprite Decomposition from Animated Graphics (ECCV2024)\n\nThis repository is an official implementation of the paper \"Fast Sprite Decomposition from Animated Graphics\".\nPlease also refer to the [project page](https://CyberAgentAILab.github.io/sprite-decompose), [paper](https://arxiv.org/abs/2408.03923), and our [Crello Animation](https://huggingface.co/datasets/cyberagent/crello-animation) dataset.\n\n\u003cimg src=\"./assets/teaser.png\"\u003e\n\n## Setup\n\nWe check the reproducibility under this environment.\n\n- Python 3.10\n- CUDA 11.3\n\nAfter constructing Python environment, install this project with the following command:\n\n```bash\npip install -e fsd\n```\n\n## Decomposition\n\n### Sample-level\n\nYou can decompose a video by running the following command:\n\n```bash\n./scripts/optimize_sample.sh \\\n    in_dir=\"./inputs/62a87468079dcd93634632e7\" \\\n    out_dir=\"./_outputs/62a87468079dcd93634632e7\" \\\n    engine.n_iters=60000 \\\n    device=cuda\n```\n\nWe provide some examples of inputs (video and foreground box) from [Crello Animation](https://huggingface.co/datasets/cyberagent/crello-animation) in [`./inputs`](./inputs/) and corresponding outputs (sprites) in [`./outputs`](./outputs/).\nYou can specify the input data by changing the `in_dir` argument.\n\nWe manage configurations using [hydra](https://github.com/facebookresearch/hydra) and you can override the [default configuration](https://github.com/CyberAgentAILab/video-layer-decomposition/blob/publication/fsd/fsd/configs/crello_sample.yaml) (our best) by specifying the arguments. We illustrate some useful arguments to adjust your optimization as follows:\n\n- `engine.n_iters`: number of iterations for optimization.\n- `engine.save_interval`: iteration interval for saving the optimization progress.\n- `engine.eval_interval`: iteration interval for evaluating the decomposition quality, which is logged in `progress.json`.\n- `engine.timelimit_minute`: time limit for each optimization in minutes. If the optimization exceeds this limit, it will be terminated even if the number of iterations is not reached.\n- `engine.resume`: if `True`, resume the optimization from the last checkpoint in `out_dir`.\n- `device`: device to run the optimization. You can specify `cuda` or `cpu`.\n\nOutputs are stored in the following directory structure:\n\n```bash\n{out_dir}\n├── sprites  # Decomposed sprites\n│   ├── iter_*\n│   │   ├── params.json  # Parameters including affine matrices and opacity\n│   │   └── textures  # Textures\n│   │       ├── 0.png\n│   │       ├── 1.png\n│   │       └── 2.png\n│   └── iter_*.png  # Output visualization\n├── last_checkpoint.pth  # Last checkpoint of optimization\n└── progress.json   # Progress log including evaluation results and elapsed time\n```\n\nYou can refer to `./inputs/{VIDEO_ID}/{VIDEO_ID}.png` and `./outputs/{VIDEO_ID}/animation/iter_*.png` for a brief understanding of the decomposition result. Below is an example:\n\nInput:\n\n\u003cimg src=\"./inputs/6462144da2e859a5ba334c50/6462144da2e859a5ba334c50.png\" height=\"70px\"\u003e\n\nOutput visualization:\n\n\u003cimg src=\"./outputs/6462144da2e859a5ba334c50/sprites/iter_000047542.png\" height=\"70px\"\u003e\nThe leftmost is the reconstruction result, the next is the background, and the rest are sprites.\n\n### on Crello Animation\n\nYou can also conduct decomposition and evaluation on [Crello Animation](https://huggingface.co/datasets/cyberagent/crello-animation) by running the following command:\n\n```bash\n./scripts/optimize.sh \\\n    out_dir=\"./_outputs/crello-animation/\" \\\n    data.split=\"val\" \\\n    device=cuda\n```\n\n`data.split` can be `val` or `test`.\nYou will obtain the same output structure as the sample-level decomposition for all samples in Crello Animation.\n\n## Citation\n\nIf you find this code useful for your research, please cite our paper.\n\n```bibtex\n@inproceedings{suzuki2024fast,\n  title={Fast Sprite Decomposition from Animated Graphics},\n  author={Suzuki, Tomoyuki and Kikuchi, Kotaro and Yamaguchi, Kota},\n  booktitle={ECCV},\n  year={2024}\n}\n```\n\n### Acknowledgments\n\nThanks to the following repositories that we referred to in part: [Track-Anything](https://github.com/gaomingqi/Track-Anything/tree/master) and [deformable-sprites](https://github.com/vye16/deformable-sprites)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcyberagentailab%2Fsprite-decompose","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcyberagentailab%2Fsprite-decompose","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcyberagentailab%2Fsprite-decompose/lists"}