{"id":19256705,"url":"https://github.com/markmohr/jostc","last_synced_at":"2026-02-27T20:37:33.667Z","repository":{"id":231878435,"uuid":"782937870","full_name":"MarkMoHR/JoSTC","owner":"MarkMoHR","description":"[TOG \u0026 SIGGRAPH 2024] Joint Stroke Tracing and Correspondence for 2D Animation","archived":false,"fork":false,"pushed_at":"2024-07-02T13:58:06.000Z","size":14941,"stargazers_count":68,"open_issues_count":5,"forks_count":12,"subscribers_count":7,"default_branch":"main","last_synced_at":"2025-07-28T15:41:29.036Z","etag":null,"topics":["2d-animation","line-art","siggraph","sketch","stroke-correspondence"],"latest_commit_sha":null,"homepage":"https://markmohr.github.io/JoSTC/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/MarkMoHR.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-04-06T13:26:43.000Z","updated_at":"2025-07-25T03:31:50.000Z","dependencies_parsed_at":"2024-06-12T13:11:57.474Z","dependency_job_id":"f7a0d262-c214-4274-b941-8c8e79ee17c9","html_url":"https://github.com/MarkMoHR/JoSTC","commit_stats":null,"previous_names":["markmohr/jostc"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/MarkMoHR/JoSTC","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MarkMoHR%2FJoSTC","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MarkMoHR%2FJoSTC/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MarkMoHR%2FJoSTC/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MarkMoHR%2FJoSTC/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/MarkMoHR","download_url":"https://codeload.github.com/MarkMoHR/JoSTC/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MarkMoHR%2FJoSTC/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29912391,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-27T19:37:42.220Z","status":"ssl_error","status_checked_at":"2026-02-27T19:37:41.463Z","response_time":57,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["2d-animation","line-art","siggraph","sketch","stroke-correspondence"],"created_at":"2024-11-09T19:06:41.113Z","updated_at":"2026-02-27T20:37:33.637Z","avatar_url":"https://github.com/MarkMoHR.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Joint Stroke Tracing and Correspondence for 2D Animation - TOG \u0026 SIGGRAPH 2024\n\n[[Paper]](https://www.sysu-imsl.com/files/TOG2024/SketchTracing_TOG2024_personal.pdf) | [[Paper (ACM)]](https://dl.acm.org/doi/10.1145/3649890) | [[Project Page]](https://markmohr.github.io/JoSTC/)\n\nThis code is used for producing stroke tracing and correspondence results, which can be imported into an inbetweening product named [CACANi](https://cacani.sg) for making 2D animations.\n\n\u003cimg src='docs/figures/teaser-sub1.png' height=180\u003e\u003cimg src='docs/figures/dynamic1.gif' height=180\u003e\n\n\u003cimg src='docs/figures/teaser-sub2.png' height=180\u003e\u003cimg src='docs/figures/dynamic2.gif' height=180\u003e\n\n## Outline\n- [Dependencies](#dependencies)\n- [Quick Start](#quick-start)\n- [Vector Stroke Correspondence Dataset](#vector-stroke-correspondence-dataset)\n- [Citation](#citation)\n\n## Dependencies\n - [cudatoolkit](https://www.anaconda.com/download/) == 11.0.3\n - [cudnn](https://www.anaconda.com/download/) == 8.4.1.50\n - [pytorch](https://pytorch.org/) == 1.9.0\n - [torchvision](https://pytorch.org/vision/0.9/) == 0.9.0\n - [diffvg](https://github.com/BachiLi/diffvg)\n - [Krita](https://krita.org/en/): for making reference vector frame\n - [CACANi](https://cacani.sg/): for making inbetweening and 2D animation\n\n## Quick Start\n\n### Model Preparation\n\nDownload the models [here](https://drive.google.com/drive/folders/15oAP7YbNKx4Cx1AmzC16wuoyQoAai2YV?usp=sharing), and place them in this file structure:\n```\nmodels/\n    quickdraw-perceptual.pth\n    point_matching_model/\n        sketch_correspondence_50000.pkl\n        transform_module/\n            sketch_transform_30000.pkl\n    stroke_tracing_model/\n        sketch_tracing_30000.pkl\n```\n\n### Create Reference Vector Frames with Krita\n\nOur method takes as inputs consecutive raster keyframes and a single vector drawing from the starting keyframe, and then generates vector images for the remaining keyframes with one-to-one stroke correspondence. So we have to create the vector image for the reference frame here. \n\n**Note**: We provide several examples for testing in directory `sample_inputs/`. If you use them, you can skip step-1 and step-2 below and execute step-3 directly.\n\n1. Our method takes squared images as input, so please preprocess the images first using [tools/image_squaring.py](https://github.com/MarkMoHR/JoSTC/blob/main/tools/image_squaring.py):\n```\npython3 tools/image_squaring.py --file path/to/the/image.png\n```\n\n2. Follow tutorial [here](https://github.com/MarkMoHR/JoSTC/blob/main/tutorials/Krita_vector_generation.md) to make vector frames as a reference in svg format with [Krita](https://krita.org/en/).\n3. Place the svg files in `sample_inputs/*/svg/`. Then, convert them into npz format using [tools/svg_to_npz.py](https://github.com/MarkMoHR/JoSTC/blob/main/tools/svg_to_npz.py):\n```\ncd tools/\npython3 svg_to_npz.py --database ../sample_inputs/rough/ --reference 23-0.png\n```\n\n### Execute the Main Code\n\nPerform joint stroke tracing and correspondence using [sketch_tracing_inference.py](https://github.com/MarkMoHR/JoSTC/blob/main/sketch_tracing_inference.py). We provide several examples for testing in directory `sample_inputs/`.\n```\npython3 sketch_tracing_inference.py --dataset_base sample_inputs --data_type rough --img_seq 23-0.png 23-1.png 23-2.png\n```\n  - `--data_type`: specify the image type with `clean` or `rough`.\n  - `--img_seq`: specify the animation frames here. The first one should be the reference frame.\n  - The results are placed in `outputs/inference/*`. Inside this folder:\n    - `raster/` stores rendered line drawings of the target vector frames.\n    - `rgb/` stores visualization (with target images underneath) of the vector stroke correspondence to the reference frame.\n    - `rgb-wo-bg/` stores visualization without target images underneath.\n    - `parameter/` stores vector stroke parameters.\n\n### Create Inbetweening and Animation with CACANi\n\n1. Convert the output npz file (vector stroke parameters) into svg format using [tools/npz_to_svg.py](https://github.com/MarkMoHR/JoSTC/blob/main/tools/npz_to_svg.py):\n```\ncd tools/\npython3 npz_to_svg.py --database_input ../sample_inputs/ --database_output ../outputs/inference/ --data_type rough --file_names 23-1.png 23-2.png\n```\n  - The results are placed in `outputs/inference/*/svg`. There are two kinds of results:\n    - `chain/`: the svg files store stroke chains, defining each path as a chain.\n    - `separate/`: the svg files store separated strokes, defining each path as a single stroke. **Note that the automatic inbetweening in [CACANi](https://cacani.sg/) relies on this format.**\n\n2. Follow tutorial [here](https://github.com/MarkMoHR/JoSTC/blob/main/tutorials/CACANi_inbetweening_generation.md) to generate inbetweening and 2D animation with [CACANi](https://cacani.sg/).\n\n### More Tools\n\n- [tools/vis_difference.py](https://github.com/MarkMoHR/JoSTC/blob/main/tools/vis_difference.py): visualize difference between the reference image and the target one. The results are placed in `sample_inputs/*/raster_diff/`\n```\ncd tools/\npython3 vis_difference.py --database_input ../sample_inputs --data_type rough --reference_image 23-0.png --target_image 23-1.png\n```\n\n- [tools/make_inbetweening.py](https://github.com/MarkMoHR/JoSTC/blob/main/tools/make_inbetweening.py): visualize the inbetweening in a single image or a gif file.\n\n\u003cbr\u003e\n\n## Vector Stroke Correspondence Dataset\n\nWe collect a dataset for training with 10k+ pairs of raster frames and their vector drawings with stroke correspondence. Please download it [here](https://drive.google.com/drive/folders/15oAP7YbNKx4Cx1AmzC16wuoyQoAai2YV?usp=sharing). We provide a reference code [dataset_utils/tuberlin_dataset_util.py](https://github.com/MarkMoHR/JoSTC/blob/main/dataset_utils/tuberlin_dataset_util.py) showing how to use the data.\n\n\u003cbr\u003e\n\n## Citation\n\nIf you use the code and models please cite:\n\n```\n@article{mo2024joint,\n  title={Joint Stroke Tracing and Correspondence for 2D Animation},\n  author={Mo, Haoran and Gao, Chengying and Wang, Ruomei},\n  journal={ACM Transactions on Graphics},\n  volume={43},\n  number={3},\n  pages={1--17},\n  year={2024},\n  publisher={ACM New York, NY}\n}\n```\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmarkmohr%2Fjostc","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmarkmohr%2Fjostc","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmarkmohr%2Fjostc/lists"}