{"id":28145095,"url":"https://github.com/hmchuong/maggie","last_synced_at":"2025-08-01T05:04:07.261Z","repository":{"id":244474201,"uuid":"780606583","full_name":"hmchuong/MaGGIe","owner":"hmchuong","description":"[CVPR24] MaGGIe: Mask Guided Gradual Human Instance Matting","archived":false,"fork":false,"pushed_at":"2024-12-26T23:27:57.000Z","size":108343,"stargazers_count":65,"open_issues_count":0,"forks_count":3,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-05-14T22:14:08.129Z","etag":null,"topics":["image-matting","mask-guided","matting","progressive-refinement","sparse-convolution","video-matting"],"latest_commit_sha":null,"homepage":"https://maggie-matt.github.io/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/hmchuong.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-04-01T20:27:20.000Z","updated_at":"2025-05-04T16:35:30.000Z","dependencies_parsed_at":"2024-12-27T00:29:35.287Z","dependency_job_id":null,"html_url":"https://github.com/hmchuong/MaGGIe","commit_stats":null,"previous_names":["hmchuong/maggie"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/hmchuong/MaGGIe","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hmchuong%2FMaGGIe","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hmchuong%2FMaGGIe/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hmchuong%2FMaGGIe/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hmchuong%2FMaGGIe/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/hmchuong","download_url":"https://codeload.github.com/hmchuong/MaGGIe/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hmchuong%2FMaGGIe/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":268171961,"owners_count":24207435,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-01T02:00:08.611Z","response_time":67,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["image-matting","mask-guided","matting","progressive-refinement","sparse-convolution","video-matting"],"created_at":"2025-05-14T22:14:00.296Z","updated_at":"2025-08-01T05:04:07.208Z","avatar_url":"https://github.com/hmchuong.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# \u003cimg src=\"figs/maggie.png\" alt=\"maggie\" width=\"28\"/\u003e MaGGIe: Mask Guided Gradual Human Instance Matting\n[[Project Page](https://maggie-matt.github.io/)] [[Hugging Face](https://huggingface.co/collections/chuonghm/maggie-cvpr24-662b9cac66c46de6b38b28e6)] [[Paper](https://arxiv.org/abs/2404.16035)] [[Model Zoo](docs/MODEL_ZOO.md)] [[Datasets](docs/DATASET.md)]\n\n*Instance-awareness alpha human matting with binary mask guidance for images and video*\n\n**Accepted at CVPR 2024**\n\n**[Chuong Huynh](https://hmchuong.github.io/), [Seoung Wug Oh](https://sites.google.com/view/seoungwugoh/), [Abhinav Shrivastava](https://www.cs.umd.edu/~abhinav/), [Joon-Young Lee](https://joonyoung-cv.github.io/)**\n\nWork is a part of Summer Internship 2023 at [Adobe Research](https://research.adobe.com/)\n\n\u003cimg src=\"figs/teaser_hd.gif\" alt=\"maggie\" width=\"800\"/\u003e\n\n## Release\n- [2024/12/26] Update the demo with [Samurai](https://github.com/educelab/samurai) - SOTA Video Object Segmentation method!\n- [2024/04/10] Demo with Gradio is ready!\n- [2024/04/07] Code, dataset and paper are released!\n- [2024/04/04] Webpage is up!\n\n\n## Contents\n- [Install](#install)\n- [MaGGIe Weights](#maggie-weights)\n- [Demo](#demo)\n- [Evaluation](#evaluation)\n- [Training](#training)\n- [Citation](#citation)\n\n## Install\n\nWe tested our model on Linux CUDA 12.0, for other OS, the framework should work fine!\n\n1. Clone this repository and navigate to `MaGGIe` folder:\n```bash\ngit clone https://github.com/hmchuong/MaGGIe.git\ncd MaGGIe\n```\n\n2. Make sure you install CUDA 12.0 and install dependencies via:\n\n```\nconda create -n maggie python=3.8 pip\nconda activate maggie\nconda install -y pytorch torchvision pytorch-cuda=12.1 -c pytorch -c nvidia\npip install -r requirements.txt\n```\n\n## MaGGIe Weights\nPlease check our [Model Zoo](docs/MODEL_ZOO.md) for all public MaGGIe checkpoints, and instructions for how to use weights.\n\n## Demo\nPlease check [Demo](demo) for more information.\n\n## Evaluation\n\nPlease check the [Model Zoo](docs/MODEL_ZOO.md) for all model weight information.\n### M-HIM2K and HIM2K\nThe script [scripts/test_maggie_image.sh](scripts/test_maggie_image.sh) contains the full evaluation on the whole M-HIM2K. The `results.csv` in the log directory contains all the results needed. To get the number in the paper, you can run this command on 4 GPUs:\n```bash\nsh scripts/eval_image.sh configs/maggie_image.yaml 4  maggie\n```\n\n\nYou can also run one subset (e.g, `natural`) with one model mask (e.g, `r50_c4_3x`) by:\n```bash\nNGPUS=4\nCONFIG=configs/maggie_image.yaml\nSUBSET=natural\nMODEL=r50_c4_3x\ntorchrun --standalone --nproc_per_node=$NGPUS tools/main.py --config $CONFIG --eval-only \\\n                                                name eval_full \\\n                                                dataset.test.split $SUBSET \\\n                                                dataset.test.downscale_mask False \\\n                                                dataset.test.mask_dir_name masks_matched_${MODEL} \\\n                                                test.save_results False \\\n                                                test.postprocessing False \\\n                                                test.log_iter 10\n```\nIf you want to save the alpha mattes, please set `test.save_results True` and change the `test.save_dir`\n### V-HIM60\nThe script [scripts/test_maggie_video.sh](scripts/test_maggie_video.sh) contains the full evaluation on the V-HIM60. This evaluation is only compatible with a single GPU. To get the number in the paper, you can run this command:\n```bash\nsh scripts/eval_video.sh configs/maggie_video.yaml maggie\n```\n\nIf you want to evaluate on a subset (e.g, `easy`), you can run:\n```bash\nCONFIG=configs/maggie_video.yaml\nSUBSET=easy\ntorchrun --standalone --nproc_per_node=1 tools/main.py --config $CONFIG --eval-only \\\n                    name eval_full \\\n                    dataset.test.split comp_$SUBSET \\\n                    test.save_results False \\\n                    test.log_iter 10\n```\nIf you want to save the alpha mattes, please set `test.save_results True` and change the `test.save_dir`.\n\n## Training\n1. Please firstly follow [DATASET](docs/DATASET.md) to prepare the training data.\n\n2. Download [pretrained weights](https://drive.google.com/file/d/1kNj33D7x7tR-5hXOvxO53QeCEC8ih3-A/view) of the encoder from [GCA-Matting](https://github.com/Yaoyi-Li/GCA-Matting?tab=readme-ov-file#models)\n\n3. Training the image instance matting. \n\nIt is recommended to use 4 A100-40GB GPUs or (any GPUs with VRAM\u003e=24GB) for this step. \nPlease check the [config](configs/maggie_image.yaml) and set `wandb` settings to your project.\n```bash\nNAME=\u003cname of the experiment\u003e\nNGPUS=4\ntorchrun --standalone --nproc_per_node=$NGPUS tools/main.py \\\n                    --config configs/maggie_image.yaml \\\n                    --precision 16 name $NAME model.weights ''\n```\nIf you want to resume training from the last checkpoint, you can turn on `train.resume_last` or set `train.resume` to the checkpoint folder you want to resume from. You can also set `wandb.id` to continue logging to the same experiment id.\n\n\n4. Training the video instance matting\n\nIt is recommend to use 8 A100-80GB GPUs for this step. Please check the [config](configs/maggie_video.yaml) and set `wandb to your project.\n```bash\nNAME=\u003cname of the experiment\u003e\nPRETRAINED=\u003cbest weight from previous step\u003e\nNGPUS=8\ntorchrun --standalone --nproc_per_node=$NGPUS tools/main.py \\\n                    --config configs/maggie_video.yaml \\\n                    --precision 16 name $NAME model.weights $PRETRAINED\n```\nIf you want to resume training from the last checkpoint, you can turn on `train.resume_last` or set `train.resume` to the checkpoint folder you want to resume from. You can also set `wandb.id` to continue logging to the same experiment id.\n\n## Citation\nIf you find MaGGIE useful for your research and applications, please cite using this BibTeX:\n```bibtex\n@inproceedings{huynh2024maggie,\n  title={MaGGIe: Masked Guided Gradual Human Instance Matting},\n  author={Huynh, Chuong and Oh, Seoung Wug and and Shrivastava, Abhinav and Lee, Joon-Young},\n  booktitle={Proceedings of the {IEEE} Conference on Computer Vision and Pattern Recognition (CVPR)},\n  year={2024}\n}\n```\n\n## Baselines\nWe also provide baselines' training and evaluation scripts at [BASELINES](docs/BASELINES.md)\n\n## Terms of Use\nThe project is under the [CC BY-NC 4.0](LICENSE.md) License for non-commercial purpose only.\n\n## Acknowledgement\nWe thank Markus Woodson for his early project discussion. Our code is based on the [OTVM](https://github.com/Hongje/OTVM) and [MGM](https://github.com/yucornetto/MGMatting).\n \n## Contact\nIf you have any question, please drop an email to chuonghm@umd.edu or create an issue on this repository.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhmchuong%2Fmaggie","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhmchuong%2Fmaggie","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhmchuong%2Fmaggie/lists"}