{"id":20493724,"url":"https://github.com/guanyingc/tom-net","last_synced_at":"2025-04-14T00:24:35.267Z","repository":{"id":52273113,"uuid":"124494582","full_name":"guanyingc/TOM-Net","owner":"guanyingc","description":"TOM-Net: Learning Transparent Object Matting from a Single Image (CVPR 2018 Spotlight)","archived":false,"fork":false,"pushed_at":"2024-07-08T10:55:34.000Z","size":23801,"stargazers_count":92,"open_issues_count":1,"forks_count":15,"subscribers_count":8,"default_branch":"master","last_synced_at":"2025-03-27T14:46:50.818Z","etag":null,"topics":["matting","torch","transparent"],"latest_commit_sha":null,"homepage":"https://guanyingc.github.io/TOM-Net","language":"Lua","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/guanyingc.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-03-09T05:56:15.000Z","updated_at":"2024-12-17T12:43:17.000Z","dependencies_parsed_at":"2024-11-15T17:47:28.464Z","dependency_job_id":null,"html_url":"https://github.com/guanyingc/TOM-Net","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/guanyingc%2FTOM-Net","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/guanyingc%2FTOM-Net/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/guanyingc%2FTOM-Net/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/guanyingc%2FTOM-Net/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/guanyingc","download_url":"https://codeload.github.com/guanyingc/TOM-Net/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248800087,"owners_count":21163404,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["matting","torch","transparent"],"created_at":"2024-11-15T17:36:31.489Z","updated_at":"2025-04-14T00:24:35.234Z","avatar_url":"https://github.com/guanyingc.png","language":"Lua","funding_links":[],"categories":[],"sub_categories":[],"readme":"# TOM-Net\n**[TOM-Net: Learning Transparent Object Matting from a Single Image, CVPR 2018 (Spotlight)](https://guanyingc.github.io/TOM-Net/)**,\n\u003cbr\u003e\n[Guanying Chen](https://guanyingc.github.io)\\*, [Kai Han](http://www.hankai.org/)\\*, [Kwan-Yee K. Wong](http://i.cs.hku.hk/~kykwong/)\n\u003cbr\u003e\n(\\* equal contribution)\n\nThis paper addresses the problem of transparent object matting from a single image.\n\u003cbr\u003e\n\u003cp align=\"center\"\u003e\n    \u003cimg src='images/cvpr2018_tom-net.jpg' width=\"600\" \u003e\n\u003c/p\u003e\n\n\n## Dependencies\nTOM-Net is implemented in [Torch](http://torch.ch/) and tested with Ubuntu 14.04. Please install Torch first following the [official document](http://torch.ch/docs/getting-started.html#_). \n- python 2.7 \n- numpy\n- cv2 \n- CUDA-8.0  \n- CUDNN v5.1\n- Torch STN ([qassemoquab/stnbhwd](https://github.com/qassemoquab/stnbhwd))\n    ```shell\n    # Basic installation steps for stn\n    git clone https://github.com/qassemoquab/stnbhwd.git\n    cd stnbhwd\n    luarocks make\n    ```\n\n## Overview\nWe provide:\n\n- Pretrained model\n- Datasets: Train (40GB), Validation (196MB), Test (179MB)\n- Code to test model on new images\n- Evaluation code on both the validation and testing data\n- Instructions to train the model \n- Example code for synthetic data rendering\n- Code and models used in the journal extension \u003cb\u003e(New!)\u003c/b\u003e\n\nIf the automatic downloading scripts are not working, please download the trained models and the introduced dataset from BaiduYun ([Models and Datasets](https://pan.baidu.com/s/1EHiiwK1BBlrXWGkqyYweaw?pwd=837t)).\n\n## Testing\n#### Download Pretrained Model\n```\nsh scripts/download_pretrained_model.sh\n```\nIf the above command is not working, please manually download the trained models from BaiduYun (Models, Datasets) and put them in `./data/models/`.\n\n#### Test on New Images\n```shell\n# Replace ${gpu} with the selected GPU ID (starting from 0)\n\n# Test a single image without having the background image\nCUDA_VISIBLE_DEVICES=${gpu} th eval/run_model.lua -input_img images/bull.jpg \n\n# You can find the results in data/TOM-Net_model/\n```\n\n#### Evaluation on Synthetic Validation Data\n```shell\n# Download synthetic validation dataset\nsh scripts/download_validation_dataset.sh\n\n# Quantitatively evaluate TOM-Net on different categories of synthetic object \n# Replace ${class} with one of the four object categories (glass, water, lens, cplx)\nCUDA_VISIBLE_DEVICES=${gpu} th eval/run_synth_data.lua -img_list ${class}.txt\n\n# Similarly, you can find the results in data/TOM-Net_model/\n```\n\n#### Evaluation on Real Testing Data\n```shell\n# Download real testing dataset, \nsh scripts/download_testing_dataset.sh\n\n# Test on sample images used in the paper\nCUDA_VISIBLE_DEVICES=${gpu} th eval/run_model.lua -img_list Sample_paper.txt\n\n# Quantitatively evaluate TOM-Net on different categories of real-world object \n# Replace ${class} with one of the four object categories (Glass, Water, Lens, Cplx)\nCUDA_VISIBLE_DEVICES=${gpu} th eval/run_model.lua -img_list ${class}.txt  \n```\n\n## Training\nTo train a new TOM-Net model, please follow the following steps:\n- Download the training data\n```shell\n# The size of the zipped training dataset is 40 GB and you need about 207 GB to unzip it.\nsh scripts/download_training_dataset.sh\n```\n\n- Train CoarseNet on simple objects\n```shell\nCUDA_VISIBLE_DEVICES=$gpu th main.lua -train_list train_simple_98k.txt -nEpochs 13 -prefix 'simple'\n# Please refer to opt.lua for more information about the training options\n\n# You can find log file, checkpoints and visualization results in data/training/simple_*\n```\n\n- Train CoarseNet on both simple and complex objects\n```shell\n# Finetune CoarseNet with all of the data\nCUDA_VISIBLE_DEVICES=$gpu th main.lua -train_list train_all_178k.txt -nEpochs 7 -prefix 'all' -retrain data/training/simple_*/checkpointdir/checkpoint13.t7\n\n# You can find log file, checkpoints and visualization results in data/training/all_*\n```\n\n- Train RefineNet on both simple and complex objects\n```shell\nCUDA_VISIBLE_DEVICES=$gpu th refine/main_refine.lua -train_list train_all_178k.txt -nEpochs 20 -coarse_net data/training/all_*/checkpointdir/checkpoint7.t7 \n# Train RefineNet with all of the data\n# Please refer to refine/opt_refine.lua for more information about the training options\n\n# You can find log file, checkpoints and visualization results in data/training/all_*/refinement/\n```\n\n## Synthetic Data Rendering\nPlease refer to [TOM-Net_Rendering](https://github.com/guanyingc/TOM-Net_Rendering) for sample rendering codes.\n\n## Codes and Models Used in the Journal Extension (IJCV)\n#### Test TOM-Net\u003csup\u003e+Bg\u003c/sup\u003e and TOM-Net\u003csup\u003e+Trimap\u003c/sup\u003e on Sample Images\n```shell\n# Download pretrained models\nsh scripts/download_pretrained_models_IJCV.sh\n\n# Test TOM-Net+Bg on sample images\nCUDA_VISIBLE_DEVICES=${gpu} th eval/run_model.lua -input_root images/TOM-Net_with_Trimap_Bg_Samples/ -img_list img_bg_trimap_list.txt -in_bg -c_net data/TOM-Net_plus_Bg_Model/CoarseNet_plus_Bg.t7 -r_net data/TOM-Net_plus_Bg_Model/RefineNet_plus_Bg.t7 \n# You can find the results in data/TOM-Net_plus_Bg_Model/*\n\n# Test TOM-Net+Trimap on sample images\nCUDA_VISIBLE_DEVICES=${gpu} th eval/run_model.lua -input_root images/TOM-Net_with_Trimap_Bg_Samples/ -img_list img_bg_trimap_list.txt -in_trimap -c_net data/TOM-Net_plus_Trimap_Model/CoarseNet_plus_Trimap.t7 -r_net data/TOM-Net_plus_Trimap_Model/RefineNet_plus_Trimap.t7 \n# You can find the results in data/TOM-Net_plus_Trimap_Model/*\n```\n\n#### Train TOM-Net\u003csup\u003e+Bg\u003c/sup\u003e and TOM-Net\u003csup\u003e+Trimap\u003c/sup\u003e \nTo train a new TOM-Net\u003csup\u003e+Bg\u003c/sup\u003e or TOM-Net\u003csup\u003e+Trimap\u003c/sup\u003e model, please follow the same procedures as training TOM-Net, except that you need to append `-in_bg` or `-in_trimap` at the end of the commands.\n\n## Citation\nIf you find this code or the provided data useful in your research, please consider cite the following relevant paper(s): \n\n```\n@inproceedings{chen2018tomnet,\n  title={TOM-Net: Learning Transparent Object Matting from a Single Image},\n  author={Chen, Guanying and Han, Kai and Wong, Kwan-Yee~K.},\n  booktitle={CVPR},\n  year={2018}\n}\n\n@inproceedings{chen2019LTOM,\n  title={Learning Transparent Object Matting},\n  author={Chen, Guanying and Han, Kai and Wong, Kwan-Yee~K.},\n  booktitle={IJCV},\n  year={2019}\n}\n```\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fguanyingc%2Ftom-net","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fguanyingc%2Ftom-net","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fguanyingc%2Ftom-net/lists"}