{"id":22306517,"url":"https://github.com/myungsub/cain","last_synced_at":"2025-10-19T09:39:32.333Z","repository":{"id":43787667,"uuid":"224426856","full_name":"myungsub/CAIN","owner":"myungsub","description":"Source code for AAAI 2020 paper \"Channel Attention Is All You Need for Video Frame Interpolation\"","archived":false,"fork":false,"pushed_at":"2023-05-26T09:18:35.000Z","size":6292,"stargazers_count":338,"open_issues_count":11,"forks_count":44,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-03-31T07:07:15.008Z","etag":null,"topics":["aaai2020","channel-attention","dataset","deep-convolutional-networks","frame-interpolation","pytorch","video-frame-interpolation"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/myungsub.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2019-11-27T12:33:33.000Z","updated_at":"2025-03-21T05:37:57.000Z","dependencies_parsed_at":"2022-08-26T23:10:50.312Z","dependency_job_id":"87554511-56aa-4aa1-91d3-ec3de4c83efb","html_url":"https://github.com/myungsub/CAIN","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/myungsub%2FCAIN","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/myungsub%2FCAIN/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/myungsub%2FCAIN/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/myungsub%2FCAIN/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/myungsub","download_url":"https://codeload.github.com/myungsub/CAIN/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247622983,"owners_count":20968575,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aaai2020","channel-attention","dataset","deep-convolutional-networks","frame-interpolation","pytorch","video-frame-interpolation"],"created_at":"2024-12-03T19:23:12.974Z","updated_at":"2025-10-19T09:39:27.273Z","avatar_url":"https://github.com/myungsub.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Channel Attention Is All You Need for Video Frame Interpolation\n\n#### Myungsub Choi, Heewon Kim, Bohyung Han, Ning Xu, Kyoung Mu Lee\n\n#### 2nd place in [[AIM 2019 ICCV Workshop](http://www.vision.ee.ethz.ch/aim19/)] - Video Temporal Super-Resolution Challenge\n\n[Project](https://myungsub.github.io/CAIN) | [Paper-AAAI](https://aaai.org/ojs/index.php/AAAI/article/view/6693/6547) (Download the paper [[here](https://www.dropbox.com/s/b62wnroqdd5lhfc/AAAI-ChoiM.4773.pdf?dl=0)] in case the AAAI link is broken) | [Poster](https://www.dropbox.com/s/7lxwka16qkuacvh/AAAI-ChoiM.4773.pdf)\n\n\u003ca href=\"https://aaai.org/Papers/AAAI/2020GB/AAAI-ChoiM.4773.pdf\" rel=\"Video\"\u003e\u003cimg src=\"./figures/CAIN_paper_thumb.jpg\" alt=\"Paper\" width=\"75%\"\u003e\u003c/a\u003e\n\u003ca href=\"https://www.dropbox.com/s/7lxwka16qkuacvh/AAAI-ChoiM.4773.pdf\" rel=\"Poster\"\u003e\u003cembed src=\"./figures/CAIN_poster.pdf\" alt=\"Poster\" width=\"20%\"\u003e\u003c/a\u003e\n\n\n## Directory Structure\n\n``` text\nproject\n│   README.md\n|   run.sh - main script to train CAIN model\n|   run_noca.sh - script to train CAIN_NoCA model\n|   test_custom.sh - script to run interpolation on custom dataset\n|   eval.sh - script to evaluate on SNU-FILM benchmark\n|   main.py - main file to run train/val\n|   config.py - check \u0026 change training/testing configurations here\n|   loss.py - defines different loss functions\n|   utils.py - misc.\n└───model\n│   │   common.py\n│   │   cain.py - main model\n|   |   cain_noca.py - model without channel attention\n|   |   cain_encdec.py - model with additional encoder-decoder\n└───data - implements dataloaders for each dataset\n│   |   vimeo90k.py - main training / testing dataset\n|   |   video.py - custom data for testing\n│   └───symbolic links to each dataset\n|       | ...\n```\n\n## Dependencies\n\nCurrent version is tested on:\n\n- Ubuntu 18.04\n- Python==3.7.5\n- numpy==1.17\n- [PyTorch](http://pytorch.org/)==1.3.1, torchvision==0.4.2, cudatoolkit==10.1\n- tensorboard==2.0.0 (If you want training logs)\n- opencv==3.4.2\n- tqdm==4.39.0\n\n``` text\n# Easy installation (using Anaconda environment)\nconda create -n cain\nconda activate cain\nconda install python=3.7\nconda install pip numpy\nconda install pytorch torchvision cudatoolkit=10.1 -c pytorch\nconda install tqdm opencv tensorboard\n```\n\n## Model\n\n\u003ccenter\u003e\u003cimg src=\"./figures/overall_architecture.png\" width=\"90%\"\u003e\u003c/center\u003e\n\n## Dataset Preparation\n\n- We use **[Vimeo90K Triplet dataset](http://toflow.csail.mit.edu/)** for training + testing.\n  - After downloading the full dataset, make symbolic links in `data/` folder : \n    - `ln -s /path/to/vimeo_triplet_data/ ./data/vimeo_triplet`\n  - Then you're done!\n- For more thorough evaluation, we built **[SNU-FILM (SNU Frame Interpolation with Large Motion)](https://myungsub.github.io/CAIN)** benchmark.\n  - Download links can be found in the [project page](https://myungsub.github.io/CAIN).\n  - Also make symbolic links after download :\n    - `ln -s /path/to/SNU-FILM_data/ ./data/SNU-FILM`\n  - Done!\n\n## Usage\n\n#### Training / Testing with Vimeo90K dataset\n- First make symbolic links in `data/` folder : `ln -s /path/to/vimeo_triplet_data/ ./data/vimeo_triplet`\n  - [Vimeo90K dataset](http://toflow.csail.mit.edu/)\n- For training: `CUDA_VISIBLE_DEVICES=0 python main.py --exp_name EXPNAME --batch_size 16 --test_batch_size 16 --dataset vimeo90k --model cain --loss 1*L1 --max_epoch 200 --lr 0.0002`\n- Or, just run `./run.sh`\n- For testing performance on Vimeo90K dataset, just add `--mode test` option\n- For testing on SNU-FILM dataset, run `./eval.sh`\n  - Testing mode (choose from ['easy', 'medium', 'hard', 'extreme']) can be modified by changing `--test_mode` option in `eval.sh`.\n\n#### Interpolating with custom video\n- Download pretrained models from [[Here](https://www.dropbox.com/s/y1xf46m2cbwk7yf/pretrained_cain.pth?dl=0)]\n- Prepare frame sequences in `data/frame_seq`\n- run `test_custom.sh`\n\n## Results\n\n\u003ccenter\u003e\u003cimg src=\"./figures/qualitative_vimeo.png\" width=\"100%\"\u003e\u003c/center\u003e\n\n### Video\n\n\u003ca href=\"https://www.dropbox.com/s/es2geeg26wwgezk/demo_combined_hires.mp4\" rel=\"Video\"\u003e\u003cimg src=\"./figures/CAIN_spotlight_thumb.jpg\" alt=\"Video\" width=\"100%\"\u003e\u003c/a\u003e\n\n## Citation\n\nIf you find this code useful for your research, please consider citing the following paper:\n\n``` text\n@inproceedings{choi2020cain,\n    author = {Choi, Myungsub and Kim, Heewon and Han, Bohyung and Xu, Ning and Lee, Kyoung Mu},\n    title = {Channel Attention Is All You Need for Video Frame Interpolation},\n    booktitle = {AAAI},\n    year = {2020}\n}\n```\n\n## Acknowledgement\n\nMany parts of this code is adapted from:\n\n- [EDSR-Pytorch](https://github.com/thstkdgus35/EDSR-PyTorch)\n- [RCAN](https://github.com/yulunzhang/RCAN)\n\nWe thank the authors for sharing codes for their great works.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmyungsub%2Fcain","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmyungsub%2Fcain","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmyungsub%2Fcain/lists"}