{"id":20693024,"url":"https://github.com/deepcs233/tin","last_synced_at":"2025-04-22T17:42:15.460Z","repository":{"id":111776465,"uuid":"222444385","full_name":"deepcs233/TIN","owner":"deepcs233","description":"[AAAI 2020] Temporal Interlacing Network","archived":false,"fork":false,"pushed_at":"2020-11-25T17:17:30.000Z","size":77,"stargazers_count":84,"open_issues_count":2,"forks_count":6,"subscribers_count":8,"default_branch":"master","last_synced_at":"2025-03-29T17:22:20.824Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/deepcs233.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-11-18T12:30:34.000Z","updated_at":"2025-02-17T04:34:26.000Z","dependencies_parsed_at":"2023-04-18T16:04:43.127Z","dependency_job_id":null,"html_url":"https://github.com/deepcs233/TIN","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deepcs233%2FTIN","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deepcs233%2FTIN/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deepcs233%2FTIN/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deepcs233%2FTIN/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/deepcs233","download_url":"https://codeload.github.com/deepcs233/TIN/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250289147,"owners_count":21405925,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-16T23:24:56.252Z","updated_at":"2025-04-22T17:42:15.454Z","avatar_url":"https://github.com/deepcs233.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Temporal Interlacing Network [pdf](http://thesouthfrog.com/about.me/Shengju_files/TIN.pdf) [arxiv]\n\nShao Hao, [Shengju Qian](http://thesouthfrog.com/about.me/) and [Yu Liu](http://liuyu.us/)\n\n## Overview\n\n![framework](http://thesouthfrog.com/about.me/Shengju_files/TIN.png)\n\nWe present Temporal Interlacing Network(TIN), a simple interlacing operator. TIN performs equivalently to the regularized temporal convolution network (r-TCN), but gains 4% more accuracy with 6x less latency on 6 challenging benchmarks. These results push the state-of-the-art performances of video understanding by a considerable margin. \n\nMeanwhile, TIN severs as the key component of the winning entry in ICCV 2019 - Multi Moments in Time challenge. You can find more detailed solutions in the [report](http://moments.csail.mit.edu/challenge2019/efficient_challenge_report.pdf).  [Challenge Website](http://moments.csail.mit.edu/results2019.html)\n\n\n## Prerequisites\n\nThe code is built with following libraries:\n\n- [PyTorch](https://pytorch.org/) 0.4(switch to branch 0.4), 1.0 or higher\n- [TensorboardX](https://github.com/lanpa/tensorboardX)\n- [tqdm](https://github.com/tqdm/tqdm.git)\n- [sklearn](https://github.com/scikit-learn/scikit-learn)\n- [scikit-learn](https://scikit-learn.org/stable/)\n\nFor extracting frames from video data, you may need [ffmpeg](https://www.ffmpeg.org/).\n\n### Data Preparation\nPlease refer to [TSN](https://github.com/yjxiong/temporal-segment-networks) and [TSM](https://github.com/mit-han-lab/temporal-shift-module#prerequisites) repo for the detailed guide of data pre-processing. We provide the example pre-processing scripts for Kinetics and Something-Something datasets.\n\n![cuda](http://thesouthfrog.com/about.me/Shengju_files/cuda.png)\n\n## Get Started(CUDA operator)\nWe provide the CUDA operator of TIN to provide acceleration. The pure Python version runs about 1.1s on 1080TI and the optimized code version runs about 0.7s. You may first build the provided library before getting started. \n\n```bash\ncd cuda_shift\nbash make.sh\n```\n\n### Train\nHere we provide several examples to train with this repo at Something-Something v1 dataset:\n\n**Train from scratch (with imagenet pretrain)** \n```bash\npython -u main.py something RGB \\\n     --arch resnet50 --num_segments 8 \\\n     --gd 20 --lr 0.02 --lr_type cos  --epochs 40 --use_warmup --warmup_epochs 1 \\\n     --store_name \"tin_example\"  \\\n     --batch-size 48 -j 4 --dropout 0.8 --consensus_type=avg --eval-freq=1 \\\n     --tin --shift_div=4 --gpus 0 1 2 3 4 5 6 7 --workers 36  \\\n```\n\n**Train from pretrained model (with Step lr)**\n```bash\npython -u main.py something RGB \\\n     --arch resnet50 --num_segments 8 \\\n     --gd 20 --lr 0.008 --lr_type step --lr_steps 10 20 30 --epochs 40 --use_warmup --warmup_epochs 1 \\\n     --store_name \"tin_example\"  \\\n     --batch-size 48 -j 4 --dropout 0.8 --consensus_type=avg --eval-freq=1 \\\n     --tin --shift_div=4 --gpus 0 1 2 3 4 5 6 7 --workers 36  \\\n     --tune_from=pretrained/xxx.pth.tar \\\n```\n\n**Resume training**\n```bash\npython -u main.py something RGB \\\n     --arch resnet50 --num_segments 8 \\\n     --gd 20 --lr 0.008 --lr_type cos  --epochs 40 --use_warmup --warmup_epochs 1 \\\n     --store_name \"tin_example\"  \\\n     --batch-size 48 -j 4 --dropout 0.8 --consensus_type=avg --eval-freq=1 \\\n     --tin --shift_div=4 --gpus 0 1 2 3 4 5 6 7 --workers 36  \\\n     --resume=True \\\n     --resume_path=checkpoint/xx/xxx.pth.tar \\\n```\n\n### Test\nHere we provide several examples to test with this repo at Something-Something v1 dataset:\n\n**Test with center crop and 1 temporal clips (same as training setting)**\n```bash\npython -u  ignore test.py something RGB \\\n     --arch resnet50 --num_segments 8 --print-freq 50 --npb \\\n     --batch-size 24 -j 5 --dropout 0.8 --consensus_type=avg  \\\n     --tin --shift_div=4 \\\n     --temporal_clips=1 --random_crops=1 \\\n     --scale_size 256 --crop_size 224 \\\n     --gpus 0 1 2 3 4 5 6 7 \\\n     --resume_path=checkpoint/xxx/ckpt.best.pth.tar \\\n```\n\n**Test with Full resolution, 2 temporal clips and 3 spatial crops**\n```bash\npython -u  ignore test.py something RGB \\\n     --arch resnet50 --num_segments 8 --print-freq 50 --npb \\\n     --batch-size 24 -j 5 --dropout 0.8 --consensus_type=avg  \\\n     --tin --shift_div=4 \\\n     --temporal_clips=2  --random_crops=3 \\\n     --scale_size 256 --crop_size 256 \\\n     --gpus 0 1 2 3 4 5 6 7 \\\n     --resume_path=checkpoint/xxx/ckpt.best.pth.tar \\\n```\n## ModelZoo\nPretrain model can be found in [X-Temporal](https://github.com/Sense-X/X-Temporal).\n\n### Acknowledge \n\nThis code is based on the original TSN and TSM codebased. Thanks to their great work!\n\n## Citation\n\nIf you use our code, please consider citing our paper or challenge report:\n\n```\n@article{shao2020temporal,\n    title={Temporal Interlacing Network},\n    author={Hao Shao and Shengju Qian and Yu Liu},\n    year={2020},\n    journal={AAAI},\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdeepcs233%2Ftin","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdeepcs233%2Ftin","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdeepcs233%2Ftin/lists"}