{"id":43238239,"url":"https://github.com/miccunifi/tape","last_synced_at":"2026-02-01T11:13:05.660Z","repository":{"id":202642448,"uuid":"707123736","full_name":"miccunifi/TAPE","owner":"miccunifi","description":"[WACV 2024] - Reference-based Restoration of Digitized Analog Videotapes","archived":false,"fork":false,"pushed_at":"2024-02-11T15:00:31.000Z","size":6848,"stargazers_count":32,"open_issues_count":1,"forks_count":0,"subscribers_count":12,"default_branch":"main","last_synced_at":"2024-05-14T00:14:28.671Z","etag":null,"topics":["analog-videos","clip","computer-vision","convolutional-neural-networks","deep-learning","degradation-model","low-level-vision","swin-transformer","transformer","unet","vgg","video-processing","video-quality","video-restoration"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/miccunifi.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2023-10-19T09:14:40.000Z","updated_at":"2024-05-12T07:50:01.000Z","dependencies_parsed_at":"2023-10-24T07:27:16.409Z","dependency_job_id":"35ce7f29-1d0f-4abe-91f0-25684c3c8811","html_url":"https://github.com/miccunifi/TAPE","commit_stats":{"total_commits":22,"total_committers":1,"mean_commits":22.0,"dds":0.0,"last_synced_commit":"3923083d7fbeeb2130eaa16518b976b91f2ce083"},"previous_names":["miccunifi/tape"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/miccunifi/TAPE","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/miccunifi%2FTAPE","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/miccunifi%2FTAPE/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/miccunifi%2FTAPE/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/miccunifi%2FTAPE/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/miccunifi","download_url":"https://codeload.github.com/miccunifi/TAPE/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/miccunifi%2FTAPE/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28977317,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-01T09:57:52.632Z","status":"ssl_error","status_checked_at":"2026-02-01T09:57:49.143Z","response_time":56,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["analog-videos","clip","computer-vision","convolutional-neural-networks","deep-learning","degradation-model","low-level-vision","swin-transformer","transformer","unet","vgg","video-processing","video-quality","video-restoration"],"created_at":"2026-02-01T11:13:04.985Z","updated_at":"2026-02-01T11:13:05.655Z","avatar_url":"https://github.com/miccunifi.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# TAPE (WACV 2024)\n\n### Reference-based Restoration of Digitized Analog Videotapes\n\n[![arXiv](https://img.shields.io/badge/arXiv-Paper-\u003cCOLOR\u003e.svg)](https://arxiv.org/abs/2310.14926)\n[![Generic badge](https://img.shields.io/badge/Video-YouTube-red.svg)](https://youtu.be/QLMmw3qiZeU)\n[![Generic badge](https://img.shields.io/badge/Slides-Link-orange.svg)](https://docs.google.com/presentation/d/1Mq2q68jkiHCfIrJRIZgeZ_D3nH-p2sCY/edit?usp=drive_link\u0026ouid=109868310223697047246\u0026rtpof=true\u0026sd=true)\n[![Generic badge](https://img.shields.io/badge/Poster-Link-purple.svg)](/assets/Poster.pdf)\n[![GitHub Stars](https://img.shields.io/github/stars/miccunifi/TAPE?style=social)](https://github.com/miccunifi/TAPE)\n\n[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/reference-based-restoration-of-digitized/analog-video-restoration-on-tape)](https://paperswithcode.com/sota/analog-video-restoration-on-tape?p=reference-based-restoration-of-digitized)\n\n**🔥 🔥 🔥 [22/12/2023] The pre-trained model and the code for real-world inference, training and testing are now available**\n\nThis is the **official repository** of the [**paper**](https://arxiv.org/abs/2310.14926) \"*Reference-based Restoration of Digitized Analog Videotapes*\".\n\n## Overview\n\n### Abstract\n\nAnalog magnetic tapes have been the main video data storage device for several decades. Videos stored on analog videotapes exhibit unique degradation patterns caused by tape aging and reader device malfunctioning that are different from those observed in film and digital video restoration tasks. In this work, we present a reference-based approach for the resToration of digitized Analog videotaPEs (TAPE). We leverage CLIP for zero-shot artifact detection to identify the cleanest frames of each video through textual prompts describing different artifacts. Then, we select the clean frames most similar to the input ones and employ them as references. We design a transformer-based Swin-UNet network that exploits both neighboring and reference frames via our Multi-Reference Spatial Feature Fusion (MRSFF) blocks. MRSFF blocks rely on cross-attention and attention pooling to take advantage of the most useful parts of each reference frame. To address the absence of ground truth in real-world videos, we create a synthetic dataset of videos exhibiting artifacts that closely resemble those commonly found in analog videotapes. Both quantitative and qualitative experiments show the effectiveness of our approach compared to other state-of-the-art methods.\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"assets/tape_teaser.png\" width=\"100%\" alt=\"Overview of the proposed approach\"\u003e\n\u003c/p\u003e\n\nOverview of the proposed approach. *Left* given a video, we identify the cleanest frames with CLIP. First, we measure the similarity between the frames and textual prompts that describe different artifacts. Then, we employ Otsu's method to define a threshold for classifying the frames based on their similarity scores, resulting in a set of clean frames. *Right* given a window of $T$ degraded input frames, we select the most similar $D$ clean frames based on the CLIP image features and employ them as references. The proposed Swin-UNet then restores the input frames while effectively leveraging the references.\n\n### Dataset\n\n\u003cp align='center'\u003e\n  \u003cimg src=\"assets/dataset_thumbnail.jpg\" width=\"25%\" alt=\"Dataset frame example\"\u003e\n  \u003cimg src=\"assets/dataset_thumbnail_2.jpg\" width=\"25%\" alt=\"Dataset frame example\"\u003e\n  \u003cimg src=\"assets/dataset_thumbnail_3.jpg\" width=\"25%\" alt=\"Dataset frame example\"\u003e\n\u003c/p\u003e\n\nWe release a dataset of videos synthetically degraded with Adobe After Effects to exhibit artifacts resembling those of real-world analog videotapes. The original high-quality videos belong to the Venice scene of the Harmonic dataset. The artifacts taken into account are: 1) tape mistracking; 2) VHS edge waving; 3) chroma loss along the scanlines; 4) tape noise; 5) undersaturation. The dataset comprises a total of 26,392 frames corresponding to 40 clips. The clips are randomly divided into training and test sets with a 75%-25% ratio.\n\nThe dataset can be downloaded [here](https://drive.google.com/drive/folders/1NjTiXOSf8_FVofvGFBGFwndomjTsThF-?usp=sharing). We release both the ```mp4``` videos and the [LMDB](https://lmdb.readthedocs.io/en/release/) files associated with each split.\n\n\n## Citation\n\n```bibtex\n@inproceedings{agnolucci2024reference,\n  title={Reference-based Restoration of Digitized Analog Videotapes},\n  author={Agnolucci, Lorenzo and Galteri, Leonardo and Bertini, Marco and Del Bimbo, Alberto},\n  booktitle={Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision},\n  pages={1659--1668},\n  year={2024}\n}\n```\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003ch2\u003eGetting Started\u003c/h2\u003e\u003c/summary\u003e\n\n### Installation\nWe recommend using the [**Anaconda**](https://www.anaconda.com/) package manager to avoid dependency/reproducibility\nproblems. For Linux systems, you can find a conda installation guide [here](https://docs.conda.io/projects/conda/en/latest/user-guide/install/linux.html).\n\n1. Clone the repository\n\n```sh\ngit clone https://github.com/miccunifi/TAPE\n```\n\n2. Install Python dependencies\n\n```sh\nconda create -n TAPE -y python=3.10\nconda activate TAPE\ncd TAPE\nchmod +x install_requirements.sh\n./install_requirements.sh\n```\n\n3. (Optional) If you want to compute the [VMAF](https://github.com/Netflix/vmaf) score, you first need to install [ffmpeg](https://ffmpeg.org/).\nThen, follow the instructions reported [here](https://github.com/Netflix/vmaf/blob/master/resource/doc/python.md) to install\nthe VMAF Python library. Finally, place the ```vmaf``` folder inside the ```utils``` directory.\n\n\n### Data Preparation\n\nDownload the dataset from [here](https://drive.google.com/drive/folders/1NjTiXOSf8_FVofvGFBGFwndomjTsThF-?usp=sharing). \nAt the end, the directory structure should look like this:\n\n```\n├── data_base_path\n|\n|    ├── train\n|    |   ├── input\n|    |   |   ├── input.lmdb\n|    |   |   ├── videos\n|    |   ├── gt\n|    |   |   ├── gt.lmdb\n|    |   |   ├── videos\n|\n|    ├── test\n|    |   ├── input\n|    |   |   ├── input.lmdb\n|    |   |   ├── videos\n|    |   ├── gt\n|    |   |   ├── gt.lmdb\n|    |   |   ├── videos\n```\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003ch2\u003eReal-world Inference\u003c/h2\u003e\u003c/summary\u003e\n\nTo use our method for restoring a real-world video, download the pre-trained model from the \n[release](https://github.com/miccunifi/TAPE/releases/tag/weights) and place it under\nthe ```experiments/pretrained_model``` directory. Then, run the following command:\n\n```python\n python real_world_inference.py --input-path \u003cpath_to_video\u003e --output-path \u003cpath_to_output_folder\u003e\n```\n\n```\n--input-path \u003cstr\u003e                           Path to the video to restore\n--output-path \u003cstr\u003e                          Path to the output folder\n--checkpoint-path \u003cstr\u003e                      Path to the pretrained model checkpoint (default=experiments/pretrained_model/checkpoint.pth)\n--num-input-frames \u003cint\u003e                     Number of input frames T for each input window (default=5)\n--num-reference-frames \u003cint\u003e                 Number of reference frames D for each input window (default=5)\n--preprocess-mode \u003cstr\u003e                      Preprocessing mode, options: ['crop', 'resize', 'none']. 'crop' extracts the --patch-size center\n                                             crop, 'resize' resizes the longest side to --patch-size while keeping the aspect ratio, 'none'\n                                             applies no preprocessing  (default=crop)\n--patch-size \u003cint\u003e                           Maximum patch size for --preprocess-mode ['crop', 'resize'] (default=512)\n    \n--frame-format \u003cstr\u003e                         Frame format of the extracted and restored frames (default=jpg)\n--generate-combined-video \u003cstore_true\u003e       Whether to generate the combined video (i.e. input and restored videos side by side)\n--no-intermediate-products \u003cstore_true\u003e      Whether to delete intermediate products (i.e. input frames, restored frames, references)\n--batch-size \u003cint\u003e                           Batch size (default=1)\n--num-workers \u003cint\u003e                          Number of workers of the data loader (default=20)\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003ch2\u003eTraining and Testing\u003c/h2\u003e\u003c/summary\u003e\n\nTo train our model from scratch, run the following command:\n\n```python\npython main.py --experiment-name \u003cname_of_the_experiment\u003e --data-base-path \u003cdata_base_path\u003e --comet-api-key \u003ccomet_api_key\u003e --comet-project-name \u003ccomet_project_name\u003e\n```\n\nYou need a [Comet ML](https://www.comet.com/site/) for logging. See ```main.py``` for all the available options. The\ncheckpoints will be saved inside the ```experiments/\u003cname_of_the_experiment\u003e/checkpoints``` folder. After training, ```main.py```\nwill run the evaluation on the test set and save the results inside the ```experiments/\u003cname_of_the_experiment\u003e/results``` folder.\n\nIf you want to skip the training and just run the evaluation on the test set, add the ```--test-only``` flag to the command\nabove. In addition, if you want to avoid computing the VMAF score, add the ```--no-vmaf``` flag.\n\nYou can test our pre-trained model by adding the ```--eval-type pretrained``` flag. Note that you first need to download the pre-trained model from the\n[release](https://github.com/miccunifi/TAPE/releases/tag/weights) and to place it under\nthe ```experiments/pretrained_model``` directory.\n\n\u003c/details\u003e\n\n## Authors\n\n* [**Lorenzo Agnolucci**](https://scholar.google.com/citations?user=hsCt4ZAAAAAJ\u0026hl=en)\n* [**Leonardo Galteri**](https://scholar.google.com/citations?user=_n2R2bUAAAAJ\u0026hl=en)\n* [**Marco Bertini**](https://scholar.google.com/citations?user=SBm9ZpYAAAAJ\u0026hl=en)\n* [**Alberto Del Bimbo**](https://scholar.google.com/citations?user=bf2ZrFcAAAAJ\u0026hl=en)\n\n## Acknowledgements\n\nThis work was partially supported by the European Commission under European Horizon 2020 Programme, grant number 101004545 - ReInHerit.\n\n## LICENSE\n\u003ca rel=\"license\" href=\"http://creativecommons.org/licenses/by-nc/4.0/\"\u003e\u003cimg alt=\"Creative Commons License\" style=\"border-width:0\" src=\"https://i.creativecommons.org/l/by-nc/4.0/88x31.png\" /\u003e\u003c/a\u003e\u003cbr /\u003eAll material is made available under [Creative Commons BY-NC 4.0](https://creativecommons.org/licenses/by-nc/4.0/). You can **use, redistribute, and adapt** the material for **non-commercial purposes**, as long as you give appropriate credit by **citing our paper** and **indicate any changes** that you've made.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmiccunifi%2Ftape","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmiccunifi%2Ftape","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmiccunifi%2Ftape/lists"}