{"id":16562791,"url":"https://github.com/charmve/timewarp","last_synced_at":"2025-03-21T11:32:37.318Z","repository":{"id":45514822,"uuid":"322352414","full_name":"Charmve/TimeWarp","owner":"Charmve","description":"🐶 「Updating ...」Replacing Real-Time the High-Resolution Meeting Background Freely ","archived":false,"fork":false,"pushed_at":"2023-05-12T06:22:29.000Z","size":102670,"stargazers_count":30,"open_issues_count":1,"forks_count":3,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-04-15T09:05:26.754Z","etag":null,"topics":["background-matting","background-replacing","charmve","communication","matting","meeting","online-meetings","real-time","realtime-messaging","video-call","video-chat","video-conferencing","virtual-reality","zoom"],"latest_commit_sha":null,"homepage":"https://gradio.app/g/BackgroundMattingV2","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Charmve.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null},"funding":{"github":"Charmve","patreon":null,"open_collective":"Charmve","ko_fi":"Charmve","tidelift":null,"community_bridge":null,"liberapay":null,"issuehunt":null,"otechie":null,"custom":["https://charmve.github.io/sponsor.html"]}},"created_at":"2020-12-17T16:33:28.000Z","updated_at":"2024-04-09T08:52:13.000Z","dependencies_parsed_at":"2022-07-17T09:00:45.184Z","dependency_job_id":"8750e2a2-44ee-4723-a71e-bc18b4f3b174","html_url":"https://github.com/Charmve/TimeWarp","commit_stats":{"total_commits":91,"total_committers":2,"mean_commits":45.5,"dds":0.03296703296703296,"last_synced_commit":"00642a7c5b49062485555c54fb130cb0cd50cfde"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Charmve%2FTimeWarp","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Charmve%2FTimeWarp/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Charmve%2FTimeWarp/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Charmve%2FTimeWarp/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Charmve","download_url":"https://codeload.github.com/Charmve/TimeWarp/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":221814961,"owners_count":16885086,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["background-matting","background-replacing","charmve","communication","matting","meeting","online-meetings","real-time","realtime-messaging","video-call","video-chat","video-conferencing","virtual-reality","zoom"],"created_at":"2024-10-11T20:37:10.254Z","updated_at":"2024-10-28T10:04:40.085Z","avatar_url":"https://github.com/Charmve.png","language":"Python","funding_links":["https://github.com/sponsors/Charmve","https://opencollective.com/Charmve","https://ko-fi.com/Charmve","https://charmve.github.io/sponsor.html","https://opencollective.com/charmve/donate"],"categories":[],"sub_categories":[],"readme":"\u003cimg src=\"images/ui/logo_H.png\" align=\"right\" alt=\"TimeWarp\" width=\"280\"/\u003e\n\n# TimeWarp - 在线视频背景替换\nReplacing Real-Time the High-Resolution Meeting Background Freely. \n\u003c!--\nhttps://grail.cs.washington.edu/projects/background-matting-v2/visuals/teaser.jpg\n--\u003e\n\u003cbr\u003e\n\u003cdiv align=\"center\"\u003e\n    \u003cimg src=\"https://user-images.githubusercontent.com/29084184/118346542-f36d7900-b56e-11eb-938d-22c1e1bcce94.png\" alt=\"Example comparison in a video call\"\u003e\n\u003c/div\u003e\n\u003cdiv class=\"teaser-caption\"\u003e\n    \u003ci\u003eCurrent video conferencing tools like Zoom can take an input feed (left) and replace the background, often introducing artifacts, as shown in the center result with close-ups of hair and glasses that still have the residual of the original background. Leveraging a frame of video without the subject (far left inset), our method produces real-time, high-resolution background matting without those common artifacts. The image on the right is our result with the corresponding close-ups, screenshot from our Zoom plugin implementation.\u003c/i\u003e\n\u003c/div\u003e\n\n\u003cbr\u003e\n👉 \u003ca href=\"https://grail.cs.washington.edu/projects/background-matting-v2/img_results.html\" target=\"_blank\"\u003eSee image results with interactive zoom-in\u003c/a\u003e\n\n## ✨ Features\n- \u003cb\u003eReal-Time\u003c/b\u003e High-Resolution Background Matting, \u003cimg src=\"https://img.shields.io/badge/4K-30%20FPS-green\" alt=\"4K 30fps\"\u003e \u0026nbsp;\u003cimg src=\"https://img.shields.io/badge/HD-60%20FPS-blue\" alt=\"HD 60fps\"\u003e;\n- Replacing our background in meeting screen;\n- Replace background with \u003cb\u003eany size\u003c/b\u003e photo you want;\n- Cross mutil-platform, like zoom plugin, web-browser, miniprogram in PC, mobile and iPad, any platform with web service;\n- IFLYTEK voice translation between Chinese and English;\n- ...\n\n## Overview\n* [Updates](#updates)\n* [Download](#download)\n    * [Model / Weights](#model--weights)\n    * [Video / Image Examples](#video--image-examples)\n    * [Datasets](#datasets)\n* [Build](#-build)\n* [Demo](#-demo)\n    * [Beta](#-beta)\n    * [Scripts](#scripts)\n    * [Notebooks](#notebooks)\n* [Usage / Documentation](#usage--documentation)\n* [Training](#training)\n* [License](#-license)\n* [Reference](#-reference)\n* [Community](#-community)\n\n\n## Updates\n\n* [Jan 09 2021] [PhotoMatte85](https://drive.google.com/file/d/1KpHKYW986Dax9-ZIM7I-HyBoWVcLPuaQ/view) dataset is now published.\n* [Dec 21 2020] We updated our project to MIT License, which permits commercial use!\n\n## Download\n\n### Model / Weights\n\n* [Download model / weights](https://drive.google.com/drive/folders/1cbetlrKREitIgjnIikG1HdM4x72FtgBh?usp=sharing)\n\n### Video / Image Examples\n\n* [HD videos](https://drive.google.com/drive/folders/1j3BMrRFhFpfzJAe6P2WDtfanoeSCLPiq) (by [Sengupta et al.](https://github.com/senguptaumd/Background-Matting)) (Our model is more robust on HD footage)\n* [4K videos and images](https://drive.google.com/drive/folders/16H6Vz3294J-DEzauw06j4IUARRqYGgRD?usp=sharing)\n\n\n### Datasets\n\n* VideoMatte240K (Coming soon)\n* [PhotoMatte85](https://drive.google.com/file/d/1KpHKYW986Dax9-ZIM7I-HyBoWVcLPuaQ/view?usp=sharing)\n\n## 🔨 Build\n\n1. setup enviroment\n\n```shell\n# git clone https://github.com/Charmve/TimeWarp.git\n# cd TimeWarp\n\n# pip install -r requirements.txt\n```\n\n2. download dataset and model [``Download``](#download)\n\n3. run test code\n- Video \n```\n# cd TimeWarp\n# bash test_video.sh\n```\n\n- Image\n```\n# cd TimeWarp\n# python3 test_image.py\n```\n\n## Demo\n\n### 🎧 Beta\n\n\u003cdiv align=\"center\"\u003e\n    \u003cimg src=\"https://github.com/PeterL1n/Matting-PyTorch/raw/master/images/teaser.gif?raw=true\"\u003e\n    \u003cbr\u003e\u003cb\u003edemo\u003c/b\u003e Real-Time High-Resolution Background Matting\u003csup\u003e[1]\u003c/sup\u003e\n\u003c/div\u003e\n\n#### Have a \u003cb\u003eTry\u003c/b\u003e ! \n\n➡️ [Web Demo by Gradio](https://gradio.app/g/BackgroundMattingV2): Matting your own images from your browser. \n\n#### Scripts\n\nWe provide several scripts in this repo for you to experiment with our model. More detailed instructions are included in the files.\n* `inference_images.py`: Perform matting on a directory of images.\n* `inference_video.py`: Perform matting on a video.\n* `inference_webcam.py`: An interactive matting demo using your webcam.\n\n#### Notebooks\nAdditionally, you can try our notebooks in Google Colab for performing matting on images and videos.\n\n* [Image matting (Colab)](https://colab.research.google.com/drive/1cTxFq1YuoJ5QPqaTcnskwlHDolnjBkB9?usp=sharing)\n* [Video matting (Colab)](https://colab.research.google.com/drive/1Y9zWfULc8-DDTSsCH-pX6Utw8skiJG5s?usp=sharing)\n\n#### Virtual Camera\nWe provide a demo application that pipes webcam video through our model and outputs to a virtual camera. The script only works on Linux system and can be used in Zoom meetings. For more information, checkout:\n* [Webcam plugin](https://github.com/andreyryabtsev/BGMv2-webcam-plugin-linux)\n\n#### Web Demo\nDevelopers in the community has helped us build a web demo. See [Community Projects](#community-projects) section below. \n\n## Usage / Documentation\n\nYou can run our model using **PyTorch**, **TorchScript**, **TensorFlow**, and **ONNX**. For detail about using our model, please check out the [Usage / Documentation](doc/model_usage.md) page.\n\n#### 中文解读 -\u003e\n- [单块GPU实现4K分辨率每秒30帧，华盛顿大学实时视频抠图再升级，毛发细节到位](https://mp.weixin.qq.com/s/9ow2xAKrDecEGl2Gn0Y5MA)\n- [整个世界都是你的绿幕：这个视频抠图换背景的方法着实真假难辨](https://mp.weixin.qq.com/s?__biz=MzA3MzI4MjgzMw==\u0026mid=2650784247\u0026idx=2\u0026sn=b8808700fdeaa0e6eda7430d0f29d2ce\u0026chksm=871a0589b06d8c9f17cc3374049773cf70f4a39ffbb5d39a1da32882b1ecd6e02453336414d9)\n\n## Training\n\nTraining code will be released upon acceptance of the paper.\n\n## ☕ Processing\n\n```mermaid\ngantt\n        dateFormat  YYYY-MM-DD\n        title TimeWarp Development Processing\n        section image matting\n        architecture design          :done,    des1, 2020-11-06,2020-12-20\n        video matting                :active,  des2, 2021-01-07, 3d\n        chrome extension plus        :         des3, after des2, 7d\n        video meeting architecture   :         des4, after des3, 5d\n```\n\n- More update processing details, please click [HERE](./docs/Changelog.md) to see \u003cb\u003eCHANGLOG\u003c/b\u003e. \n\n## 🔔 Update Reminder\n\n[Subscribe --\u003e](https://github.com/Charmve/TimeWarp/edit/main/README.md)\n\nBusiness ✉️ email: yidazhang1@gmail.com\n\n## 🍮 Community\n- GitHub \u003ca href=\"https://github.com/Charmve/TimeWarp/discussions\" target=\"_blank\"\u003ediscussions 💬\u003c/a\u003e or \u003ca href=\"https://github.com/Charmve/TimeWarp/issues\" target=\"_blank\"\u003eissues 💭\u003c/a\u003e\n\n- QQ Group: 697948168 (password：TimeWarp)\n- Email: yidazhang1#gmail.com \n\n\u003c!--\n## 💖 Sponsors and Backers\n\u003ca href=\"https://opencollective.com/charmve/donate\" target=\"_blank\"\u003e\n  \u003cimg src=\"https://opencollective.com/webpack/donate/button@2x.png?color=blue\" width=300 /\u003e\n\u003c/a\u003e\n--\u003e\n\n## 🔗 Reference\n\n[1] Real-Time High-Resolution Background Matting. Shanchuan Lin*, Andrey Ryabtsev*, Soumyadip Sengupta. [[GitHub]](https://github.com/PeterL1n/BackgroundMattingV2) | [[arXiv]](https://arxiv.org/pdf/2012.07810.pdf) | [[data]](https://grail.cs.washington.edu/projects/background-matting-v2/)\n\n[2] Background Matting: The World is Your Green Screen. Soumyadip Sengupta, Vivek Jayaram, Brian Curless, et.al. CVPR 2020. [[GitHub]](https://github.com/senguptaumd/Background-Matting) | [[arXiv]](https://arxiv.org/abs/2004.00626)\n\n[3] MODNet: Is a Green Screen Really Necessary for Real-Time Portrait Matting? [[GitHub](https://github.com/ZHKKKe/MODNet)] | [[arXiv]](https://arxiv.org/pdf/2011.11961.pdf)\n\n[4] High-Resolution Deep Image Matting. Haichao Yu, Ning Xu, Zilong Huang. [GitHub] | [[arXiv]](https://arxiv.org/pdf/2009.06613.pdf)\n\n[5] End-to-end Animal Image Matting. Jizhizi Li, Jing Zhang, Stephen J. Maybank. [[GitHub]](https://github.com/JizhiziLi/animal-matting) | [[arXiv]](https://arxiv.org/pdf/2010.16188.pdf).\n\n[6] BGMv2-webcam-plugin-linux. https://github.com/andreyryabtsev/BGMv2-webcam-plugin-linux\n\n[7] Chrome-extensions-samples. https://github.com/GoogleChrome/chrome-extensions-samples\n\n[8] Zoom Video Ingester. https://github.com/harvard-dce/zoom-recording-ingester\n\n[9] Zoom Recording Downloader. https://github.com/ricardorodrigues-ca/zoom-recording-downloader\n\n[10] PaperswithCode. Image Matting. https://paperswithcode.com/task/image-matting \u003ca href=\"https://paperswithcode.com/task/image-matting\"\u003e\u003cimg src=\"https://img.shields.io/badge/PaperswithCode-matting-000000.svg?logo=paperwithcode\u0026color=lightblue\" alt=\"PaperswithCode image-matting\"\u003e\u003c/a\u003e\n\n## 📜 License\n\nThis work is licensed under the [Creative Commons Attribution NonCommercial ShareAlike 4.0 License](https://creativecommons.org/licenses/by-nc-sa/4.0/legalcode).\n\n## 💝 Acknowledge\n\nThanks for [PeterL1n](https://github.com/PeterL1n), et al. \n\n\u003cbr\u003e\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"images/ui/logo_V.png\" alt=\"TimeWarp\" width=\"280\"/\u003e\n\u003cbr\u003eFeel free to ask any questions, open a PR if you feel something can be done differently!\u003c/p\u003e\n\u003ch2 align=\"center\"\u003e🌟Star this repository🌟\u003c/h2\u003e\n\u003cp align=\"center\"\u003eCreated by \u003ca href=\"https://github.com/Charmve\"\u003eCharmve\u003c/a\u003e \u0026 \u003ca href=\"https://github.com/MaiweiAI\"\u003emaiwei.ai\u003c/a\u003e Community | Deployed on \u003ca href=\"https://gradio.app/g/BackgroundMattingV2\"\u003eGradio.app\u003c/a\u003e\u003c/p\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcharmve%2Ftimewarp","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcharmve%2Ftimewarp","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcharmve%2Ftimewarp/lists"}