Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/amrzv/awesome-colab-notebooks
Collection of google colaboratory notebooks for fast and easy experiments
https://github.com/amrzv/awesome-colab-notebooks
List: awesome-colab-notebooks
cnn colab-notebooks deep-learning deep-neural-networks generative-adversarial-network google-colab google-colab-notebook google-colab-notebooks google-colab-tutorial google-colaboratory google-colabs jupyter-notebooks machine-learning pytorch tensorflow tensorflow-tutorials
Last synced: 2 months ago
JSON representation
Collection of google colaboratory notebooks for fast and easy experiments
- Host: GitHub
- URL: https://github.com/amrzv/awesome-colab-notebooks
- Owner: amrzv
- License: mit
- Created: 2020-12-27T11:47:18.000Z (over 3 years ago)
- Default Branch: main
- Last Pushed: 2024-04-01T20:34:01.000Z (3 months ago)
- Last Synced: 2024-04-13T19:53:41.117Z (2 months ago)
- Topics: cnn, colab-notebooks, deep-learning, deep-neural-networks, generative-adversarial-network, google-colab, google-colab-notebook, google-colab-notebooks, google-colab-tutorial, google-colaboratory, google-colabs, jupyter-notebooks, machine-learning, pytorch, tensorflow, tensorflow-tutorials
- Language: Python
- Homepage:
- Size: 1.26 MB
- Stars: 1,197
- Watchers: 41
- Forks: 219
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Lists
- awesome-stars - awesome-colab-notebooks
- awesome-stars - amrzv/awesome-colab-notebooks - Collection of google colaboratory notebooks for fast and easy experiments (Python)
- my-awesome-stars - amrzv/awesome-colab-notebooks - Collection of google colaboratory notebooks for fast and easy experiments (Python)
- ultimate-awesome - awesome-colab-notebooks - Collection of google colaboratory notebooks for fast and easy experiments. (Other Lists / Julia Lists)
- awesome-generative-ai - 🔥
README
[](https://hits.seeyoufarm.com)
# Awesome colab notebooks collection for ML experiments
## Trending
| repositories | papers |
|---|---|
|
- Vahe1994/AQLM [](https://github.com/Vahe1994/AQLM)
- Doubiiu/DynamiCrafter [](https://github.com/Doubiiu/DynamiCrafter)
- Artiprocher/DiffSynth-Studio [](https://github.com/Artiprocher/DiffSynth-Studio)
- wilson1yan/VideoGPT [](https://github.com/wilson1yan/VideoGPT)
- horseee/DeepCache [](https://github.com/horseee/DeepCache)
- metavoiceio/metavoice-src [](https://github.com/metavoiceio/metavoice-src)
- QwenLM/Qwen-VL [](https://github.com/QwenLM/Qwen-VL)
- EleutherAI/lm-evaluation-harness [](https://github.com/EleutherAI/lm-evaluation-harness)
- KillianLucas/open-interpreter [](https://github.com/KillianLucas/open-interpreter)
- comfyanonymous/ComfyUI [](https://github.com/comfyanonymous/ComfyUI)
- piddnad/DDColor [](https://github.com/piddnad/DDColor)
- google-research/big_vision [](https://github.com/google-research/big_vision)
- microsoft/xr-development-for-beginners [](https://github.com/microsoft/xr-development-for-beginners)
- OpenTalker/video-retalking [](https://github.com/OpenTalker/video-retalking)
- ultralytics/ultralytics [](https://github.com/ultralytics/ultralytics)
- nd-ball/py-irt [](https://github.com/nd-ball/py-irt)
- drengskapur/colab2pdf [](https://github.com/drengskapur/colab2pdf)
- graphdeco-inria/gaussian-splatting [](https://github.com/graphdeco-inria/gaussian-splatting)
- haotian-liu/LLaVA [](https://github.com/haotian-liu/LLaVA)
- microsoft/autogen [](https://github.com/microsoft/autogen)
- ShishirPatil/gorilla [](https://github.com/ShishirPatil/gorilla)
- wenquanlu/HandRefiner [](https://github.com/wenquanlu/HandRefiner)
- facebookresearch/home-robot [](https://github.com/facebookresearch/home-robot)
- guoyww/animatediff [](https://github.com/guoyww/animatediff)
- IDEA-Research/GroundingDINO [](https://github.com/IDEA-Research/GroundingDINO)
- mistralai/mistral-src [](https://github.com/mistralai/mistral-src)
- TencentARC/PhotoMaker [](https://github.com/TencentARC/PhotoMaker)
- Stability-AI/StableCascade [](https://github.com/Stability-AI/StableCascade)
- Algolzw/daclip-uir [](https://github.com/Algolzw/daclip-uir)
- rohitgandikota/sliders [](https://github.com/rohitgandikota/sliders)
- sczhou/CodeFormer [](https://github.com/sczhou/CodeFormer)
- damian0815/compel [](https://github.com/damian0815/compel)
- LIDA [](https://doi.org/10.18653/v1/2023.acl-demo.11)
- SkyAR [](https://doi.org/10.1109/TIP.2022.3192717)
- DFL-Colab [](https://doi.org/10.1016/j.patcog.2023.109628)
- GHOST [](https://doi.org/10.1109/ACCESS.2022.3196668)
- SadTalker [](https://doi.org/10.1109/CVPR52729.2023.00836)
- py-irt [](https://doi.org/10.18653/v1/2021.acl-long.346)
- Gaussian Splatting [](https://doi.org/10.1145/3592433)
- PyTerrier [](https://doi.org/10.1145/3459637.3482013)
- ByteTrack [](https://doi.org/10.1007/978-3-031-20047-2_1)
- Motion Supervised co-part Segmentation [](https://doi.org/10.1109/ICPR48806.2021.9412520)
- VToonify [](https://doi.org/10.1145/3550454.3555437)
- DragGAN [](https://doi.org/10.1145/3588432.3591500)
- OWL-ViT [](https://doi.org/10.1007/978-3-031-20080-9_42)
- FILM [](https://doi.org/10.1007/978-3-031-20071-7_15)
- Rethinking Style Transfer: From Pixels to Parameterized Brushstrokes [](https://doi.org/10.1109/CVPR46437.2021.01202)
- LDM [](https://doi.org/10.1109/CVPR52688.2022.01042)
- LaSAFT [](https://doi.org/10.1109/ICASSP39728.2021.9413896)
- ConvNeXt [](https://doi.org/10.1109/CVPR52688.2022.01167)
- Mask2Former [](https://doi.org/10.1109/CVPR52688.2022.00135)
- SimSwap [](https://doi.org/10.1145/3394171.3413630)
- NAFNet [](https://doi.org/10.1007/978-3-031-20071-7_2)
- MSG-Net [](https://doi.org/10.1007/978-3-030-11018-5_32)
- Geometry-Free View Synthesis [](https://doi.org/10.1109/ICCV48922.2021.01409)
- Detic [](https://doi.org/10.1007/978-3-031-20077-9_21)
- AnimeGANv2 [](https://doi.org/10.1007/978-981-15-5577-0_18)
- deep-significance [](https://doi.org/10.18653/v1/p19-1266)
- FGVC [](https://doi.org/10.1007/978-3-030-58610-2_42)
- AlphaTensor [](https://doi.org/10.1038/s41586-022-05172-4)
- AlphaFold [](https://doi.org/10.1038/s41586-021-03819-2)
- Taming Transformers for High-Resolution Image Synthesis [](https://doi.org/10.1109/CVPR46437.2021.01268)
- NeRViS [](https://doi.org/10.1109/ICCV48922.2021.00230)
- EfficientDet [](https://doi.org/10.1109/CVPR42600.2020.01079)
## Research
| name | description | authors | links | colaboratory | update |
|------|-------------|:--------|:------|:------------:|:------:|
| Würstchen | Architecture for text-to-image synthesis that combines competitive performance with unprecedented cost-effectiveness for large-scale text-to-image diffusion models |
- [Pablo Pernias](https://github.com/pabloppp)
- [Dominic Rampas](https://github.com/dome272)
- [Mats Richter](https://scholar.google.com/citations?user=xtlV5SAAAAAJ)
- [Christopher Pal](https://www.polymtl.ca/expertises/pal-christopher-j)
- [Marc Aubreville](https://lme.tf.fau.de/person/aubreville/)
- [
](https://arxiv.org/abs/2306.00637) - [
](https://huggingface.co/blog/wuerstchen) - [
](https://www.reddit.com/r/StableDiffusion/comments/16hsklt/w%C3%BCrstchen_is_here_a_game_changing_fastest/) - [
](https://youtu.be/ogJsCPqgFMk)
| AQLM | Extreme Compression of Large Language Models via Additive Quantization |
- [Vage Egiazarian](https://github.com/Vahe1994)
- [Andrei Panferov](https://blog.panferov.org/)
- [Denis Kuznedelev](https://github.com/Godofnothing)
- [Elias Frantar](https://efrantar.github.io/) others
- [Artem Babenko](https://scholar.google.com/citations?user=2Kv3JP0AAAAJ)
- [Dan Alistarh](https://github.com/dalistarh)
- [](https://arxiv.org/abs/2401.06118)
- [](https://huggingface.co/docs/datasets/main/en/cache#cache-directory), [](https://huggingface.co/datasets/togethercomputer/RedPajama-Data-1T-Sample), [](https://huggingface.co/datasets/Vahe1994/AQLM)
- [](https://www.reddit.com/r/LearningMachines/comments/1atvrnl/240106118_extreme_compression_of_large_language/)
- [](https://youtu.be/Qx8PNk4OkUA), [](https://youtu.be/hAHBKAXO-88)
| Multi-LoRA Composition | LoRA Switch and LoRA Composite, approaches that aim to surpass traditional techniques in terms of accuracy and image quality, especially in complex compositions |
- [Ming Zhong](https://maszhongming.github.io/)
- [Yelong Shen](https://scholar.google.com/citations?user=S6OFEFEAAAAJ)
- [Shuohang Wang](https://www.microsoft.com/en-us/research/people/shuowa/)
- [Yadong Lu](https://adamlu123.github.io/) others
- [Yizhu Jiao](https://yzjiao.github.io/)
- [Siru Ouyang](https://ozyyshr.github.io/)
- [Donghan Yu](https://plusross.github.io/)
- [Jiawei Han](https://hanj.cs.illinois.edu/)
- [Weizhu Chen](https://www.microsoft.com/en-us/research/people/wzchen/)
- [](https://arxiv.org/abs/2402.16843)
- [
](https://medium.com/@letscodeai/multi-lora-composition-for-image-generation-f2706528c590) - [](https://www.reddit.com/r/ninjasaid13/comments/1b13q8s/multilora_composition_for_image_generation/)
- [
](https://x.com/MingZhong_/status/1762347881812443575?s=20) - [website](https://maszhongming.github.io/Multi-LoRA-Composition/)
| AlphaFold | Highly accurate protein structure prediction |
- [John Jumper](https://scholar.google.com/citations?user=a5goOh8AAAAJ)
- [Richard Evans](http://www.doc.ic.ac.uk/~re14/)
- [Alexander Pritzel](https://scholar.google.com/citations?user=GPgAyU0AAAAJ)
- [Tim Green](http://tfgg.me/) others
- [Michael Figurnov](https://figurnov.ru/)
- [Olaf Ronneberger](https://lmb.informatik.uni-freiburg.de/people/ronneber/)
- [Kathryn Tunyasuvunakool](https://scholar.google.com/citations?user=eEqNGagAAAAJ)
- [Russ Bates](https://scholar.google.com/citations?user=Koes5ewAAAAJ)
- [Augustin Žídek](https://augustin.zidek.eu/)
- [Anna Potapenko](http://apotapenko.com/)
- [Alex Bridgland](https://scholar.google.com/citations?user=VWmXKPMAAAAJ)
- [Clemens Meyer](https://scholar.google.com/citations?user=EWLZiM8AAAAJ)
- [Simon Kohl](https://www.simonkohl.com/)
- [Andrew Ballard](https://scholar.google.com/citations?user=syjQhAMAAAAJ)
- [Bernardino Romera-Paredes](https://sites.google.com/site/romeraparedes/)
- [Stanislav Nikolov](https://scholar.google.co.uk/citations?user=O-b7pBEAAAAJ)
- [Rishub Jain](http://rishub.me/)
- [blog post](https://deepmind.com/blog/article/alphafold-a-solution-to-a-50-year-old-grand-challenge-in-biology), [blog post](https://deepmind.com/blog/article/putting-the-power-of-alphafold-into-the-worlds-hands)
- [
](https://github.com/deepmind/tree), [](https://github.com/deepmind/chex) - [paper](https://www.nature.com/articles/s41586-021-03828-1)
- [
](https://paperswithcode.com/method/alphafold) - [
](https://en.wikipedia.org/wiki/AlphaFold) - [](https://www.youtube.com/watch?v=gg7WjuFs8F4), [](https://www.youtube.com/watch?v=B9PL__gVxLI)
| AMARETTO | Multiscale and multimodal inference of regulatory networks to identify cell circuits and their drivers shared and distinct within and across biological systems of human disease |
- [Nathalie Pochet](http://portals.broadinstitute.org/pochetlab/)
- [Olivier Gevaert](https://profiles.stanford.edu/olivier-gevaert)
- [Mohsen Nabian](https://github.com/monabiyan)
- [Jayendra Shinde](https://jayendrashinde91.github.io/) others
- [Celine Everaert](http://www.crig.ugent.be/en/node/510)
- [Thorin Tabor](http://thorin.tabcreations.com/)
- [bioconductor](https://bioconductor.org/packages/release/bioc/html/AMARETTO.html)
- [project](http://portals.broadinstitute.org/pochetlab/amaretto.html)
| TAPIR | Tracking Any Point with per-frame Initialization and temporal Refinement |
- [Carl Doersch](http://www.carldoersch.com/)
- [Yi Yang](https://yangyi02.github.io/)
- [Mel Vecerik](https://scholar.google.com/citations?user=Jvi_XPAAAAAJ)
- [Dilara Gokay](https://scholar.google.com/citations?user=cnbENAEAAAAJ) others
- [Ankush Gupta](https://ankushgupta.org/)
- [Yusuf Aytar](https://people.csail.mit.edu/yusuf/)
- [Joao Carreira](https://scholar.google.com/citations?user=IUZ-7_cAAAAJ)
- [Andrew Zisserman](https://www.robots.ox.ac.uk/~az/)
- [](https://arxiv.org/abs/2306.08637), [](https://arxiv.org/abs/2308.15975)
- [blog post](https://deepmind-tapir.github.io/), [blog post](https://deepmind-tapir.github.io/blogpost.html)
- [
](https://www.deepmind.com/open-source/kinetics) - [](https://github.com/google-research/kubric/tree/main/challenges/point_tracking)
- [](https://medium.com/@jumabek4044/what-is-tapir-tracking-any-point-with-per-frame-initialization-and-temporal-refinement-and-how-it-bdad9946dc53)
- [
](https://proceedings.neurips.cc/paper_files/paper/2022/hash/58168e8a92994655d6da3939e7cc0918-Abstract-Datasets_and_Benchmarks.html) - [](https://youtu.be/2HSHofqoJ9M), [](https://youtu.be/I1DQJH3v7Nk)
| LIDA | Tool for generating grammar-agnostic visualizations and infographics | [Victor Dibia](https://victordibia.com/) | [](https://doi.org/10.18653/v1/2023.acl-demo.11) [](https://github.com/microsoft/lida)
- [](https://arxiv.org/abs/2303.02927)
- [](https://github.com/victordibia/llmx), [](https://github.com/lida-project/lida-streamlit)
- [](https://medium.com/@c17hawke/lida-automatically-generate-visualization-and-with-llms-the-future-of-data-visualization-6bc556876b46)
- [project](https://microsoft.github.io/lida/)
- [](https://youtu.be/exYi9W-dhME), [](https://youtu.be/U9K1Cu45nMQ), [](https://youtu.be/6xcCwlDx6f8)
| ViT | Vision Transformer and MLP-Mixer Architectures |
- [Alexey Dosovitskiy](https://scholar.google.com/citations?user=FXNJRDoAAAAJ)
- [Lucas Beyer](http://lucasb.eyer.be)
- [Alexander Kolesnikov](https://github.com/akolesnikoff)
- [Dirk Weissenborn](https://github.com/dirkweissenborn) others
- [Xiaohua Zhai](https://github.com/xiaohuazhai)
- [Thomas Unterthiner](https://github.com/untom)
- [Mostafa Dehghani](https://www.mostafadehghani.com/)
- [Matthias Minderer](https://matthias.minderer.net/)
- [Georg Heigold](https://scholar.google.com/citations?user=WwqlChAAAAAJ)
- [Sylvain Gelly](https://scholar.google.com/citations?user=m7LvuTkAAAAJ)
- [Jakob Uszkoreit](https://scholar.google.com/citations?user=mOG0bwsAAAAJ)
- [Neil Houlsby](https://neilhoulsby.github.io/)
- [](https://arxiv.org/abs/2010.11929), [](https://arxiv.org/abs/2105.01601), [](https://arxiv.org/abs/2105.01601), [](https://arxiv.org/abs/2106.10270), [](https://arxiv.org/abs/2106.01548), [](https://arxiv.org/abs/2111.07991), [](https://arxiv.org/abs/2203.08065)
- [blog post](https://blog.research.google/2022/04/locked-image-tuning-adding-language.html)
- [](https://github.com/huggingface/pytorch-image-models), [](https://github.com/google/flaxformer)
- [
](https://www.kaggle.com/models) - [](https://medium.com/@weiwen21/an-image-is-worth-16x16-words-transformers-for-image-recognition-at-scale-957f88e53726)
- [](https://youtu.be/TrdevFK_am4), [](https://youtu.be/HZ4j_U3FC94), [](https://youtu.be/7K4Z8RqjWIk), [](https://youtu.be/oDtcobGQ7xU?si=C2EgZTESzhTXFSq6), [](https://youtu.be/v6xj_DG-UEo)
| DeepLabCut | Efficient method for markerless pose estimation based on transfer learning with deep neural networks that achieves excellent results with minimal training data |
- [Alexander Mathis](https://github.com/AlexEMG)
- [Pranav Mamidanna](https://pranavm19.github.io/)
- [Kevin Cury](https://kevincury.com/)
- [Taiga Abe](https://cellistigs.github.io/) others
- [Venkatesh Murthy](https://github.com/venkateshnmurthy)
- [Mackenzie Mathis](https://github.com/MMathisLab)
- [Matthias Bethge](https://bethgelab.org/)
- [](https://arxiv.org/abs/1605.03170), [](https://arxiv.org/abs/1804.03142), [](https://arxiv.org/abs/1909.11229), [](https://arxiv.org/abs/2009.00564), [](https://arxiv.org/abs/1909.13868), [](https://arxiv.org/abs/1909.13868)
- [
](https://hub.docker.com/r/deeplabcut/deeplabcut) - [forum](https://forum.image.sc/tag/deeplabcut)
- [](https://github.com/DeepLabCut/DLCutils), [](https://github.com/DeepLabCut/DeepLabCut-Workshop-Materials)
- [](https://medium.com/@cziscience/how-open-source-software-contributors-are-accelerating-biomedicine-1a5f50f6846a)
- [](https://twitter.com/DeepLabCut)
- [website](https://www.deeplabcut.org/)
- [](https://www.youtube.com/@deeplabcut7702), [](https://youtu.be/uWZu3rnj-kQ), [](https://youtu.be/Teb5r2TNAYs)
| 3D Ken Burns | A reference implementation of 3D Ken Burns Effect from a Single Image using PyTorch - given a single input image, it animates this still image with a virtual camera scan and zoom subject to motion parallax | [Manuel Romero](https://mrm8488.github.io/) | [](https://doi.org/10.1145/3355089.3356528) [](https://github.com/sniklaus/3d-ken-burns)
- [](https://arxiv.org/abs/1909.05483)
- [](https://www.youtube.com/watch?v=WrajxHHfRBA)
| VALL-E X | Cross-lingual neural codec language model for cross-lingual speech synthesis |
- [Ziqiang Zhang](https://github.com/onisac-K)
- [Long Zhou](https://long-zhou.github.io/)
- [Chengyi Wang](https://cywang97.github.io/)
- [Sanyuan Chen](https://sanyuan-chen.github.io/) others
- [Yu Wu](https://www.microsoft.com/en-us/research/people/yuwu1/)
- [Shujie Liu](https://www.microsoft.com/en-us/research/people/shujliu/)
- [Zhuo Chen](https://www.microsoft.com/en-us/research/people/zhuc/)
- [Yanqing Liu](https://scholar.google.com/citations?user=dIJFz4UAAAAJ)
- [Huaming Wang](https://scholar.google.com/citations?user=aJDLg5IAAAAJ)
- [Jinyu Li](https://www.microsoft.com/en-us/research/people/jinyli/)
- [Lei He](https://scholar.google.com/citations?user=EKl9yY8AAAAJ)
- [Sheng Zhao](https://scholar.google.com/citations?user=689bIIwAAAAJ)
- [Furu Wei](https://www.microsoft.com/en-us/research/people/fuwei/)
- [](https://arxiv.org/abs/2303.03926), [](https://arxiv.org/abs/2301.02111), [](https://arxiv.org/abs/2209.03143)
- [demo](https://plachtaa.github.io/)
- [
](https://discord.gg/qCBRmAnTxg) - [](https://github.com/lifeiteng/vall-e)
- [](https://huggingface.co/Plachta/VALL-E-X)
- [](https://medium.com/syncedreview/speak-a-foreign-language-in-your-own-voice-1dafa42f78d9)
- [project](https://www.microsoft.com/en-us/research/project/vall-e-x)
- [](https://youtu.be/7qgfoVFQmvk)
| PhotoMaker | Efficient personalized text-to-image generation method, which mainly encodes an arbitrary number of input ID images into a stack ID embedding for preserving ID information |
- [Zhen Li](https://paper99.github.io/)
- [Mingdeng Cao](https://github.com/ljzycmd)
- [Xintao Wang](https://xinntao.github.io/)
- [Zhongang Qi](https://scholar.google.com/citations?user=zJvrrusAAAAJ) others
- [Ming-Ming Cheng](https://mmcheng.net/cmm/)
- [Ying Shan](https://scholar.google.com/citations?user=4oXBp9UAAAAJ)
- [](https://arxiv.org/abs/2312.04461)
- [](https://github.com/bmaltais/PhotoMaker), [](https://github.com/sdbds/PhotoMaker-for-windows), [](https://github.com/ZHO-ZHO-ZHO/ComfyUI-PhotoMaker), [](https://github.com/mit-han-lab/fastcomposer), [](https://github.com/TencentARC/T2I-Adapter), [](https://github.com/tencent-ailab/IP-Adapter)
- [](https://huggingface.co/TencentARC/PhotoMaker)
- [](https://medium.com/@christopheverdier/photomaker-the-art-of-ai-consistent-characters-generation-cf2cd037bc3e)
- [project](https://photo-maker.github.io/)
- [](https://www.reddit.com/r/StableDiffusion/comments/197bfj9/tencentarc_releases_photomaker/)
- [](https://youtu.be/NWIdzTEk5O4), [](https://youtu.be/ZTck128jfFY)
| DDColor | End-to-end method with dual decoders for image colorization |
- [Xiaoyang Kang](https://piddnad.github.io/xiaoyangkang)
- [Tao Yang](https://cg.cs.tsinghua.edu.cn/people/~tyang/)
- [Wenqi Ouyang](https://vicky0522.github.io/Wenqi-Ouyang/)
- [Peiran Ren](https://scholar.google.com/citations?user=x5dEuxsAAAAJ) others
- [Lingzhi Li](https://lingzhili.com/)
- [Xuansong Xie](https://github.com/xungie)
- [](https://arxiv.org/abs/2212.11613)
- [](https://github.com/jixiaozhong/ColorFormer), [](https://github.com/KIMGEONUNG/BigColor)
| PASD | Pixel-aware stable diffusion network to achieve robust Real-ISR as well as personalized stylization |
- [Tao Yang](https://cg.cs.tsinghua.edu.cn/people/~tyang)
- [Peiran Ren](http://renpr.org/)
- [Xuansong Xie](https://github.com/xungie)
- [Lei Zhang](https://www4.comp.polyu.edu.hk/~cslzhang)
- [](https://arxiv.org/abs/2308.14469)
- [](https://github.com/pkuliyi2015/multidiffusion-upscaler-for-automatic1111)
- [](https://huggingface.co/runwayml/stable-diffusion-v1-5), [](https://huggingface.co/nitrosocke/mo-di-diffusion)
- [](https://www.reddit.com/r/StableDiffusion/comments/18qxe5q/pixelaware_stable_diffusion_for_realistic_image/)
| HandRefiner | Refining Malformed Hands in Generated Images by Diffusion-based Conditional Inpainting |
- [Wenquan Lu](https://github.com/wenquanlu)
- [Yufei Xu](https://scholar.google.com/citations?user=hlYWxX8AAAAJ)
- [Jing Zhang](https://scholar.google.com/citations?user=9jH5v74AAAAJ)
- [Chaoyue Wang](https://wang-chaoyue.github.io/)
- [Dacheng Tao](https://scholar.google.com/citations?user=RwlJNLcAAAAJ)
- [](https://arxiv.org/abs/2311.17957)
- [](https://github.com/Fannovel16/comfyui_controlnet_aux), [](https://github.com/Mikubill/sd-webui-controlnet), [](https://github.com/microsoft/MeshGraphormer)
- [](https://www.reddit.com/r/StableDiffusion/comments/1881z4v/handrefiner_refining_malformed_hands_in_generated/)
- [](https://youtu.be/Tt-Fyn1RA6c)
| audio2photoreal | Framework for generating full-bodied photorealistic avatars that gesture according to the conversational dynamics of a dyadic interaction |
- [Evonne Ng](https://people.eecs.berkeley.edu/~evonne_ng/)
- [Javier Romero](https://scholar.google.com/citations?user=Wx62iOsAAAAJ)
- [Timur Bagautdinov](https://scholar.google.ch/citations?user=oLi7xJ0AAAAJ)
- [Shaojie Bai](https://jerrybai1995.github.io/) others
- [Trevor Darrell](https://people.eecs.berkeley.edu/~trevor/)
- [Angjoo Kanazawa](https://people.eecs.berkeley.edu/~kanazawa/)
- [Alexander Richard](https://alexanderrichard.github.io/)
- [](https://arxiv.org/abs/2401.01885)
- [](https://github.com/facebookresearch/ca_body)
- [project](https://people.eecs.berkeley.edu/~evonne_ng/projects/audio2photoreal/)
- [](https://youtu.be/Y0GMaMtUynQ)
| GraphCast | Learning skillful medium-range global weather forecasting |
- [Remi Lam](https://github.com/remilam)
- [Alvaro Sanchez-Gonzalez](https://github.com/alvarosg)
- [Matthew Willson](https://github.com/mjwillson)
- [Peter Wirnsberger](https://pewi.org/) others
- [Meire Fortunato](https://scholar.google.com/citations?user=_fMHSIUAAAAJ)
- [Ferran Alet](https://scholar.google.com/citations?user=1lmBq3QAAAAJ)
- [Suman Ravuri](https://www.linkedin.com/in/suman-ravuri-81928082)
- [Timo Ewalds](https://github.com/tewalds)
- [Zach Eaton-Rosen](https://scholar.google.com/citations?user=mQ3zD_wAAAAJ)
- [Weihua Hu](https://weihua916.github.io/)
- [Alexander Merose](https://alex.merose.com/)
- [Stephan Hoyer](https://stephanhoyer.com/)
- [George Holland](https://www.linkedin.com/in/g-aracil-holland)
- [Oriol Vinyals](https://research.google/people/oriol-vinyals/)
- [Jacklynn Stott](https://linkedin.com/in/jacklynnstott)
- [Alexander Pritzel](https://github.com/a-pritzel)
- [Shakir Mohamed](https://www.shakirm.com/)
- [Peter Battaglia](https://scholar.google.com/citations?user=nQ7Ij30AAAAJ)
- [](https://arxiv.org/abs/2212.12794)
- [data](https://www.ecmwf.int/en/forecasts/datasets/reanalysis-datasets/era5)
- [](https://deepmind.google/discover/blog/graphcast-ai-model-for-faster-and-more-accurate-global-weather-forecasting/)
- [](https://github.com/google-deepmind/chex), [](https://github.com/dask/dask), [](https://github.com/google-deepmind/jaxline), [](https://github.com/google-deepmind/tree), [](https://github.com/mikedh/trimesh)
- [](https://towardsdatascience.com/graphcast-how-to-get-things-done-f2fd5630c5fb)
- [](https://youtu.be/BufUW7h9TB8), [](https://youtu.be/PD1v5PCJs_o), [](https://youtu.be/Eul-JN9Nwb0), [](https://youtu.be/BTyhgp9Hugc), [](https://youtu.be/aJ_H4exg0xU)
| ESM | Evolutionary Scale Modeling: Pretrained language models for proteins |
- [Zeming Lin](https://research.facebook.com/people/lin-zeming/)
- [Roshan Rao](https://rmrao.github.io/)
- [Brian Hie](https://brianhie.com/)
- [Zhongkai Zhu](https://www.linkedin.com/in/zhongkai-zhu-03a27424) others
- [Allan dos Santos Costa](https://scholar.google.com/citations?user=Zb4RsFsAAAAJ)
- [Maryam Fazel-Zarandi](https://www.maryamfazel.com/)
- [Tom Sercu](https://tom.sercu.me/)
- [Salvatore Candido](https://scholar.google.com/citations?user=BDgbhmEAAAAJ)
- [Alexander Rives](https://scholar.google.com/citations?user=vqb78-gAAAAJ)
- [Joshua Meier](https://scholar.google.com/citations?user=2M0OltAAAAAJ)
- [Robert Verkuil](https://dblp.org/pid/296/8930.html)
- [Jason Liu](https://www.linkedin.com/in/liujiayi/)
- [Chloe Hsu](https://chloe-hsu.com/)
- [Adam Lerer](https://scholar.google.com/citations?user=Ad6O4-0AAAAJ)
- [ESM Atlas](https://esmatlas.com/)
- [FSDP](https://fairscale.readthedocs.io/en/stable/api/nn/fsdp.html)
- [ICML](https://proceedings.mlr.press/v139/rao21a.html)
- [data](https://ftp.uniprot.org/pub/databases/uniprot/previous_releases/release-2018_03/uniref/)
- [](https://github.com/sokrypton/ColabFold)
- [](https://huggingface.co/docs/transformers/model_doc/esm)
- [paper](https://doi.org/10.1101/2022.07.20.500902), [paper](https://doi.org/10.1101/2021.07.09.450648), [paper](https://doi.org/10.1101/2022.04.10.487779), [paper](https://doi.org/10.1101/2022.12.21.521521)
- [pubmed](https://pubmed.ncbi.nlm.nih.gov/33876751/)
- [](https://youtu.be/N-eisTvUYrk), [](https://youtu.be/GHoE4VkDehY)
| CoTracker | Architecture that jointly tracks multiple points throughout an entire video |
- [Nikita Karaev](https://nikitakaraevv.github.io/)
- [Ignacio Rocco](https://www.irocco.info/)
- [Benjamin Graham](https://ai.meta.com/people/benjamin-graham/)
- [Natalia Neverova](https://nneverova.github.io/) others
- [Andrea Vedaldi](https://www.robots.ox.ac.uk/~vedaldi/)
- [Christian Rupprecht](https://chrirupp.github.io/)
- [](https://arxiv.org/abs/2307.07635), [](https://arxiv.org/abs/2303.11898)
- [](https://github.com/benjiebob/BADJA)
- [project](https://co-tracker.github.io/)
- [](https://youtu.be/w5QVc7BVGPA)
| LLaVA | Large Language and Vision Assistant, an end-to-end trained large multimodal model that connects a vision encoder and LLM for general-purpose visual and language understanding |
- [Haotian Liu](https://hliu.cc/)
- [Chunyuan Li](https://chunyuan.li/)
- [Qingyang Wu](https://qywu.github.io/)
- [Yong Jae Lee](https://pages.cs.wisc.edu/~yongjaelee/)
- [Yuheng Li](https://yuheng-li.github.io/)
- [](https://arxiv.org/abs/2304.08485), [](https://arxiv.org/abs/2310.03744), [](https://arxiv.org/abs/2306.00890), [](https://arxiv.org/abs/2309.09958), [](https://arxiv.org/abs/2306.14895)
- [demo](https://llava.hliu.cc/)
- [](https://github.com/ggerganov/llama.cpp/pull/3436), [](https://github.com/microsoft/LLaVA-Med), [](https://github.com/lm-sys/FastChat), [](https://github.com/UX-Decoder/Segment-Everything-Everywhere-All-At-Once), [](https://github.com/Luodian/Otter), [](https://github.com/Instruction-Tuning-with-GPT-4/GPT-4-LLM)
- [](https://huggingface.co/datasets/liuhaotian/LLaVA-Pretrain), [](https://huggingface.co/liuhaotian/LLaVA-Pretrained-Projectors)
- [](https://xthemadgenius.medium.com/how-to-use-llava-large-language-and-vision-assistant-732c666b5ed0)
- [project](https://llava-vl.github.io/)
- [](https://youtu.be/mkI7EPD1vp8), [](https://youtu.be/kx1VpI6JzsY), [](https://youtu.be/RxBSmbdJ1I8), [](https://youtu.be/mdYycY4lsuE), [](https://youtu.be/t7I46dxfmWs), [](https://youtu.be/KRAQkJC-XJU)
| Background Matting V2 | Real-time, high-resolution background replacement technique which operates at 30fps in 4K resolution, and 60fps for HD on a modern GPU |
- [Shanchuan Lin](https://github.com/PeterL1n)
- [Andrey Ryabtsev](https://github.com/andreyryabtsev)
- [Soumyadip Sengupta](https://github.com/senguptaumd)
- [Brian Curless](https://homes.cs.washington.edu/~curless/) others
- [Steve Seitz](https://www.smseitz.com/)
- [Ira Kemelmacher-Shlizerman](https://www.irakemelmacher.com/)
- [](https://arxiv.org/abs/2012.07810)
- [](https://github.com/senguptaumd/Background-Matting), [](https://github.com/andreyryabtsev/BGMv2-webcam-plugin-linux)
- [project](https://grail.cs.washington.edu/projects/background-matting-v2/)
- [](https://youtu.be/oMfPTeYDF9g), [](https://youtu.be/b7ps21MVyTA)
| Gaussian Splatting | State-of-the-art visual quality while maintaining competitive training times and importantly allow high-quality real-time (≥ 100 fps) novel-view synthesis at 1080p resolution |
- [Bernhard Kerbl](https://www.cg.tuwien.ac.at/staff/BernhardKerbl)
- [Georgios Kopanas](https://grgkopanas.github.io/)
- [Thomas Leimkühler](https://people.mpi-inf.mpg.de/~tleimkue/)
- [George Drettakis](http://www-sop.inria.fr/members/George.Drettakis/)
- [](https://arxiv.org/abs/2308.04079)
- [](https://huggingface.co/camenduru/gaussian-splatting)
- [](https://medium.com/axinc-ai/3d-gaussian-splatting-real-time-rendering-of-photorealistic-scenes-f7f1a47f060)
- [project](https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/)
- [](https://www.reddit.com/r/singularity/comments/163jeqa/3d_gaussian_splatting_for_realtime_radiance_field/)
- [](https://youtu.be/T_kXY43VZnk), [](https://youtu.be/UXtuigy_wYc), [](https://youtu.be/HVv_IQKlafQ), [](https://youtu.be/w43KV79LsFw), [](https://youtu.be/TLK3TDDcJFU), [](https://youtu.be/kShNYOuDnlI), [](https://youtu.be/juRMRej2d5c)
| DeepCache | Training-free paradigm that accelerates diffusion models from the perspective of model architecture |
- [Xinyin Ma](https://horseee.github.io/)
- [Gongfan Fang](https://fangggf.github.io/)
- [Xinchao Wang](https://sites.google.com/site/sitexinchaowang/)
- [](https://arxiv.org/abs/2312.00858)
- [](https://huggingface.co/docs/diffusers/v0.24.0/en/api/pipelines/stable_diffusion/text2img#diffusers.StableDiffusionPipeline)
- [project](https://horseee.github.io/Diffusion_DeepCache/)
- [](https://www.reddit.com/r/StableDiffusion/comments/18b40hh/deepcache_accelerating_diffusion_models_for_free/)
| MagicAnimate | Diffusion-based framework that aims at enhancing temporal consistency, preserving reference image faithfully, and improving animation fidelity |
- [Zhongcong Xu](https://scholar.google.com/citations?user=-4iADzMAAAAJ)
- [Jianfeng Zhang](http://jeff95.me/)
- [Jun Hao Liew](https://scholar.google.com/citations?user=8gm-CYYAAAAJ)
- [Hanshu Yan](https://hanshuyan.github.io/) others
- [Jiawei Liu](https://jia-wei-liu.github.io/)
- [Chenxu Zhang](https://zhangchenxu528.github.io/)
- [Jiashi Feng](https://sites.google.com/site/jshfeng/home)
- [Mike Shou](https://sites.google.com/view/showlab)
- [](https://arxiv.org/abs/2311.16498)
- [](https://huggingface.co/zcxu-eric/MagicAnimate), [](https://huggingface.co/runwayml/stable-diffusion-v1-5), [](https://huggingface.co/stabilityai/sd-vae-ft-mse)
- [](https://medium.com/@AIWorldBlog/revolutionizing-image-animation-with-magicanimate-technology-78cc94151915)
- [project](https://showlab.github.io/magicanimate/)
- [website](https://www.magicanimate.org/)
- [](https://youtu.be/td27SyA9M80), [](https://youtu.be/1pATjLFvNtY), [](https://youtu.be/HeXknItbMM8)
| DiffBIR | Towards Blind Image Restoration with Generative Diffusion Prior |
- [Xinqi Lin](https://github.com/0x3f3f3f3fun)
- [Jingwen He](https://github.com/hejingwenhejingwen)
- [Ziyan Chen](https://github.com/ziyannchen)
- [Zhaoyang Lyu](https://zhaoyanglyu.github.io/) others
- [Ben Fei](https://scholar.google.com/citations?user=skQROj8AAAAJ)
- [Bo Dai](http://daibo.info/)
- [Wanli Ouyang](https://wlouyang.github.io/)
- [Yu Qiao](https://mmlab.siat.ac.cn/yuqiao)
- [Chao Dong](http://xpixel.group/2010/01/20/chaodong.html)
- [](https://arxiv.org/abs/2308.15070)
- [](https://github.com/albarji/mixture-of-diffusers)
- [](https://huggingface.co/stabilityai/stable-diffusion-2-1-base)
- [project](https://0x3f3f3f3fun.github.io/projects/diffbir/)
- [](https://youtu.be/rGnrpxWjBOg), [](https://youtu.be/MIRiJGuGqsg)
| Parallel WaveGAN | State-of-the-art non-autoregressive models to build your own great vocoder | [Tomoki Hayashi](https://kan-bayashi.github.io/) | [](https://doi.org/10.1109/ICASSP40776.2020.9053795) [](https://github.com/kan-bayashi/ParallelWaveGAN)
- [](https://arxiv.org/abs/1910.11480), [](https://arxiv.org/abs/1910.06711), [](https://arxiv.org/abs/2005.05106)
- [demo](https://kan-bayashi.github.io/ParallelWaveGAN/)
- [](https://github.com/NVIDIA/tacotron2), [](https://github.com/espnet/espnet)
| AudioLDM | Text-to-audio system that is built on a latent space to learn the continuous audio representations from contrastive language-audio pretraining latents |
- [Haohe Liu](https://haoheliu.github.io/)
- [Zehua Chen](https://github.com/zehuachenImperial)
- [Yi Yuan](https://www.surrey.ac.uk/people/yi-yuan)
- [Xinhao Mei](https://xinhaomei.github.io/) others
- [Xubo Liu](https://liuxubo717.github.io/)
- [Danilo Mandic](https://www.imperial.ac.uk/people/d.mandic)
- [Wenwu Wang](http://personal.ee.surrey.ac.uk/Personal/W.Wang/)
- [Mark Plumbley](https://www.surrey.ac.uk/people/mark-plumbley)
- [](https://arxiv.org/abs/2301.12503)
- [](https://github.com/LAION-AI/CLAP), [](https://github.com/CompVis/stable-diffusion), [](https://github.com/toshas/torch-fidelity)
- [project](https://audioldm.github.io/)
- [](https://youtu.be/_0VTltNYhao)
| TabPFN | Neural network that learned to do tabular data prediction |
- [Noah Hollmann](https://github.com/noahho)
- [Samuel Müller](https://scholar.google.com/citations?user=pevYEjAAAAAJ)
- [Katharina Eggensperger](https://github.com/KEggensperger)
- [Frank Hutter](https://ml.informatik.uni-freiburg.de/profile/hutter/)
- [](https://arxiv.org/abs/2207.01848), [](https://arxiv.org/abs/2106.11189), [](https://arxiv.org/abs/2106.01342), [](https://arxiv.org/abs/2106.03253), [](https://arxiv.org/abs/2106.11189), [](https://arxiv.org/abs/2112.10510)
- [blog post](https://www.automl.org/tabpfn-a-transformer-that-solves-small-tabular-classification-problems-in-a-second/)
- [](https://twitter.com/tunguz/status/1578730907711655937)
- [](https://youtu.be/BGTO5N5-ack)
| Concept Sliders | Plug-and-play low rank adaptors applied on top of pretrained models |
- [Rohit Gandikota](https://rohitgandikota.github.io/)
- [Joanna Materzyńska](https://joaanna.github.io/)
- [Tingrui Zhou](https://www.p1at.dev/)
- [Antonio Torralba](https://groups.csail.mit.edu/vision/torralbalab/)
- [David Bau](https://baulab.info/)
- [](https://arxiv.org/abs/2311.12092), [](https://arxiv.org/abs/2207.12598)
- [](https://medium.com/@furkangozukara/concept-sliders-lora-adaptors-for-precise-control-in-diffusion-models-b7f6b36fabee)
- [](https://proceedings.neurips.cc/paper/2020/hash/49856ed476ad01fcff881d57e161d73f-Abstract.html)
- [project](https://sliders.baulab.info/)
- [](https://www.reddit.com/r/StableDiffusion/comments/180zon7/concept_sliders_lora_adaptors_for_precise_control/)
| Qwen-VL | Set of large-scale vision-language models designed to perceive and understand both text and images |
- [Jinze Bai](https://github.com/jinze1994)
- [Shuai Bai](https://github.com/ShuaiBai623)
- [Shusheng Yang](https://shushengyang.com/)
- [Shijie Wang](https://scholar.google.com/citations?user=DuAqyTwAAAAJ) others
- [Sinan Tan](https://www.tinytangent.com/)
- [Peng Wang](https://scholar.google.com/citations?user=7fjqA0YAAAAJ)
- [Junyang Lin](https://justinlin610.github.io/)
- [Chang Zhou](https://scholar.google.com/citations?user=QeSoG3sAAAAJ)
- [Jingren Zhou](http://www.cs.columbia.edu/~jrzhou/)
- [](https://arxiv.org/abs/2308.12966), [](https://arxiv.org/abs/2106.09685), [](https://arxiv.org/abs/2305.14314)
- [demo](https://modelscope.cn/studios/qwen/Qwen-VL-Chat-Demo/summary)
- [](https://discord.gg/z3GAxXZ9Ce)
- [](https://github.com/BradyFU/Awesome-Multimodal-Large-Language-Models/tree/Evaluation), [](https://github.com/OFA-Sys/TouchStone), [](https://github.com/PanQiWei/AutoGPTQ)
- [](https://huggingface.co/spaces/AILab-CVC/SEED-Bench_Leaderboard), [](https://huggingface.co/Qwen/Qwen-VL)
- [](https://youtu.be/ElrSJDg23Po), [](https://youtu.be/E3MS8GfGWj4), [](https://youtu.be/ju09YaO7BGA)
| AnimeGANv3 | Double-tail generative adversarial network for fast photo animation |
- [Gang Liu](https://github.com/lg0061408)
- [Xin Chen](https://github.com/TachibanaYoshino)
- [project](https://tachibanayoshino.github.io/AnimeGANv3/)
- [](https://youtu.be/EosubeJmAnE), [](https://youtu.be/5qLUflWb45E), [](https://youtu.be/iFjiaPlhVm4), [](https://youtu.be/vJqQQMRYKh0), [](https://youtu.be/0KaScDxgyBw), [](https://youtu.be/6WXhjXb5a-o)
| PixArt-Σ | Weak-to-Strong Training of Diffusion Transformer for 4K Text-to-Image Generation |
- [Junsong Chen](https://lawrence-cj.github.io/)
- [Chongjian Ge](https://chongjiange.github.io/)
- [Enze Xie](https://xieenze.github.io/)
- [Yue Wu](https://yuewuhkust.github.io/) others
- [Lewei Yao](https://scholar.google.com/citations?user=hqDyTg8AAAAJ)
- [Xiaozhe Ren](https://scholar.google.com/citations?user=3t2j87YAAAAJ)
- [Zhongdao Wang](https://zhongdao.github.io/)
- [Ping Luo](http://luoping.me/)
- [Huchuan Lu](https://scholar.google.com/citations?user=D3nE0agAAAAJ)
- [Zhenguo Li](https://scholar.google.com/citations?user=XboZC1AAAAAJ)
- [](https://arxiv.org/abs/2403.04692), [](https://arxiv.org/abs/2310.00426), [](https://arxiv.org/abs/2401.05252)
- [](https://discord.gg/rde6eaE5Ta)
- [](https://huggingface.co/spaces/PixArt-alpha/PixArt-alpha), [](https://huggingface.co/spaces/PixArt-alpha/PixArt-LCM)
- [project](https://pixart-alpha.github.io/PixArt-sigma-project/)
- [](https://www.reddit.com/r/PixArtSigma/)
| FILM | A frame interpolation algorithm that synthesizes multiple intermediate frames from two input images with large in-between motion |
- [Fitsum Reda](https://fitsumreda.github.io/)
- [Janne Kontkanen](https://scholar.google.com/citations?user=MnXc4JQAAAAJ)
- [Eric Tabellion](http://www.tabellion.org/et/)
- [Deqing Sun](https://deqings.github.io/) others
- [Caroline Pantofaru](https://scholar.google.com/citations?user=vKAKE1gAAAAJ)
- [Brian Curless](https://homes.cs.washington.edu/~curless/)
- [](https://arxiv.org/abs/2202.04901)
- [data](http://data.csail.mit.edu/tofu/testset/vimeo_interp_test.zip), [data](https://vision.middlebury.edu/flow/data), [data](https://people.cs.umass.edu/~hzjiang/projects/superslomo/UCF101_results.zip)
- [](https://github.com/sniklaus/softmax-splatting/blob/master/benchmark.py)
- [project](https://film-net.github.io/)
- [
](https://www.tensorflow.org/tutorials/load_data/tfrecord), [](https://www.tensorflow.org/api_docs/python/tf/train/Example), [](https://www.tensorflow.org/guide/saved_model) - [](https://youtu.be/OAD-BieIjH4)
| Show-1 | Hybrid model, dubbed as Show-1, which marries pixel-based and latent-based VDMs for text-to-video generation |
- [David Junhao Zhang](https://junhaozhang98.github.io/)
- [Jay Zhangjie Wu](https://zhangjiewu.github.io/)
- [Jiawei Liu](https://jia-wei-liu.github.io/)
- [Rui Zhao](https://ruizhaocv.github.io/) others
- [Lingmin Ran](https://siacorplab.nus.edu.sg/people/ran-lingmin/)
- [Yuchao Gu](https://ycgu.site/)
- [Difei Gao](https://scholar.google.com/citations?user=No9OsocAAAAJ)
- [Mike Zheng Shou](https://sites.google.com/view/showlab/home)
- [](https://arxiv.org/abs/2309.15818)
- [](https://huggingface.co/showlab/show-1-base), [](https://huggingface.co/showlab/show-1-interpolation), [](https://huggingface.co/showlab/show-1-sr1), [](https://huggingface.co/showlab/show-1-sr2), [](https://huggingface.co/damo-vilab/modelscope-damo-text-to-video-synthesis), [](https://huggingface.co/cerspense/zeroscope_v2_576w)
- [project](https://showlab.github.io/Show-1/)
| AudioSep | Foundation model for open-domain audio source separation with natural language queries |
- [Xubo Liu](https://liuxubo717.github.io/)
- [Qiuqiang Kong](https://qiuqiangkong.github.io/)
- [Yan Zhao](https://cliffzhao.github.io/)
- [Haohe Liu](https://haoheliu.github.io/) others
- [Yi Yuan](https://www.surrey.ac.uk/people/yi-yuan)
- [Yuzhuo Liu](https://github.com/redrabbit94)
- [Rui Xia](https://scholar.google.co.uk/citations?user=26oErxwAAAAJ)
- [Yuxuan Wang](https://scholar.google.com/citations?user=3RaOfJkAAAAJ)
- [Mark Plumbley](https://www.surrey.ac.uk/people/mark-plumbley)
- [Wenwu Wang](http://personal.ee.surrey.ac.uk/Personal/W.Wang/)
- [](https://arxiv.org/abs/2308.05037)
- [project](https://audio-agi.github.io/Separate-Anything-You-Describe/)
| DA-CLIP | Degradation-aware vision-language model to better transfer pretrained vision-language models to low-level vision tasks as a universal framework for image restoration |
- [Ziwei Luo](https://algolzw.github.io/)
- [Fredrik Gustafsson](http://www.fregu856.com/)
- [Zheng Zhao](https://zz.zabemon.com/)
- [Jens Sjölund](https://github.com/jsjol)
- [Thomas Schön](https://user.it.uu.se/~thosc112/index.html)
- [](https://arxiv.org/abs/2310.01018)
- [](https://github.com/Algolzw/image-restoration-sde)
- [](https://huggingface.co/weblzw/daclip-uir-ViT-B-32-irsde)
- [project](https://algolzw.github.io/daclip-uir/)
| SadTalker | Generates 3D motion coefficients of the 3DMM from audio and implicitly modulates a novel 3D-aware face render for talking head generation |
- [Wenxuan Zhang](https://github.com/Winfredy)
- [Xiaodong Cun](https://vinthony.github.io/academic/)
- [Xuan Wang](https://xuanwangvc.github.io/)
- [Yong Zhang](https://yzhang2016.github.io/) others
- [Xi Shen](https://xishen0220.github.io/)
- [Yu Guo](https://yuguo-xjtu.github.io/)
- [Ying Shan](https://scholar.google.com/citations?user=4oXBp9UAAAAJ)
- [Fei Wang](http://gr.xjtu.edu.cn/zh/web/feynmanw)
- [](https://arxiv.org/abs/2211.12194)
- [](https://discord.gg/rrayYqZ4tf)
- [](https://github.com/zhanglonghao1992/One-Shot_Free-View_Neural_Talking_Head_Synthesis), [](https://github.com/RenYurui/PIRender), [](https://github.com/microsoft/Deep3DFaceReconstruction), [](https://github.com/xinntao/facexlib), [](https://github.com/Zz-ww/SadTalker-Video-Lip-Sync), [](https://github.com/OpenTalker/DPE), [](https://github.com/FeiiYin/SPI), [](https://github.com/Mael-zys/T2M-GPT)
- [project](https://sadtalker.github.io/)
- [](https://youtu.be/AoIzJWnQw1M), [](https://youtu.be/fDgQcDL-qOc), [](https://youtu.be/BkSnM9cxkcM), [](https://youtu.be/7u0FYVPQ5rc)
| Musika | Music generation system that can be trained on hundreds of hours of music using a single consumer GPU, and that allows for much faster than real-time generation of music of arbitrary length on a consumer CPU |
- [Marco Pasini](https://github.com/marcoppasini)
- [Jan Schlüter](https://www.ofai.at/~jan.schlueter/)
- [](https://arxiv.org/abs/2208.08706), [](https://arxiv.org/abs/2005.08526)
- [data](https://magenta.tensorflow.org/datasets/maestro)
- [](https://github.com/hendriks73/tempo-cnn), [](https://github.com/CPJKU/madmom)
- [](https://huggingface.co/spaces/marcop/musika)
- [project](https://marcoppasini.github.io/musika)
- [](https://youtu.be/QBl8y2Z_i7Y), [](https://youtu.be/0l7OSM-bFvc)
| YOLOv6 | Single-stage object detection framework dedicated to industrial applications |
- [Kaiheng Weng](https://github.com/khwengXU)
- [Meng Cheng](https://github.com/MTChengMeng)
- [Yiduo Li](https://github.com/yili123123)
- [Xiangxiang Chu](https://scholar.google.com/citations?&user=jn21pUsAAAAJ)
- [Xiaolin Wei](https://scholar.google.com/citations?user=s5b7lU4AAAAJ)
- [](https://arxiv.org/abs/2209.02976), [](https://arxiv.org/abs/2301.05586)
- [blog post](https://learnopencv.com/yolov6-object-detection/)
- [data](https://cocodataset.org/#download)
- [
](https://yolov6-docs.readthedocs.io/zh_CN/latest/) - [](https://github.com/FeiGeChuanShu/ncnn-android-yolov6), [](https://github.com/DefTruth/lite.ai.toolkit/blob/main/lite/ort/cv/yolov6.cpp), [](https://github.com/Linaom1214/TensorRT-For-YOLO-Series), [](https://github.com/zhiqwang/yolov5-rt-stack/tree/main/deployment/tensorrt-yolov6)
- [](https://youtu.be/3OpwcGU7VvE), [](https://youtu.be/GJ0lVOE3a7c), [](https://youtu.be/3hqkbqJ5ag8), [](https://youtu.be/fFCWrMFH2UY)
| DreamGaussian | Algorithm to convert 3D Gaussians into textured meshes and apply a fine-tuning stage to refine the details |
- [Jiaxiang Tang](https://me.kiui.moe/)
- [Jiawei Ren](https://jiawei-ren.github.io/)
- [Hang Zhou](https://hangz-nju-cuhk.github.io/)
- [Ziwei Liu](https://liuziwei7.github.io/)
- [Gang Zeng](http://www.cis.pku.edu.cn/info/1177/1378.htm)
- [](https://arxiv.org/abs/2309.16653)
- [](https://github.com/graphdeco-inria/diff-gaussian-rasterization), [](https://github.com/NVlabs/nvdiffrast), [](https://github.com/hoffstadt/DearPyGui)
- [project](https://dreamgaussian.github.io/)
| ICON | Given a set of images, method estimates a detailed 3D surface from each image and then combines these into an animatable avatar |
- [Yuliang Xiu](https://xiuyuliang.cn/)
- [Jinlong Yang](https://is.mpg.de/~jyang)
- [Dimitrios Tzionas](https://ps.is.mpg.de/~dtzionas)
- [Michael Black](https://ps.is.mpg.de/~black)
- [](https://arxiv.org/abs/2112.09127)
- [](https://github.com/facebookresearch/KeypointNeRF), [](https://github.com/YadiraF/PIXIE), [](https://github.com/YuliangXiu/bvh-distance-queries), [](https://github.com/Project-Splinter/MonoPortDataset), [](https://github.com/ZhengZerong/PaMIR), [](https://github.com/Project-Splinter/MonoPort), [](https://github.com/shunsukesaito/SCANimate), [](https://github.com/google/aistplusplus_api)
- [](https://huggingface.co/spaces/Yuliang/ICON)
- [project](https://icon.is.tue.mpg.de/)
- [](https://youtu.be/hZd6AYin2DE)
| DINOv2 | Produce high-performance visual features that can be directly employed with classifiers as simple as linear layers on a variety of computer vision tasks; these visual features are robust and perform well across domains without any requirement for fine-tuning |
- [Maxime Oquab](https://scholar.google.com/citations?user=5vteYV8AAAAJ)
- [Timothée Darcet](https://github.com/TimDarcet)
- [Théo Moutakanni](https://github.com/TheoMoutakanni)
- [Huy Vo](https://huyvvo.github.io/) others
- [Marc Szafraniec](https://github.com/MarcSzafraniec/)
- [Vasil Khalidov](https://scholar.google.com/citations?user=tjazz3AAAAAJ)
- [Pierre Fernandez](https://pierrefdz.github.io/)
- [Daniel Haziza](https://scholar.google.com/citations?user=2eSKdFMAAAAJ)
- [Francisco Massa](https://github.com/fmassa)
- [Alaaeldin El-Nouby](https://aelnouby.github.io/)
- [Mahmoud Assran](http://www.midoassran.ca/)
- [Nicolas Ballas](https://scholar.google.com/citations?user=euUV4iUAAAAJ)
- [Wojciech Galuba](https://scholar.google.com/citations?user=jyaTX64AAAAJ)
- [Russell Howes](http://www.russellhowes.net/)
- [Po-Yao Huang](https://berniebear.github.io/)
- [Shang-Wen Li](https://swdanielli.github.io/)
- [Ishan Misra](http://imisra.github.io/)
- [Michael Rabbat](https://scholar.google.com/citations?user=cMPKe9UAAAAJ)
- [Vasu Sharma](https://vasusharma.github.io/)
- [Gabriel Synnaeve](https://syhw.github.io/)
- [Hu Xu](https://howardhsu.github.io/)
- [Hervé Jegou](https://github.com/jegou)
- [Julien Mairal](http://thoth.inrialpes.fr/people/mairal/)
- [Patrick Labatut](https://github.com/patricklabatut)
- [Armand Joulin](https://scholar.google.com/citations?user=kRJkDakAAAAJ)
- [Piotr Bojanowski](https://github.com/piotr-bojanowski)
- [](https://arxiv.org/abs/2304.07193)
- [blog post](https://ai.facebook.com/blog/dino-v2-computer-vision-self-supervised-learning/)
- [demo](https://dinov2.metademolab.com/)
- [](https://huggingface.co/docs/transformers/main/model_doc/dinov2)
- [](https://purnasaigudikandula.medium.com/dinov2-image-classification-visualization-and-paper-review-745bee52c826), [](https://towardsdatascience.com/meta-ais-another-revolutionary-large-scale-model-dinov2-for-image-feature-extraction-1114b287eadd)
- [](https://youtu.be/csEgtSh7jV4), [](https://www.youtube.com/live/KSZiJ4k28b4), [](https://youtu.be/RZEkdOc3szU)
| Neuralangelo | Framework for high-fidelity 3D surface reconstruction from RGB video captures |
- [Zhaoshuo Li](https://mli0603.github.io/)
- [Thomas Müller](https://tom94.net/)
- [Alex Evans](https://scholar.google.com/citations?user=ToqGImkAAAAJ)
- [Russell Taylor](https://www.cs.jhu.edu/~rht/) others
- [Mathias Unberath](https://mathiasunberath.github.io/)
- [Ming-Yu Liu](https://mingyuliu.net/)
- [Chen-Hsuan Lin](https://chenhsuanlin.bitbucket.io/)
- [](https://arxiv.org/abs/2306.03092)
- [blog post](https://blogs.nvidia.com/blog/2023/06/01/neuralangelo-ai-research-3d-reconstruction/)
- [](https://github.com/mli0603/BlenderNeuralangelo)
- [project](https://research.nvidia.com/labs/dir/neuralangelo/)
- [](https://youtu.be/PQMNCXR-WF8), [](https://youtu.be/Qpdw3SW54kI), [](https://youtu.be/lC2uPDfaTcE)
| GHOST | One-shot pipeline for image-to-image and image-to-video face swap solutions |
- [Alexander Groshev](https://github.com/AlexanderGroshev)
- [Anastasia Maltseva](https://github.com/NastyaMittseva)
- [Daniil Chesakov](https://github.com/Danyache)
- [Andrey Kuznetsov](https://www.linkedin.com/in/andrey-kuznetsov-70ab12127)
- [Denis Dimitrov](https://github.com/denndimitrov)
- [blog post](https://habr.com/ru/company/sberbank/blog/645919/)
- [data](https://www.robots.ox.ac.uk/~vgg/data/vgg_face/)
- [](https://hub.docker.com/r/wawa9000/ghost)
| OWL-ViT | Simple Open-Vocabulary Object Detection with Vision Transformers |
- [Matthias Minderer](http://matthias.minderer.net/)
- [Alexey Gritsenko](https://github.com/AlexeyG)
- [Austin Stone](https://github.com/AustinCStone)
- [Maxim Neumann](https://github.com/maximneumann) others
- [Dirk Weissenborn](https://github.com/dirkweissenborn)
- [Alexey Dosovitskiy](https://scholar.google.com/citations?user=FXNJRDoAAAAJ)
- [Aravindh Mahendran](https://github.com/aravindhm)
- [Anurag Arnab](https://github.com/anuragarnab)
- [Mostafa Dehghani](https://mostafadehghani.com/)
- [Zhuoran Shen](https://cmsflash.github.io/)
- [Xiao Wang](https://scholar.google.com/citations?user=ukyXqzMAAAAJ)
- [Xiaohua Zhai](https://github.com/xiaohuazhai)
- [Thomas Kipf](https://tkipf.github.io/)
- [Neil Houlsby](https://neilhoulsby.github.io/)
- [](https://arxiv.org/abs/2205.06230)
- [](https://huggingface.co/docs/transformers/model_doc/owlvit)
| Wav2Lip | A Lip Sync Expert Is All You Need for Speech to Lip Generation In the Wild |
- [Prajwal Renukanand](https://github.com/prajwalkr)
- [Rudrabha Mukhopadhyay](https://rudrabha.github.io/)
- [Vinay Namboodiri](https://vinaypn.github.io/)
- [C. V. Jawahar](https://faculty.iiit.ac.in/~jawahar/)
- [](https://arxiv.org/abs/2008.10010)
- [data](https://www.robots.ox.ac.uk/~vgg/data/lip_reading/lrs2.html)
- [demo](http://bhaasha.iiit.ac.in/lipsync/)
- [project](http://cvit.iiit.ac.in/research/projects/cvit-projects/a-lip-sync-expert-is-all-you-need-for-speech-to-lip-generation-in-the-wild/)
- [](https://www.youtube.com/watch?v=0fXaDCZNOJc)
| StyleGAN3 | Alias-Free Generative Adversarial Networks |
- [Tero Karras](https://research.nvidia.com/person/tero-karras)
- [Miika Aittala](https://research.nvidia.com/person/Miika-Aittala)
- [Samuli Laine](https://research.nvidia.com/person/Samuli-Laine)
- [Erik Härkönen](https://github.com/harskish) others
- [Janne Hellsten](https://research.nvidia.com/person/Janne-Hellsten)
- [Jaakko Lehtinen](https://users.aalto.fi/~lehtinj7/)
- [Timo Aila](https://research.nvidia.com/person/timo-aila)
- [](https://arxiv.org/abs/2106.12423), [](https://arxiv.org/abs/1706.08500), [](https://arxiv.org/abs/1801.01401), [](https://arxiv.org/abs/1904.06991), [](https://arxiv.org/abs/1812.04948), [](https://arxiv.org/abs/1606.03498)
- [](https://github.com/NVlabs/stylegan3-detector), [](https://github.com/NVlabs/ffhq-dataset), [](https://github.com/NVlabs/metfaces-dataset), [](https://github.com/NVlabs/stylegan2-ada-pytorch), [](https://github.com/NVlabs/stylegan2-ada)
- [](https://proceedings.neurips.cc/paper/2021/hash/076ccd93ad68be51f23707988e934906-Abstract.html)
- [project](https://nvlabs.github.io/stylegan3)
| Big GAN | Large Scale GAN Training for High Fidelity Natural Image Synthesis |
- [Andrew Brock](https://github.com/ajbrock)
- [Jeff Donahue](https://jeffdonahue.com/)
- [Karen Simonyan](https://scholar.google.com/citations?user=L7lMQkQAAAAJ)
- [](https://arxiv.org/abs/1809.11096)
| MakeItTalk | A method that generates expressive talking-head videos from a single facial image with audio as the only input |
- [Yang Zhou](https://people.umass.edu/~yangzhou/)
- [Xintong Han](http://users.umiacs.umd.edu/~xintong/)
- [Eli Shechtman](https://research.adobe.com/person/eli-shechtman/)
- [Jose Echevarria](http://www.jiechevarria.com/) others
- [Evangelos Kalogerakis](https://people.cs.umass.edu/~kalo/)
- [Dingzeyu Li](https://dingzeyu.li/)
- [](https://arxiv.org/abs/2004.12992)
- [data](https://drive.google.com/drive/folders/1EwuAy3j1b9Zc1MsidUfxG_pJGc_cV60O)
- [project](https://people.umass.edu/~yangzhou/MakeItTalk/)
- [](https://www.youtube.com/watch?v=vUMGKASgbf8)
| HiDT | A generative image-to-image model and a new upsampling scheme that allows to apply image translation at high resolution |
- [Denis Korzhenkov](https://github.com/denkorzh)
- [Gleb Sterkin](https://github.com/belkakari)
- [Sergey Nikolenko](https://logic.pdmi.ras.ru/~sergey/)
- [Victor Lempitsky](http://sites.skoltech.ru/compvision/members/vilem/)
- [](https://arxiv.org/abs/2003.08791)
- [project](https://saic-mdal.github.io/HiDT/)
- [](https://www.youtube.com/playlist?list=PLuvGzlEQXT1KQuKrfBBEWh2f3PToxyeM5), [](https://www.youtube.com/watch?v=EWKAgwgqXB4)
| CutLER | Simple approach for training unsupervised object detection and segmentation models |
- [Xudong Wang](https://people.eecs.berkeley.edu/~xdwang/)
- [Rohit Girdhar](https://rohitgirdhar.github.io/)
- [Stella Yu](https://www1.icsi.berkeley.edu/~stellayu/)
- [Ishan Misra](https://imisra.github.io/)
- [](https://arxiv.org/abs/2301.11320), [](https://arxiv.org/abs/1706.02677)
- [](https://detectron2.readthedocs.io/en/latest/tutorials/datasets.html)
- [project](http://people.eecs.berkeley.edu/~xdwang/projects/CutLER/)
| Recognize Anything & Tag2Text | Vision language pre-training framework, which introduces image tagging into vision-language models to guide the learning of visual-linguistic features |
- [Xinyu Huang](https://xinyu1205.github.io/)
- [Youcai Zhang](https://github.com/Coler1994)
- [Jinyu Ma](https://github.com/majinyu666)
- [Zhaoyang Li](https://github.com/ZhaoyangLi-nju) others
- [Yanchun Xie](https://scholar.google.com/citations?user=T0xk9-wAAAAJ)
- [Yuzhuo Qin](https://scholar.google.com/citations?user=5ZG65AkAAAAJ)
- [Tong Luo](https://ieeexplore.ieee.org/author/37089387319)
- [Yaqian Li](https://openreview.net/profile?id=~Yaqian_Li1)
- [Yandong Guo](http://www.lsl.zone/)
- [Yandong Guo](https://scholar.google.com/citations?user=fWDoWsQAAAAJ)
- [Lei Zhang](https://www.leizhang.org/)
- [](https://arxiv.org/abs/2306.03514), [](https://arxiv.org/abs/2303.05657)
- [](https://github.com/OpenGVLab/Ask-Anything), [](https://github.com/positive666/Prompt-Can-Anything)
- [](https://artgor.medium.com/paper-review-recognize-anything-a-strong-image-tagging-model-9e5e1c6dd0af)
- [project](https://recognize-anything.github.io/), [project](https://recognize-anything.github.io/)
| Thin-Plate Spline Motion Model | End-to-end unsupervised motion transfer framework |
- [Jian Zhao](https://scholar.google.com/citations?user=OKm5CQYAAAAJ)
- [Hui Zhang](https://scholar.google.com/citations?user=w3mzCiwAAAAJ)
- [](https://arxiv.org/abs/2203.14367)
- [](https://github.com/AliaksandrSiarohin/monkey-net), [](https://github.com/AliaksandrSiarohin/video-preprocessing), [](https://github.com/AliaksandrSiarohin/pose-evaluation), [](https://github.com/TalkUHulk/Image-Animation-Turbo-Boost)
- [](https://huggingface.co/spaces/CVPR/Image-Animation-using-Thin-Plate-Spline-Motion-Model)
- [supp](https://cloud.tsinghua.edu.cn/f/f7b8573bb5b04583949f/?dl=1)
| DragGAN | Drag Your GAN: Interactive Point-based Manipulation on the Generative Image Manifold |
- [Xingang Pan](https://xingangpan.github.io/)
- [Ayush Tewari](https://ayushtewari.com/)
- [Thomas Leimkühler](https://people.mpi-inf.mpg.de/~tleimkue/)
- [Lingjie Liu](https://lingjie0206.github.io/) others
- [Abhimitra Meka](https://www.meka.page/)
- [Christian Theobalt](https://people.mpi-inf.mpg.de/~theobalt/)
- [](https://arxiv.org/abs/2305.10973)
- [](https://github.com/NVlabs/stylegan3#requirements)
- [project](https://vcai.mpi-inf.mpg.de/projects/DragGAN/)
- [](https://twitter.com/XingangP)
| Fast Segment Anything | CNN Segment Anything Model trained using only 2% of the SA-1B dataset published by SAM authors |
- [Xu Zhao](https://scholar.google.com/citations?user=F0cYEyAAAAAJ)
- [Wenchao Ding](https://github.com/berry-ding)
- [Yongqi An](https://github.com/an-yongqi)
- [Yinglong Du](https://github.com/YinglongDu) others
- [Tao Yu](https://github.com/tianjinren)
- [Min Li](https://github.com/limin2021)
- [Ming Tang](https://www.researchgate.net/profile/Ming-Tang-2)
- [Jinqiao Wang](https://scholar.google.com/citations?user=7_BkyxEAAAAJ)
- [](https://arxiv.org/abs/2306.12156), [](https://arxiv.org/abs/2112.10003)
- [](https://github.com/ChuRuaNh0/FastSam_Awsome_TensorRT)
- [](https://medium.com/@mahimairaja/so-what-exactly-is-fastsam-the-ultimate-guide-ddae21d3b486)
- [](https://youtu.be/yHNPyqazYYU), [](https://youtu.be/SslzS0AsiAw), [](https://www.youtube.com/live/qvqkjP1wCDE)
| MobileSAM | Towards Lightweight SAM for Mobile Applications |
- [Chaoning Zhang](https://github.com/ChaoningZhang)
- [Dongshen Han](https://github.com/dongshenhan)
- [Yu Qiao](https://github.com/qiaoyu1002)
- [Jung Uk Kim](https://visualai.khu.ac.kr/) others
- [Sung-Ho Bae](https://scholar.google.com/citations?user=EULut5oAAAAJ)
- [Seungkyu Lee](https://scholar.google.com/citations?user=3Pf6C6cAAAAJ)
- [Choong Seon Hong](https://scholar.google.com/citations?user=oKANWloAAAAJ)
- [](https://arxiv.org/abs/2306.14289)
- [](https://github.com/jolibrain/joliGEN), [](https://github.com/akbartus/MobileSAM-in-the-Browser), [](https://github.com/qiaoyu1002/Inpaint-Anything), [](https://github.com/qiaoyu1002/Personalize-SAM), [](https://github.com/Jumpat/SegmentAnythingin3D), [](https://github.com/vietanhdev/anylabeling), [](https://github.com/wangsssky/SonarSAM), [](https://github.com/continue-revolution/sd-webui-segment-anything)
- [](https://twitter.com/_akhaliq/status/1674410573075718145)
- [](https://youtu.be/eTEfq_kWabQ)
| Grounding DINO | Marrying DINO with Grounded Pre-Training for Open-Set Object Detection |
- [Shilong Liu](https://github.com/SlongLiu)
- [Zhaoyang Zeng](https://scholar.google.com/citations?user=U_cvvUwAAAAJ)
- [Tianhe Ren](https://rentainhe.github.io/)
- [Feng Li](https://scholar.google.com/citations?user=ybRe9GcAAAAJ) others
- [Hao Zhang](https://scholar.google.com/citations?user=B8hPxMQAAAAJ)
- [Jie Yang](https://yangjie-cv.github.io/)
- [Chunyuan Li](https://scholar.google.com/citations?user=Zd7WmXUAAAAJ)
- [Jianwei Yang](https://jwyang.github.io/)
- [Hang Su](https://www.suhangss.me/)
- [Jun Zhu](https://scholar.google.com/citations?user=axsP38wAAAAJ)
- [Lei Zhang](https://www.leizhang.org/)
- [](https://arxiv.org/abs/2303.05499)
- [](https://github.com/IDEA-Research/DINO), [](https://github.com/UX-Decoder/Semantic-SAM), [](https://github.com/OptimalScale/DetGPT), [](https://github.com/IDEA-Research/OpenSeeD), [](https://github.com/UX-Decoder/Segment-Everything-Everywhere-All-At-Once), [](https://github.com/microsoft/X-Decoder/tree/xgpt), [](https://github.com/IDEA-Research/detrex)
- [](https://paperswithcode.com/sota/zero-shot-object-detection-on-mscoco?p=grounding-dino-marrying-dino-with-grounded), [](https://paperswithcode.com/sota/zero-shot-object-detection-on-odinw?p=grounding-dino-marrying-dino-with-grounded), [](https://paperswithcode.com/sota/object-detection-on-coco-minival?p=grounding-dino-marrying-dino-with-grounded), [](https://paperswithcode.com/sota/object-detection-on-coco?p=grounding-dino-marrying-dino-with-grounded)
- [](https://youtu.be/wxWDt5UiwY8), [](https://youtu.be/cMa77r3YrDk), [](https://youtu.be/C4NqaRBz_Kw), [](https://youtu.be/oEQYStnF2l8)
| T5X | Modular, composable, research-friendly framework for high-performance, configurable, self-service training, evaluation, and inference of sequence models at many scales |
- [Adam Roberts](https://github.com/adarob)
- [Hyung Won Chung](https://github.com/hwchung27)
- [Anselm Levskaya](https://anselmlevskaya.com/)
- [Gaurav Mishra](https://research.google/people/GauravMishra/) others
- [James Bradbury](https://github.com/jekbradbury)
- [Daniel Andor](https://github.com/andorardo)
- [Sharan Narang](https://github.com/sharannarang)
- [Brian Lester](https://blester125.com/)
- [Colin Gaffney](https://github.com/cpgaffney1)
- [Afroz Mohiuddin](https://scholar.google.com/citations?user=xUtt1j0AAAAJ)
- [Curtis Hawthorne](https://github.com/cghawthorne)
- [Aitor Lewkowycz](https://scholar.google.com/citations?user=Yum1ah0AAAAJ)
- [Alex Salcianu](https://scholar.google.com/citations?user=HSrT1wsAAAAJ)
- [Marc van Zee](https://github.com/marcvanzee)
- [Jacob Austin](https://jacobaustin123.github.io/)
- [Sebastian Goodman](https://github.com/0x0539)
- [Livio Baldini Soares](https://liviosoares.github.io/)
- [Haitang Hu](https://hthu.github.io/)
- [Sasha Tsvyashchenko](https://endl.ch/)
- [Aakanksha Chowdhery](http://www.achowdhery.com/)
- [Jasmijn Bastings](https://jasmijn.ninja/)
- [Jannis Bulian](http://bulian.org/)
- [Xavier Garcia](https://scholar.google.com/citations?user=Y2Hio6MAAAAJ)
- [Jianmo Ni](https://nijianmo.github.io/)
- [Kathleen Kenealy](https://scholar.google.com/citations?&user=HgRBC5gAAAAJ)
- [Jonathan Clark](http://www.cs.cmu.edu/~jhclark/)
- [Dan Garrette](http://www.dhgarrette.com/)
- [James Lee-Thorp](https://scholar.google.com/citations?user=qsPv098AAAAJ)
- [Colin Raffel](https://colinraffel.com/)
- [Noam Shazeer](https://scholar.google.com/citations?user=wsGvgA8AAAAJ)
- [Marvin Ritter](https://scholar.google.com/citations?user=arcf5FgAAAAJ)
- [Maarten Bosma](https://scholar.google.com/citations?user=wkeFQPgAAAAJ)
- [Alexandre Passos](https://www.ic.unicamp.br/~tachard/)
- [Jeremy Maitin-Shepard](https://research.google/people/JeremyMaitinShepard/)
- [Noah Fiedel](https://scholar.google.com/citations?user=XWpV9DsAAAAJ)
- [Brennan Saeta](https://github.com/saeta)
- [Ryan Sepassi](https://ryansepassi.com/)
- [Alexander Spiridonov](https://research.google/people/AlexanderSpiridonov/)
- [Joshua Newlan](https://github.com/joshnewlan)
- [Andrea Gesmundo](https://github.com/agesmundo)
- [](https://arxiv.org/abs/2203.17189), [](https://arxiv.org/abs/1910.10683)
- [](https://t5x.readthedocs.io/en/latest/)
- [](https://github.com/tensorflow/mesh), [](https://github.com/tensorflow/serving)
- [](https://www.tensorflow.org/datasets/catalog/wmt_t2t_translate), [](https://www.tensorflow.org/guide/data), [](https://www.tensorflow.org/tensorboard)
| First Order Motion Model for Image Animation | Transferring facial movements from video to image | [Aliaksandr Siarohin](https://aliaksandrsiarohin.github.io/aliaksandr-siarohin-website/) | [](https://github.com/AliaksandrSiarohin/first-order-model)
- [](https://papers.nips.cc/paper/2019/hash/31c0b36aef265d9221af80872ceb62f9-Abstract.html)
- [project](https://aliaksandrsiarohin.github.io/first-order-model-website/)
- [](https://www.youtube.com/watch?v=u-0cQ-grXBQ)
| MMS | The Massively Multilingual Speech project expands speech technology from about 100 languages to over 1000 by building a single multilingual speech recognition model supporting over 1100 languages, language identification models able to identify over 4000 languages, pretrained models supporting over 1400 languages, and text-to-speech models for over 1100 languages |
- [Vineel Pratap](https://github.com/vineelpratap)
- [Andros Tjandra](https://github.com/androstj)
- [Bowen Shi](https://scholar.google.com/citations?user=xqyoorYAAAAJ)
- [Paden Tomasello](https://scholar.google.com/citations?user=sBtWMGYAAAAJ) others
- [Arun Babu](https://scholar.google.com/citations?user=oJfoTakAAAAJ)
- [Sayani Kundu](https://www.linkedin.com/in/sayani-kundu)
- [Ali Elkahky](https://scholar.google.com/citations?user=KB3S8RoAAAAJ)
- [Zhaoheng Ni](https://scholar.google.com/citations?user=SYFMSNsAAAAJ)
- [Apoorv Vyas](https://apoorv2904.github.io/)
- [Maryam Fazel-Zarandi](https://www.maryamfazel.com/)
- [Alexei Baevski](https://github.com/alexeib)
- [Yossi Adi](https://www.cs.huji.ac.il/~adiyoss/)
- [Xiaohui Zhang](https://github.com/xiaohui-zhang)
- [Wei-Ning Hsu](https://wnhsu.github.io/)
- [Alexis Conneau](https://github.com/aconneau)
- [Michael Auli](https://github.com/michaelauli)
- [](https://arxiv.org/abs/2305.13516)
- [blog post](https://ai.facebook.com/blog/multilingual-model-speech-recognition/)
- [](https://huggingface.co/docs/transformers/main/en/model_doc/mms), [](https://huggingface.co/facebook/mms-cclms/), [](https://huggingface.co/blog/mms_adapters)
- [](https://youtu.be/GEzxHxWys2s), [](https://youtu.be/g06agCmxS7I)
| DFL-Colab | This project provides you IPython Notebook to use DeepFaceLab | [chervonij](https://github.com/chervonij) | [](https://doi.org/10.1016/j.patcog.2023.109628) [](https://github.com/iperov/DeepFaceLab)
- [](https://arxiv.org/abs/2005.05535)
- [guide](https://mrdeepfakes.com/forums/thread-guide-deepfacelab-google-colab-tutorial)
- [](https://www.youtube.com/channel/UCTKBl8kB6DJ_qLnk1NGDGbQ)
| FAB | Flow AIS Bootstrap uses AIS to generate samples in regions where the flow is a poor approximation of the target, facilitating the discovery of new modes |
- [Laurence Midgley](https://lollcat.github.io/laurence-midgley/)
- [Vincent Stimper](https://is.mpg.de/person/vstimper)
- [Gregor N. C. Simm](https://www.gncs.me/)
- [Bernhard Schölkopf](https://scholar.google.com/citations?user=DZ-fHPgAAAAJ)
- [José Miguel Hernández-Lobato](https://jmhl.org/)
- [](https://arxiv.org/abs/2208.01893)
- [](https://github.com/lollcat/fab-jax-old), [](https://github.com/deepmind/annealed_flow_transport)
- [](https://youtu.be/xQQXvOWu9nE)
| CodeFormer | Transformer-based prediction network to model global composition and context of the low-quality faces for code prediction, enabling the discovery of natural faces that closely approximate the target faces even when the inputs are severely degraded |
- [Shangchen Zhou](https://shangchenzhou.com/)
- [Kelvin Chan](https://ckkelvinchan.github.io/)
- [Chongyi Li](https://li-chongyi.github.io/)
- [Chen Change Loy](https://www.mmlab-ntu.com/person/ccloy/)
- [](https://arxiv.org/abs/2206.11253)
- [](https://github.com/samb-t/unleashing-transformers), [](https://github.com/deepcam-cn/yolov5-face), [](https://github.com/xinntao/facexlib)
- [](https://proceedings.neurips.cc/paper_files/paper/2022/hash/c573258c38d0a3919d8c1364053c45df-Abstract-Conference.html)
- [project](https://shangchenzhou.com/projects/CodeFormer/)
- [](https://youtu.be/d3VDpkXlueI), [](https://youtu.be/PtwWu-FugbA), [](https://youtu.be/ORtYP8NW4T0), [](https://youtu.be/xc5lKOKBCcg)
| Text2Video-Zero | Text-to-Image Diffusion Models are Zero-Shot Video Generators |
- [Levon Khachatryan](https://github.com/lev1khachatryan)
- [Andranik Movsisyan](https://github.com/19and99)
- [Vahram Tadevosyan](https://www.linkedin.com/in/vtadevosian)
- [Roberto Henschel](https://github.com/rob-hen) others
- [Zhangyang Wang](https://www.ece.utexas.edu/people/faculty/atlas-wang)
- [Shant Navasardyan](https://scholar.google.com/citations?user=VJSh59sAAAAJ)
- [Humphrey Shi](https://www.humphreyshi.com/)
- [](https://arxiv.org/abs/2303.13439), [](https://arxiv.org/abs/1907.01341), [](https://arxiv.org/abs/2303.17604)
- [](https://github.com/dbolya/tomesd), [](https://github.com/JiauZhang/Text2Video-Zero), [](https://github.com/camenduru/text2video-zero-colab), [](https://github.com/SHI-Labs/Text2Video-Zero-sd-webui)
- [](https://huggingface.co/docs/diffusers/api/pipelines/text_to_video_zero)
- [project](https://text2video-zero.github.io/)
- [video](https://www.dropbox.com/s/uv90mi2z598olsq/Text2Video-Zero.MP4)
- [](https://youtu.be/beeDJJz-Q0A), [](https://youtu.be/97-1GYPtz0M)
| Segment Anything | The Segment Anything Model produces high quality object masks from input prompts such as points or boxes, and it can be used to generate masks for all objects in an image |
- [Alexander Kirillov](https://alexander-kirillov.github.io/)
- [Eric Mintun](https://ericmintun.github.io/)
- [Nikhila Ravi](https://nikhilaravi.com/)
- [Hanzi Mao](https://hanzimao.me/) others
- [Chloé Rolland](https://www.linkedin.com/in/chlo%C3%A9-rolland-223135a/)
- [Laura Gustafson](https://scholar.google.com/citations?user=c8IpF9gAAAAJ)
- [Tete Xiao](https://tetexiao.com/)
- [Spencer Whitehead](https://www.spencerwhitehead.com/)
- [Alex Berg](http://acberg.com/)
- [Wan-Yen Lo](https://github.com/wanyenlo)
- [Piotr Dollar](https://pdollar.github.io/)
- [Ross Girshick](https://www.rossgirshick.info/)
- [](https://arxiv.org/abs/2304.02643)
- [blog post](https://ai.facebook.com/research/publications/segment-anything/), [blog post](https://ai.facebook.com/blog/segment-anything-foundation-model-image-segmentation/)
- [data](https://ai.facebook.com/datasets/segment-anything/)
- [website](https://segment-anything.com/)
- [](https://youtu.be/2O_vecl28OA), [](https://youtu.be/fVeW9a6wItM), [](https://youtu.be/FjYE0tKWOiY)
| EVA3D | High-quality unconditional 3D human generative model that only requires 2D image collections for training |
- [Fangzhou Hong](https://hongfz16.github.io/)
- [Zhaoxi Chen](https://frozenburning.github.io/)
- [Yushi Lan](https://github.com/NIRVANALAN)
- [Liang Pan](https://github.com/paul007pl)
- [Ziwei Liu](https://liuziwei7.github.io/)
- [](https://arxiv.org/abs/2210.04888)
- [project](https://hongfz16.github.io/projects/EVA3D.html)
- [](https://youtu.be/JNV0FJ0aDWM), [](https://youtu.be/M-kyvzTQrBI)
| Stable Dreamfusion | Using a pretrained 2D text-to-image diffusion model to perform text-to-3D synthesis |
- [Jiaxiang Tang](https://me.kiui.moe/)
- [Ben Poole](https://cs.stanford.edu/~poole/)
- [Ajay Jain](https://ajayj.com/)
- [Jon Barron](https://jonbarron.info/)
- [Ben Mildenhall](https://bmild.github.io/)
- [](https://arxiv.org/abs/2209.14988)
- [](https://github.com/ashawkey/torch-ngp), [](https://github.com/hoffstadt/DearPyGui)
- [](https://huggingface.co/runwayml/stable-diffusion-v1-5)
- [project](https://dreamfusion3d.github.io/)
- [
](https://pytorch.org/docs/stable/cpp_extension.html#torch.utils.cpp_extension.load) - [](https://youtu.be/uM5NPodZZ1U?t=219), [](https://youtu.be/zWD5ZR5GtJM), [](https://youtu.be/L3G0dx1Q0R8), [](https://youtu.be/dIgDbBTztUM)
| UniFormer | Unified Transformer for Efficient Spatiotemporal Representation Learning |
- [Kunchang Li](https://github.com/Andy1621)
- [Yali Wang](https://scholar.google.com/citations?user=hD948dkAAAAJ)
- [Peng Gao](https://gaopengcuhk.github.io/)
- [Guanglu Song](https://songguanglu.github.io/) others
- [Yu Liu](https://liuyu.us/)
- [Hongsheng Li](http://www.ee.cuhk.edu.hk/~hsli/)
- [Yu Qiao](http://mmlab.siat.ac.cn/yuqiao/index.html)
- [](https://arxiv.org/abs/2201.04676), [](https://arxiv.org/abs/2201.09450), [](https://arxiv.org/abs/2104.10858), [](https://arxiv.org/abs/2103.17239)
- [](https://github.com/zihangJiang/TokenLabeling), [](https://github.com/facebookresearch/deit), [](https://github.com/facebookresearch/fvcore), [](https://github.com/rwightman/pytorch-image-models), [](https://github.com/facebookincubator/submitit), [](https://github.com/facebookresearch/SlowFast), [](https://github.com/SwinTransformer/Swin-Transformer-Object-Detection), [](https://github.com/whai362/PVT/tree/v2/segmentation), [](https://github.com/HRNet/HRFormer/tree/main/pose)
- [](https://huggingface.co/spaces/Sense-X/uniformer_image_demo), [](https://huggingface.co/spaces/Sense-X/uniformer_video_demo), [](https://huggingface.co/spaces/Andy1621/uniformer_image_detection), [](https://huggingface.co/spaces/Andy1621/uniformer_image_segmentation)
| PIFuHD | Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitization |
- [Shunsuke Saito](https://github.com/shunsukesaito)
- [Tomas Simon](http://www.cs.cmu.edu/~tsimon/)
- [Jason Saragih](https://scholar.google.com/citations?user=ss-IvjMAAAAJ)
- [Hanbyul Joo](https://jhugestar.github.io/)
- [](https://arxiv.org/abs/2004.00452)
- [](https://youtu.be/uEDqCxvF5yc), [](https://www.youtube.com/watch?v=8qnwbbDS8xk)
| VideoReTalking | System to edit the faces of a real-world talking head video according to input audio, producing a high-quality and lip-syncing output video even with a different emotion |
- [Kun Cheng](https://github.com/kunncheng)
- [Xiaodong Cun](https://vinthony.github.io/)
- [Yong Zhang](https://yzhang2016.github.io/)
- [Menghan Xia](https://menghanxia.github.io/) others
- [Fei Yin](https://feiiyin.github.io/)
- [Mingrui Zhu](https://web.xidian.edu.cn/mrzhu/en/index.html)
- [Xuan Wang](https://xuanwangvc.github.io/)
- [Jue Wang](https://juewang725.github.io/)
- [Nannan Wang](https://web.xidian.edu.cn/nnwang/en/index.html)
- [](https://arxiv.org/abs/2211.14758)
- [](https://github.com/donydchen/ganimation_replicate), [](https://github.com/RenYurui/PIRender), [](https://github.com/OpenTalker/StyleHEAT), [](https://github.com/FeiiYin/SPI), [](https://github.com/Mael-zys/T2M-GPT)
- [](https://xthemadgenius.medium.com/making-videos-talk-right-syncing-lips-with-sound-using-videoretalking-611428084bbc)
- [project](https://opentalker.github.io/video-retalking/)
- [](https://www.reddit.com/r/StableDiffusion/comments/178krha/videoretalking/)
- [](https://youtu.be/pttsTrQ-fko), [](https://youtu.be/2Lkw8AmmRn0), [](https://youtu.be/RJ8YK_K4Ne0)
| Visual ChatGPT | Connects ChatGPT and a series of Visual Foundation Models to enable sending and receiving images during chatting |
- [Chenfei Wu](https://github.com/chenfei-wu)
- [Shengming Yin](https://github.com/shengming-yin)
- [Weizhen Qi](https://github.com/WeizhenQ)
- [Xiaodong Wang](https://wang-xiaodong1899.github.io/) others
- [Zecheng Tang](https://github.com/CODINNLG)
- [Nan Duan](https://nanduan.github.io/)
- [](https://arxiv.org/abs/2303.04671)
- [](https://github.com/hwchase17/langchain), [](https://github.com/lllyasviel/ControlNet), [](https://github.com/timothybrooks/instruct-pix2pix), [](https://github.com/timojl/clipseg)
- [](https://youtu.be/0UfXlFUwLms), [](https://youtu.be/7YEiEyfPF5U)
| Tune-A-Video | One-Shot Tuning of Image Diffusion Models for Text-to-Video Generation |
- [Jay Zhangjie Wu](https://zhangjiewu.github.io/)
- [Yixiao Ge](https://geyixiao.com/)
- [Xintao Wang](https://xinntao.github.io/)
- [Stan Weixian Lei](https://github.com/StanLei52) others
- [Yuchao Gu](https://ycgu.site/)
- [Yufei Shi](https://scholar.google.com/citations?user=rpnlkwEAAAAJ)
- [Wynne Hsu](https://www.comp.nus.edu.sg/~whsu/)
- [Ying Shan](https://scholar.google.com/citations?user=4oXBp9UAAAAJ)
- [Xiaohu Qie](https://scholar.google.com/citations?user=mk-F69UAAAAJ)
- [Mike Zheng Shou](https://sites.google.com/view/showlab)
- [](https://arxiv.org/abs/2212.11565), [](https://arxiv.org/abs/2112.10752)
- [](https://huggingface.co/Tune-A-Video-library), [](https://huggingface.co/stabilityai/stable-diffusion-2-1), [](https://huggingface.co/sd-dreambooth-library)
- [project](https://tuneavideo.github.io/)
- [](https://youtu.be/uzF6CTtjn-g), [](https://youtu.be/uUlp1_ExsGQ)
| LaMa | Resolution-robust Large Mask Inpainting with Fourier Convolutions |
- [Roman Suvorov](https://github.com/windj007)
- [Elizaveta Logacheva](https://github.com/elimohl)
- [Anton Mashikhin](https://www.linkedin.com/in/heyt0ny/)
- [Anastasia Remizova](https://github.com/feathernox) others
- [Arsenii Ashukha](https://ashukha.com/)
- [Aleksei Silvestrov](https://www.linkedin.com/in/%D0%B0%D0%BB%D0%B5%D0%BA%D1%81%D0%B5%D0%B9-%D1%81%D0%B8%D0%BB%D1%8C%D0%B2%D0%B5%D1%81%D1%82%D1%80%D0%BE%D0%B2-141b99b6/)
- [Naejin Kong](https://github.com/naejin-kong)
- [Harshith Goka](https://github.com/h9399-goka)
- [Kiwoong Park](https://github.com/kyoong-park)
- [Victor Lempitsky](http://sites.skoltech.ru/compvision/members/vilem/)
- [](https://arxiv.org/abs/2109.07161)
- [](https://github.com/andy971022/auto-lama), [](https://github.com/richzhang/PerceptualSimilarity), [](https://github.com/Po-Hsun-Su/pytorch-ssim), [](https://github.com/mseitzer/pytorch-fid)
- [project](https://saic-mdal.github.io/lama-project/)
| GPEN | GAN Prior Embedded Network for Blind Face Restoration in the Wild |
- [Tao Yang](https://cg.cs.tsinghua.edu.cn/people/~tyang/)
- [Peiran Ren](https://scholar.google.com/citations?&user=x5dEuxsAAAAJ)
- [Xuansong Xie](https://scholar.google.com/citations?user=M0Ei1zkAAAAJ)
- [Lei Zhang](http://www4.comp.polyu.edu.hk/~cslzhang/)
- [](https://arxiv.org/abs/2105.06070)
- [demo](https://vision.aliyun.com/experience/detail?spm=a211p3.14020179.J_7524944390.17.66cd4850wVDkUQ&tagName=facebody&children=EnhanceFace)
- [](https://github.com/biubug6/Pytorch_Retinaface), [](https://github.com/rosinality/stylegan2-pytorch)
| Disco Diffusion | A frankensteinian amalgamation of notebooks, models and techniques for the generation of AI Art and Animations |
- [Max Ingham](https://github.com/somnai-dreams)
- [Adam Letts](https://linktr.ee/gandamu)
- [Daniel Russell](https://github.com/russelldc)
- [Chigozie Nri](https://github.com/chigozienri)
- [](https://github.com/openai/guided-diffusion)
- [](https://youtu.be/_DtWfh9oS54), [](https://youtu.be/gWxmtdZL8FE), [](https://youtu.be/yVJB6oD0_gM)
| Open-Unmix | A deep neural network reference implementation for music source separation, applicable for researchers, audio engineers and artists |
- [Fabian-Robert Stöter](http://faroit.com/)
- [Antoine Liutkus](https://github.com/aliutkus)
- [data](https://sigsep.github.io/datasets/musdb.html#musdb18-compressed-stems)
- [](https://github.com/sigsep/norbert)
- [project](https://sigsep.github.io/open-unmix/)
- [](https://paperswithcode.com/sota/music-source-separation-on-musdb18?p=open-unmix-a-reference-implementation-for)
- [](https://www.youtube.com/playlist?list=PLhA3b2k8R3t0VpYCpCTU2B1h604rvnV4N)
| GrooVAE | Some applications of machine learning for generating and manipulating beats and drum performances |
- [Jon Gillick](https://www.jongillick.com/)
- [Adam Roberts](https://github.com/adarob)
- [Jesse Engel](https://github.com/jesseengel)
- [](https://arxiv.org/abs/1905.06118)
- [blog post](https://g.co/magenta/groovae)
- [data](https://g.co/magenta/groove-datasets)
- [web app](https://groove-drums.glitch.me/)
- [](https://www.youtube.com/watch?v=x2YLmXzovDo)
| Multitrack MusicVAE | The models in this notebook are capable of encoding and decoding single measures of up to 8 tracks, optionally conditioned on an underlying chord |
- [Ian Simon](https://github.com/iansimon)
- [Adam Roberts](https://github.com/adarob)
- [Colin Raffel](https://colinraffel.com//)
- [Jesse Engel](https://github.com/jesseengel) others
- [Curtis Hawthorne](https://github.com/cghawthorne)
- [Douglas Eck](https://github.com/douglaseck)
- [](https://arxiv.org/abs/1806.00195)
- [blog post](http://g.co/magenta/multitrack)
| MusicVAE | A Hierarchical Latent Vector Model for Learning Long-Term Structure in Music |
- [Adam Roberts](https://github.com/adarob)
- [Jesse Engel](https://github.com/jesseengel)
- [Colin Raffel](https://colinraffel.com//)
- [Curtis Hawthorne](https://github.com/cghawthorne)
- [Douglas Eck](https://github.com/douglaseck)
- [](https://arxiv.org/abs/1803.05428)
- [blog post](https://g.co/magenta/music-vae)
- [project](https://magenta.tensorflow.org/music-vae)
- [](https://www.youtube.com/playlist?list=PLBUMAYA6kvGU8Cgqh709o5SUvo-zHGTxr)
| Learning to Paint | Learning to Paint With Model-based Deep Reinforcement Learning | [Manuel Romero](https://mrm8488.github.io/) | [](https://doi.org/10.1109/ICCV.2019.00880)
- [](https://arxiv.org/abs/1903.04411)
- [](https://www.reddit.com/r/reinforcementlearning/comments/b5lpfl/learning_to_paint_with_modelbased_deep/)
- [](https://www.youtube.com/watch?v=YmOgKZ5oipk)
| VALL-E | Language modeling approach for text to speech synthesis |
- [Chengyi Wang](https://cywang97.github.io/)
- [Sanyuan Chen](https://sanyuan-chen.github.io/)
- [Yu Wu](https://www.microsoft.com/en-us/research/people/yuwu1/)
- [Ziqiang Zhang](https://github.com/zz12375) others
- [Long Zhou](https://long-zhou.github.io/)
- [Shujie Liu](https://www.microsoft.com/en-us/research/people/shujliu/)
- [Zhuo Chen](https://www.microsoft.com/en-us/research/people/zhuc/)
- [Yanqing Liu](https://scholar.google.com/citations?user=dIJFz4UAAAAJ)
- [Huaming Wang](https://scholar.google.com/citations?user=aJDLg5IAAAAJ)
- [Jinyu Li](https://www.microsoft.com/en-us/research/people/jinyli/)
- [Lei He](https://scholar.google.com/citations?user=EKl9yY8AAAAJ)
- [Sheng Zhao](https://scholar.google.com/citations?user=689bIIwAAAAJ)
- [Furu Wei](https://www.microsoft.com/en-us/research/people/fuwei/)
- [](https://arxiv.org/abs/2301.02111)
- [](https://github.com/facebookresearch/encodec), [](https://github.com/microsoft/DeepSpeed#requirements)
- [](https://vidrihmarko.medium.com/mind-blowing-vall-e-neural-codec-language-models-are-zero-shot-text-to-speech-synthesizers-f002560ecd6)
- [project](https://valle-demo.github.io/)
- [](https://www.reddit.com/r/MachineLearning/comments/104ixvi/r_neural_codec_language_models_are_zeroshot_text/)
- [](https://youtu.be/F6HSsVIkqIU), [](https://youtu.be/ZehhrrQGmt4), [](https://youtu.be/-3MPZxRxvV4), [](https://youtu.be/ha2WjP7zfno)
| Instant-NGP | Instant Neural Graphics Primitives with a Multiresolution Hash Encoding |
- [Thomas Müller](https://tom94.net/)
- [Alex Evans](https://research.nvidia.com/person/alex-evans)
- [Christoph Schied](https://research.nvidia.com/person/christoph-schied)
- [Alexander Keller](https://research.nvidia.com/person/alex-keller)
- [](https://arxiv.org/abs/2201.05989)
- [blog post](https://developer.nvidia.com/blog/getting-started-with-nvidia-instant-nerfs/)
- [](https://github.com/NVlabs/tiny-cuda-nn), [](https://github.com/IDLabMedia/large-lightfields-dataset), [](https://github.com/nickponline/dd-nerf-dataset), [](https://github.com/ocornut/imgui), [](https://github.com/nothings/stb)
- [project](https://nvlabs.github.io/instant-ngp/)
- [tutorial](https://www.nvidia.com/en-us/on-demand/session/siggraph2022-sigg22-s-16/)
- [](https://youtu.be/j8tMk-GE8hY), [](https://youtu.be/8GbENSmdVeE), [](https://youtu.be/DJ2hcC1orc4), [](https://youtu.be/z3-fjYzd0BA)
| Fourier Feature Networks | Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains |
- [Matthew Tancik](https://www.matthewtancik.com/)
- [Pratul Srinivasan](https://pratulsrinivasan.github.io/)
- [Ben Mildenhall](https://bmild.github.io/)
- [Sara Fridovich-Keil](https://people.eecs.berkeley.edu/~sfk/) others
- [Nithin Raghavan](https://cseweb.ucsd.edu//~n2raghavan/)
- [Utkarsh Singhal](https://scholar.google.com/citations?user=lvA86MYAAAAJ)
- [Ravi Ramamoorthi](https://cseweb.ucsd.edu//~ravir/)
- [Jon Barron](https://jonbarron.info/)
- [Ren Ng](https://www2.eecs.berkeley.edu/Faculty/Homepages/yirenng.html)
- [](https://arxiv.org/abs/1806.07572)
- [](https://proceedings.neurips.cc/paper/2020/hash/55053683268957697aa39fba6f231c68-Abstract.html), [](https://papers.nips.cc/paper/2007/hash/013a006f03dbc5392effeb8f18fda755-Abstract.html)
- [project](https://bmild.github.io/fourfeat/)
- [](https://youtu.be/nVA6K6Sn2S4)
| AlphaPose | Whole-Body Regional Multi-Person Pose Estimation and Tracking in Real-Time |
- [Hao-Shu Fang](https://fang-haoshu.github.io/)
- [Jiefeng Li](https://jeffli.site/)
- [Hongyang Tang](https://github.com/tang-hy)
- [Chao Xu](https://www.isdas.cn/) others
- [Haoyi Zhu](https://www.haoyizhu.site/)
- [Yuliang Xiu](https://xiuyuliang.cn/)
- [Yong-Lu Li](https://dirtyharrylyl.github.io/)
- [Cewu Lu](https://scholar.google.com/citations?user=QZVQEWAAAAAJ)
- [](https://arxiv.org/abs/2211.03375)
- [](https://github.com/tycoer/AlphaPose_jittor), [](https://github.com/Fang-Haoshu/Halpe-FullBody)
- [project](https://www.mvig.org/research/alphapose.html)
- [](https://youtu.be/uze6chg-YeU), [](https://youtu.be/Z2WPd59pRi8), [](https://youtu.be/qW4lb9tnA3I), [](https://youtu.be/_qtNzylm1XI)
| HybrIK | Hybrid Analytical-Neural Inverse Kinematics Solution for 3D Human Pose and Shape Estimation |
- [Jiefeng Li](https://jeffli.site/)
- [Chao Xu](https://www.isdas.cn/)
- [Zhicun Chen](https://github.com/chenzhicun)
- [Siyuan Bian](https://github.com/biansy000) others
- [Lixin Yang](https://lixiny.github.io/)
- [Cewu Lu](https://www.mvig.org/)
- [](https://arxiv.org/abs/2011.14672)
- [](https://github.com/mks0601/3DMPPE_POSENET_RELEASE)
- [project](https://jeffli.site/HybrIK/)
- [](https://paperswithcode.com/sota/3d-human-pose-estimation-on-3dpw?p=hybrik-a-hybrid-analytical-neural-inverse)
- [supp](https://openaccess.thecvf.com/content/CVPR2021/supplemental/Li_HybrIK_A_Hybrid_CVPR_2021_supplemental.zip)
- [](https://youtu.be/tvwnXXH7xIw)
| Demucs | Hybrid Spectrogram and Waveform Source Separation | [Alexandre Défossez](https://ai.honu.io/) | [](https://github.com/facebookresearch/demucs)
- [](https://arxiv.org/abs/2111.03600), [](https://arxiv.org/abs/2010.01733), [](https://arxiv.org/abs/2109.05418), [](https://arxiv.org/abs/1805.02410)
- [](https://github.com/adefossez/mdx21_demucs), [](https://github.com/CarlGao4/Demucs-Gui), [](https://github.com/kuielab/mdx-net-submission), [](https://github.com/f90/Wave-U-Net)
| MotionDiffuse | The first diffusion model-based text-driven motion generation framework, which demonstrates several desired properties over existing methods |
- [Mingyuan Zhang](https://mingyuan-zhang.github.io/)
- [Zhongang Cai](https://caizhongang.github.io/)
- [Liang Pan](https://github.com/paul007pl)
- [Fangzhou Hong](https://hongfz16.github.io/) others
- [Xinying Guo](https://gxyes.github.io/)
- [Lei Yang](https://scholar.google.com/citations?user=jZH2IPYAAAAJ)
- [Ziwei Liu](https://liuziwei7.github.io/)
- [](https://arxiv.org/abs/2208.15001)
- [](https://huggingface.co/spaces/mingyuan/MotionDiffuse)
- [project](https://mingyuan-zhang.github.io/projects/MotionDiffuse.html)
- [](https://youtu.be/U5PTnw490SA)
| VToonify | Leverages the mid- and high-resolution layers of StyleGAN to render high-quality artistic portraits based on the multi-scale content features extracted by an encoder to better preserve the frame details |
- [Shuai Yang](https://williamyang1991.github.io/)
- [Liming Jiang](https://liming-jiang.com/)
- [Ziwei Liu](https://liuziwei7.github.io/)
- [Chen Change Loy](https://www.mmlab-ntu.com/person/ccloy/)
- [](https://arxiv.org/abs/2209.11224), [](https://arxiv.org/abs/2001.02890)
- [](https://github.com/rosinality/stylegan2-pytorch), [](https://github.com/zllrunning/face-parsing.PyTorch), [](https://github.com/zhujiapeng/LowRankGAN)
- [](https://huggingface.co/spaces/PKUWilliamYang/VToonify), [](https://huggingface.co/PKUWilliamYang/VToonify/tree/main/models)
- [project](https://www.mmlab-ntu.com/project/vtoonify/)
- [](https://youtu.be/0_OmVhDgYuY)
| PyMAF | Pyramidal Mesh Alignment Feedback loop in regression network for well-aligned body mesh recovery and extend it for the recovery of expressive full-body models |
- [Hongwen Zhang](https://hongwenzhang.github.io/)
- [Yating Tian](https://github.com/tinatiansjz)
- [Yuxiang Zhang](https://zhangyux15.github.io/)
- [Mengcheng Li](https://github.com/Dw1010) others
- [Liang An](https://anl13.github.io/)
- [Zhenan Sun](http://www.cbsr.ia.ac.cn/users/znsun/)
- [Yebin Liu](https://www.liuyebin.com/)
- [](https://arxiv.org/abs/2207.06400), [](https://arxiv.org/abs/2103.16507)
- [](https://github.com/facebookresearch/eft), [](https://github.com/HongwenZhang/DaNet-DensePose2SMPL), [](https://github.com/facebookresearch/DensePose), [](https://github.com/Microsoft/human-pose-estimation.pytorch)
- [project](https://www.liuyebin.com/pymaf-x/)
- [](https://youtu.be/yqEmznSKjYI), [](https://youtu.be/ylOB0wCeV34)
| AlphaTensor | Discovering faster matrix multiplication algorithms with reinforcement learning |
- [Alhussein Fawzi](http://www.alhusseinfawzi.info/)
- [Matej Balog](http://matejbalog.eu/)
- [Aja Huang](https://en.wikipedia.org/wiki/Aja_Huang)
- [Thomas Hubert](https://scholar.google.com/citations?user=WXG0QfMAAAAJ) others
- [Bernardino Romera-Paredes](https://sites.google.com/site/romeraparedes/)
- [Mohammadamin Barekatain](http://barekatain.me/)
- [Alexander Novikov](https://scholar.google.com/citations?user=jMUkLqwAAAAJ)
- [Francisco Ruiz](https://franrruiz.github.io/)
- [Julian Schrittwieser](https://www.furidamu.org/)
- [Grzegorz Swirszcz](https://sites.google.com/site/grzegorzswirszcz/home)
- [David Silver](https://www.davidsilver.uk/)
- [Demis Hassabis](https://en.wikipedia.org/wiki/Demis_Hassabis)
- [Pushmeet Kohli](https://sites.google.com/site/pushmeet/)
- [blog post](https://www.deepmind.com/blog/discovering-novel-algorithms-with-alphatensor)
- [](https://youtu.be/3N3Bl5AA5QU), [](https://youtu.be/gpYnDls4PdQ), [](https://youtu.be/IYgZS2EvnLI), [](https://youtu.be/8ILk4Wjo5rc)
| Swin2SR | Novel Swin Transformer V2, to improve SwinIR for image super-resolution, and in particular, the compressed input scenario |
- [Marcos Conde](https://mv-lab.github.io/)
- [Ui-Jin Choi](https://github.com/Choiuijin1125)
- [Maxime Burchi](https://scholar.google.com/citations?user=7S_l2eAAAAAJ)
- [Radu Timofte](https://www.informatik.uni-wuerzburg.de/computervision/home/)
- [](https://arxiv.org/abs/2209.11345), [](https://arxiv.org/abs/2108.10257), [](https://arxiv.org/abs/2208.11184), [](https://arxiv.org/abs/2111.09883)
- [](https://github.com/cszn/KAIR/), [](https://github.com/mv-lab/AISP), [](https://github.com/microsoft/Swin-Transformer)
- [](https://huggingface.co/spaces/jjourney1125/swin2sr)
- [](https://www.kaggle.com/code/jesucristo/super-resolution-demo-swin2sr-official/), [](https://www.kaggle.com/datasets/jesucristo/super-resolution-benchmarks), [](https://www.kaggle.com/jinssaa/official-swin2sr-demo-results/)
| Functa | From data to functa: Your data point is a function and you can treat it like one |
- [Emilien Dupont](https://emiliendupont.github.io/)
- [Hyunjik Kim](https://hyunjik11.github.io/)
- [Ali Eslami](http://arkitus.com/)
- [Danilo Rezende](https://danilorezende.com/about/)
- [Dan Rosenbaum](https://danrsm.github.io/)
- [](https://arxiv.org/abs/2201.12204)
- [](https://github.com/sxyu/pixel-nerf), [](https://github.com/deepmind/jaxline)
- [](https://www.tensorflow.org/datasets/catalog/celeb_a_hq)
| Whisper | Automatic speech recognition system trained on 680,000 hours of multilingual and multitask supervised data collected from the web |
- [Alec Radford](http://newmu.github.io/)
- [Jong Wook Kim](https://jongwook.kim/)
- [Tao Xu](https://github.com/bayesian)
- [Greg Brockman](https://gregbrockman.com/) others
- [Christine McLeavey](http://christinemcleavey.com/)
- [Ilya Sutskever](http://www.cs.toronto.edu/~ilya/)
- [](https://arxiv.org/abs/2212.04356)
- [blog post](https://openai.com/research/whisper)
- [](https://github.com/kkroening/ffmpeg-python)
- [](https://youtu.be/OCBZtgQGt1I), [](https://youtu.be/8SQV-B83tPU), [](https://youtu.be/nE5iVtwKerA)
| DeOldify (video) | Colorize your own videos! | [Jason Antic](https://github.com/jantic) | [](https://github.com/jantic/DeOldify)
- [](https://arxiv.org/abs/1805.08318), [](https://arxiv.org/abs/1706.08500)
- [](https://medium.com/element-ai-research-lab/stabilizing-neural-style-transfer-for-video-62675e203e42)
- [model](https://data.deepai.org/deoldify/ColorizeVideo_gen.pth)
- [](https://www.reddit.com/r/Nickelodeons/), [](https://www.reddit.com/r/silentmoviegifs/)
- [](https://twitter.com/DeOldify)
- [website](https://deoldify.ai/)
- [](http://www.youtube.com/watch?v=l3UXXid04Ys), [](http://www.youtube.com/watch?v=EXn-n2iqEjI)
| DeOldify (photo) | Colorize your own photos! |
- [Jason Antic](https://github.com/jantic)
- [Matt Robinson](https://github.com/mc-robinson)
- [María Benavente](https://github.com/mariabg)
- [](https://arxiv.org/abs/1805.08318), [](https://arxiv.org/abs/1706.08500)
- [model](https://data.deepai.org/deoldify/ColorizeArtistic_gen.pth)
- [](https://www.reddit.com/r/TheWayWeWere/)
- [](https://twitter.com/DeOldify)
- [website](https://deoldify.ai/)
| Real-ESRGAN | Extend the powerful ESRGAN to a practical restoration application, which is trained with pure synthetic data |
- [Xintao Wang](https://xinntao.github.io/)
- [Liangbin Xie](https://liangbinxie.github.io/)
- [Chao Dong](https://scholar.google.com/citations?user=OSDCB0UAAAAJ)
- [Ying Shan](https://scholar.google.com/citations?user=4oXBp9UAAAAJ)
- [](https://arxiv.org/abs/2107.10833)
- [](https://github.com/xinntao/ESRGAN), [](https://github.com/xinntao/facexlib), [](https://github.com/xinntao/HandyView), [](https://github.com/Tencent/ncnn), [](https://github.com/nihui/waifu2x-ncnn-vulkan)
| IDE-3D | Interactive Disentangled Editing for High-Resolution 3D-aware Portrait Synthesis |
- [Jingxiang Sun](https://mrtornado24.github.io/)
- [Xuan Wang](https://xuanwangvc.github.io/)
- [Yichun Shi](https://seasonsh.github.io/)
- [Lizhen Wang](https://lizhenwangt.github.io/) others
- [Jue Wang](https://juewang725.github.io/)
- [Yebin Liu](http://www.liuyebin.com/)
- [](https://arxiv.org/abs/2205.15517), [](https://github.com/NVlabs/eg3d), [](https://github.com/NVlabs/ffhq-dataset), [](https://github.com/NVlabs/stylegan3)
- [](https://youtu.be/Kj5XY_J2Alk)
| Decision Transformers | An architecture that casts the problem of RL as conditional sequence modeling |
- [Lili Chen](http://www.lilichen.me/)
- [Kevin Lu](https://kzl.github.io/)
- [Aravind Rajeswaran](https://aravindr93.github.io/)
- [Kimin Lee](https://sites.google.com/view/kiminlee) others
- [Aditya Grover](https://aditya-grover.github.io/)
- [Michael Laskin](https://www.mishalaskin.com/)
- [Pieter Abbeel](http://people.eecs.berkeley.edu/~pabbeel/)
- [Aravind Srinivas](https://github.com/aravindsrinivas)
- [Igor Mordatch](https://scholar.google.com/citations?user=Vzr1RukAAAAJ)
- [](https://arxiv.org/abs/2106.01345)
- [](https://huggingface.co/models?other=gym-continous-control), [](https://huggingface.co/edbeeching/decision-transformer-gym-hopper-expert), [](https://huggingface.co/docs/transformers/model_doc/decision_transformer)
- [project](https://sites.google.com/berkeley.edu/decision-transformer)
- [](https://en.wikipedia.org/wiki/Autoregressive_model)
- [](https://youtu.be/k08N5a0gG0A), [](https://youtu.be/-buULmf7dec), [](https://youtu.be/83QN9S-0I84), [](https://youtu.be/w4Bw8WYL8Ps)
| Dream Fields | Zero-Shot Text-Guided Object Generation |
- [Ajay Jain](https://ajayj.com/)
- [Ben Mildenhall](https://bmild.github.io/)
- [Jon Barron](https://jonbarron.info/)
- [Pieter Abbeel](https://people.eecs.berkeley.edu/~pabbeel/)
- [Ben Poole](https://cs.stanford.edu/~poole/)
- [](https://arxiv.org/abs/2112.01455), [](https://arxiv.org/abs/2104.00677), [](https://arxiv.org/abs/2103.13415)
- [](https://github.com/ajayjain/DietNeRF), [](https://github.com/google/mipnerf)
- [project](https://ajayj.com/dreamfields)
- [](https://youtu.be/1Fke6w46tv4)
| GANgealing | Framework for learning discriminative models and their GAN-generated training data jointly end-to-end |
- [William Peebles](https://www.wpeebles.com/)
- [Jun-Yan Zhu](https://www.cs.cmu.edu/~junyanz/)
- [Richard Zhang](https://richzhang.github.io/)
- [Antonio Torralba](https://groups.csail.mit.edu/vision/torralbalab/) others
- [Alexei Efros](https://people.eecs.berkeley.edu/~efros/)
- [Eli Shechtman](https://research.adobe.com/person/eli-shechtman/)
- [](https://arxiv.org/abs/2112.05143)
- [](https://github.com/nileshkulkarni/acsm), [](https://github.com/rosinality/stylegan2-pytorch), [](https://jitengmu.github.io/CoordGAN/)
- [project](https://www.wpeebles.com/gangealing)
- [](https://youtu.be/Qa1ASS_NuzE), [](https://youtu.be/qtOkktTNs-k)
| textual-inversion | An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion |
- [Rinon Gal](https://rinongal.github.io/)
- [Yuval Alaluf](https://yuval-alaluf.github.io/)
- [Yuval Atzmon](https://research.nvidia.com/person/yuval-atzmon)
- [Or Patashnik](https://orpatashnik.github.io/) others
- [Amit Bermano](https://www.cs.tau.ac.il/~amberman/)
- [Gal Chechik](https://research.nvidia.com/person/gal-chechik)
- [Daniel Cohen-Or](https://danielcohenor.com/)
- [](https://arxiv.org/abs/2208.01618)
- [project](https://textual-inversion.github.io/)
- [](https://youtu.be/f3oXa7_SYek), [](https://youtu.be/opD_H9bED9Y)
| StyleGAN-Human | A Data-Centric Odyssey of Human Generation |
- [Jianglin Fu](https://github.com/arleneF)
- [Shikai Li](https://github.com/leeskyed)
- [Yuming Jiang](https://yumingj.github.io/)
- [Kwan-Yee Lin](https://kwanyeelin.github.io/) others
- [Chen Qian](https://scholar.google.com/citations?user=AerkT0YAAAAJ)
- [Chen Change Loy](https://www.mmlab-ntu.com/person/ccloy/)
- [Wayne Wu](https://wywu.github.io/)
- [Ziwei Liu](https://liuziwei7.github.io/)
- [](https://arxiv.org/abs/2204.11823)
- [](https://github.com/NVlabs/stylegan), [](https://github.com/NVlabs/stylegan2-ada-pytorch), [](https://github.com/NVlabs/stylegan3)
- [project](https://stylegan-human.github.io/)
- [](https://paperswithcode.com/dataset/market-1501)
- [](https://youtu.be/nIrb9hwsdcI), [](https://youtu.be/86b49sCz0Gg), [](https://youtu.be/g3nmM6MdxwY), [](https://youtu.be/p2uwqh_SFL8)
| Make-A-Scene | Scene-Based Text-to-Image Generation with Human Priors |
- [Oran Gafni](https://github.com/ogafni)
- [Adam Polyak](https://scholar.google.com/citations?user=CP62OTMAAAAJ)
- [Oron Ashual](https://scholar.google.com/citations?user=CUA9JCkAAAAJ)
- [Shelly Sheynin](https://github.com/shellysheynin) others
- [Devi Parikh](https://faculty.cc.gatech.edu/~parikh/)
- [Yaniv Taigman](https://ytaigman.github.io/)
- [](https://arxiv.org/abs/2203.13131)
- [](https://youtu.be/ZM06MjPdoxw)
| StyleGAN-NADA | Zero-Shot non-adversarial domain adaptation of pre-trained generators |
- [Rinon Gal](https://rinongal.github.io/)
- [Or Patashnik](https://orpatashnik.github.io/)
- [Haggai Maron](https://haggaim.github.io/)
- [Gal Chechik](https://research.nvidia.com/person/gal-chechik)
- [Daniel Cohen-Or](https://danielcohenor.com/)
- [](https://arxiv.org/abs/2108.00946), [](https://arxiv.org/abs/2103.17249), [](https://arxiv.org/abs/2104.02699)
- [](https://github.com/rosinality/stylegan2-pytorch/), [](https://github.com/NVlabs/stylegan2-ada)
- [project](https://stylegan-nada.github.io/)
| YOLOv7 | Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors |
- [Chien-Yao Wang](https://scholar.google.com/citations?user=DkQh4M4AAAAJ)
- [Alexey Bochkovskiy](http://www.alexeyab.com/)
- [Mark Liao](https://www.iis.sinica.edu.tw/pages/liao/)
- [](https://arxiv.org/abs/2207.02696)
- [data](http://images.cocodataset.org/annotations/annotations_trainval2017.zip), [data](http://images.cocodataset.org/zips/train2017.zip), [data](http://images.cocodataset.org/zips/val2017.zip), [data](https://github.com/WongKinYiu/yolov7/releases/download/v0.1/coco2017labels-segments.zip)
- [](https://github.com/WongKinYiu/yolor), [](https://github.com/WongKinYiu/PyTorch_YOLOv4), [](https://github.com/WongKinYiu/ScaledYOLOv4), [](https://github.com/Megvii-BaseDetection/YOLOX), [](https://github.com/DingXiaoH/RepVGG), [](https://github.com/JUGGHM/OREPA_CVPR2022), [](https://github.com/TexasInstruments/edgeai-yolov5/tree/yolo-pose)
- [](https://paperswithcode.com/sota/real-time-object-detection-on-coco?p=yolov7-trainable-bag-of-freebies-sets-new)
- [](https://www.youtube.com/playlist?list=PL_Nji0JOuXg2QMohGK7wfzgJ-MavzXRHW), [](https://youtu.be/-QWxJ0j9EY8)
| GLIP | Grounded language-image pre-training model for learning object-level, language-aware, and semantic-rich visual representations |
- [Liunian Harold Li](https://liunian-harold-li.github.io/)
- [Pengchuan Zhang](https://pzzhang.github.io/pzzhang/)
- [Haotian Zhang](https://haotian-zhang.github.io/)
- [Jianwei Yang](https://jwyang.github.io/) others
- [Chunyuan Li](https://chunyuan.li/)
- [Yiwu Zhong](https://pages.cs.wisc.edu/~yiwuzhong/)
- [Lijuan Wang](https://github.com/LijuanWang)
- [Lu Yuan](https://scholar.google.com/citations?user=k9TsUVsAAAAJ)
- [Lei Zhang](https://www.leizhang.org/)
- [Jenq-Neng Hwang](https://people.ece.uw.edu/hwang/)
- [Kai-Wei Chang](http://web.cs.ucla.edu/~kwchang/)
- [Jianfeng Gao](https://www.microsoft.com/en-us/research/people/jfgao/)
- [](https://arxiv.org/abs/2112.03857), [](https://arxiv.org/abs/2206.05836), [](https://arxiv.org/abs/2102.01066), [](https://arxiv.org/abs/2204.08790)
- [blog post](https://www.microsoft.com/en-us/research/project/project-florence-vl/articles/object-detection-in-the-wild-via-grounded-language-image-pre-training/)
- [](https://github.com/gligen/GLIGEN)
- [](https://huggingface.co/harold/GLIP)
- [](https://sh-tsang.medium.com/glip-grounded-language-image-pre-training-2be2483295b3), [](https://towardsdatascience.com/glip-introducing-language-image-pre-training-to-object-detection-5ddb601873aa)
- [](https://youtu.be/zu1BGQBI4dU)
| Anycost GAN | Interactive natural image editing |
- [Ji Lin](http://linji.me/)
- [Richard Zhang](https://richzhang.github.io/)
- [Frieder Ganz](https://scholar.google.com/citations?user=u9ySZkUAAAAJ)
- [Song Han](https://songhan.mit.edu/)
- [Jun-Yan Zhu](https://www.cs.cmu.edu/~junyanz/)
- [](https://arxiv.org/abs/2103.03243)
- [](https://github.com/NVlabs/stylegan2), [](https://github.com/rosinality/stylegan2-pytorch), [](https://github.com/NVlabs/ffhq-dataset), [](https://github.com/switchablenorms/CelebAMask-HQ), [](https://github.com/fyu/lsun)
- [project](https://hanlab.mit.edu/projects/anycost-gan/)
- [](https://www.youtube.com/watch?v=_yEziPl9AkM)
| GFPGAN | Towards Real-World Blind Face Restoration with Generative Facial Prior |
- [Xintao Wang](https://xinntao.github.io/)
- [Yu Li](https://yu-li.github.io/)
- [Honglun Zhang](https://scholar.google.com/citations?user=KjQLROoAAAAJ)
- [Ying Shan](https://scholar.google.com/citations?user=4oXBp9UAAAAJ)
- [](https://arxiv.org/abs/2101.04061)
- [](https://github.com/xinntao/facexlib), [](https://github.com/xinntao/HandyView), [](https://github.com/NVlabs/ffhq-dataset)
- [project](https://xinntao.github.io/projects/gfpgan)
| EPro-PnP | Generalized End-to-End Probabilistic Perspective-n-Points for Monocular Object Pose Estimation |
- [Hansheng Chen](https://lakonik.github.io/)
- [Pichao Wang](https://wangpichao.github.io/)
- [Fan Wang](https://scholar.google.com/citations?user=WCRGTHsAAAAJ)
- [Wei Tian](https://scholar.google.com/citations?user=aYKQn88AAAAJ) others
- [Lu Xiong](https://ieeexplore.ieee.org/author/37401835800)
- [Hao Li](https://scholar.google.com/citations?user=pHN-QIwAAAAJ)
- [](https://arxiv.org/abs/2203.13254)
- [](https://github.com/megvii-research/petr), [](https://github.com/HuangJunJie2017/BEVDet), [](https://github.com/fudan-zvg/PolarFormer), [](https://github.com/zhiqi-li/BEVFormer), [](https://github.com/open-mmlab/mmdetection3d)
- [nuScenes](https://www.nuscenes.org/object-detection?externalData=no&mapData=no&modalities=Camera)
- [](https://youtu.be/TonBodQ6EUU)
| VQ-Diffusion | Based on a VQ-VAE whose latent space is modeled by a conditional variant of the recently developed Denoising Diffusion Probabilistic Model |
- [Shuyang Gu](https://github.com/cientgu)
- [Dong Chen](http://www.dongchen.pro/)
- [Jianmin Bao](https://jianminbao.github.io/)
- [Fang Wen](https://www.microsoft.com/en-us/research/people/fangwen/) others
- [Bo Zhang](https://bo-zhang.me/)
- [Dongdong Chen](http://www.dongdongchen.bid/)
- [Lu Yuan](https://scholar.google.com/citations?&user=k9TsUVsAAAAJ)
- [Baining Guo](https://scholar.google.com/citations?user=h4kYmRYAAAAJ)
- [Shuyang Gu](https://github.com/cientgu)
- [Zhicong Tang](https://github.com/zzctan)
- [](https://arxiv.org/abs/2111.14822), [](https://arxiv.org/abs/2205.16007)
- [](https://github.com/ehoogeboom/multinomial_diffusion), [](https://github.com/openai/improved-diffusion)
| OPT | Open Pre-trained Transformers is a family of NLP models trained on billions of tokens of text obtained from the internet |
- [Susan Zhang](https://github.com/suchenzang)
- [Stephen Roller](https://stephenroller.com/)
- [Naman Goyal](https://github.com/ngoyal2707)
- [Mikel Artetxe](https://github.com/artetxem) others
- [Moya Chen](https://moyachen.com/)
- [Christopher Dewan](https://github.com/m3rlin45)
- [Mona Diab](https://scholar.google.com/citations?user=-y6SIhQAAAAJ)
- [Xi Victoria Lin](http://victorialin.net/)
- [Todor Mihaylov](https://github.com/tbmihailov)
- [Myle Ott](https://myleott.com/)
- [Sam Shleifer](https://github.com/sshleifer)
- [Kurt Shuster](https://github.com/klshuster)
- [Daniel Simig](https://scholar.google.com/citations?user=TtWU9fsAAAAJ)
- [Punit Singh Koura](https://github.com/punitkoura)
- [Anjali Sridhar](https://www.linkedin.com/in/anjalisridhar/)
- [Tianlu Wang](https://tianlu-wang.github.io/)
- [Luke Zettlemoyer](https://www.cs.washington.edu/people/faculty/lsz/)
- [](https://arxiv.org/abs/2205.01068), [](https://arxiv.org/abs/1906.02243), [](https://arxiv.org/abs/2104.10350), [](https://arxiv.org/abs/2201.11990)
- [blog post](https://ai.facebook.com/blog/democratizing-access-to-large-scale-language-models-with-opt-175b/)
- [](https://github.com/NVIDIA/Megatron-LM)
- [](https://youtu.be/Ejg0OunCi9U)
| Customizing a Transformer Encoder | We will learn how to customize the encoder to employ new network architectures | [Chen Chen](https://github.com/chenGitHuber) | [](https://github.com/tensorflow/models/tree/master/official/nlp/modeling)
- [](https://arxiv.org/abs/1706.03762)
- [](https://github.com/tensorflow/models/blob/master/official/nlp/modeling/networks/encoder_scaffold.py)
| MTTR | End-to-End Referring Video Object Segmentation with Multimodal Transformers |
- [Adam Botach](https://www.linkedin.com/in/adam-botach)
- [Evgenii Zheltonozhskii](https://evgeniizh.com/)
- [Chaim Baskin](https://github.com/chaimbaskin)
- [](https://arxiv.org/abs/2111.14821), [](https://arxiv.org/abs/1907.11692), [](https://arxiv.org/abs/2106.13230)
- [](https://github.com/SwinTransformer/Video-Swin-Transformer)
- [](https://huggingface.co/spaces/MTTR/MTTR-Referring-Video-Object-Segmentation)
- [](https://youtu.be/YqlhXgq6hcs)
| SwinIR | Image Restoration Using Swin Transformer |
- [Jingyun Liang](https://jingyunliang.github.io/)
- [Jiezhang Cao](https://github.com/caojiezhang)
- [Guolei Sun](https://github.com/GuoleiSun)
- [Kai Zhang](https://cszn.github.io/) others
- [Luc Van Gool](https://scholar.google.com/citations?user=TwMib_QAAAAJ)
- [Radu Timofte](https://www.informatik.uni-wuerzburg.de/computervision/home/)
- [](https://arxiv.org/abs/2108.10257), [](https://arxiv.org/abs/2107.10833)
- [](https://github.com/cszn/BSRGAN), [](https://github.com/microsoft/Swin-Transformer), [](https://github.com/cszn/KAIR)
| VRT | A Video Restoration Transformer |
- [Jingyun Liang](https://jingyunliang.github.io/)
- [Jiezhang Cao](https://github.com/caojiezhang)
- [Yuchen Fan](https://ychfan.github.io/)
- [Kai Zhang](https://cszn.github.io/) others
- [Yawei Li](https://ofsoundof.github.io/)
- [Radu Timofte](https://www.informatik.uni-wuerzburg.de/computervision/home/)
- [Luc Van Gool](https://scholar.google.com/citations?user=TwMib_QAAAAJ)
- [](https://arxiv.org/abs/2201.12288)
- [](https://github.com/cszn/KAIR), [](https://github.com/SwinTransformer/Video-Swin-Transformer), [](https://github.com/open-mmlab/mmediting)
| Omnivore | A single model which excels at classifying images, videos, and single-view 3D data using exactly the same model parameters |
- [Rohit Girdhar](http://rohitgirdhar.github.io/)
- [Mannat Singh](https://scholar.google.com/citations?user=QOO8OCcAAAAJ)
- [Nikhila Ravi](https://nikhilaravi.com/)
- [Laurens Maaten](https://lvdmaaten.github.io/) others
- [Armand Joulin](https://ai.facebook.com/people/armand-joulin/)
- [Ishan Misra](https://imisra.github.io/)
- [](https://arxiv.org/abs/2201.08377), [](https://arxiv.org/abs/2206.08356)
- [](https://huggingface.co/spaces/akhaliq/omnivore)
- [project](https://facebookresearch.github.io/omnivore/)
- [](https://paperswithcode.com/dataset/epic-kitchens-100)
| Detic | Detecting Twenty-thousand Classes using Image-level Supervision |
- [Xingyi Zhou](https://www.cs.utexas.edu/~zhouxy/)
- [Rohit Girdhar](https://rohitgirdhar.github.io/)
- [Armand Joulin](https://ai.facebook.com/people/armand-joulin/)
- [Philipp Krähenbühl](https://github.com/philkr)
- [Ishan Misra](https://imisra.github.io/)
- [](https://arxiv.org/abs/2201.02605)
- [](https://github.com/lvis-dataset/lvis-api)
| T0 | Multitask Prompted Training Enables Zero-Shot Task Generalization |
- [Victor Sanh](https://github.com/VictorSanh)
- [Albert Webson](https://representation.ai/)
- [Colin Raffel](https://colinraffel.com//)
- [Stephen Bach](http://cs.brown.edu/people/sbach/) others
- [Lintang Sutawika](https://github.com/lintangsutawika)
- [Zaid Alyafeai](https://github.com/zaidalyafeai)
- [Antoine Chaffin](https://antoine.chaffin.fr/)
- [Arnaud Stiegler](https://github.com/arnaudstiegler)
- [Teven Scao](https://scholar.google.com/citations?user=ik0_vxsAAAAJ)
- [Arun Raja](https://www.arunraja.dev/)
- [Manan Dey](https://github.com/manandey)
- [M Saiful Bari](https://sbmaruf.github.io/)
- [Canwen Xu](https://www.canwenxu.net/)
- [Urmish Thakker](https://github.com/Urmish)
- [Shanya Sharma](https://shanyas10.github.io/)
- [Eliza Szczechla](https://elsanns.github.io/)
- [Taewoon Kim](https://tae898.github.io/)
- [Gunjan Chhablani](https://gchhablani.github.io/)
- [Nihal Nayak](https://nihalnayak.github.io/)
- [Debajyoti Datta](http://debajyotidatta.github.io/)
- [Jonathan Chang](https://github.com/cccntu/)
- [Mike Tian-Jian Jiang](https://github.com/tianjianjiang)
- [Matteo Manica](https://github.com/drugilsberg)
- [Sheng Shen](https://sincerass.github.io/)
- [Zheng Xin Yong](https://yongzx.github.io/)
- [Harshit Pandey](https://scholar.google.com/citations?user=BPIs78gAAAAJ)
- [Rachel Bawden](https://rbawden.github.io/)
- [Trishala Neeraj](https://github.com/trishalaneeraj)
- [Jos Rozen](https://scholar.google.com/citations?user=OxEDKogAAAAJ)
- [Abheesht Sharma](https://github.com/abheesht-sharma)
- [Andrea Santilli](https://teelinsan.github.io/)
- [Thibault Fevry](http://thibaultfevry.com/)
- [Jason Alan Fries](https://web.stanford.edu/~jfries/)
- [Ryan Teehan](https://github.com/rteehas)
- [Stella Biderman](https://www.stellabiderman.com/)
- [Leo Gao](https://github.com/leogao2)
- [Tali Bers](https://github.com/tbers-coursera)
- [Thomas Wolf](https://thomwolf.io/)
- [Alexander M. Rush](https://scholar.google.com/citations?user=LIjnUGgAAAAJ)
- [](https://arxiv.org/abs/2110.08207)
- [](https://youtu.be/iJ0IVZgGjTM), [](https://youtu.be/YToXXfrIu6w)
| AvatarCLIP | A zero-shot text-driven framework for 3D avatar generation and animation |
- [Fangzhou Hong](https://hongfz16.github.io/)
- [Mingyuan Zhang](https://scholar.google.com/citations?user=2QLD4fAAAAAJ)
- [Liang Pan](https://scholar.google.com/citations?user=lSDISOcAAAAJ)
- [Zhongang Cai](https://caizhongang.github.io/) others
- [Lei Yang](https://scholar.google.com/citations?user=jZH2IPYAAAAJ)
- [Ziwei Liu](https://liuziwei7.github.io/)
- [](https://arxiv.org/abs/2205.08535), [](https://arxiv.org/abs/2112.01455), [](https://arxiv.org/abs/2112.03221), [](https://arxiv.org/abs/2112.05139), [](https://arxiv.org/abs/2203.13333)
- [data](https://www.di.ens.fr/willow/research/surreal/data/)
- [](https://github.com/daniilidis-group/neural_renderer), [](https://github.com/GuyTevet/MotionCLIP), [](https://github.com/Totoro97/NeuS), [](https://github.com/vchoutas/smplx), [](https://github.com/nghorbani/human_body_prior)
- [project](https://hongfz16.github.io/projects/AvatarCLIP.html)
- [](https://youtu.be/-l2ZMeoASGY)
| Text2Mesh | Text-Driven Neural Stylization for Meshes |
- [Oscar Michel](https://ojmichel.github.io/)
- [Roi Bar-On](https://github.com/roibaron)
- [Richard Liu](https://github.com/factoryofthesun)
- [Sagie Benaim](https://sagiebenaim.github.io/)
- [Rana Hanocka](http://people.cs.uchicago.edu/~ranahanocka/)
- [CLIP](https://openai.com/blog/clip/)
- [](https://arxiv.org/abs/2112.03221)
- [](https://www.kaggle.com/code/neverix/text2mesh/notebook)
- [project](https://threedle.github.io/text2mesh/)
| T5 | Text-To-Text Transfer Transformer |
- [Colin Raffel](https://colinraffel.com/)
- [Noam Shazeer](https://scholar.google.com/citations?user=wsGvgA8AAAAJ)
- [Adam Roberts](https://github.com/adarob)
- [Katherine Lee](https://github.com/katelee168) others
- [Sharan Narang](https://github.com/sharannarang)
- [Michael Matena](https://scholar.google.com/citations?user=rN_9vroAAAAJ)
- [Yanqi Zhou](https://zhouyanqi.github.io)
- [Wei Li](https://research.google/people/106528/)
- [Peter J. Liu](https://scholar.google.com/citations?user=1EPxhywAAAAJ)
- [](https://arxiv.org/abs/1910.10683)
- [](https://github.com/tensorflow/mesh/tree/master/mesh_tensorflow/transformer)
- [](https://www.tensorflow.org/datasets)
| XLS-R | Self-supervised Cross-lingual Speech Representation Learning at Scale |
- [Arun Babu](https://github.com/arbabu123)
- [Changhan Wang](https://www.changhan.me/)
- [Andros Tjandra](https://github.com/androstj)
- [Kushal Lakhotia](https://about.me/hikushalhere) others
- [Qiantong Xu](https://github.com/xuqiantong)
- [Naman Goyal](https://github.com/ngoyal2707)
- [Kritika Singh](https://scholar.google.com/citations?user=Ltk3SykAAAAJ)
- [Patrick von Platen](https://github.com/patrickvonplaten)
- [Yatharth Saraf](https://scholar.google.com/citations?user=KJTtNJwAAAAJ)
- [Juan Pino](https://scholar.google.com/citations?user=weU_-4IAAAAJ)
- [Alexei Baevski](https://github.com/alexeib)
- [Alexis Conneau](https://github.com/aconneau)
- [Michael Auli](https://github.com/michaelauli)
- [](https://arxiv.org/abs/2111.09296)
- [blog post](https://huggingface.co/blog/fine-tune-xlsr-wav2vec2)
- [](https://github.com/facebookresearch/fairscale)
| DiffCSE | Unsupervised contrastive learning framework for learning sentence embeddings |
- [Yung-Sung Chuang](https://people.csail.mit.edu/yungsung/)
- [Rumen Dangovski](http://super-ms.mit.edu/rumen.html)
- [Hongyin Luo](https://luohongyin.github.io/)
- [Yang Zhang](https://mitibmwatsonailab.mit.edu/people/yang-zhang/) others
- [Shiyu Chang](https://code-terminator.github.io/)
- [Marin Soljačić](http://www.mit.edu/~soljacic/marin.html)
- [Shang-Wen Li](https://swdanielli.github.io/)
- [Scott Wen-tau Yih](https://scottyih.org/)
- [Yoon Kim](https://people.csail.mit.edu/yoonkim/)
- [James Glass](http://groups.csail.mit.edu/sls/people/glass.shtml)
- [](https://arxiv.org/abs/2204.10298), [](https://arxiv.org/abs/2104.08821), [](https://arxiv.org/abs/2111.00899)
- [](https://github.com/princeton-nlp/SimCSE)
- [](https://huggingface.co/voidism)
- [](https://twitter.com/YungSungChuang/status/1517518077902000129)
| ViDT+ | An Extendable, Efficient and Effective Transformer-based Object Detector |
- [Hwanjun Song](https://songhwanjun.github.io/)
- [Deqing Sun](https://deqings.github.io/)
- [Sanghyuk Chun](https://sanghyukchun.github.io/home/)
- [Varun Jampani](https://varunjampani.github.io/) others
- [Dongyoon Han](https://sites.google.com/site/dyhan0920/)
- [Byeongho Heo](https://sites.google.com/view/byeongho-heo/home)
- [Wonjae Kim](https://wonjae.kim/)
- [Ming-Hsuan Yang](http://faculty.ucmerced.edu/mhyang/)
- [](https://arxiv.org/abs/2204.07962), [](https://arxiv.org/abs/2110.03921)
- [](https://github.com/fundamentalvision/Deformable-DETR), [](https://github.com/EherSenaw/ViDT_colab)
| NAFNet | Nonlinear Activation Free Network for Image Restoration |
- [Liangyu Chen](https://github.com/mayorx)
- [Xiaojie Chu](https://github.com/chuxiaojie)
- [Xiangyu Zhang](https://scholar.google.com/citations?user=yuB-cfoAAAAJ)
- [Jian Sun](http://www.jiansun.org/)
- [](https://arxiv.org/abs/2204.04676), [](https://arxiv.org/abs/2204.08714)
- [](https://paperswithcode.com/sota/image-deblurring-on-gopro?p=simple-baselines-for-image-restoration), [](https://paperswithcode.com/sota/image-denoising-on-sidd?p=simple-baselines-for-image-restoration)
| Panini-Net | GAN Prior based Degradation-Aware Feature Interpolation for Face Restoration |
- [Yinhuai Wang](https://github.com/wyhuai)
- [Yujie Hu](https://villa.jianzhang.tech/people/yujie-hu/)
- [Jian Zhang](http://jianzhang.tech/)
- [](https://arxiv.org/abs/2203.08444)
- [](https://github.com/NVlabs/ffhq-dataset), [](https://github.com/tkarras/progressive_growing_of_gans)
| Deep Painterly Harmonization | Algorithm produces significantly better results than photo compositing or global stylization techniques and that it enables creative painterly edits that would be otherwise difficult to achieve |
- [Fujun Luan](https://luanfujun.github.io/)
- [Sylvain Paris](http://people.csail.mit.edu/sparis/)
- [Eli Shechtman](https://research.adobe.com/person/eli-shechtman/)
- [Kavita Bala](https://www.cs.cornell.edu/~kb/)
- [](https://arxiv.org/abs/1804.03189), [](https://arxiv.org/abs/1701.08893)
- [](https://github.com/jcjohnson/neural-style), [](https://github.com/torch/torch7), [](https://github.com/szagoruyko/loadcaffe)
| E2FGVI | An End-to-End framework for Flow-Guided Video Inpainting through elaborately designed three trainable modules, namely, flow completion, feature propagation, and content hallucination modules |
- [Zhen Li](https://paper99.github.io/)
- [Cheng-Ze Lu](https://github.com/LGYoung)
- [Jianhua Qin](https://scholar.google.com/citations?&user=TAr7TU4AAAAJ)
- [Chun-Le Guo](https://scholar.google.com/citations?user=RZLYwR0AAAAJ)
- [Ming-Ming Cheng](https://mmcheng.net/)
- [](https://arxiv.org/abs/2204.02663)
- [data](https://competitions.codalab.org/competitions/19544#participate-get-data), [data](https://data.vision.ee.ethz.ch/csergi/share/davis/DAVIS-2017-trainval-480p.zip)
- [](https://github.com/researchmm/STTN), [](https://github.com/microsoft/Focal-Transformer), [](https://github.com/ruiliu-ai/FuseFormer), [](https://github.com/phoenix104104/fast_blind_video_consistency#evaluation)
- [](https://medium.com/mlearning-ai/end-to-end-framework-for-flow-guided-video-inpainting-c5e2d8b61d20)
- [](https://youtu.be/N--qC3T2wc4), [](https://youtu.be/3eH3Fm6gOFk)
| LDM | High-Resolution Image Synthesis with Latent Diffusion Models |
- [Robin Rombach](https://github.com/rromb)
- [Andreas Blattmann](https://github.com/ablattmann)
- [Dominik Lorenz](https://github.com/qp-qp)
- [Patrick Esser](https://github.com/pesser)
- [Björn Ommer](https://ommer-lab.com/people/ommer/)
- [](https://arxiv.org/abs/2112.10752), [](https://arxiv.org/abs/2202.09778), [](https://arxiv.org/abs/2111.02114)
- [](https://github.com/fyu/lsun), [](https://github.com/openai/guided-diffusion), [](https://github.com/lucidrains/denoising-diffusion-pytorch), [](https://github.com/lucidrains/x-transformers)
- [](https://huggingface.co/spaces/multimodalart/latentdiffusion)
| GP-UNIT | Novel framework, Generative Prior-guided UNsupervised Image-to-image Translation, to improve the overall quality and applicability of the translation algorithm |
- [Shuai Yang](https://williamyang1991.github.io/)
- [Liming Jiang](https://liming-jiang.com/)
- [Ziwei Liu](https://liuziwei7.github.io/)
- [Chen Change Loy](https://www.mmlab-ntu.com/person/ccloy/)
- [ImageNet](https://image-net.org/download.php)
- [](https://arxiv.org/abs/2204.03641)
- [](https://github.com/clovaai/stargan-v2#datasets-and-pre-trained-networks), [](https://github.com/switchablenorms/CelebAMask-HQ), [](https://github.com/NVlabs/metfaces-dataset), [](https://github.com/TreB1eN/InsightFace_Pytorch), [](https://github.com/NVlabs/SPADE), [](https://github.com/nvlabs/imaginaire), [](https://doi.org/10.1109/CVPR52688.2022.01779)
- [project](https://www.mmlab-ntu.com/project/gpunit/)
- [](https://youtu.be/dDApWs_oDrM)
| DualStyleGAN | More challenging exemplar-based high-resolution portrait style transfer by introducing a novel DualStyleGAN with flexible control of dual styles of the original face domain and the extended artistic portrait domain |
- [Shuai Yang](https://williamyang1991.github.io/)
- [Liming Jiang](https://liming-jiang.com/)
- [Ziwei Liu](https://liuziwei7.github.io/)
- [Chen Change Loy](https://www.mmlab-ntu.com/person/ccloy/)
- [](https://arxiv.org/abs/2203.13248)
- [data](https://cs.nju.edu.cn/rl/WebCaricature.htm), [data](https://www.gwern.net/Crops#danbooru2019-portraits)
- [](https://github.com/lowfuel/progrock-stable), [](https://github.com/rosinality/stylegan2-pytorch), [](https://github.com/TreB1eN/InsightFace_Pytorch)
- [](https://huggingface.co/spaces/Gradio-Blocks/DualStyleGAN), [](https://huggingface.co/spaces/hysts/DualStyleGAN)
- [project](https://www.mmlab-ntu.com/project/dualstylegan/)
- [](https://youtu.be/scZTu77jixI)
| CLIPasso | Semantically-Aware Object Sketching |
- [Yael Vinker](https://yaelvi116.wixsite.com/mysite)
- [Ehsan Pajouheshgar](https://pajouheshgar.github.io/)
- [Jessica Y. Bo](https://jessica-bo.github.io/)
- [Roman Bachmann](https://roman-bachmann.github.io/) others
- [Amit Bermano](https://www.cs.tau.ac.il/~amberman/)
- [Daniel Cohen-Or](https://danielcohenor.com/)
- [Amir Zamir](https://vilab.epfl.ch/zamir/)
- [Ariel Shamir](https://faculty.runi.ac.il/arik/site/index.asp)
- [](https://arxiv.org/abs/2202.05822), [](https://arxiv.org/abs/2106.14843)
- [demo](https://replicate.com/yael-vinker/clipasso)
- [](https://github.com/BachiLi/diffvg)
- [project](https://clipasso.github.io/clipasso/)
| StyleSDF | A high resolution, 3D-consistent image and shape generation technique |
- [Roy Or-El](https://homes.cs.washington.edu/~royorel/)
- [Xuan Luo](https://roxanneluo.github.io/)
- [Mengyi Shan](https://shanmy.github.io/)
- [Eli Shechtman](https://research.adobe.com/person/eli-shechtman/) others
- [Jeong Joon Park](https://jjparkcv.github.io/)
- [Ira Kemelmacher-Shlizerman](https://www.irakemelmacher.com/)
- [](https://arxiv.org/abs/2112.11427)
- [](https://github.com/rosinality/stylegan2-pytorch), [](https://github.com/yenchenlin/nerf-pytorch)
- [](https://huggingface.co/spaces/SerdarHelli/StyleSDF-3D)
- [project](https://stylesdf.github.io/)
| Disentangled Lifespan Face Synthesis | LFS model is proposed to disentangle the key face characteristics including shape, texture and identity so that the unique shape and texture age transformations can be modeled effectively |
- [Sen He](https://senhe.github.io/)
- [Wentong Liao](https://www.tnt.uni-hannover.de/en/staff/liao/)
- [Michael Yang](https://sites.google.com/site/michaelyingyang/)
- [Yi-Zhe Song](http://personal.ee.surrey.ac.uk/Personal/Y.Song/) others
- [Bodo Rosenhahn](https://scholar.google.com/citations?user=qq3TxtcAAAAJ)
- [Tao Xiang](http://personal.ee.surrey.ac.uk/Personal/T.Xiang/index.html)
- [](https://arxiv.org/abs/2108.02874)
- [project](https://senhe.github.io/projects/iccv_2021_lifespan_face/)
- [](https://www.youtube.com/watch?v=uklX03ns0m0)
| Mask2Former | Masked-attention Mask Transformer for Universal Image Segmentation |
- [Bowen Cheng](https://bowenc0221.github.io/)
- [Ishan Misra](https://imisra.github.io/)
- [Alexander Schwing](https://alexander-schwing.de/)
- [Alexander Kirillov](https://alexander-kirillov.github.io/)
- [Rohit Girdhar](https://rohitgirdhar.github.io/)
- [](https://arxiv.org/abs/2112.01527), [](https://arxiv.org/abs/2112.10764)
- [demo](https://replicate.com/facebookresearch/mask2former)
- [](https://github.com/facebookresearch/MaskFormer)
- [](https://huggingface.co/spaces/akhaliq/Mask2Former)
- [project](https://bowenc0221.github.io/mask2former/)
| SpecVQGAN | Taming the visually guided sound generation by shrinking a training dataset to a set of representative vectors |
- [Vladimir Iashin](https://iashin.ai/)
- [Esa Rahtu](https://esa.rahtu.fi/)
- [](http://arxiv.org/abs/2110.08791), [](https://arxiv.org/abs/2012.09841), [](https://arxiv.org/abs/1711.00937), [](https://arxiv.org/abs/2008.00820), [](https://arxiv.org/abs/1712.01393), [](https://arxiv.org/abs/1512.08512)
- [](https://github.com/PeihaoChen/regnet), [](https://github.com/toshas/torch-fidelity), [](https://github.com/descriptinc/melgan-neurips), [](https://github.com/google/lyra)
- [project](https://iashin.ai/SpecVQGAN)
- [](https://en.wikipedia.org/wiki/Foley_(filmmaking)), [](https://en.wikipedia.org/wiki/Row-_and_column-major_order), [](https://en.wikipedia.org/wiki/Kullback%E2%80%93Leibler_divergence)
- [](https://www.youtube.com/watch?v=Bucb3nAa398)
| JoJoGAN | One Shot Face Stylization |
- [Min Jin Chong](https://mchong6.github.io/)
- [David Forsyth](http://luthuli.cs.uiuc.edu/~daf/)
- [](https://arxiv.org/abs/2112.11641)
- [](https://github.com/rosinality/stylegan2-pytorch), [](https://github.com/replicate/cog)
| Pose with Style | Detail-Preserving Pose-Guided Image Synthesis with Conditional StyleGAN |
- [Badour AlBahar](https://badouralbahar.github.io/)
- [Jingwan Lu](https://research.adobe.com/person/jingwan-lu/)
- [Jimei Yang](https://github.com/jimeiyang)
- [Zhixin Shu](https://zhixinshu.github.io/) others
- [Eli Shechtman](https://research.adobe.com/person/eli-shechtman/)
- [Jia-Bin Huang](https://jbhuang0604.github.io/)
- [](https://arxiv.org/abs/2109.06166)
- [](https://github.com/rosinality/stylegan2-pytorch)
- [project](https://pose-with-style.github.io/)
- [](https://youtu.be/d_ETeAVLilw)
| ConvNeXt | A pure ConvNet model constructed entirely from standard ConvNet modules |
- [Zhuang Liu](https://liuzhuang13.github.io/)
- [Hanzi Mao](https://hanzimao.me/)
- [Chao-Yuan Wu](https://chaoyuan.org/)
- [Christoph Feichtenhofer](https://feichtenhofer.github.io/) others
- [Trevor Darrell](https://people.eecs.berkeley.edu/~trevor/)
- [Saining Xie](https://www.sainingxie.com/)
- [](https://arxiv.org/abs/2201.03545)
- [](https://github.com/rwightman/pytorch-image-models), [](https://github.com/facebookresearch/deit), [](https://github.com/microsoft/unilm/tree/master/beit)
- [](https://huggingface.co/spaces/akhaliq/convnext)
- [](https://youtu.be/QzCjXqFnWPE), [](https://youtu.be/idiIllIQOfU), [](https://youtu.be/QqejV0LNDHA)
| diffsort | Differentiable Sorting Networks |
- [Felix Petersen](http://petersen.ai/)
- [Christian Borgelt](https://borgelt.net/)
- [Hilde Kuehne](https://hildekuehne.github.io/)
- [Oliver Deussen](https://www.cgmi.uni-konstanz.de/personen/prof-dr-oliver-deussen/)
- [](https://arxiv.org/abs/2105.04019), [](https://arxiv.org/abs/2203.09630)
- [](https://youtu.be/Rl-sFaE1z4M)
| Taming Transformers for High-Resolution Image Synthesis | We combine the efficiancy of convolutional approaches with the expressivity of transformers by introducing a convolutional VQGAN, which learns a codebook of context-rich visual parts, whose composition is modeled with an autoregressive transformer |
- [Patrick Esser](https://github.com/pesser)
- [Robin Rombach](https://github.com/rromb)
- [Björn Ommer](https://ommer-lab.com/people/ommer/)
- [](https://arxiv.org/abs/2012.09841)
- [project](https://compvis.github.io/taming-transformers/)
| FuseDream | Training-Free Text-to-Image Generation with Improved CLIP+GAN Space Optimization |
- [Xingchao Liu](https://scholar.google.com/citations?user=VOTVE0UAAAAJ)
- [Chengyue Gong](https://github.com/ChengyueGongR)
- [Lemeng Wu](https://github.com/klightz)
- [Hao Su](https://cseweb.ucsd.edu//~haosu/)
- [Qiang Liu](https://www.cs.utexas.edu/~lqiang/)
- [](https://arxiv.org/abs/2112.01573)
| GLIDE | Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models |
- [Alex Nichol](https://aqnichol.com/)
- [Prafulla Dhariwal](https://github.com/prafullasd)
- [Aditya Ramesh](http://adityaramesh.com/)
- [Pranav Shyam](https://github.com/pranv) others
- [Pamela Mishkin](https://manlikemishap.github.io/)
- [Bob McGrew](https://github.com/bmcgrew)
- [Ilya Sutskever](http://www.cs.utoronto.ca/~ilya/)
- [Mark Chen](https://scholar.google.com/citations?user=5fU-QMwAAAAJ)
- [](https://arxiv.org/abs/2112.10741)
- [](https://youtu.be/ItKi3h7IY2o)
| Nerfies | First method capable of photorealistically reconstructing deformable scenes using photos/videos captured casually from mobile phones |
- [Keunhong Park](https://keunhong.com/)
- [Utkarsh Sinha](https://utkarshsinha.com/)
- [Jon Barron](https://jonbarron.info/)
- [Sofien Bouaziz](http://sofienbouaziz.com/) others
- [Dan Goldman](https://www.danbgoldman.com/home/)
- [Steven Seitz](https://www.smseitz.com/)
- [Ricardo Martin-Brualla](https://ricardomartinbrualla.com/)
- [](https://arxiv.org/abs/2011.12948)
- [](https://github.com/google-research/google-research/tree/master/jaxnerf)
- [project](https://nerfies.github.io/)
- [](https://www.reddit.com/r/photogrammetry/comments/k1i0ct/deformable_neural_radiance_fields_nerfies/)
- [](https://youtu.be/MrKrnHhk8IA), [](https://youtu.be/IDMiMKWucaI)
| PoolFormer | MetaFormer Is Actually What You Need for Vision |
- [Weihao Yu](https://whyu.me/)
- [Mi Luo](https://luomi97.github.io/)
- [Pan Zhou](https://panzhous.github.io/)
- [Chenyang Si](https://github.com/ChenyangSi) others
- [Yichen Zhou](https://dblp.org/pid/55/10422.html)
- [Xinchao Wang](https://sites.google.com/site/sitexinchaowang/)
- [Jiashi Feng](https://sites.google.com/site/jshfeng/)
- [Shuicheng Yan](https://yanshuicheng.ai/)
- [](https://arxiv.org/abs/2111.11418)
- [](https://github.com/rwightman/pytorch-image-models), [](https://github.com/facebookresearch/fvcore), [](https://github.com/NVIDIA/apex)
- [](https://huggingface.co/spaces/akhaliq/poolformer)
| HyperStyle | A hypernetwork that learns to modulate StyleGAN's weights to faithfully express a given image in editable regions of the latent space |
- [Yuval Alaluf](https://yuval-alaluf.github.io/)
- [Omer Tov](https://github.com/omertov)
- [Ron Mokady](https://rmokady.github.io/)
- [Rinon Gal](https://rinongal.github.io/)
- [Amit Bermano](https://www.cs.tau.ac.il/~amberman/)
- [](https://arxiv.org/abs/2111.15666), [](https://arxiv.org/abs/1904.03189), [](https://arxiv.org/abs/2012.09036), [](https://arxiv.org/abs/2005.07727)
- [data](https://ai.stanford.edu/~jkrause/cars/car_dataset.html)
- [](https://github.com/NVlabs/ffhq-dataset), [](https://github.com/clovaai/stargan-v2), [](https://github.com/rosinality/stylegan2-pytorch), [](https://github.com/TreB1eN/InsightFace_Pytorch), [](https://github.com/HuangYG123/CurricularFace), [](https://github.com/lessw2020/Ranger-Deep-Learning-Optimizer), [](https://github.com/pytorch/vision/blob/main/torchvision/models/resnet.py), [](https://github.com/dvschultz/stylegan2-ada-pytorch)
- [project](https://yuval-alaluf.github.io/hyperstyle/)
- [](https://youtu.be/_sbXmLY2jMw)
| encoder4editing | Designing an Encoder for StyleGAN Image Manipulation |
- [Omer Tov](https://github.com/omertov)
- [Yuval Alaluf](https://yuval-alaluf.github.io/)
- [Yotam Nitzan](https://yotamnitzan.github.io/)
- [Or Patashnik](https://orpatashnik.github.io/)
- [Daniel Cohen-Or](https://danielcohenor.com/)
- [](https://arxiv.org/abs/2102.02766)
- [](https://github.com/eladrich/pixel2style2pixel)
| StyleCariGAN | Caricature Generation via StyleGAN Feature Map Modulation |
- [Wonjong Jang](https://wonjongg.github.io/)
- [Gwangjin Ju](https://github.com/jugwangjin)
- [Yucheol Jung](https://ycjung.info/)
- [Jiaolong Yang](https://jlyang.org/) others
- [Xin Tong](https://www.microsoft.com/en-us/research/people/xtong/)
- [Seungyong Lee](https://scholar.google.com/citations?user=yGPH-nAAAAAJ)
- [](https://arxiv.org/abs/2107.04331)
- [](https://github.com/NVlabs/stylegan2), [](https://github.com/rosinality/stylegan2-pytorch)
- [project](https://wonjongg.github.io/StyleCariGAN/)
- [](https://www.youtube.com/watch?v=kpHbGOlI-BU)
| CartoonGAN | The implementation of the cartoon GAN model with PyTorch | [Tobias Sunderdiek](https://github.com/TobiasSunderdiek) | [](https://doi.org/10.1109/CVPR.2018.00986)
- [](https://www.kaggle.com/alamson/safebooru)
- [project](https://tobiassunderdiek.github.io/cartoon-gan/)
| SimSwap | An efficient framework, called Simple Swap, aiming for generalized and high fidelity face swapping |
- [Xuanhong Chen](https://github.com/neuralchen)
- [Bingbing Ni](https://scholar.google.com/citations?user=eUbmKwYAAAAJ)
- [Yanhao Ge](https://scholar.google.com/citations?user=h6tuBAcAAAAJ)
- [](https://arxiv.org/abs/2106.06340)
- [](https://github.com/deepinsight/insightface)
| RVM | Robust High-Resolution Video Matting with Temporal Guidance |
- [Shanchuan Lin](https://github.com/PeterL1n)
- [Linjie Yang](https://sites.google.com/site/linjieyang89/)
- [Imran Saleemi](http://www.cs.ucf.edu/~imran/)
- [Soumyadip Sengupta](https://homes.cs.washington.edu/~soumya91/)
- [](http://arxiv.org/abs/2108.11515)
- [](https://github.com/NVIDIA/VideoProcessingFramework), [](https://github.com/FeiGeChuanShu/ncnn_Android_RobustVideoMatting)
- [project](https://peterl1n.github.io/RobustVideoMatting)
- [](https://youtu.be/Jvzltozpbpk), [](https://youtu.be/Ay-mGCEYEzM)
| RVM | Robust, real-time, high-resolution human video matting method that achieves new state-of-the-art performance |
- [Shanchuan Lin](https://github.com/PeterL1n)
- [Linjie Yang](https://sites.google.com/site/linjieyang89)
- [Imran Saleemi](https://github.com/imran-saleemi)
- [Soumyadip Sengupta](https://github.com/senguptaumd)
- [](https://arxiv.org/abs/2108.11515)
- [project](https://peterl1n.github.io/RobustVideoMatting)
- [](https://www.reddit.com/r/MachineLearning/comments/pdbpmg/r_robust_highresolution_video_matting_with/)
- [](https://youtu.be/Jvzltozpbpk), [](https://youtu.be/Ay-mGCEYEzM), [](https://youtu.be/VL-0K6HjhvQ), [](https://youtu.be/Jhuf6M_VrBI), [](https://youtu.be/_oN9yyRi3HY)
| AnimeGANv2 | An improved version of AnimeGAN - it prevents the generation of high-frequency artifacts by simply changing the normalization of features in the network |
- [Xin Chen](https://github.com/TachibanaYoshino)
- [Gang Liu](https://github.com/lg0061408)
- [bryandlee](https://github.com/bryandlee)
- [](https://github.com/TachibanaYoshino/AnimeGANv2), [](https://github.com/TachibanaYoshino/AnimeGAN)
- [](https://huggingface.co/spaces/akhaliq/AnimeGANv2)
- [project](https://tachibanayoshino.github.io/AnimeGANv2/)
| SOAT | StyleGAN of All Trades: Image Manipulation with Only Pretrained StyleGAN |
- [Min Jin Chong](https://mchong6.github.io/)
- [Hsin-Ying Lee](http://hsinyinglee.com/)
- [David Forsyth](http://luthuli.cs.uiuc.edu/~daf/)
- [](https://arxiv.org/abs/2111.01619)
- [](https://github.com/justinpinkney/toonify), [](https://github.com/rosinality/stylegan2-pytorch)
- [](https://huggingface.co/spaces/akhaliq/SOAT)
| Arnheim | Generative Art Using Neural Visual Grammars and Dual Encoders |
- [Chrisantha Fernando](https://www.chrisantha.co.uk/)
- [Ali Eslami](http://arkitus.com/)
- [Jean-Baptiste Alayrac](https://www.jbalayrac.com/)
- [Piotr Mirowski](https://piotrmirowski.com/) others
- [Dylan Banarse](https://www.2ne1.com/)
- [Simon Osindero](https://scholar.google.com/citations?user=Jq8ZS5kAAAAJ)
- [](https://arxiv.org/abs/2105.00162), [](https://arxiv.org/abs/2106.14843), [](https://arxiv.org/abs/1801.07729), [](https://arxiv.org/abs/1606.02580), [](https://arxiv.org/abs/1609.09106)
- [](https://github.com/openai/dall-e)
- [](https://en.wikipedia.org/wiki/Compositional_pattern-producing_network)
- [](https://www.youtube.com/watch?v=U7guaMdeF4g), [](https://www.youtube.com/watch?v=zh0goLbS-l0), [](https://www.youtube.com/watch?v=SYJGNt7yu6M), [](https://www.youtube.com/watch?v=MxkYKa0x5AU)
| StyleGAN 2 | Generation of faces, cars, etc. | [Mikael Christensen](https://github.com/Syntopia) | [](https://doi.org/10.1109/CVPR42600.2020.00813) [](https://github.com/NVlabs/stylegan2)
- [](http://arxiv.org/abs/1912.04958)
- [](https://github.com/NVlabs/ffhq-dataset)
- [](https://youtu.be/c-NJtV9Jvp0)
| ByteTrack | Multi-Object Tracking by Associating Every Detection Box |
- [Yifu Zhang](https://github.com/ifzhang)
- [Peize Sun](https://peizesun.github.io/)
- [Yi Jiang](https://github.com/iFighting)
- [Dongdong Yu](https://miracle-fmh.github.io/) others
- [Ping Luo](http://luoping.me/)
- [Xinggang Wang](https://xinggangw.info/)
- [](https://arxiv.org/abs/2110.06864)
- [data](https://motchallenge.net/), [data](https://www.crowdhuman.org/)
- [](https://github.com/Megvii-BaseDetection/YOLOX), [](https://github.com/ifzhang/FairMOT), [](https://github.com/PeizeSun/TransTrack), [](https://github.com/samylee/Towards-Realtime-MOT-Cpp)
- [](https://paperswithcode.com/task/multi-object-tracking)
| GPT-2 | Retrain an advanced text generating neural network on any text dataset using gpt-2-simple! | [Max Woolf](https://minimaxir.com/) | [](https://github.com/openai/gpt-2)
- [blog post](https://minimaxir.com/2019/09/howto-gpt2/), [blog post](https://openai.com/research/better-language-models)
- [](https://github.com/minimaxir/gpt-2-simple)
- [](https://www.reddit.com/r/MachineLearning/comments/aqlzde/r_openai_better_language_models_and_their/)
| ConvMixer | An extremely simple model that is similar in spirit to the ViT and the even-more-basic MLP-Mixer in that it operates directly on patches as input, separates the mixing of spatial and channel dimensions, and maintains equal size and resolution throughout the network |
- [Asher Trockman](http://ashertrockman.com/)
- [Zico Kolter](http://zicokolter.com/)
- [](https://arxiv.org/abs/2201.09792)
- [](https://github.com/locuslab/convmixer-cifar10), [](https://github.com/rwightman/pytorch-image-models)
- [](https://medium.com/codex/an-overview-on-convmixer-patches-are-all-you-need-8502a8d87011)
- [](https://youtu.be/Gl0s0GDqN3c?t=990)
| IC-GAN | Instance-Conditioned GAN |
- [Arantxa Casanova](https://github.com/ArantxaCasanova)
- [Marlène Careil](https://www.linkedin.com/in/marl%C3%A8ne-careil-901804155)
- [Jakob Verbeek](http://thoth.inrialpes.fr/~verbeek/)
- [Michał Drożdżal](https://scholar.google.com/citations?user=XK_ktwQAAAAJ)
- [Adriana Romero-Soriano](https://sites.google.com/site/adriromsor)
- [](https://arxiv.org/abs/2109.05070)
- [blog post](https://ai.facebook.com/blog/instance-conditioned-gans/)
- [](https://github.com/facebookresearch/faiss), [](https://github.com/ajbrock/BigGAN-PyTorch), [](https://github.com/NVlabs/stylegan2-ada-pytorch), [](https://github.com/bioinf-jku/TTUR), [](https://github.com/mit-han-lab/data-efficient-gans)
- [](https://proceedings.neurips.cc/paper/2021/hash/e7ac288b0f2d41445904d071ba37aaff-Abstract.html)
| Skillful Precipitation Nowcasting Using Deep Generative Models of Radar | Open-sourced dataset and model snapshot for precipitation nowcasting |
- [Suman Ravuri](https://www.linkedin.com/in/suman-ravuri-81928082)
- [Karel Lenc](https://www.robots.ox.ac.uk/~karel/)
- [Matthew Willson](https://www.linkedin.com/in/matthew-willson-6a1b422)
- [Dmitry Kangin](https://scholar.google.com/citations?user=vv-leaMAAAAJ) others
- [Rémi Lam](https://scholar.google.com/citations?user=Sm7xCbEAAAAJ)
- [Piotr Mirowski](https://piotrmirowski.com/)
- [Maria Athanassiadou](https://scholar.google.com/citations?user=VtkgHP0AAAAJ)
- [Sheleem Kashem](https://www.linkedin.com/in/sheleemkashem/)
- [Rachel Prudden](https://computerscience.exeter.ac.uk/staff/rep218)
- [Amol Mandhane](https://github.com/amol-mandhane)
- [Aidan Clark](https://scholar.google.com/citations?user=_19DrfIAAAAJ)
- [Andrew Brock](https://github.com/ajbrock)
- [Karen Simonyan](https://scholar.google.com/citations?user=L7lMQkQAAAAJ)
- [Raia Hadsell](https://github.com/raiah)
- [Niall Robinson](https://github.com/niallrobinson)
- [Ellen Clancy](https://www.linkedin.com/in/ellen-clancy-815967124)
- [Shakir Mohamed](https://www.shakirm.com/)
- [](https://arxiv.org/abs/2104.00954)
- [blog post](https://deepmind.com/blog/article/nowcasting)
- [local kernel](https://research.google.com/colaboratory/local-runtimes.html)
- [](https://www.tensorflow.org/hub)
| Live Speech Portraits | Real-Time Photorealistic Talking-Head Animation |
- [Yuanxun Lu](https://github.com/YuanxunLu)
- [Jinxiang Chai](https://scholar.google.com/citations?user=OcN1_gwAAAAJ)
- [Xun Cao](https://cite.nju.edu.cn/People/Faculty/20190621/i5054.html)
- [](https://arxiv.org/abs/2109.10595)
- [](https://github.com/lelechen63/ATVGnet), [](https://github.com/lelechen63/Talking-head-Generation-with-Rhythmic-Head-Motion), [](https://github.com/DinoMan/speech-driven-animation), [](https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix)
- [project](https://yuanxunlu.github.io/projects/LiveSpeechPortraits/)
| StylEx | Training a GAN to explain a classifier in StyleSpace |
- [Oran Lang](https://research.google/people/105975/)
- [Yossi Gandelsman](https://yossigandelsman.github.io/)
- [Michal Yarom](https://scholar.google.com/citations?user=GMVxiYgAAAAJ)
- [Yoav Wald](https://scholar.google.com/citations?user=hh5nOn4AAAAJ) others
- [Gal Elidan](https://research.google/people/105719/)
- [Avinatan Hassidim](https://research.google/people/105831/)
- [William Freeman](https://billf.mit.edu/)
- [Phillip Isola](http://web.mit.edu/phillipi/)
- [Amir Globerso](https://cs3801.wixsite.com/amirgloberson)
- [Michal Irani](http://www.weizmann.ac.il/math/irani/)
- [Inbar Mosseri](https://research.google/people/InbarMosseri/)
- [](https://arxiv.org/abs/2104.13369), [](https://arxiv.org/abs/1906.10112), [](https://arxiv.org/abs/2011.12799), [](https://arxiv.org/abs/1912.04958), [](https://arxiv.org/abs/1710.01711)
- [blog post](https://ai.googleblog.com/2022/01/introducing-stylex-new-approach-for.html)
- [project](https://explaining-in-style.github.io/)
- [supplementary](https://explaining-in-style.github.io/supmat.html)
- [](https://youtu.be/wLk2eBdXH4M)
| VITS | Parallel end-to-end TTS method that generates more natural sounding audio than current two-stage models |
- [Jaehyeon Kim](https://jaywalnut310.github.io/)
- [Jungil Kong](https://github.com/jik876)
- [Juhee Son](https://juheeuu.github.io/)
- [](https://arxiv.org/abs/2106.06103)
- [demo](https://jaywalnut310.github.io/vits-demo/)
| Bringing Old Photo Back to Life | Restoring old photos that suffer from severe degradation through a deep learning approach |
- [Ziyu Wan](http://raywzy.com/)
- [Bo Zhang](https://bo-zhang.me/)
- [Dongdong Chen](http://www.dongdongchen.bid/)
- [Pan Zhang](https://panzhang0212.github.io/) others
- [Dong Chen](http://www.dongchen.pro/)
- [Jing Liao](https://liaojing.github.io/html/)
- [Fang Wen](https://www.microsoft.com/en-us/research/people/fangwen/)
- [](https://arxiv.org/abs/2004.09484)
- [demo](https://replicate.com/microsoft/bringing-old-photos-back-to-life)
- [project](http://raywzy.com/Old_Photo/)
- [](https://youtu.be/Q5bhszQq9eA)
| PTI | Pivotal Tuning Inversion enables employing off-the-shelf latent based semantic editing techniques on real images using StyleGAN |
- [Daniel Roich](https://github.com/danielroich)
- [Ron Mokady](https://rmokady.github.io/)
- [Amit Bermano](https://www.cs.tau.ac.il/~amberman/)
- [Daniel Cohen-Or](https://danielcohenor.com/)
- [](https://arxiv.org/abs/2106.05744)
- [](https://github.com/NVlabs/stylegan2-ada-pytorch), [](https://github.com/richzhang/PerceptualSimilarity)
| TediGAN | Framework for multi-modal image generation and manipulation with textual descriptions |
- [Weihao Xia](https://github.com/weihaox)
- [Yujiu Yang](http://www.fiesta.tsinghua.edu.cn/pi/3/24)
- [Jing-Hao Xue](http://www.homepages.ucl.ac.uk/~ucakjxu/)
- [Baoyuan Wu](https://sites.google.com/site/baoyuanwu2015/home)
- [](https://arxiv.org/abs/2012.03308), [](https://arxiv.org/abs/2104.08910)
- [](https://github.com/weihaox/Multi-Modal-CelebA-HQ), [](https://github.com/NVlabs/ffhq-dataset), [](https://github.com/rosinality/stylegan2-pytorch/), [](https://github.com/fyu/lsun)
- [](https://youtu.be/L8Na2f5viAM)
| CogView | Mastering Text-to-Image Generation via Transformers |
- [Ming Ding](https://scholar.google.com/citations?user=Va50YzkAAAAJ)
- [Zhuoyi Yang](https://scholar.google.com/citations?user=tgAt-gEAAAAJ)
- [Wenyi Hong](https://github.com/wenyihong)
- [Wendi Zheng](https://github.com/minkowski0125) others
- [Chang Zhou](https://scholar.google.com/citations?user=QeSoG3sAAAAJ)
- [Junyang Lin](https://justinlin610.github.io/)
- [Xu Zou](http://xuzou.cn/)
- [Zhou Shao](https://www.researchgate.net/profile/Shao_Zhou4)
- [Hongxia Yang](https://sites.google.com/site/hystatistics/home)
- [Jie Tang](https://keg.cs.tsinghua.edu.cn/jietang/)
- [](https://arxiv.org/abs/2105.13290)
- [demo](https://thudm.github.io/CogView/index.html)
- [](https://github.com/NVIDIA/apex), [](https://github.com/Sleepychord/cogdata)
- [](https://towardsdatascience.com/cogview-image-generation-and-language-modelling-at-scale-8d358a0686d2)
- [](https://proceedings.neurips.cc/paper/2021/hash/a4d92e2cd541fca87e4620aba658316d-Abstract.html)
- [](https://www.reddit.com/r/MachineLearning/comments/nmxsd8/r_cogview_mastering_texttoimage_generation_via/)
- [](https://youtu.be/Cw1r8ACIj8U)
| GANs N' Roses | Stable, Controllable, Diverse Image to Image Translation |
- [Min Jin Chong](https://mchong6.github.io/)
- [David Forsyth](http://luthuli.cs.uiuc.edu/~daf/)
- [](https://arxiv.org/abs/2106.06561), [](https://arxiv.org/abs/2007.06600)
- [](https://github.com/rosinality/stylegan2-pytorch), [](https://github.com/znxlwm/UGATIT-pytorch)
- [](https://youtu.be/VNg0NyCGl_4)
| Rethinking Style Transfer: From Pixels to Parameterized Brushstrokes | A method to stylize images by optimizing parameterized brushstrokes instead of pixels |
- [Dmytro Kotovenko](https://scholar.google.de/citations?user=T_U8yxwAAAAJ)
- [Matthias Wright](https://matthias-wright.github.io/)
- [Arthur Heimbrecht](https://github.com/arwehei)
- [Björn Ommer](https://ommer-lab.com/people/ommer/)
- [](https://arxiv.org/abs/2103.17185)
- [project](https://compvis.github.io/brushstroke-parameterized-style-transfer/)
| Pixel2Style2Pixel | Encoding in Style: A StyleGAN Encoder for Image-to-Image Translation |
- [Elad Richardson](https://github.com/eladrich)
- [Yuval Alaluf](https://yuval-alaluf.github.io/)
- [Yotam Nitzan](https://yotamnitzan.github.io/)
- [Daniel Cohen-Or](https://danielcohenor.com/)
- [](https://arxiv.org/abs/2008.00951)
- [](https://github.com/rosinality/stylegan2-pytorch), [](https://github.com/HuangYG123/CurricularFace)
- [project](https://eladrich.github.io/pixel2style2pixel/)
- [](https://youtu.be/bfvSwhqsTgM)
| Fine-tuning a BERT | We will work through fine-tuning a BERT model using the tensorflow-models PIP package |
- [Chen Chen](https://github.com/chenGitHuber)
- [Claire Yao](https://github.com/claireyao-fen)
- [](https://arxiv.org/abs/1810.04805)
- [](https://tensorflow.org/hub)
| ReStyle | A Residual-Based StyleGAN Encoder via Iterative Refinement |
- [Yuval Alaluf](https://yuval-alaluf.github.io/)
- [Or Patashnik](https://orpatashnik.github.io/)
- [Daniel Cohen-Or](https://danielcohenor.com/)
- [](https://arxiv.org/abs/2104.02699), [](https://arxiv.org/abs/2008.00951), [](https://arxiv.org/abs/2102.02766)
- [](https://github.com/rosinality/stylegan2-pytorch), [](https://github.com/TreB1eN/InsightFace_Pytorch)
- [project](https://yuval-alaluf.github.io/restyle-encoder/)
| Motion Representations for Articulated Animation | Novel motion representations for animating articulated objects consisting of distinct parts |
- [Aliaksandr Siarohin](https://aliaksandrsiarohin.github.io/aliaksandr-siarohin-website/)
- [Oliver Woodford](https://ojwoodford.github.io/)
- [Jian Ren](https://alanspike.github.io/)
- [Menglei Chai](https://mlchai.com/)
- [Sergey Tulyakov](http://www.stulyakov.com/)
- [](https://arxiv.org/abs/2104.11280)
- [project](https://snap-research.github.io/articulated-animation/)
- [](https://www.youtube.com/watch?v=gpBYN8t8_yY)
| SAM | Age Transformation Using a Style-Based Regression Model |
- [Yuval Alaluf](https://yuval-alaluf.github.io/)
- [Or Patashnik](https://orpatashnik.github.io/)
- [Daniel Cohen-Or](https://danielcohenor.com/)
- [](https://arxiv.org/abs/2102.02754)
- [](https://github.com/eladrich/pixel2style2pixel), [](https://github.com/rosinality/stylegan2-pytorch)
- [project](https://yuval-alaluf.github.io/SAM/)
- [](https://youtu.be/X_pYC_LtBFw)
| Geometry-Free View Synthesis | Is a geometric model required to synthesize novel views from a single image? |
- [Robin Rombach](https://github.com/rromb)
- [Patrick Esser](https://github.com/pesser)
- [Björn Ommer](https://ommer-lab.com/people/ommer/)
- [](https://arxiv.org/abs/2104.07652)
- [data](https://google.github.io/realestate10k/)
- [](https://github.com/colmap/colmap)
| NeRViS | An algorithm for full-frame video stabilization by first estimating dense warp fields |
- [Yu-Lun Liu](http://www.cmlab.csie.ntu.edu.tw/~yulunliu/)
- [Wei-Sheng Lai](https://www.wslai.net/)
- [Ming-Hsuan Yang](https://faculty.ucmerced.edu/mhyang/)
- [Yung-Yu Chuang](https://www.csie.ntu.edu.tw/~cyy/)
- [Jia-Bin Huang](https://jbhuang0604.github.io/)
- [](https://arxiv.org/abs/2102.06205)
- [data](http://liushuaicheng.org/SIGGRAPH2013/database.html)
- [](https://github.com/cxjyxxme/deep-online-video-stabilization), [](https://github.com/jinsc37/DIFRINT)
- [project](https://alex04072000.github.io/NeRViS/)
- [](https://youtu.be/KO3sULs4hso)
| NeX | View synthesis based on enhancements of multiplane image that can reproduce NeXt-level view-dependent effects in real time |
- [Suttisak Wizadwongsa](https://www.linkedin.com/in/suttisak-wizadwongsa-763a931a5/)
- [Pakkapon Phongthawee](http://pureexe.github.io/)
- [Jiraphon Yenphraphai](https://www.linkedin.com/in/jiraphon-yenphraphai-990ba6175/)
- [Supasorn Suwajanakorn](https://www.supasorn.com/)
- [](https://arxiv.org/abs/2103.05606)
- [data](https://vistec-my.sharepoint.com/personal/pakkapon_p_s19_vistec_ac_th/_layouts/15/onedrive.aspx?id=%2Fpersonal%2Fpakkapon%5Fp%5Fs19%5Fvistec%5Fac%5Fth%2FDocuments%2Fpublic%2FVLL%2FNeX%2Fshiny%5Fdatasets&originalPath=aHR0cHM6Ly92aXN0ZWMtbXkuc2hhcmVwb2ludC5jb20vOmY6L2cvcGVyc29uYWwvcGFra2Fwb25fcF9zMTlfdmlzdGVjX2FjX3RoL0VuSVVoc1JWSk9kTnNaXzRzbWRoeWUwQjh6MFZseHFPUjM1SVIzYnAwdUd1cFE%5FcnRpbWU9WXRVQTQtQTcyVWc), [data](https://vistec-my.sharepoint.com/personal/pakkapon_p_s19_vistec_ac_th/_layouts/15/onedrive.aspx?originalPath=aHR0cHM6Ly92aXN0ZWMtbXkuc2hhcmVwb2ludC5jb20vOmY6L2cvcGVyc29uYWwvcGFra2Fwb25fcF9zMTlfdmlzdGVjX2FjX3RoL0VyalBSUkw5Sm5GSXA4TU42ZDFqRXVvQjNYVm94SmtmZlBqZm9QeWhIa2owZGc%5FcnRpbWU9bC0yYWctRTcyVWc&id=%2Fpersonal%2Fpakkapon%5Fp%5Fs19%5Fvistec%5Fac%5Fth%2FDocuments%2Fpublic%2FVLL%2FNeX%2Fmodified%5Fdataset)
- [](https://github.com/Fyusion/LLFF)
- [project](https://nex-mpi.github.io/)
- [vistec](https://vistec.ist/)
- [](https://www.youtube.com/watch?v=HyfkF7Z-ddA)
| Score SDE | Score-Based Generative Modeling through Stochastic Differential Equations |
- [Yang Song](https://yang-song.net/)
- [Jascha Sohl-Dickstein](http://www.sohldickstein.com/)
- [Diederik Kingma](http://dpkingma.com/)
- [Abhishek Kumar](https://abhishek.umiacs.io/) others
- [Stefano Ermon](https://cs.stanford.edu/~ermon/)
- [Ben Poole](https://cs.stanford.edu/~poole/)
- [](https://arxiv.org/abs/2011.13456), [](https://arxiv.org/abs/1907.05600), [](https://arxiv.org/abs/2006.09011), [](https://arxiv.org/abs/2006.11239)
- [](https://github.com/yang-song/score_sde_pytorch), [](https://github.com/google/ml_collections)
- [](https://youtu.be/L9ZegT87QK8)
| Talking Head Anime from a Single Image | The network takes as input an image of an anime character's face and a desired pose, and it outputs another image of the same character in the given pose | [Pramook Khungurn](https://pkhungurn.github.io/) | [](https://github.com/pkhungurn/talking-head-anime-demo)
- [](https://github.com/lincolnhard/head-pose-estimation)
- [project](https://pkhungurn.github.io/talking-head-anime/)
- [](https://en.wikipedia.org/wiki/Virtual_YouTuber), [](https://en.wikipedia.org/wiki/MikuMikuDance)
- [](https://youtu.be/kMQCERkTdO0), [](https://youtu.be/T1Gp-RxFZwU), [](https://youtu.be/FioRJ6x_RbI)
| NFNet | An adaptive gradient clipping technique, a significantly improved class of Normalizer-Free ResNets |
- [Andrew Brock](https://github.com/ajbrock)
- [Soham De](https://sohamde.github.io/)
- [Samuel L. Smith](https://scholar.google.co.uk/citations?user=fyEqU5oAAAAJ)
- [Karen Simonyan](https://scholar.google.com/citations?user=L7lMQkQAAAAJ)
- [](https://arxiv.org/abs/2102.06171), [](https://arxiv.org/abs/2101.08692)
- [](https://github.com/deepmind/jaxline)
- [](https://youtu.be/rNkHjZtH0RQ), [](https://www.youtube.com/live/qyy2WhRRSI4?feature=share)
| RITM | Simple feedforward model for click-based interactive segmentation that employs the segmentation masks from previous steps |
- [Konstantin Sofiiuk](https://github.com/ksofiyuk)
- [Ilia Petrov](https://virtualhumans.mpi-inf.mpg.de/people/Petrov.html)
- [Anton Konushin](https://scholar.google.com/citations?user=ZT_k-wMAAAAJ)
- [](https://arxiv.org/abs/2102.06583)
- [](https://github.com/HRNet/HRNet-Image-Classification)
- [](https://paperswithcode.com/sota/interactive-segmentation-on-grabcut?p=reviving-iterative-training-with-mask), [](https://paperswithcode.com/sota/interactive-segmentation-on-berkeley?p=reviving-iterative-training-with-mask)
| CLIP | A neural network which efficiently learns visual concepts from natural language supervision |
- [Jong Wook Kim](https://jongwook.kim/)
- [Alec Radford](http://newmu.github.io/)
- [Ilya Sutskever](http://www.cs.utoronto.ca/~ilya/)
- [](https://arxiv.org/abs/2103.00020)
- [data](https://www.cs.toronto.edu/~kriz/cifar.html)
- [paper](https://cdn.openai.com/papers/Learning_Transferable_Visual_Models_From_Natural_Language_Supervision.pdf)
- [project](https://openai.com/blog/clip/)
- [slides](https://icml.cc/media/icml-2021/Slides/9193.pdf)
| Adversarial Patch | A method to create universal, robust, targeted adversarial image patches in the real world | [Tom Brown](https://github.com/nottombrown) |
- [](https://arxiv.org/abs/1712.09665)
| MSG-Net | Multi-style Generative Network with a novel Inspiration Layer, which retains the functionality of optimization-based approaches and has the fast speed of feed-forward networks |
- [Hang Zhang](https://hangzhang.org/)
- [Kristin Dana](https://www.ece.rutgers.edu/~kdana/dana.html)
- [](https://arxiv.org/abs/1703.06953)
- [project](http://computervisionrutgers.github.io/MSG-Net/)
- [](https://www.youtube.com/watch?v=oy6pWNWBt4Y)
| f-BRS | Feature backpropagating refinement scheme that solves an optimization problem with respect to auxiliary variables instead of the network inputs, and requires running forward and backward pass just for a small part of a network |
- [Konstantin Sofiiuk](https://github.com/ksofiyuk)
- [Ilia Petrov](https://virtualhumans.mpi-inf.mpg.de/people/Petrov.html)
- [Olga Barinova](https://github.com/OlgaBarinova)
- [Anton Konushin](https://scholar.google.com/citations?user=ZT_k-wMAAAAJ)
- [](https://arxiv.org/abs/2001.10331)
- [](https://github.com/HRNet/HRNet-Image-Classification)
- [](https://youtu.be/ArcZ5xtyMCk), [](https://youtu.be/xg-5J9gLuXA)
| Neural Style Transfer | Implementation of Neural Style Transfer in Keras 2.0+ | [Somshubra Majumdar](http://titu1994.github.io/) | [](https://doi.org/10.1167/16.12.326) [](https://github.com/titu1994/Neural-Style-Transfer)
- [](http://arxiv.org/abs/1508.06576), [](http://arxiv.org/abs/1605.04603), [](https://arxiv.org/abs/1606.05897)
| SkyAR | A vision-based method for video sky replacement and harmonization, which can automatically generate realistic and dramatic sky backgrounds in videos with controllable styles | [Zhengxia Zou](http://www-personal.umich.edu/~zzhengxi/) | [](https://doi.org/10.1109/TIP.2022.3192717) [](https://github.com/jiupinjia/SkyAR)
- [](https://arxiv.org/abs/2010.11800)
- [project](https://jiupinjia.github.io/skyar/)
- [](https://www.youtube.com/watch?v=zal9Ues0aOQ)
| MusicXML Documentation | The goal of this notebook is to explore one of the magenta libraries for music |
- [Prakruti Joshi](https://github.com/prakruti-joshi)
- [Falak Shah](https://falaktheoptimist.github.io/)
- [Twisha Naik](https://github.com/twisha96)
- [magenta](https://magenta.tensorflow.org/)
- [music theory](http://musictheoryblog.blogspot.com/2008/02/learn-music-theory.html)
- [musicXML](https://www.musicxml.com/for-developers/)
| SVG VAE | A colab demo for the SVG VAE model | [Raphael Gontijo Lopes](https://raphagl.com/) | [](https://doi.org/10.1109/ICCV.2019.00802)
- [](https://arxiv.org/abs/1904.02632)
- [blog post](https://magenta.tensorflow.org/svg-vae)
| Neural Magic Eye | Learning to See and Understand the Scene Behind an Autostereogram |
- [Zhengxia Zou](http://www-personal.umich.edu/~zzhengxi/)
- [Tianyang Shi](https://www.shitianyang.tech/)
- [Yi Yuan](https://yiyuan1991.github.io/)
- [Zhenwei Shi](http://levir.buaa.edu.cn/)
- [](https://arxiv.org/abs/2012.15692)
- [project](https://jiupinjia.github.io/neuralmagiceye/)
- [](https://www.youtube.com/watch?v=Fkh7DEblqJ8)
| FGVC | Method first extracts and completes motion edges, and then uses them to guide piecewise-smooth flow completion with sharp edges |
- [Chen Gao](http://chengao.vision/)
- [Ayush Saraf](https://github.com/ayush29feb)
- [Johannes Kopf](https://johanneskopf.de/)
- [Jia-Bin Huang](https://jbhuang0604.github.io/)
- [](https://arxiv.org/abs/2009.01835)
- [project](http://chengao.vision/FGVC/)
- [](https://www.youtube.com/watch?v=CHHVPxHT7rc)
| VIBE | Video Inference for Body Pose and Shape Estimation, which makes use of an existing large-scale motion capture dataset together with unpaired, in-the-wild, 2D keypoint annotations |
- [Muhammed Kocabas](https://ps.is.mpg.de/person/mkocabas)
- [Nikos Athanasiou](https://github.com/athn-nik)
- [Michael Black](https://ps.is.mpg.de/person/black)
- [](https://arxiv.org/abs/1912.05656)
- [](https://github.com/carlosedubarreto/vibe_win_install), [](https://github.com/vchoutas/smplx), [](https://github.com/akanazawa/human_dynamics), [](https://github.com/MandyMo/pytorch_HMR), [](https://github.com/soulslicer/STAF/tree/staf)
- [](https://paperswithcode.com/sota/3d-human-pose-estimation-on-3dpw?p=vibe-video-inference-for-human-body-pose-and)
- [](https://youtu.be/3qhs5IRJ1LI), [](https://youtu.be/w1biKeiQThY), [](https://youtu.be/rIr-nX63dUA), [](https://youtu.be/fW0sIZfQcIs), [](https://youtu.be/8Qt0wA16kTo), [](https://youtu.be/xyo5gl5GLEI), [](https://youtu.be/XNzgUhxKC38), [](https://youtu.be/hErK0MamTY4), [](https://youtu.be/Gfmm8uMfMq0)
| SeFa | A closed-form approach for unsupervised latent semantic factorization in GANs |
- [Yujun Shen](https://shenyujun.github.io/)
- [Bolei Zhou](https://boleizhou.github.io/)
- [](https://arxiv.org/abs/2007.06600)
- [project](https://genforce.github.io/sefa/)
- [](https://www.youtube.com/watch?v=OFHW2WbXXIQ)
| Stylized Neural Painting | An image-to-painting translation method that generates vivid and realistic painting artworks with controllable styles |
- [Zhengxia Zou](http://www-personal.umich.edu/~zzhengxi/)
- [Tianyang Shi](https://www.shitianyang.tech/)
- [Yi Yuan](https://yiyuan1991.github.io/)
- [Zhenwei Shi](http://levir.buaa.edu.cn/)
- [](https://arxiv.org/abs/2011.08114)
- [project](https://jiupinjia.github.io/neuralpainter/)
- [](https://www.youtube.com/watch?v=oerb-nwrXhk)
| BiT | Big Transfer: General Visual Representation Learning |
- [Alexander Kolesnikov](https://github.com/akolesnikoff)
- [Lucas Beyer](http://lucasb.eyer.be)
- [Xiaohua Zhai](https://github.com/xiaohuazhai)
- [Joan Puigcerver](https://www.jpuigcerver.net/) others
- [Jessica Yung](https://github.com/jessicayung)
- [Sylvain Gelly](https://scholar.google.com/citations?user=m7LvuTkAAAAJ)
- [Neil Houlsby](https://neilhoulsby.github.io/)
- [](https://arxiv.org/abs/1912.11370), [](https://arxiv.org/abs/2106.05237)
- [](https://huggingface.co/google/bit-50)
- [](https://sh-tsang.medium.com/review-big-transfer-bit-general-visual-representation-learning-cb4bf8ed9732)
- [](https://youtu.be/k1GOF2jmX7c), [](https://youtu.be/0iTgt5-SOsU), [](https://youtu.be/X5Rhm__OxvA)
| LaSAFT | Latent Source Attentive Frequency Transformation for Conditioned Source Separation | [Woosung Choi](https://ws-choi.github.io/) | [](https://doi.org/10.1109/ICASSP39728.2021.9413896) [](https://github.com/ws-choi/Conditioned-Source-Separation-LaSAFT)
- [](https://arxiv.org/abs/2010.11631)
- [data](https://sigsep.github.io/datasets/musdb.html)
- [project](https://lasaft.github.io/)
| Lifespan Age Transformation Synthesis | Multi-domain image-to-image generative adversarial network architecture, whose learned latent space models a continuous bi-directional aging process |
- [Roy Or-El](https://homes.cs.washington.edu/~royorel/)
- [Soumyadip Sengupta](https://homes.cs.washington.edu/~soumya91/)
- [Ohad Fried](https://www.ohadf.com/)
- [Eli Shechtman](https://research.adobe.com/person/eli-shechtman/)
- [Ira Kemelmacher-Shlizerman](https://www.irakemelmacher.com/)
- [](https://arxiv.org/abs/2003.09764)
- [](https://github.com/royorel/FFHQ-Aging-Dataset), [](https://github.com/NVIDIA/pix2pixHD), [](https://github.com/rosinality/style-based-gan-pytorch)
- [project](https://grail.cs.washington.edu/projects/lifespan_age_transformation_synthesis/)
- [](https://youtu.be/_jTFcjN2hBk), [](https://youtu.be/9fulnt2_q_Y)
| HiGAN | Semantic Hierarchy Emerges in Deep Generative Representations for Scene Synthesis |
- [Ceyuan Yang](https://ceyuan.me/)
- [Yujun Shen](https://shenyujun.github.io/)
- [Bolei Zhou](https://boleizhou.github.io/)
- [](https://arxiv.org/abs/1911.09267), [](https://arxiv.org/abs/1412.6856), [](https://arxiv.org/abs/1906.10112)
- [project](https://genforce.github.io/higan/)
- [](https://www.youtube.com/watch?v=X5yWu2Jwjpg)
| InterFaceGAN | Interpreting the Latent Space of GANs for Semantic Face Editing |
- [Yujun Shen](https://shenyujun.github.io/)
- [Jinjin Gu](https://www.jasongt.com/)
- [Xiaoou Tang](https://www.ie.cuhk.edu.hk/people/xotang.shtml)
- [Bolei Zhou](https://boleizhou.github.io/)
- [](https://arxiv.org/abs/1907.10786), [](https://arxiv.org/abs/2005.09635), [](https://arxiv.org/abs/1710.10196)
- [](https://github.com/tkarras/progressive_growing_of_gans), [](https://github.com/NVlabs/stylegan)
- [project](https://genforce.github.io/interfacegan/)
- [](https://www.youtube.com/watch?v=uoftpl3Bj6w)
| Instance-aware Image Colorization | Novel deep learning framework to achieve instance-aware colorization | [Jheng-Wei Su](https://github.com/ericsujw) | [](https://doi.org/10.1109/CVPR42600.2020.00799) [](https://github.com/ericsujw/InstColorization)
- [](https://arxiv.org/abs/2005.10825)
- [project](https://ericsujw.github.io/InstColorization/)
- [](https://www.youtube.com/watch?v=Zj1N4uE1ehk)
| MoCo | Momentum Contrast for unsupervised visual representation learning |
- [Kaiming He](https://kaiminghe.github.io/)
- [Haoqi Fan](https://haoqifan.github.io/)
- [Yuxin Wu](https://ppwwyyxx.com/)
- [Saining Xie](http://sainingxie.com/)
- [Ross Girshick](https://www.rossgirshick.info/)
- [](https://arxiv.org/abs/1911.05722), [](https://arxiv.org/abs/2003.04297), [](https://arxiv.org/abs/1706.02677)
- [](https://github.com/ppwwyyxx/moco.tensorflow)
- [](https://youtu.be/LvHwBQF14zs), [](https://youtu.be/4VVGtYPM8JE), [](https://youtu.be/o5Qh61dLDf0)
| Rewriting a Deep Generative Model | We ask if a deep network can be reprogrammed to follow different rules, by enabling a user to directly change the weights, instead of training with a data set |
- [David Bau](https://people.csail.mit.edu/davidbau/home/)
- [Steven Liu](http://people.csail.mit.edu/stevenliu/)
- [Tongzhou Wang](https://ssnl.github.io/)
- [Jun-Yan Zhu](https://www.cs.cmu.edu/~junyanz/)
- [Antonio Torralba](https://groups.csail.mit.edu/vision/torralbalab/)
- [](https://arxiv.org/abs/2007.15646), [](https://arxiv.org/abs/1912.04958)
- [](https://github.com/NVlabs/stylegan2), [](https://github.com/rosinality/stylegan2-pytorch)
- [project](https://rewriting.csail.mit.edu/)
- [](https://www.youtube.com/watch?v=i2_-zNqtEPk), [](https://rewriting.csail.mit.edu/video/)
| SIREN | Implicit Neural Representations with Periodic Activation Functions |
- [Vincent Sitzmann](https://vsitzmann.github.io/)
- [Julien Martel](http://web.stanford.edu/~jnmartel/)
- [](https://arxiv.org/abs/2006.09661)
- [data](https://drive.google.com/drive/folders/1_iq__37-hw7FJOEUK1tX7mdp8SKB368K)
- [](https://proceedings.neurips.cc/paper/2020/hash/53c04118df112c13a8c34b38343b9c10-Abstract.html)
- [project](https://vsitzmann.github.io/siren/)
- [](https://www.youtube.com/watch?v=Q2fLWGBeaiI)
| PIFu | Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization |
- [Ryota Natsume](https://github.com/nanopoteto)
- [Shunsuke Saito](https://github.com/shunsukesaito)
- [Zeng Huang](https://zeng.science/)
- [Angjoo Kanazawa](https://people.eecs.berkeley.edu/~kanazawa/)
- [Hao Li](http://hao.li)
- [](https://arxiv.org/abs/1905.05172)
- [](https://www.youtube.com/watch?v=S1FpjwKqtPs)
| 3D Photo Inpainting | Method for converting a single RGB-D input image into a 3D photo, i.e., a multi-layer representation for novel view synthesis that contains hallucinated color and depth structures in regions occluded in the original view |
- [Meng-Li Shih](https://shihmengli.github.io/)
- [Shih-Yang Su](https://lemonatsu.github.io/)
- [Johannes Kopf](https://johanneskopf.de/)
- [Jia-Bin Huang](https://jbhuang0604.github.io/)
- [](https://arxiv.org/abs/2004.04727)
- [project](https://shihmengli.github.io/3D-Photo-Inpainting/)
| Motion Supervised co-part Segmentation | A self-supervised deep learning method for co-part segmentation |
- [Aliaksandr Siarohin](https://aliaksandrsiarohin.github.io/aliaksandr-siarohin-website/)
- [Subhankar Roy](https://github.com/roysubhankar)
- [](http://arxiv.org/abs/2004.03234)
- [](https://github.com/AliaksandrSiarohin/video-preprocessing)
- [](https://www.youtube.com/watch?v=RJ4Nj1wV5iA)
| Onsets and Frames | Onsets and Frames is an automatic music transcription framework with piano and drums models |
- [Curtis Hawthorne](https://github.com/cghawthorne)
- [Erich Elsen](https://github.com/ekelsen)
- [](https://arxiv.org/abs/1710.11153), [](https://arxiv.org/abs/1810.12247), [](https://arxiv.org/abs/2004.00188)
- [blog post](http://g.co/magenta/onsets-frames)
- [data](https://g.co/magenta/maestro-wave2midi2wave), [data](https://magenta.tensorflow.org/datasets/e-gmd)
| BERT score | An automatic evaluation metric for text generation | [Tianyi Zhang](https://tiiiger.github.io/) | [](https://github.com/Tiiiger/bert_score)
- [](https://arxiv.org/abs/1904.09675)
| Generating Piano Music with Transformer | This Colab notebook lets you play with pretrained Transformer models for piano music generation, based on the Music Transformer |
- [Ian Simon](https://github.com/iansimon)
- [Anna Huang](https://github.com/czhuang)
- [Jesse Engel](https://github.com/jesseengel)
- [Curtis Hawthorne](https://github.com/cghawthorne)
- [](https://arxiv.org/abs/1706.03762), [](https://arxiv.org/abs/1809.04281)
- [blog post](http://g.co/magenta/music-transformer)
| GANSynth | This notebook is a demo GANSynth, which generates audio with Generative Adversarial Networks | [Jesse Engel](https://github.com/jesseengel) | [](https://github.com/magenta/magenta/tree/main/magenta/models/gansynth)
- [](https://arxiv.org/abs/1902.08710), [](https://arxiv.org/abs/1809.11096)
- [project](https://storage.googleapis.com/magentadata/papers/gansynth/index.html)
| Latent Constraints | Conditional Generation from Unconditional Generative Models |
- [Jesse Engel](https://github.com/jesseengel)
- [Matthew Hoffman](http://matthewdhoffman.com/)
- [Adam Roberts](https://github.com/adarob)
- [](https://arxiv.org/abs/1711.05772)
- [data](http://mmlab.ie.cuhk.edu.hk/projects/CelebA.html)
| Performance RNN | This notebook shows you how to generate new performed compositions from a trained model |
- [Ian Simon](https://github.com/iansimon)
- [Sageev Oore](https://github.com/osageev)
- [Curtis Hawthorne](https://github.com/cghawthorne)
- [blog post](https://magenta.tensorflow.org/performance-rnn)
- [data](http://www.piano-e-competition.com/)
| NSynth | This colab notebook has everything you need to upload your own sounds and use NSynth models to reconstruct and interpolate between them |
- [Jesse Engel](https://github.com/jesseengel)
- [Cinjon Resnick](https://github.com/cinjon)
- [Adam Roberts](https://github.com/adarob)
- [Sander Dieleman](https://benanne.github.io/) others
- [Karen Simonyan](https://scholar.google.com/citations?user=L7lMQkQAAAAJ)
- [Mohammad Norouzi](https://norouzi.github.io/)
- [Douglas Eck](https://github.com/douglaseck)
- [](https://arxiv.org/abs/1704.01279)
- [blog post](https://magenta.tensorflow.org/nsynth)
- [data](https://magenta.tensorflow.org/datasets/nsynth)
- [tutorial](https://magenta.tensorflow.org/nsynth-fastgen)
- [](https://www.youtube.com/watch?v=AaALLWQmCdI), [](https://www.youtube.com/watch?v=BOoSy-Pg8is)
## Tutorials
| name | description | authors | links | colaboratory | update |
|------|-------------|:--------|:------|:------------:|:------:|
| Cleanlab | Helps you clean data and labels by automatically detecting issues in a ML dataset |
- [Curtis Northcutt](https://www.curtisnorthcutt.com/)
- [Lu Jiang](http://www.lujiang.info/)
- [Isaac Chuang](http://feynman.mit.edu/ike/homepage/index.html)
- [](https://arxiv.org/abs/1911.00068)
- [blog post](https://l7.curtisnorthcutt.com/confident-learning)
- [](https://docs.cleanlab.ai/)
- [](https://medium.com/@sujathamudadla1213/cleanlab-python-library-34e0a37720ef)
- [
](https://cleanlab.ai/slack) - [](https://twitter.com/CleanlabAI)
- [](https://youtu.be/BnOTv0f9Msk), [](https://youtu.be/nGye-lrsLRc), [](https://youtu.be/QHaT_AiUljw)
| dm_control | DeepMind Infrastructure for Physics-Based Simulation |
- [Saran Tunyasuvunakool](https://github.com/saran-t)
- [Alistair Muldal](https://github.com/alimuldal)
- [Yotam Doron](http://www.yotamdoron.com/)
- [Siqi Liu](http://siqi.fr/) others
- [Steven Bohez](https://github.com/sbohez)
- [Josh Merel](https://sites.google.com/site/jsmerel/)
- [Tom Erez](https://github.com/erez-tom)
- [Timothy Lillicrap](https://contrastiveconvergence.net/~timothylillicrap/index.php)
- [Nicolas Heess](https://scholar.google.com/citations?user=79k7bGEAAAAJ)
- [Yuval Tassa](https://github.com/yuvaltassa)
- [](https://arxiv.org/abs/2006.12983), [](https://arxiv.org/abs/1801.00690), [](https://arxiv.org/abs/1902.07151), [](https://arxiv.org/abs/1707.02286), [](https://arxiv.org/abs/1802.09564), [](https://arxiv.org/abs/1802.10567)
- [blog post](https://www.deepmind.com/publications/dm-control-software-and-tasks-for-continuous-control)
- [](https://en.wikipedia.org/wiki/Tippe_top)
- [](https://youtu.be/CMjoiU482Jk), [](https://youtu.be/rAai4QzcYbs), [](https://youtu.be/WhaRsrlaXLk)
| MuJoCo | A general purpose physics engine that aims to facilitate research and development in robotics, biomechanics, graphics and animation, machine learning, and other areas which demand fast and accurate simulation of articulated structures interacting with their environment |
- [Emo Todorov](https://homes.cs.washington.edu/~todorov/)
- [Tom Erez](https://github.com/erez-tom)
- [Yuval Tassa](https://github.com/yuvaltassa)
- [](https://arxiv.org/abs/2006.12983)
- [blog post](https://www.deepmind.com/blog/opening-up-a-physics-simulator-for-robotics), [blog post](https://www.deepmind.com/blog/open-sourcing-mujoco)
- [](https://mujoco.readthedocs.io/en/latest/overview.html)
- [website](https://mujoco.org/)
- [](https://en.wikipedia.org/wiki/Tippe_top), [](https://en.wikipedia.org/wiki/Chaos_theory), [](https://en.wikipedia.org/wiki/3D_projection#Mathematical_formula)
- [](https://youtu.be/0ORsj_E17B0), [](https://youtu.be/yHZVVfsJ8mc), [](https://youtu.be/eyzzsGJ1iic)
| PyTorch3D | Library for deep learning with 3D data |
- [Nikhila Ravi](https://nikhilaravi.com/)
- [Jeremy Reizenstein](https://github.com/bottler)
- [David Novotny](https://d-novotny.github.io/)
- [Taylor Gordon](https://scholar.google.com/citations?user=CNOoeQ0AAAAJ) others
- [Wan-Yen Lo](https://github.com/wanyenlo)
- [Justin Johnson](https://web.eecs.umich.edu/~justincj/)
- [Georgia Gkioxari](https://gkioxari.github.io/)
- [](https://arxiv.org/abs/2007.08501), [](https://arxiv.org/abs/1906.02739)
- [blog post](https://ai.meta.com/blog/implicitron-a-new-modular-extensible-framework-for-neural-implicit-representations-in-pytorch3d/), [blog post](https://ai.meta.com/blog/-introducing-pytorch3d-an-open-source-library-for-3d-deep-learning/)
- [](https://pytorch3d.readthedocs.org/)
- [](https://www.kaggle.com/code/sohonjit/rendering-with-pytorch3d)
- [](https://towardsdatascience.com/glimpse-into-pytorch3d-an-open-source-3d-deep-learning-library-291a4beba30f), [](https://medium.com/@phamtdong0406/crafting-realistic-renderings-with-pytorch3d-947a38194f0a), [](https://towardsdatascience.com/how-to-render-3d-files-using-pytorch3d-ef9de72483f8)
- [website](https://pytorch3d.org/)
- [](https://youtu.be/0JEb7knenps), [](https://youtu.be/Pph1r-x9nyY), [](https://youtu.be/eCDBA_SbxCE), [](https://youtu.be/MOBAJb5nJRI), [](https://youtu.be/g50RiDnfIfY), [](https://youtu.be/hgBk9WlF-XA), [](https://youtu.be/Sb9gCCnSAUg), [](https://youtu.be/ZLqJ33Ey-MU)
| Kornia | Library is composed by a subset of packages containing operators that can be inserted within neural networks to train models to perform image transformations, epipolar geometry, depth estimation, and low-level image processing such as filtering and edge detection that operate directly on tensors |
- [Edgar Riba](https://github.com/edgarriba)
- [Dmytro Mishkin](https://dmytro.ai/)
- [Daniel Ponsa](https://github.com/DanielPonsa)
- [Ethan Rublee](https://github.com/ethanrublee)
- [Gary Bradski](https://github.com/garybradski)
- [](https://arxiv.org/abs/1910.02190)
- [blog post](https://opencv.org/kornia-an-open-source-differentiable-computer-vision-library-for-pytorch/)
- [](https://kornia.readthedocs.io/en/latest/)
- [](https://join.slack.com/t/kornia/shared_invite/zt-csobk21g-2AQRi~X9Uu6PLMuUZdvfjA)
- [](https://twitter.com/kornia_foss)
- [website](https://kornia.github.io/)
- [](https://www.youtube.com/channel/UCI1SE1Ij2Fast5BSKxoa7Ag), [](https://youtu.be/3RmCYFhwclE), [](https://youtu.be/AAZa-mXjYF0)
| AniPortrait | Framework for generating high-quality animation driven by audio and a reference portrait image |
- [Huawei Wei]()
- [Zejun Yang](https://github.com/Zejun-Yang)
- [Zhisheng Wang](https://scholar.google.com/citations?user=XrK2HNcAAAAJ)
- [](https://arxiv.org/abs/2403.17694)
- [](https://github.com/CelebV-HQ/CelebV-HQ), [](https://github.com/HumanAIGC/EMO), [](https://github.com/MooreThreads/Moore-AnimateAnyone), [](https://github.com/magic-research/magic-animate), [](https://github.com/guoqincode/Open-AnimateAnyone)
- [](https://huggingface.co/ZJYang/AniPortrait), [](https://huggingface.co/runwayml/stable-diffusion-v1-5), [](https://huggingface.co/stabilityai/sd-vae-ft-mse), [](https://huggingface.co/lambdalabs/sd-image-variations-diffusers/tree/main/image_encoder), [](https://huggingface.co/facebook/wav2vec2-base-960h)
- [](https://www.reddit.com/r/StableDiffusion/comments/1bp7rnj/aniportrait_audiodriven_synthesis_of/)
- [](https://youtu.be/wdRhYLQFQH8), [](https://youtu.be/T-B6xJRG6fQ)
| OpenVINO | Open-source toolkit for optimizing and deploying AI inference | [intel](https://www.intel.com/content/www/us/en/developer/topic-technology/open/overview.html) | [](https://github.com/openvinotoolkit/openvino)
- [blog post](https://blog.openvino.ai/)
- [](https://discord.gg/7pVRxUwdWG)
- [](https://docs.openvino.ai/)
- [forum](https://software.intel.com/en-us/forums/computer-vision)
- [](https://github.com/openvinotoolkit/open_model_zoo), [](https://github.com/Tencent/TNN), [](https://github.com/openvinotoolkit/openvino_contrib), [](https://github.com/openvinotoolkit/training_extensions), [](https://github.com/openvinotoolkit/model_server), [](https://github.com/opencv/cvat), [](https://github.com/openvinotoolkit/datumaro)
- [](https://huggingface.co/OpenVINO)
- [](https://medium.com/@openvino), [](https://medium.com/openvino-toolkit)
- [](https://en.wikipedia.org/wiki/OpenVINO)
- [](https://www.youtube.com/playlist?list=PLg-UKERBljNxdIQir1wrirZJ50yTp4eHv), [](https://youtu.be/Je8n8M0OwxQ), [](https://youtu.be/Ru51DELfc-Q), [](https://youtu.be/5X0RmlH6JI4), [](https://youtu.be/hhVRSLbpI5Q), [](https://youtu.be/JH8fsEAIaXo), [](https://www.youtube.com/playlist?list=PLWw98q-Xe7iH06qxEW5a22SBsSNsGnYjZ)
| PEFT | Parameter-Efficient Fine-Tuning methods enable efficient adaptation of pre-trained language models to various downstream applications without fine-tuning all the model's parameters |
- [Sourab Mangrulkar](https://github.com/pacman100)
- [Sylvain Gugger](https://github.com/sgugger)
- [Lysandre Debut](http://lysand.re/)
- [Younes Belkada](https://github.com/younesbelkada)
- [Sayak Paul](https://sayak.dev/)
- [blog post](https://www.philschmid.de/fine-tune-flan-t5-peft)
- [](https://huggingface.co/docs/peft)
- [](https://github.com/microsoft/DeepSpeed/issues/3002)
- [](https://huggingface.co/datasets/ought/raft/viewer/twitter_complaints), [](https://huggingface.co/bigscience/T0_3B), [](https://huggingface.co/bigscience/mt0-xxl), [](https://huggingface.co/facebook/opt-6.7b), [](https://huggingface.co/roberta-large), [](https://huggingface.co/datasets/glue/viewer/mrpc)
- [](https://youtu.be/YVU5wAA6Txo), [](https://youtu.be/Us5ZFp16PaU), [](https://youtu.be/YKCtbIJC3kQ)
| Building Your Own Federated Learning Algorithm | We discuss how to implement federated learning algorithms without deferring to the tff.learning API | [Zachary Charles](https://zachcharles.com/) |
- [](https://arxiv.org/abs/1907.08610)
- [blog post](https://ai.googleblog.com/2020/05/federated-analytics-collaborative-data.html)
- [](https://paperswithcode.com/task/federated-learning)
- [](https://www.tensorflow.org/federated/api_docs/python/tff/learning/Model)
| Federated Learning for Image Classification | We use the classic MNIST training example to introduce the Federated Learning API layer of TFF, tff.learning - a set of higher-level interfaces that can be used to perform common types of federated learning tasks, such as federated training, against user-supplied models implemented in TensorFlow | [Krzysztof Ostrowski](https://github.com/krzys-ostrowski) |
- [](https://arxiv.org/abs/1602.05629)
- [data](https://www.nist.gov/srd/nist-special-database-19)
- [](https://medium.com/tensorflow/standardizing-on-keras-guidance-on-high-level-apis-in-tensorflow-2-0-bad2b04c819a)
- [](https://paperswithcode.com/task/federated-learning), [](https://paperswithcode.com/task/image-classification)
| Federated Learning for Text Generation | We start with a RNN that generates ASCII characters, and refine it via federated learning | [Krzysztof Ostrowski](https://github.com/krzys-ostrowski) |
- [](https://arxiv.org/abs/1812.01097), [](https://arxiv.org/abs/1602.05629)
- [data](http://www.ibiblio.org/pub/docs/books/gutenberg/9/98/98.txt), [data](http://www.ibiblio.org/pub/docs/books/gutenberg/4/46/46.txt)
- [](https://www.tensorflow.org/hub)
| Custom Federated Algorithms, Part 1: Introduction to the Federated Core | This tutorial is the first part of a two-part series that demonstrates how to implement custom types of federated algorithms in TensorFlow Federated using the Federated Core - a set of lower-level interfaces that serve as a foundation upon which we have implemented the Federated Learning layer | [Krzysztof Ostrowski](https://github.com/krzys-ostrowski) |
- [](https://arxiv.org/abs/1602.05629)
- [](https://paperswithcode.com/task/federated-learning)
- [](https://www.tensorflow.org/federated/federated_core), [](https://www.tensorflow.org/federated/federated_learning)
| Custom Federated Algorithms, Part 2: Implementing Federated Averaging | This tutorial is the second part of a two-part series that demonstrates how to implement custom types of federated algorithms in TFF using the Federated Core, which serves as a foundation for the Federated Learning layer | [Krzysztof Ostrowski](https://github.com/krzys-ostrowski) | [](https://github.com/tensorflow/federated/blob/master/tensorflow_federated/python/learning/federated_averaging.py)
- [](https://paperswithcode.com/task/federated-learning)
- [](https://www.tensorflow.org/federated/federated_core), [](https://www.tensorflow.org/federated/federated_learning)
| TFF for Federated Learning Research: Model and Update Compression | We use the EMNIST dataset to demonstrate how to enable lossy compression algorithms to reduce communication cost in the Federated Averaging algorithm | [Weikang Song](https://github.com/swkpku) |
- [](https://arxiv.org/abs/1602.05629)
- [](https://paperswithcode.com/task/federated-learning)
- [tensor encoding](http://jakubkonecny.com/files/tensor_encoding.pdf)
- [](https://www.tensorflow.org/federated/api_docs/python/tff/simulation/datasets/emnist), [](https://www.tensorflow.org/federated/api_docs/python/tff/learning/build_federated_averaging_process)
| High-performance simulations with TFF | This tutorial will describe how to setup high-performance simulations with TFF in a variety of common scenarios | [Krzysztof Ostrowski](https://github.com/krzys-ostrowski) |
- [](https://paperswithcode.com/task/federated-learning)
| Intel® Extension for Transformers | Transformer-based Toolkit to Accelerate GenAI/LLM Everywhere | [intel](https://www.intel.com/content/www/us/en/developer/topic-technology/open/overview.html) | [](https://github.com/intel/intel-extension-for-transformers)
- [](https://arxiv.org/abs/2309.17453), [](https://arxiv.org/abs/2311.00502), [](https://arxiv.org/abs/2211.07715), [](https://arxiv.org/abs/2210.17114), [](https://arxiv.org/abs/2111.05754)
- [](https://discord.gg/Wxk3J3ZJkU)
- [](https://intel.github.io/intel-extension-for-transformers/latest/docs/Welcome.html)
- [](https://github.com/ggerganov/ggml), [](https://github.com/ggerganov/llama.cpp), [](https://github.com/TimDettmers/bitsandbytes), [](https://github.com/lm-sys/FastChat), [](https://github.com/IntelLabs/fastRAG), [](https://github.com/IST-DASLab/gptq), [](https://github.com/mit-han-lab/streaming-llm)
- [](https://huggingface.co/blog/assisted-generation), [](https://huggingface.co/Intel/neural-chat-7b-v3-1), [](https://huggingface.co/blog/Andyrasika/neural-chat-intel)
- [](https://medium.com/@NeuralCompressor/creating-your-own-llms-on-your-laptop-a08cc4f7c91b), [](https://medium.com/@NeuralCompressor/the-practice-of-supervised-finetuning-and-direct-preference-optimization-on-habana-gaudi2-a1197d8a3cd3), [](https://medium.com/@NeuralCompressor/llm-performance-of-intel-extension-for-transformers-f7d061556176), [](https://medium.com/@NeuralCompressor/high-performance-low-bit-layer-wise-weight-only-quantization-on-a-laptop-712580899396), [](https://medium.com/intel-analytics-software/reduce-large-language-model-carbon-footprint-with-intel-neural-compressor-and-intel-extension-for-dfadec3af76a)
- [](https://youtu.be/bWhZ1u_1rlc), [](https://www.youtube.com/watch?v=RbKRELWP9y8&t=2954s), [](https://youtu.be/7_urstS-noU), [](https://youtu.be/bWhZ1u_1rlc), [](https://youtu.be/KWT6yKfu4n0)
| Deforum Stable Diffusion | Open source project is designed to be free to use and easy to modify for custom needs and pipelines |
- [EnzymeZoo](https://linktr.ee/enzymezoo)
- [Артем Храпов](https://github.com/kabachuha)
- [Forest Star Walz](https://github.com/reallybigname)
- [pharmapsychotic](https://github.com/pharmapsychotic)
- [](https://discord.gg/deforum)
- [](https://docs.google.com/document/d/1RrQv7FntzOuLg4ohjRZPVL7iptIyBhwwbcEYEW2OfcI)
- [project](https://deforum.github.io/)
- [](https://youtu.be/w_sxuDMt_V0), [](https://youtu.be/bicPayZDI60), [](https://youtu.be/dqkQo2alZvU)
| AutoGen | Framework that enables development of LLM applications using multiple agents that can converse with each other to solve tasks | [microsoft](https://github.com/microsoft) | [](https://github.com/microsoft/autogen)
- [blog post](https://www.microsoft.com/en-us/research/blog/autogen-enabling-next-generation-large-language-model-applications/)
- [](https://discord.gg/pAbnFJrkgZ)
- [](https://medium.com/@multiplatform.ai/microsoft-autogen-transforming-ai-frameworks-for-enhanced-problem-solving-video-ac2655e7cdf)
- [project](https://microsoft.github.io/autogen/)
- [](https://youtu.be/zdcCD--IieY), [](https://youtu.be/dCCr52uT0W8), [](https://youtu.be/JMpgsx74XDI)
| Supervision | Reusable computer vision tools | [Roboflow](https://roboflow.com/about) | [](https://github.com/roboflow/supervision)
- [](https://discord.gg/GbfgXGJ8Bk)
- [](https://github.com/roboflow/inference), [](https://docs.roboflow.com/)
- [](https://github.com/roboflow/notebooks)
- [](https://huggingface.co/spaces/Roboflow/Annotators)
- [](https://www.kaggle.com/code/leoroboflow/inferring-on-a-dataset-with-a-roboflow-model)
- [website](https://supervision.roboflow.com/)
- [](https://youtu.be/uWP6UjDeZvY), [](https://youtu.be/4Q3ut7vqD5o), [](https://youtube.com/roboflow)
| Transfer learning and fine-tuning | You will learn how to classify images of cats and dogs by using transfer learning from a pre-trained network | [François Chollet](https://fchollet.com/) |
- [](https://paperswithcode.com/task/transfer-learning)
- [](https://en.wikipedia.org/wiki/Transfer_learning)
| The Autodiff Cookbook | You'll go through a whole bunch of neat autodiff ideas that you can cherry pick for your own work, starting with the basics |
- [Alex Wiltschko](https://github.com/alexbw)
- [Matthew Johnson](http://people.csail.mit.edu/mattjj/)
- [](https://arxiv.org/abs/1406.2572), [](https://arxiv.org/abs/1706.04454), [](https://arxiv.org/abs/1802.03451), [](https://arxiv.org/abs/1811.07062)
- [book](https://mitpress.mit.edu/sites/default/files/titles/content/sicm_edition_2/book.html), [book](https://mitpress.mit.edu/books/functional-differential-geometry)
- [](https://github.com/google/jax#auto-vectorization-with-vmap), [](https://github.com/hips/autograd)
- [tutorial](http://videolectures.net/deeplearning2017_johnson_automatic_differentiation/)
- [](https://en.wikipedia.org/wiki/Truncated_Newton_method), [](https://en.wikipedia.org/wiki/Pullback_(differential_geometry)), [](https://en.wikipedia.org/wiki/Holomorphic_function), [](https://en.wikipedia.org/wiki/Cauchy%E2%80%93Riemann_equations)
| YOLOv8 | State-of-the-art model that builds upon the success of previous YOLO versions and introduces new features and improvements to further boost performance and flexibility | [Glenn Jocher](https://github.com/glenn-jocher) | [](https://github.com/ultralytics/ultralytics)
- [COCO](http://cocodataset.org/)
- [ImageNet](https://www.image-net.org/)
- [blog post](https://habr.com/ru/articles/710016/)
- [](https://ultralytics.com/discord)
- [](https://hub.docker.com/r/ultralytics/ultralytics)
- [](https://docs.ultralytics.com/)
- [](https://www.kaggle.com/ultralytics/yolov8)
- [](https://twitter.com/ultralytics)
- [](https://youtube.com/ultralytics), [](https://youtu.be/m9fH9OWn8YM), [](https://youtu.be/wuZtUMEiKWY), [](https://youtu.be/gRAyOPjQ9_s), [](https://youtu.be/fhzCwJkDONE), [](https://youtu.be/IHbJcOex6dk)
| Deep RL Course | The Hugging Face Deep Reinforcement Learning Course |
- [Thomas Simonini](https://www.simoninithomas.com/)
- [Omar Sanseviero](https://osanseviero.github.io/hackerllama/)
- [Sayak Paul](https://sayak.dev/)
- [](https://github.com/alex-petrenko/sample-factory)
- [](https://huggingface.co/deep-rl-course/unit0/introduction), [](https://huggingface.co/spaces/huggingface-projects/Deep-Reinforcement-Learning-Leaderboard)
- [](https://pytorch.org/tutorials/beginner/deep_learning_60min_blitz.html)
- [syllabus](https://simoninithomas.github.io/deep-rl-course)
- [](https://youtu.be/2GwBez0D20A), [](https://youtu.be/CsuIANBnSq8), [](https://youtu.be/AQKAOXJa6qg)
| Anomalib | Deep learning library that aims to collect state-of-the-art anomaly detection algorithms for benchmarking on both public and private datasets |
- [Samet Akcay](https://github.com/samet-akcay)
- [Dick Ameln](https://github.com/djdameln)
- [Ashwin Vaidya](https://ashwinvaidya.com/)
- [Barath Lakshmanan](https://github.com/blakshma) others
- [Nilesh Ahuja](https://github.com/nahuja-intel)
- [Utku Genc](https://github.com/ugenc-intel)
- [](https://arxiv.org/abs/2011.08785)
- [data](https://www.mvtec.com/company/research/datasets/mvtec-ad)
- [](https://openvinotoolkit.github.io/anomalib/)
- [](https://github.com/rwightman/pytorch-image-models), [](https://github.com/vnk8071/anomaly-detection-in-industry-manufacturing/tree/master/anomalib_contribute)
- [](https://towardsdatascience.com/getting-started-with-pytorch-image-models-timm-a-practitioners-guide-4e77b4bf9055)
- [](https://paperswithcode.com/lib/timm)
| Nerfstudio | API that allows for a simplified end-to-end process of creating, training, and testing NeRFs |
- [Matthew Tancik](https://github.com/tancik)
- [Ethan Weber](https://ethanweber.me/)
- [Evonne Ng](http://people.eecs.berkeley.edu/~evonne_ng/)
- [Ruilong Li](http://www.liruilong.cn/) others
- [Brent Yi](https://github.com/brentyi)
- [Justin Kerr](https://kerrj.github.io/)
- [Terrance Wang](https://github.com/terrancewang)
- [Alexander Kristoffersen](https://akristoffersen.com/)
- [Jake Austin](https://github.com/jake-austin)
- [Kamyar Salahi](https://github.com/TheQuantumFractal)
- [Abhik Ahuja](https://abhikahuja.com/)
- [David McAllister](https://github.com/mcallisterdavid)
- [Angjoo Kanazawa](https://github.com/akanazawa)
- [Viewer](https://viewer.nerf.studio/)
- [](https://arxiv.org/abs/2302.04264)
- [](https://discord.gg/uMbNqcraFc)
- [](https://docs.nerf.studio/en/latest/)
- [](https://github.com/NVlabs/tiny-cuda-nn)
- [](https://twitter.com/nerfstudioteam)
- [](https://youtu.be/XwKq7qDQCQk), [](https://youtu.be/nSFsugarWzk), [](https://youtu.be/h5EWiRRxYEQ), [](https://youtu.be/8cv9G7izdPY)
| Feast | An open source feature store for machine learning |
- [Willem Pienaar](https://github.com/woop)
- [Danny Chiao](https://github.com/adchia)
- [Achal Shah](http://achals.com/)
- [Terence Lim](https://terryyylim.github.io/portfolio/) others
- [Ches Martin](https://github.com/ches)
- [Judah Rand](https://github.com/judahrand)
- [Matt Delacour](https://github.com/MattDelac)
- [Miguel Trejo Marrufo](https://github.com/TremaMiguel)
- [Francisco Javier Arceo](https://franciscojavierarceo.github.io/)
- [](https://docs.feast.dev/)
- [](https://github.com/baineng/feast-hive), [](https://github.com/Shopify/feast-trino), [](https://github.com/Azure/feast-azure), [](https://github.com/amundsen-io/amundsen/blob/main/databuilder/databuilder/extractor/feast_extractor.py)
- [website](https://feast.dev/)
- [](https://youtu.be/DaNv-Wf1MBA), [](https://youtu.be/p2cuq4eJ2BY)
| FiftyOne | Open-source tool for building high-quality datasets and computer vision models |
- [Brian Moore](https://github.com/brimoor)
- [Jason Corso](https://web.eecs.umich.edu/~jjcorso/)
- [blog post](https://voxel51.com/blog/)
- [](https://docs.voxel51.com/)
- [](https://github.com/voxel51/fiftyone-examples)
- [](https://medium.com/voxel51), [](https://towardsdatascience.com/open-source-tools-for-fast-computer-vision-model-building-b39755aab490)
- [](https://slack.voxel51.com/)
- [](https://twitter.com/voxel51)
- [website](https://voxel51.com/fiftyone/)
- [](https://www.youtube.com/playlist?list=PLuREAXoPgT0SJLKsgFzKxffMApbXp90Gi)
| MetaVoice | 1.2B parameter base model trained on 100K hours of speech for TTS | [MetaVoice](https://themetavoice.xyz/) | [](https://github.com/metavoiceio/metavoice-src)
- [demo](https://ttsdemo.themetavoice.xyz/)
- [](https://huggingface.co/metavoiceio)
- [](https://twitter.com/MetaVoiceAI)
- [](https://youtu.be/Y_k3bHPcPTo), [](https://youtu.be/gVKbf31hrYs)
| VC | Client software for performing real-time voice conversion using various Voice Conversion AI | [w-okada](https://github.com/w-okada) | [](https://github.com/w-okada/voice-changer)
- [](https://github.com/yxlllc/DDSP-SVC)
- [](https://huggingface.co/wok000/vcclient000)
- [](https://youtu.be/POo_Cg0eFMU), [](https://youtu.be/fba9Zhsukqw), [](https://youtu.be/s_GirFEGvaA), [](https://youtu.be/Q7bbEC4aeKM), [](https://youtu.be/_JXbvSTGPoo), [](https://youtu.be/pHhjg2JwdPI), [](https://youtu.be/We5oYpCR3WQ), [](https://youtu.be/aVfoC1EHlVs), [](https://youtu.be/YF1lBaqeyt8)
| Generative AI for Beginners - A Course | A 12 Lesson course teaching everything you need to know to start building Generative AI applications | [microsoft](https://www.microsoft.com/) | [](https://github.com/microsoft/xr-development-for-beginners)
- [](https://aka.ms/genai-discord)
- [](https://github.com/microsoft/Web-Dev-For-Beginners)
- [project](https://microsoft.github.io/generative-ai-for-beginners/)
| OmegaConf | Hierarchical configuration system, with support for merging configurations from multiple sources providing a consistent API regardless of how the configuration was created | [Omry Yadan](https://github.com/omry) | [](https://github.com/omry/omegaconf)
- [](https://omegaconf.readthedocs.io/)
- [](https://majianglin2003.medium.com/python-omegaconf-a33be1b748ab)
- [slides](https://docs.google.com/presentation/d/e/2PACX-1vT_UIV7hCnquIbLUm4NnkUpXvPEh33IKiUEvPRF850WKA8opOlZOszjKdZ3tPmf8u7hGNP6HpqS-NT5/pub)
| Optuna | An automatic hyperparameter optimization software framework, particularly designed for machine learning |
- [Takuya Akiba](https://iwiwi.github.io/)
- [Shotaro Sano](https://github.com/g-votte)
- [Toshihiko Yanase](https://github.com/toshihikoyanase)
- [Takeru Ohta](https://github.com/sile)
- [Masanori Koyama](https://scholar.google.com/citations?user=oY1gA10AAAAJ)
- [](https://arxiv.org/abs/1907.10902)
- [](https://hub.docker.com/r/optuna/optuna)
- [](https://optuna.readthedocs.io/en/stable/)
- [](https://github.com/optuna/optuna-dashboard)
- [website](https://optuna.org/)
- [](https://youtu.be/J_aymk4YXhg), [](https://youtu.be/tcrcLRopTX0), [](https://youtu.be/-UeC4MR3PHM), [](https://youtu.be/oC8zFYcfYXU)
| Data augmentation | This tutorial demonstrates data augmentation: a technique to increase the diversity of your training set by applying random transformations such as image rotation | [Billy Lamberta](https://github.com/lamberta) |
- [](https://paperswithcode.com/task/data-augmentation)
- [](https://www.tensorflow.org/datasets/catalog/tf_flowers)
- [](https://en.wikipedia.org/wiki/Data_augmentation)
| Stable Cascade | Text to image model introduces an interesting three-stage approach, setting new benchmarks for quality, flexibility, fine-tuning, and efficiency with a focus on further eliminating hardware barriers | [Stability AI](https://stability.ai/research) | [](https://github.com/Stability-AI/StableCascade)
- [](https://arxiv.org/abs/2306.00637)
- [blog post](https://stability.ai/news/introducing-stable-cascade)
- [](https://discord.gg/stablediffusion)
- [](https://huggingface.co/stabilityai/stable-cascade), [](https://huggingface.co/datasets/nateraw/parti-prompts)
- [](https://medium.com/intelligent-art/stable-cascade-a-super-easy-local-installation-guide-ce0cbd06d800), [](https://medium.com/@yushantripleseven/stable-cascade-training-inference-a52e12ecc5fa)
- [](https://twitter.com/stabilityai)
- [](https://youtu.be/Ybu6qTbEsewc), [](https://youtu.be/JuX-uukwdkI), [](https://youtu.be/YMxXtaiVHks), [](https://youtu.be/UgM-z2q3Xe0), [](https://youtu.be/W6YLIyA3Kco), [](https://youtu.be/X1rLWFRagIw)
| CleanVision | Automatically detects potential issues in image datasets like images that are: blurry, under/over-exposed, (near) duplicates, etc | [cleanlab](https://cleanlab.ai/about/) | [](https://github.com/cleanlab/cleanvision)
- [blog post](https://cleanlab.ai/blog/cleanvision/)
- [](https://cleanvision.readthedocs.io/)
- [](https://github.com/cleanlab/cleanvision-examples)
- [](https://cleanlab.ai/slack)
- [](https://twitter.com/CleanlabAI)
| DynamiCrafter | Animating Open-domain Images with Video Diffusion Priors |
- [Jinbo Xing](https://doubiiu.github.io/)
- [Menghan Xia](https://menghanxia.github.io/)
- [Yong Zhang](https://yzhang2016.github.io/)
- [Haoxin Chen](https://scutpaul.github.io/) others
- [Wangbo Yu](https://github.com/GooDrYu)
- [Hanyuan Liu](https://github.com/hyliu)
- [Xintao Wang](https://xinntao.github.io/)
- [Tien-Tsin Wong](https://ttwong12.github.io/myself.html)
- [Ying Shan](https://scholar.google.com/citations?user=4oXBp9UAAAAJ)
- [](https://arxiv.org/abs/2310.12190)
- [](https://github.com/chaojie/ComfyUI-DynamiCrafter), [](https://github.com/AILab-CVC/VideoCrafter), [](https://github.com/YingqingHe/ScaleCrafter), [](https://github.com/AILab-CVC/TaleCrafter), [](https://github.com/AILab-CVC/FreeNoise)
- [](https://huggingface.co/Doubiiu/DynamiCrafter_1024)
- [project](https://doubiiu.github.io/projects/DynamiCrafter/)
- [](https://www.reddit.com/r/StableDiffusion/comments/1aj7gcw/dynamicrafter_gets_updated/)
- [](https://x.com/noguchis/status/1754488826016432341?s=20)
- [](https://youtu.be/0NfmIsNAg-g), [](https://youtu.be/PtW7hjCawbo)
| DiffSynth | Restructured architectures including Text Encoder, UNet, VAE, among others, maintaining compatibility with models from the open-source community while enhancing computational performance | [Artiprocher](https://github.com/Artiprocher) | [](https://github.com/Artiprocher/DiffSynth-Studio)
- [](https://arxiv.org/abs/2401.16224)
- [](https://huggingface.co/Helsinki-NLP/opus-mt-en-zh), [](https://huggingface.co/alibaba-pai/pai-bloom-1b1-text2prompt-sd)
| Composer | PyTorch library that enables you to train neural networks faster, at lower cost, and to higher accuracy | [The Mosaic ML Team](https://www.mosaicml.com/team) | [](https://github.com/mosaicml/composer)
- [app](https://app.mosaicml.com/)
- [](https://arxiv.org/abs/2202.05924), [](https://arxiv.org/abs/2002.04688)
- [blog post](https://www.mosaicml.com/blog/5-best-practices-for-efficient-model-training)
- [](http://docs.mosaicml.com/)
- [](https://join.slack.com/t/mosaicml-community/shared_invite/zt-w0tiddn9-WGTlRpfjcO9J5jyrMub1dg)
- [](https://twitter.com/mosaicml)
- [website](https://www.mosaicml.com/composer)
- [](https://en.wikipedia.org/wiki/Amdahl's_law)
- [](https://www.youtube.com/@mosaicml6047/videos), [](https://youtu.be/n-1WV5QdIDc), [](https://youtu.be/Xi_5wq2MpOw)
| Brax | A differentiable physics engine that simulates environments made up of rigid bodies, joints, and actuators |
- [Daniel Freeman](https://github.com/cdfreeman-google)
- [Erik Frey](https://fawx.com/)
- [Anton Raichuk](https://scholar.google.com/citations?user=fquIpvgAAAAJ)
- [Sertan Girgin](https://sites.google.com/site/girgint/home) others
- [Igor Mordatch](https://scholar.google.com/citations?user=Vzr1RukAAAAJ)
- [Olivier Bachem](http://olivierbachem.ch/)
- [](https://arxiv.org/abs/2106.13281)
- [](https://neurips.cc/Conferences/2021/CallForDatasetsBenchmarks)
| Autodistill | Uses big, slower foundation models to train small, faster supervised models | [autodistill](https://github.com/autodistill) | [](https://github.com/autodistill/autodistill)
- [blog post](https://blog.roboflow.com/autodistill/)
- [](https://docs.autodistill.com/)
- [](https://github.com/autodistill/autodistill-grounded-sam), [](https://github.com/autodistill/autodistill-yolov8), [](https://github.com/autodistill/autodistill-yolonas), [](https://github.com/autodistill/autodistill-yolov5), [](https://github.com/autodistill/autodistill-detr), [](https://github.com/autodistill/autodistill-detic), [](https://github.com/autodistill/autodistill-grounding-dino), [](https://github.com/autodistill/autodistill-owl-vit), [](https://github.com/autodistill/autodistill-sam-clip), [](https://github.com/autodistill/autodistill-llava), [](https://github.com/autodistill/autodistill-kosmos-2), [](https://github.com/autodistill/autodistill-owlv2), [](https://github.com/autodistill/autodistill-roboflow-universe), [](https://github.com/autodistill/autodistill-azure-vision), [](https://github.com/autodistill/autodistill-rekognition), [](https://github.com/autodistill/autodistill-gcp-vision), [](https://github.com/roboflow/inference)
- [](https://youtu.be/gKTYMfwPo4M), [](https://youtu.be/M_QZ_Q0zT0k), [](https://youtube.com/roboflow)
| CycleGAN | This notebook demonstrates unpaired image to image translation using conditional GAN's | [Billy Lamberta](https://github.com/lamberta) |
- [](https://arxiv.org/abs/1703.10593)
- [](https://www.tensorflow.org/datasets/catalog/cycle_gan)
| Pix2Pix | This notebook demonstrates image to image translation using conditional GAN's | [Billy Lamberta](https://github.com/lamberta) |
- [](https://arxiv.org/abs/1611.07004)
- [data](https://people.eecs.berkeley.edu/~tinghuiz/projects/pix2pix/datasets/)
| Integrated gradients | This tutorial demonstrates how to implement Integrated Gradients, an Explainable AI technique | [Google](https://www.tensorflow.org/) | [](https://github.com/GoogleCloudPlatform/training-data-analyst/tree/master/blogs/integrated_gradients)
- [](https://arxiv.org/abs/1703.01365)
- [visualizing](https://distill.pub/2020/attribution-baselines/)
- [](https://en.wikipedia.org/wiki/Explainable_artificial_intelligence), [](https://en.wikipedia.org/wiki/Linear_interpolation), [](https://en.wikipedia.org/wiki/Riemann_sum)
| MAGNeT | Masked generative sequence modeling method that operates directly over several streams of audio tokens |
- [Alon Ziv](https://www.cs.huji.ac.il/w~alonzi/)
- [Itai Gat](https://itaigat.com/)
- [Gaël Le Lan](https://github.com/gl3lan)
- [Tal Remez](https://talremez.github.io/) others
- [Felix Kreuk](https://felixkreuk.github.io/)
- [Alexandre Défossez](https://ai.honu.io/)
- [Jade Copet](https://scholar.google.com/citations?&user=GRMLwjAAAAAJ)
- [Gabriel Synnaeve](https://syhw.github.io/)
- [Yossi Adi](https://www.cs.huji.ac.il/~adiyoss/)
- [](https://arxiv.org/abs/2401.04577), [](https://arxiv.org/abs/2305.09636), [](https://arxiv.org/abs/2307.04686)
- [](https://github.com/FurkanGozukara/Stable-Diffusion/blob/main/Tutorials/AI-Music-Generation-Audiocraft-Tutorial.md#more-info-about-top-k-top-p-temperature-and-classifier-free-guidance-from-chatgpt)
- [](https://huggingface.co/facebook/magnet-medium-10secs), [](https://huggingface.co/facebook/magnet-medium-30secs), [](https://huggingface.co/facebook/audio-magnet-medium)
- [](https://generativeai.pub/metas-ai-magnet-the-next-big-thing-in-text-to-audio-technology-7d524d9459ef)
- [project](https://pages.cs.huji.ac.il/adiyoss-lab/MAGNeT/)
- [](https://www.reddit.com/r/ArtificialInteligence/comments/19808gf/magnet_masked_audio_generation_using_a_single/)
| Retrieval based Voice Conversion WebUI | An easy-to-use Voice Conversion framework based on VITS | [RVC-Project](https://github.com/RVC-Project) | [](https://github.com/RVC-Project/Retrieval-based-Voice-Conversion-WebUI)
- [](https://discord.gg/HcsmBBGyVk)
- [](https://github.com/auspicious3000/contentvec), [](https://github.com/jik876/hifi-gan), [](https://github.com/FFmpeg/FFmpeg), [](https://github.com/Anjok07/ultimatevocalremovergui), [](https://github.com/openvpi/audio-slicer), [](https://github.com/Dream-High/RMVPE)
- [](https://huggingface.co/lj1995/VoiceConversionWebUI)
- [](https://medium.com/@ja.harr91/decoding-the-sound-of-virality-a-deep-dive-into-adversarial-ai-for-voice-conversion-tasks-on-m1-d60d32cfb2d4)
- [](https://youtu.be/-JcvdDErkAU), [](https://youtu.be/9TroP5mR3CM), [](https://youtu.be/Y8IxVVQBEpc), [](https://youtu.be/qZ12-Vm2ryc), [](https://youtu.be/5i_Pyw0gH-M)
| Flax | Neural network library and ecosystem for JAX designed for flexibility |
- [Jonathan Heek](https://github.com/jheek)
- [Anselm Levskaya](https://anselmlevskaya.com/)
- [Avital Oliver](https://github.com/avital)
- [Marvin Ritter](https://github.com/Marvin182) others
- [Bertrand Rondepierre](https://github.com/BertrandRdp)
- [Andreas Steiner](https://github.com/andsteing)
- [Marc van Zee](https://research.google/people/marc-van-zee/)
- [](https://flax.readthedocs.io/)
- [](https://github.com/huggingface/transformers/tree/main/examples/flax)
- [](https://medium.com/syncedreview/google-introduces-flax-a-neural-network-library-for-jax-84bdc6f8f160)
- [](https://www.reddit.com/r/MachineLearning/comments/erpdf7/p_flax_a_neural_network_library_for_jax_designed/)
- [](https://youtu.be/e8StU6WQCqw), [](https://youtu.be/HOlQzrn84A4), [](https://youtu.be/5eUSmJvK8WA)
| Big Vision | This codebase is designed for training large-scale vision models using Cloud TPU VMs or GPU machines |
- [Lucas Beyer](http://lucasb.eyer.be/)
- [Xiaohua Zhai](https://github.com/xiaohuazhai)
- [Alexander Kolesnikov](https://github.com/akolesnikoff)
- [](https://arxiv.org/abs/2010.11929), [](https://arxiv.org/abs/2106.04560), [](https://arxiv.org/abs/2105.01601), [](https://arxiv.org/abs/2205.01580), [](https://arxiv.org/abs/2212.08013), [](https://arxiv.org/abs/2305.13035), [](https://arxiv.org/abs/2303.17376), [](https://arxiv.org/abs/2306.07915), [](https://arxiv.org/abs/2305.16999), [](https://arxiv.org/abs/2302.08242), [](https://arxiv.org/abs/2006.07159)
- [](https://www.tensorflow.org/guide/data), [](https://www.tensorflow.org/datasets)
| Open Interpreter | An open-source, locally running implementation of OpenAI's Code Interpreter | [Killian Lucas](https://github.com/KillianLucas) | [](https://github.com/KillianLucas/open-interpreter)
- [](https://discord.gg/6p3fD6rBVm)
- [](https://docs.openinterpreter.com/)
- [website](https://openinterpreter.com/)
- [](https://youtu.be/SqnXUHwIa3c), [](https://youtu.be/s-f4lCETxu0), [](https://youtu.be/J-H2un1Adr0), [](https://youtu.be/jaijpff58vw), [](https://youtu.be/7KFbG_3dKKs), [](https://youtu.be/4OhuFjPyZNQ), [](https://youtu.be/01tQLn_RRcE), [](https://youtu.be/uyfoHQVgeY0)
| highway-env | A collection of environments for autonomous driving and tactical decision-making tasks | [Edouard Leurent](https://edouardleurent.com/) | [](https://github.com/eleurent/highway-env)
- [](https://arxiv.org/abs/2102.03483), [](https://arxiv.org/abs/2105.05701), [](https://arxiv.org/abs/2101.07140)
- [](https://highway-env.readthedocs.io/en/latest/)
- [](https://github.com/eleurent/rl-agents), [](https://github.com/eleurent/finite-mdp), [](https://github.com/openai/baselines/tree/master/baselines/her)
| Seamless Communication | Family of AI models that enable more natural and authentic communication across languages |
- [Loïc Barrault](https://loicbarrault.github.io/)
- [Yu-An Chung](https://iamyuanchung.github.io/)
- [Mariano Coria](https://www.linkedin.com/in/marianocoria)
- [David Dale](https://daviddale.ru/) others
- [Ning Dong](https://scholar.google.com/citations?user=gg1hvjoAAAAJ)
- [Mark Duppenthaler](https://github.com/mduppes)
- [Paul-Ambroise Duquenne](https://scholar.google.com/citations?user=Uah8IcAAAAAJ)
- [Hady Elsahar](https://www.hadyelsahar.io/)
- [Min-Jae Hwang](https://mjhwang93.github.io/)
- [Hirofumi Inaguma](https://hirofumi0810.github.io/)
- [Ilia Kulikov](https://github.com/uralik)
- [Pengwei Li](https://scholar.google.com/citations?user=hQB3YsYAAAAJ)
- [Daniel Licht](https://github.com/Lichtphyz)
- [Jean Maillard](https://scholar.google.com/citations?user=_ewOoK0AAAAJ)
- [Ruslan Mavlyutov](https://github.com/mavlyutovr)
- [Kaushik Ram Sadagopan](https://github.com/kauterry)
- [Abinesh Ramakrishnan](https://github.com/ibanesh)
- [Tuan Tran](https://antoine-tran.github.io/)
- [Guillaume Wenzek](https://github.com/gwenzek)
- [Yilin Yang](https://yilinyang7.github.io/)
- [Ethan Ye](https://github.com/yeyinthtoon)
- [Ivan Evtimov](https://ivanevtimov.eu/)
- [Pierre Fernandez](https://pierrefdz.github.io/)
- [Robin San Roman](https://scholar.google.com/citations?user=AJ3ir84AAAAJ)
- [Bokai Yu](https://scholar.google.com/citations?user=7jNmPwUAAAAJ)
- [Pierre Andrews](https://github.com/Mortimerp9)
- [Can Balioglu](http://canbalioglu.com/)
- [Peng-Jen Chen](https://scholar.google.com/citations?user=rOXs9VMAAAAJ)
- [Marta Costa-jussà](https://costa-jussa.com/)
- [Maha Elbayad](http://elbayadm.github.io/)
- [Hongyu Gong](https://github.com/hygong-fb)
- [Francisco Guzmán](https://guzmanhe.github.io/)
- [Kevin Heffernan](https://github.com/heffernankevin)
- [Somya Jain](https://scholar.google.com/citations?user=AmBxU3kAAAAJ)
- [Justine Kao](https://scholar.google.com/citations?user=Y9BLeTAAAAAJ)
- [Ann Lee](https://www.stat.cmu.edu/~annlee/)
- [Xutai Ma](https://github.com/xutaima)
- [Benjamin Peloquin](https://scholar.google.com/citations?user=5GNAjB8AAAAJ)
- [Juan Pino](https://scholar.google.com/citations?user=weU_-4IAAAAJ)
- [Sravya Popuri](https://scholar.google.com/citations?user=MtmqG3UAAAAJ)
- [Holger Schwenk](https://github.com/hoschwenk)
- [Anna Sun](https://github.com/annasun28)
- [Paden Tomasello](https://scholar.google.com/citations?user=sBtWMGYAAAAJ)
- [Changhan Wang](https://www.changhan.me/)
- [Skyler Wang](https://www.skylerwang.com/)
- [Mary Williamson](https://scholar.google.com/citations?user=Ys4xB-QAAAAJ)
- [](https://arxiv.org/abs/2312.05187)
- [blog post](https://ai.meta.com/research/seamless-communication/)
- [](https://github.com/libsndfile/libsndfile), [](https://github.com/facebookresearch/fairseq2), [](https://github.com/facebookresearch/SimulEval), [](https://github.com/facebookresearch/stopes), [](https://github.com/facebookresearch/SONAR)
- [](https://huggingface.co/facebook/seamless-m4t-v2-large), [](https://huggingface.co/facebook/seamless-expressive), [](https://huggingface.co/facebook/seamless-streaming)
- [](https://ngwaifoong92.medium.com/beginners-guide-to-seamlessm4t-81efad6e8ca6)
- [](https://www.youtube.com/watch?v=0padjtkHXTE), [](https://youtu.be/rNN7qsoCKBo), [](https://youtu.be/RKEFZ44YOcc)
| colab2pdf | Convert your Colab notebook to a PDF | [Drengskapur](https://github.com/drengskapur) | [](https://github.com/drengskapur/colab2pdf) | [](https://colab.research.google.com/drive/1zqrIYC0iQ_CZkRqGXgZggrwjtt_4BmpL) | 11.12.2023 |
| Sentence Transformers | Multilingual Sentence, Paragraph, and Image Embeddings using BERT & Co |
- [Nils Reimers](https://www.nils-reimers.de/)
- [Iryna Gurevych](https://www.informatik.tu-darmstadt.de/ukp/ukp_home/head_ukp/index.en.jsp)
- [](https://arxiv.org/abs/1908.10084), [](https://arxiv.org/abs/2004.09813), [](https://arxiv.org/abs/2010.08240)
- [](https://www.sbert.net/)
| LM Evaluation Harness | Framework for few-shot evaluation of language models. | [EleutherAI](https://www.eleuther.ai/) | [](https://github.com/EleutherAI/lm-evaluation-harness)
- [](https://arxiv.org/abs/2005.14165)
- [](https://discord.gg/eleutherai)
- [](https://github.com/AutoGPTQ/AutoGPTQ), [](https://github.com/EleutherAI/gpt-neox), [](https://github.com/microsoft/Megatron-DeepSpeed), [](https://github.com/vllm-project/vllm)
- [project](https://www.eleuther.ai/projects/large-language-model-evaluation)
| CleanRL | Deep Reinforcement Learning library that provides high-quality single-file implementation with research-friendly features |
- [Shengyi Huang](https://costa.sh/)
- [Rousslan Dossa](https://dosssman.github.io/)
- [Chang Ye](https://github.com/yooceii)
- [Jeff Braga](https://github.com/bragajj) others
- [Dipam Chakraborty](https://github.com/dipamc)
- [Kinal Mehta](https://kinalmehta.github.io/)
- [João Araújo](https://github.com/joaogui1)
- [](https://arxiv.org/abs/1707.06347), [](https://arxiv.org/abs/1707.06887), [](https://arxiv.org/abs/1812.05905), [](https://arxiv.org/abs/1509.02971), [](https://arxiv.org/abs/1802.09477), [](https://arxiv.org/abs/2009.04416), [](https://arxiv.org/abs/1810.12894)
- [](https://docs.cleanrl.dev/)
- [](https://github.com/tinkoff-ai/CORL), [](https://github.com/Farama-Foundation/Gymnasium), [](https://github.com/openai/baselines), [](https://github.com/ikostrikov/jaxrl)
- [](https://huggingface.co/cleanrl)
- [paper](https://www.jmlr.org/papers/v23/21-1342.html)
- [](https://www.youtube.com/channel/UCDdC6BIFRI0jvcwuhi3aI6w), [](https://youtu.be/dm4HdGujpPs)
| YOLOv5 | You Only Look Once | [Glenn Jocher](https://github.com/glenn-jocher) | [](https://github.com/ultralytics/yolov5)
- [data](http://cocodataset.org/#upload)
- [](https://www.kaggle.com/ultralytics/yolov5), [](https://www.kaggle.com/ultralytics/coco128)
| Distil-Whisper | Maintains the robustness of the Whisper model to difficult acoustic conditions, while being less prone to hallucination errors on long-form audio |
- [Sanchit Gandhi](https://github.com/sanchit-gandhi)
- [Patrick von Platen](https://github.com/patrickvonplaten)
- [Alexander Rush](https://scholar.google.com/citations?&user=LIjnUGgAAAAJ)
- [](https://arxiv.org/abs/2311.00430), [](https://arxiv.org/abs/2211.17192)
- [](https://github.com/huggingface/safetensors), [](https://github.com/Dao-AILab/flash-attention)
- [](https://huggingface.co/collections/distil-whisper/training-datasets-6538d05c69721489d1db1e49), [](https://huggingface.co/docs/transformers/model_doc/auto#transformers.AutoModelForSpeechSeq2Seq), [](https://huggingface.co/docs/transformers/model_doc/auto#transformers.AutoProcessor), [](https://huggingface.co/docs/transformers/main_classes/pipelines#transformers.AutomaticSpeechRecognitionPipeline), [](https://huggingface.co/docs/transformers/v4.34.1/en/model_doc/whisper#transformers.WhisperForConditionalGeneration.forward.example), [](https://huggingface.co/docs/transformers/main/en/main_classes/text_generation#transformers.GenerationMixin.generate.assistant_model), [](https://huggingface.co/docs/transformers/main/en/perf_infer_gpu_one#flashattention-2), [](https://huggingface.co/docs/transformers/main/en/perf_infer_gpu_one#bettertransformer)
- [](https://medium.com/prompt-engineering/transcribing-audio-with-python-and-distil-whisper-9b4fec3d53bf)
- [](https://www.reddit.com/r/MachineLearning/comments/17vqtcb/p_distilwhisper_a_distilled_variant_of_whisper/)
- [](https://youtu.be/46Q6fbdUCbg), [](https://youtu.be/SZtHEKyvuug), [](https://www.youtube.com/live/kI1pA1CADxM)
| Gorilla | Finetuned LLaMA-based model that surpasses the performance of GPT-4 on writing API calls |
- [Shishir Patil](https://shishirpatil.github.io/)
- [Tianjun Zhang](https://github.com/tianjunz)
- [Xin Wang](https://xinw.ai/)
- [Joseph Gonzalez](https://people.eecs.berkeley.edu/~jegonzal/)
- [](https://arxiv.org/abs/2305.15334)
- [](https://discord.gg/SwTyuTAxX3)
- [](https://github.com/gorilla-llm/gorilla-cli)
- [](https://medium.com/latinxinai/try-gorilla-a-large-language-model-connected-with-massive-apis-442f3b554ffb)
- [project](http://gorilla.cs.berkeley.edu/)
- [](https://youtu.be/4EdyWkcddPc), [](https://youtu.be/RMgM3tPTpXI), [](https://youtu.be/CX1Kzijq2TI), [](https://youtu.be/8AqQBPI4CFI), [](https://youtu.be/iQwYoii4YiI), [](https://youtu.be/alDArqcxSvw), [](https://youtu.be/EypdTAlmoo4), [](https://youtu.be/LkV5DTRNxAg)
| AnimateDiff | Practical framework to animate most of the existing personalized text-to-image models once and for all, saving efforts in model-specific tuning |
- [Yuwei Guo](https://github.com/guoyww)
- [Ceyuan Yang](https://github.com/limbo0000)
- [Anyi Rao](https://anyirao.com/)
- [Yaohui Wang](https://wyhsirius.github.io/) others
- [Yu Qiao](https://mmlab.siat.ac.cn/yuqiao/)
- [Dahua Lin](http://dahua.site/)
- [Bo Dai](https://daibo.info/)
- [](https://arxiv.org/abs/2307.04725)
- [](https://github.com/continue-revolution/sd-webui-animatediff), [](https://github.com/talesofai/AnimateDiff), [](https://youtu.be/-wki7IrQ_sU)
- [project](https://animatediff.github.io/)
- [](https://youtu.be/rdnOhM8L8nE), [](https://youtu.be/LcHAZaJjA5k), [](https://www.youtube.com/live/66JgpI3a650?feature=share)
| Intel® Neural Compressor | Aims to provide popular model compression techniques such as quantization, pruning (sparsity), distillation, and neural architecture search on mainstream frameworks such as TensorFlow, PyTorch, ONNX Runtime, and MXNet, as well as Intel extensions such as Intel Extension for TensorFlow and Intel Extension for PyTorch | [intel](https://www.intel.com/content/www/us/en/developer/topic-technology/open/overview.html) | [](https://github.com/intel/neural-compressor)
- [](https://arxiv.org/abs/2309.14592), [](https://arxiv.org/abs/2309.05516), [](https://arxiv.org/abs/2211.07715)
- [](https://discord.com/invite/Wxk3J3ZJkU)
- [](https://github.com/intel/neural-compressor)
- [](https://github.com/intel/intel-extension-for-tensorflow), [](https://github.com/intel/intel-extension-for-pytorch), [](https://github.com/Lightning-AI/pytorch-lightning/blob/master/docs/source-pytorch/advanced/post_training_quantization.rst)
- [](https://medium.com/pytorch/pytorch-inference-acceleration-with-intel-neural-compressor-842ef4210d7d), [](https://medium.com/intel-analytics-software/efficient-text-classification-with-intel-neural-compressor-4853296deeac)
- [](https://neurips.cc/virtual/2022/59433)
- [](https://pytorch.org/tutorials/recipes/intel_neural_compressor_for_pytorch.html)
- [](https://youtu.be/SswQbIHUrvQ), [](https://youtu.be/5xHKe4wWLes), [](https://youtu.be/H7Gg-EmGpAI), [](https://youtu.be/ie3w_j0Ntsk), [](https://youtu.be/m2LokuUdeVg), [](https://youtu.be/38wrDHEQZuM)
| Bark | Transformer-based text-to-audio model | [suno](https://www.suno.ai/) | [](https://github.com/suno-ai/bark)
- [](https://arxiv.org/abs/2209.03143), [](https://arxiv.org/abs/2301.02111)
- [](https://discord.gg/J2B2vsjKuE)
- [examples](https://suno-ai.notion.site/Bark-Examples-5edae8b02a604b54a42244ba45ebc2e2)
- [](https://github.com/facebookresearch/encodec), [](https://github.com/karpathy/nanoGPT)
- [](https://huggingface.co/docs/huggingface_hub/package_reference/environment_variables#hfhome)
- [](https://twitter.com/OnusFM)
- [](https://youtu.be/84LzaXAo6vE), [](https://youtu.be/rU5Do9yHbwM), [](https://youtu.be/w41-MUfxIWo), [](https://youtu.be/_m-MxEpHUQY)
| ComfyUI | Powerful and modular stable diffusion GUI and backend | [comfyanonymous](https://github.com/comfyanonymous) | [](https://github.com/comfyanonymous/ComfyUI)
- [examples](https://comfyanonymous.github.io/ComfyUI_examples/)
- [](https://github.com/madebyollin/taesd)
- [pytorch](https://developer.apple.com/metal/pytorch/)
- [](https://www.reddit.com/r/StableDiffusion/comments/10lzgze/i_figured_out_a_way_to_apply_different_prompts_to/)
- [](https://youtu.be/vUTV85D51yk), [](https://youtu.be/gySLXbe7WZQ), [](https://youtu.be/ovjeVGmy6ZM)
| SAGE | Methodology for generative spelling correction, which was tested on English and Russian languages and potentially can be extended to any language with minor changes |
- [Nikita Martynov](https://github.com/meduzick)
- [Mark Baushenko](https://github.com/e0xextazy)
- [Anastasia Kozlova](https://github.com/anastasia-kozlova)
- [Katerina Kolomeytseva](https://www.linkedin.com/in/katerina-kolomeytseva-394a7a21a) others
- [Aleksandr Abramov](https://github.com/Ab1992ao)
- [Alena Fenogenova](https://github.com/Alenush)
- [](https://arxiv.org/abs/2308.09435)
- [](https://github.com/ai-forever/augmentex)
- [](https://huggingface.co/ai-forever/RuM2M100-1.2B), [](https://huggingface.co/ai-forever/FRED-T5-large-spell), [](https://huggingface.co/ai-forever/RuM2M100-418M), [](https://huggingface.co/ai-forever/T5-large-spell), [](https://huggingface.co/datasets/ai-forever/spellcheck_benchmark)
- [](https://en.wikipedia.org/wiki/Levenshtein_distance)
- [](https://youtu.be/yFfkV0Qjuu0)
| Mistral Transformer | The most powerful language model for its size to date |
- [Albert Jiang](https://albertqjiang.github.io/)
- [Alexandre Sablayrolles](https://github.com/alexandresablayrolles)
- [Arthur Mensch](https://github.com/arthurmensch)
- [Chris Bamford](https://griddly.ai/) others
- [Devendra Chaplot](https://devendrachaplot.github.io/)
- [Diego Casas](https://github.com/diegolascasas)
- [Florian Bressand](https://www.linkedin.com/in/florianbressand)
- [Gianna Lengyel](https://www.linkedin.com/in/gianna-maria-lengyel)
- [Guillaume Lample](https://github.com/glample)
- [Lucile Saulnier](https://scholar.google.com/citations?user=Baj_9IsAAAAJ)
- [Lélio Renard Lavaud](https://github.com/lerela)
- [Marie-Anne Lachaux](https://scholar.google.com/citations?user=dSEMIJ8AAAAJ)
- [Pierre Stock](https://github.com/pierrestock)
- [Teven Scao](https://scholar.google.com/citations?user=ik0_vxsAAAAJ)
- [Thibaut Lavril](https://scholar.google.com/citations?user=9nPunCEAAAAJ)
- [Thomas Wang](https://github.com/thomasw21)
- [Timothée Lacroix](https://scholar.google.com/citations?&user=tZGS6dIAAAAJ)
- [William Sayed](https://www.linkedin.com/in/william-el-sayed-48672312a)
- [](https://arxiv.org/abs/2310.06825), [](https://arxiv.org/abs/1904.10509), [](https://arxiv.org/abs/2004.05150), [](https://arxiv.org/abs/2306.05685)
- [blog post](https://mistral.ai/news/announcing-mistral-7b/)
- [](https://discord.com/invite/mistralai)
- [](https://docs.mistral.ai/)
- [](https://github.com/vllm-project/vllm), [](https://github.com/lm-sys/FastChat), [](https://github.com/ggerganov/ggml), [](https://github.com/Dao-AILab/flash-attention), [](https://github.com/skypilot-org/skypilot)
- [](https://huggingface.co/mistralai)
- [](https://towardsdatascience.com/mistral-7b-recipes-for-fine-tuning-and-quantization-on-your-computer-631401583f77)
- [](https://youtu.be/g7kVVBlCGo0), [](https://youtu.be/ASpageg8nPw), [](https://youtu.be/OMIuP6lQXe4), [](https://youtu.be/jnPZApwtE4I), [](https://youtu.be/3SdopNwQJ-c)
| Fooocus | Image generating software | [Lvmin Zhang](https://lllyasviel.github.io/Style2PaintsResearch/lvmin) | [](https://github.com/lllyasviel/Fooocus)
- [](https://arxiv.org/abs/2210.00939)
- [](https://youtu.be/8krykSwOz3E), [](https://youtu.be/558W8rfnP-Q), [](https://youtu.be/TJkrzuPdmvE), [](https://youtu.be/NfNwmKM3sxc)
| Actor-Critic | This tutorial demonstrates how to implement the Actor-Critic method using TensorFlow to train an agent on the Open AI Gym CartPole-V0 environment | [Mark Daoust](https://github.com/MarkDaoust) |
- [gym](https://gym.openai.com/)
- [](https://papers.nips.cc/paper/1786-actor-critic-algorithms.pdf), [](https://papers.nips.cc/paper/1713-policy-gradient-methods-for-reinforcement-learning-with-function-approximation.pdf)
- [](https://en.wikipedia.org/wiki/Temporal_difference_learning)
| Simple audio recognition | This tutorial will show you how to build a basic speech recognition network that recognizes ten different words | [Google](https://www.tensorflow.org/) |
- [coursera](https://www.coursera.org/lecture/audio-signal-processing/stft-2-tjEQe)
- [](https://paperswithcode.com/task/speech-recognition)
- [](https://www.tensorflow.org/datasets/catalog/speech_commands)
- [tf.js](https://codelabs.developers.google.com/codelabs/tensorflowjs-audio-codelab/index.html#0)
| MMagic | AIGC toolbox for professional AI researchers and machine learning engineers to explore image and video processing, editing and generation | [OpenMMLab](https://openmmlab.com/) | [](https://github.com/open-mmlab/mmagic)
- [](https://discord.gg/raweFPmdzG)
- [](https://mmagic.readthedocs.io/en/latest/)
- [](https://github.com/open-mmlab/mmgeneration), [](https://github.com/open-mmlab/mmengine/blob/main/mmengine/model/wrappers/seperate_distributed.py), [](https://github.com/open-mmlab/mmcv), [](https://github.com/open-mmlab/mim)
- [](https://openmmlab.medium.com/)
- [](https://twitter.com/OpenMMLab)
- [](https://www.youtube.com/openmmlab)
| YOLOv3 | You Only Look Once | [Glenn Jocher](https://github.com/glenn-jocher) | [](https://github.com/ultralytics/yolov3)
- [data](http://cocodataset.org/#upload)
- [](https://www.kaggle.com/ultralytics/yolov3), [](https://www.kaggle.com/ultralytics/coco128)
| MMAction2 | An open-source toolbox for video understanding based on PyTorch | [MMAction2 Contributors](https://openmmlab.com/aboutus) | [](https://github.com/open-mmlab/mmaction2)
- [](https://arxiv.org/abs/2106.13230), [](https://arxiv.org/abs/2107.10161), [](https://arxiv.org/abs/2103.17263), [](https://arxiv.org/abs/2104.13586), [](https://arxiv.org/abs/2102.05095), [](https://arxiv.org/abs/2003.13042)
- [data](https://sdolivia.github.io/FineGym/), [data](http://www.svcl.ucsd.edu/projects/resound/dataset.html), [data](https://research.google.com/ava/index.html), [data](https://www.deepmind.com/open-source/kinetics)
- [](https://mmaction2.readthedocs.io/)
- [](https://github.com/open-mmlab/mmcv), [](https://github.com/SwinTransformer/Video-Swin-Transformer), [](https://github.com/Cogito2012/DEAR), [](https://github.com/xvjiarui/VFS), [](https://github.com/holistic-video-understanding/HVU-Dataset)
| Ray | Unified framework for scaling AI and Python applications |
- [Philipp Moritz](https://github.com/pcmoritz)
- [Robert Nishihara](https://github.com/robertnishihara)
- [Stephanie Wang](https://stephanie-wang.github.io/)
- [Alexey Tumanov](https://faculty.cc.gatech.edu/~atumanov/) others
- [Richard Liaw](https://github.com/richardliaw)
- [Eric Liang](https://github.com/ericl)
- [Melih Elibol](https://research.nvidia.com/person/melih-elibol)
- [Zongheng Yang](https://zongheng.me/)
- [William Paul](https://github.com/Wapaul1)
- [Michael Jordan](https://people.eecs.berkeley.edu/~jordan/)
- [Ion Stoica](https://people.eecs.berkeley.edu/~istoica/)
- [](https://arxiv.org/abs/1712.05889), [](https://arxiv.org/abs/2203.05072), [](https://arxiv.org/abs/1712.09381), [](https://arxiv.org/abs/1807.05118), [](https://arxiv.org/abs/1703.03924)
- [](https://docs.ray.io/en/latest/index.html)
- [website](https://www.ray.io/)
- [](https://youtu.be/LmROEotKhJA), [](https://youtu.be/uzt-CwohQC8), [](https://youtu.be/XME90SGL6Vs)
| Neural style transfer | This tutorial uses deep learning to compose one image in the style of another image | [Billy Lamberta](https://github.com/lamberta) |
- [](https://arxiv.org/abs/1508.06576)
| Image classification | This tutorial shows how to classify images of flowers | [Billy Lamberta](https://github.com/lamberta) |
- [](https://paperswithcode.com/task/image-classification)
| Home Robot | Low-level API for controlling various home robots | [Chris Paxton](https://cpaxton.github.io/) | [](https://github.com/facebookresearch/home-robot)
- [](https://github.com/cpaxton/contact_graspnet/tree/cpaxton/devel), [](https://github.com/facebookresearch/fairo), [](https://github.com/hello-robot/stretch_body), [](https://github.com/hello-robot/stretch_firmware), [](https://github.com/hello-robot/stretch_ros), [](https://github.com/hello-robot/stretch_ros2), [](https://github.com/hello-robot/stretch_web_interface), [](https://github.com/RoboStack/ros-noetic), [](https://github.com/codekansas/stretch-robot)
| Stable Diffusion 2 | New stable diffusion model at 768x768 resolution. Same number of parameters in the U-Net as 1.5, but uses OpenCLIP-ViT/H as the text encoder and is trained from scratch |
- [Robin Rombach](https://github.com/rromb)
- [Andreas Blattmann](https://github.com/ablattmann)
- [Dominik Lorenz](https://github.com/qp-qp)
- [Patrick Esser](https://github.com/pesser) others
- [Björn Ommer](https://ommer-lab.com/people/ommer/)
- [qunash](https://github.com/qunash)
- [](https://arxiv.org/abs/2112.10752), [](https://arxiv.org/abs/2202.00512), [](https://arxiv.org/abs/2010.02502), [](https://arxiv.org/abs/2108.01073), [](https://arxiv.org/abs/2202.09778), [](https://arxiv.org/abs/2206.00927)
- [](https://github.com/qunash/stable-diffusion-2-gui), [](https://github.com/isl-org/MiDaS), [](https://github.com/lucidrains/denoising-diffusion-pytorch), [](https://github.com/runwayml/stable-diffusion/blob/main/scripts/inpaint_st.py), [](https://github.com/crowsonkb/k-diffusion)
- [](https://huggingface.co/stabilityai/stable-diffusion-2-1), [](https://huggingface.co/stabilityai/stable-diffusion-2-1-base), [](https://huggingface.co/stabilityai/stable-diffusion-2-depth), [](https://huggingface.co/stabilityai/stable-diffusion-2-inpainting)
- [](https://youtu.be/HytucGhwTRs)
| DALL·E Mini | Generate images from a text prompt |
- [Boris Dayma](https://github.com/borisdayma)
- [Suraj Patil](https://github.com/patil-suraj)
- [Pedro Cuenca](https://github.com/pcuenca)
- [Khalid Saifullah](https://khalidsaifullaah.github.io/) others
- [Tanishq Abraham](https://github.com/tmabraham)
- [Phúc H. Lê Khắc](https://lkhphuc.com/)
- [Luke Melas](https://lukemelas.github.io/)
- [Ritobrata Ghosh](https://ghosh-r.github.io/)
- [](https://arxiv.org/abs/2102.08981), [](https://arxiv.org/abs/2012.09841), [](https://arxiv.org/abs/1910.13461), [](https://arxiv.org/abs/2103.00020), [](https://arxiv.org/abs/2012.09841), [](https://arxiv.org/abs/1807.04015)
- [blog post](https://wandb.ai/dalle-mini/dalle-mini/reports/DALL-E-mini--Vmlldzo4NjIxODA)
- [data](https://aclanthology.org/P18-1238/)
- [](https://github.com/huggingface/transformers/tree/master/examples/research_projects/jax-projects), [](https://github.com/openai/CLIP/blob/main/data/yfcc100m.md)
- [](https://huggingface.co/spaces/flax-community/dalle-mini)
| Autoencoders | This tutorial introduces autoencoders with three examples: the basics, image denoising, and anomaly detection | [Google](https://www.tensorflow.org/) |
- [blog post](https://blog.keras.io/building-autoencoders-in-keras.html)
- [book](https://www.deeplearningbook.org/contents/autoencoders.html)
- [data](http://www.timeseriesclassification.com/description.php?Dataset=ECG5000)
- [examples](https://anomagram.fastforwardlabs.com/#/)
- [](https://paperswithcode.com/method/autoencoder)
| xFormers | Toolbox to Accelerate Research on Transformers |
- [Benjamin Lefaudeux](https://github.com/blefaudeux)
- [Francisco Massa](https://github.com/fmassa)
- [Diana Liskovich](https://www.linkedin.com/in/dianaliskovich)
- [Wenhan Xiong](https://xwhan.github.io/) others
- [Vittorio Caggiano](https://vittorio-caggiano.github.io/)
- [Sean Naren](https://github.com/SeanNaren)
- [Min Xu](https://github.com/min-xu-ai)
- [Jieru Hu](https://github.com/jieru-hu)
- [Marta Tintore](https://github.com/MartaTintore)
- [Susan Zhang](https://suchenzang.github.io/)
- [Patrick Labatut](https://github.com/patricklabatut)
- [Daniel Haziza](https://scholar.google.com/citations?user=2eSKdFMAAAAJ)
- [](https://facebookresearch.github.io/xformers/)
- [](https://github.com/google-research/sputnik), [](https://github.com/hgyhungry/ge-spmm), [](https://github.com/openai/triton), [](https://github.com/RobinBruegger/RevTorch), [](https://github.com/mlpen/Nystromformer), [](https://github.com/facebookresearch/fairscale), [](https://github.com/huggingface/pytorch-image-models), [](https://github.com/Dao-AILab/flash-attention)
- [](https://youtu.be/NJyZCdxnGe4)
| Classify text with BERT | This tutorial contains complete code to fine-tune BERT to perform sentiment analysis on a dataset of plain-text IMDB movie reviews | [Google](https://www.tensorflow.org/) | [](https://doi.org/10.18653/v1/N19-1423)
- [](https://arxiv.org/abs/1810.04805), [](https://arxiv.org/abs/1711.05101)
- [data](https://ai.stanford.edu/~amaas/data/sentiment/)
- [](https://paperswithcode.com/task/text-classification)
- [](https://tfhub.dev/google/collections/bert/1)
| Kandinsky 2.1 | As text and image encoder it uses CLIP model and diffusion image prior between latent spaces of CLIP modalities |
- [Arseniy Shakhmatov](https://github.com/cene555)
- [Anton Razzhigaev](https://github.com/razzant)
- [Aleksandr Nikolich](https://github.com/AlexWortega)
- [Vladimir Arkhipkin](https://github.com/oriBetelgeuse) others
- [Igor Pavlov](https://github.com/boomb0om)
- [Andrey Kuznetsov](https://github.com/kuznetsoffandrey)
- [Denis Dimitrov](https://github.com/denndimitrov)
- [blog post](https://habr.com/ru/companies/sberbank/articles/725282/)
- [demo](https://editor.fusionbrain.ai/)
- [](https://huggingface.co/sberbank-ai/Kandinsky_2.1)
- [](https://youtu.be/LZvp4SWcCao), [](https://youtu.be/IoPhRE37XSU), [](https://youtu.be/dYt9xJ7dnpU), [](https://youtu.be/rN2J5TL2RZ0)
| SoftVC VITS | Singing Voice Conversion | [svc develop team](https://github.com/svc-develop-team) | [](https://github.com/svc-develop-team/so-vits-svc)
- [](https://github.com/NaruseMioShirakana/MoeVoiceStudio), [](https://github.com/openvpi/DiffSinger/tree/refactor/modules/nsf_hifigan), [](https://github.com/auspicious3000/contentvec), [](https://github.com/yxlllc/DDSP-SVC), [](https://github.com/flutydeer/audio-slicer), [](https://github.com/openvpi/audio-slicer)
- [](https://huggingface.co/NaruseMioShirakana/MoeSS-SUBModel/tree/main)
| threestudio | Unified framework for 3D content creation from text prompts, single images, and few-shot images, by lifting 2D text-to-image generation models |
- [Yuan-Chen Guo](https://github.com/bennyguo)
- [Ying-Tian Liu](https://github.com/thuliu-yt16)
- [Ruizhi Shao](https://github.com/DSaurus)
- [Christian Laforte](https://github.com/claforte) others
- [Vikram Voleti](https://github.com/voletiv)
- [Guan Luo](https://github.com/logan0601)
- [Chia-Hao Chen](https://scholar.google.com/citations?user=X0zirvMAAAAJ)
- [Zi-Xin Zou](https://github.com/zouzx)
- [Chen Wang](https://cwchenwang.github.io/)
- [Yanpei Cao](https://yanpei.me/)
- [Song-Hai Zhang](https://scholar.google.com/citations?user=AWtV-EQAAAAJ)
- [](https://arxiv.org/abs/2303.15413), [](https://arxiv.org/abs/2305.16213), [](https://arxiv.org/abs/2211.10440)
- [](https://discord.gg/ejer2MAB8N)
- [](https://github.com/DSaurus/Tensor4D), [](https://github.com/eladrich/latent-nerf), [](https://github.com/Gorilla-Lab-SCUT/Fantasia3D), [](https://github.com/cvlab-columbia/zero123), [](https://github.com/guochengqian/Magic123), [](https://github.com/ayaanzhaque/instruct-nerf2nerf), [](https://github.com/KAIR-BAIR/nerfacc), [](https://github.com/Lightning-AI/lightning), [](https://github.com/ashawkey/fantasia3d.unofficial)
- [](https://huggingface.co/DeepFloyd/IF-I-XL-v1.0), [](https://huggingface.co/docs/huggingface_hub/v0.14.1/guides/download#download-an-entire-repository)
- [](https://www.reddit.com/r/StableDiffusion/comments/1635cb0/threestudio_a_unified_framework_for_3d_content/)
- [](https://youtu.be/gT8Xvx5b6IE)
| Image captioning | Given an image our goal is to generate a caption | [Billy Lamberta](https://github.com/lamberta) |
- [](https://arxiv.org/abs/1502.03044)
- [data](https://cocodataset.org/#home)
| Word2Vec | Word2Vec is not a singular algorithm, rather, it is a family of model architectures and optimizations that can be used to learn word embeddings from large datasets | [Google](https://www.tensorflow.org/) |
- [](https://arxiv.org/abs/1301.3781)
- [link](https://web.stanford.edu/class/cs224n/readings/cs224n-2019-notes01-wordvecs1.pdf)
- [](https://papers.nips.cc/paper/5021-distributed-representations-of-words-and-phrases-and-their-compositionality.pdf)
- [projector](http://projector.tensorflow.org/)
- [](https://paperswithcode.com/method/cbow-word2vec), [](https://paperswithcode.com/method/skip-gram-word2vec)
- [](https://en.wikipedia.org/wiki/Zipf%27s_law)
| Word embeddings | This tutorial contains an introduction to word embeddings | [Billy Lamberta](https://github.com/lamberta) |
- [data](http://ai.stanford.edu/~amaas/data/sentiment/)
- [projector](http://projector.tensorflow.org/)
| Tortoise | A multi-voice TTS system trained with an emphasis on quality | [James Betker](https://nonint.com/) | [](https://github.com/neonbjb/tortoise-tts)
- [](https://arxiv.org/abs/2102.12092), [](https://arxiv.org/abs/2102.09672), [](https://arxiv.org/abs/2106.07889)
- [examples](https://nonint.com/static/tortoise_v2_examples.html)
- [](https://github.com/neonbjb/DL-Art-School)
- [](https://huggingface.co/patrickvonplaten), [](https://huggingface.co/spaces/osanseviero/tortoisse-tts)
- [](https://youtu.be/J3-jfS29RF4)
| TRL | Set of tools to train transformer language models with Reinforcement Learning, from the Supervised Fine-tuning step, Reward Modeling step to the Proximal Policy Optimization step |
- [Leandro von Werra](https://github.com/lvwerra)
- [Younes Belkada](https://github.com/younesbelkada)
- [Lewis Tunstall](https://lewtun.github.io/blog/)
- [Edward Beeching](https://edbeeching.github.io/) others
- [Tristan Thrush](http://www.tristanthrush.com/)
- [Nathan Lambert](https://www.natolambert.com/)
- [](https://arxiv.org/abs/1909.08593)
- [](http://hf.co/docs/trl)
- [](https://github.com/openai/lm-human-preferences)
- [](https://youtu.be/xQ5nc1CF7iQ), [](https://youtu.be/67SO20dszNA)
| Petals | Run 100B+ language models at home, BitTorrent-style | [BigScience](https://bigscience.huggingface.co/) | [](https://github.com/bigscience-workshop/petals)
- [](https://arxiv.org/abs/2209.01188), [](https://arxiv.org/abs/2108.07258)
- [](https://github.com/borzunov/chat.petals.ml), [](https://github.com/timDettmers/bitsandbytes)
- [](https://huggingface.co/bigscience/bloom)
- [project](https://petals.ml/)
- [](https://en.wikipedia.org/wiki/BitTorrent)
| Epistemic Neural Networks | A library for neural networks that know what they don't know |
- [Ian Osband](http://iosband.github.io/)
- [Zheng Wen](http://zheng-wen.com/)
- [Seyed Mohammad Asghari](https://github.com/mohammadasghari)
- [Vikranth Dwaracherla](https://github.com/dvikranth) others
- [Morteza Ibrahimi](https://github.com/mibrahimi)
- [Xiuyuan Lu](https://scholar.google.com/citations?user=SPL_2lIAAAAJ)
- [Benjamin Van Roy](https://web.stanford.edu/~bvr/)
- [](https://arxiv.org/abs/2107.08924)
- [](https://medium.com/syncedreview/deepminds-epistemic-neural-networks-open-new-avenues-for-uncertainty-modelling-in-large-and-fa83ab00aba3)
- [](https://youtu.be/j8an0dKcX4A)
| DeepFloyd IF | State-of-the-art open-source text-to-image model with a high degree of photorealism and language understanding |
- [Alex Shonenkov](https://linktr.ee/shonenkovAI)
- [Misha Konstantinov](https://github.com/zeroshot-ai)
- [Daria Bakshandaeva](https://github.com/Gugutse)
- [Christoph Schuhmann](http://christoph-schuhmann.de/) others
- [Ksenia Ivanova](https://github.com/ivksu)
- [Nadiia Klokova](https://github.com/vauimpuls)
- [](https://arxiv.org/abs/2205.11487)
- [](https://discord.gg/umz62Mgr)
- [](https://huggingface.co/DeepFloyd), [](https://huggingface.co/docs/diffusers/optimization/fp16#model-offloading-for-fast-inference-and-memory-savings), [](https://huggingface.co/docs/diffusers/api/pipelines/if#optimizing-for-speed), [](https://huggingface.co/docs/diffusers/api/pipelines/if#optimizing-for-memory), [](https://huggingface.co/blog/if), [](https://huggingface.co/docs/diffusers/main/en/api/pipelines/if)
- [](https://www.kaggle.com/code/shonenkov/deepfloyd-if-4-3b-generator-of-pictures)
- [](https://twitter.com/deepfloydai)
- [website](https://deepfloyd.ai/deepfloyd-if)
- [](https://youtu.be/4Zkipll5Rjc), [](https://youtu.be/tq5ZXZWwTPA), [](https://youtu.be/rLtfd1TvYJk)
| normflows | PyTorch implementation of discrete normalizing flows |
- [Vincent Stimper](https://is.mpg.de/person/vstimper)
- [David Liu](https://davindicode.github.io/)
- [Andrew Campbell](https://github.com/andrew-cr)
- [Vincent Berenz](http://vincentberenz.is.tuebingen.mpg.de/) others
- [Lukas Ryll](https://github.com/lukasryll)
- [Bernhard Schölkopf](https://scholar.google.com/citations?user=DZ-fHPgAAAAJ)
- [José Miguel Hernández-Lobato](https://jmhl.org/)
- [](https://arxiv.org/abs/2302.12014)
- [](https://vincentstimper.github.io/normalizing-flows/)
- [](https://github.com/VincentStimper/resampled-base-flows), [](https://github.com/VincentStimper/hmc-hyperparameter-tuning)
- [](https://en.wikipedia.org/wiki/Von_Mises_distribution)
| MyoSuite | A collection of musculoskeletal environments and tasks simulated with the MuJoCo physics engine and wrapped in the OpenAI gym API to enable the application of Machine Learning to bio-mechanic control problems |
- [Vittorio Caggiano](https://github.com/Vittorio-Caggiano)
- [Huawei Wang](https://huaweiwang.github.io/)
- [Guillaume Durandau](https://people.utwente.nl/g.v.durandau)
- [Massimo Sartori](https://people.utwente.nl/m.sartori)
- [Vikash Kumar](https://vikashplus.github.io/)
- [](https://arxiv.org/abs/2205.13600)
- [](https://myosuite.readthedocs.io/en/latest/)
| Audiocraft | PyTorch library for deep learning research on audio generation |
- [Jade Copet](https://scholar.google.com/citations?&user=GRMLwjAAAAAJ)
- [Felix Kreuk](https://felixkreuk.github.io/)
- [Itai Gat](https://itaigat.com/)
- [Tal Remez](https://talremez.github.io/) others
- [David Kant](https://www.linkedin.com/in/david-kant-339a3b1b7)
- [Gabriel Synnaeve](https://syhw.github.io/)
- [Yossi Adi](https://www.cs.huji.ac.il/~adiyoss/)
- [Alexandre Défossez](https://ai.honu.io/)
- [](https://arxiv.org/abs/2306.05284), [](https://arxiv.org/abs/2301.11325)
- [](https://github.com/facebookresearch/encodec), [](https://github.com/camenduru/MusicGen-colab)
- [](https://huggingface.co/facebook/musicgen-large)
- [project](https://ai.honu.io/papers/musicgen/)
- [](https://youtu.be/v-YpvPkhdO4), [](https://www.youtube.com/watch?v=EGfxuTy9Eeo), [](https://youtu.be/la2fGS0dW98), [](https://youtu.be/v-YpvPkhdO4)
| Transformer | This tutorial trains a Transformer model to translate Portuguese to English | [Billy Lamberta](https://github.com/lamberta) | [](https://github.com/neulab/word-embeddings-for-nmt)
- [](https://arxiv.org/abs/1706.03762), [](https://arxiv.org/abs/1903.03878)
- [link](https://deepmind.com/blog/article/alphastar-mastering-real-time-strategy-game-starcraft-ii)
| Detectron2 | FAIR's next-generation platform for object detection and segmentation | [Yuxin Wu](http://ppwwyyxx.com/) | [](https://github.com/facebookresearch/detectron2)
- [blog post](https://ai.facebook.com/blog/-detectron2-a-pytorch-based-modular-object-detection-library-/)
- [](https://detectron2.readthedocs.io/en/latest/)
- [](https://github.com/matterport/Mask_RCNN/tree/master/samples/balloon)
| ChatRWKV | Like ChatGPT but powered by RWKV (100% RNN) language model, which is the only RNN that can match transformers in quality and scaling, while being faster and saves VRAM |
- [Bo Peng](https://github.com/BlinkDL)
- [Eric Alcaide](https://hypnopump.github.io/)
- [Quentin Anthony](https://quentin-anthony.github.io/)
- [Alon Albalak](https://alon-albalak.github.io/) others
- [Samuel Arcadinho](https://github.com/SSamDav)
- [Matteo Grella](http://www.matteogrella.com/)
- [Kranthi Kiran](https://kranthigv.github.io/)
- [Haowen Hou](https://github.com/howard-hou)
- [Przemyslaw Kazienko](https://kazienko.eu/en)
- [Jan Kocon](https://github.com/KoconJan)
- [Bartlomiej Koptyra](https://github.com/bkoptyra)
- [Ipsit Mantri](https://ipsitmantri.github.io/)
- [Ferdinand Mom](https://3outeille.github.io/)
- [Xiangru Tang](https://github.com/tangxiangru)
- [Johan Wind](https://johanwind.github.io/)
- [Stanisław Woźniak](https://www.researchgate.net/profile/Stanislaw-Wozniak-3)
- [Qihang Zhao](https://www.researchgate.net/profile/Qihang-Zhao-2)
- [Peng Zhou](https://pengzhou.sites.ucsc.edu/)
- [Jian Zhu](https://lingjzhu.github.io/)
- [Rui-Jie Zhu](https://scholar.google.com/citations?user=08ITzJsAAAAJ)
- [](https://arxiv.org/abs/2305.13048)
- [](https://discord.gg/bDSBUMeFpc)
- [](https://github.com/saharNooby/rwkv.cpp), [](https://github.com/harrisonvanderbyl/rwkv-cpp-cuda), [](https://github.com/Blealtan/RWKV-LM-LoRA), [](https://github.com/josStorer/RWKV-Runner)
- [](https://huggingface.co/BlinkDL)
- [](https://www.reddit.com/r/MachineLearning/comments/1135aew/r_rwkv4_14b_release_and_chatrwkv_a_surprisingly/)
- [](https://twitter.com/BlinkDL_AI)
- [website](https://www.rwkv.com/)
- [](https://youtu.be/UeAD1qWNb1U)
| Python Data Science Handbook | Jupyter notebook version of the Python Data Science Handbook by Jake VanderPlas | [Jake Vanderplas](http://vanderplas.com/) | [](https://github.com/jakevdp/PythonDataScienceHandbook)
- [project](https://jakevdp.github.io/PythonDataScienceHandbook/)
| PGMax | General factor graphs for discrete probabilistic graphical models, and hardware-accelerated differentiable loopy belief propagation in JAX |
- [Guangyao Zhou](https://stanniszhou.github.io/)
- [Nishanth Kumar](http://nishanthjkumar.com/)
- [Antoine Dedieu](https://github.com/antoine-dedieu)
- [Miguel Lázaro-Gredilla](https://www.tsc.uc3m.es/~miguel/) others
- [Shrinu Kushagra](https://cs.uwaterloo.ca/~skushagr/)
- [Dileep George](https://dileeplearning.github.io/)
- [](https://arxiv.org/abs/2202.04110)
- [](https://en.wikipedia.org/wiki/Belief_propagation)
| StableLM | Stability AI Language Models | [Stability AI](https://stability.ai/research) | [](https://github.com/Stability-AI/StableLM)
- [blog post](https://stability.ai/blog/stability-ai-launches-the-first-of-its-stablelm-suite-of-language-models)
- [](https://github.com/facebookresearch/llama), [](https://github.com/tatsu-lab/stanford_alpaca), [](https://github.com/nomic-ai/gpt4all), [](https://github.com/databrickslabs/dolly), [](https://github.com/anthropics/hh-rlhf), [](https://github.com/ggerganov/llama.cpp)
- [](https://huggingface.co/lmsys/vicuna-13b-delta-v0), [](https://huggingface.co/datasets/RyokoAI/ShareGPT52K), [](https://huggingface.co/stabilityai)
- [](https://youtu.be/dypPSs4t77g), [](https://youtu.be/nWf1StvtoRw), [](https://youtu.be/Hg-s2RTaTFE), [](https://youtu.be/qXtJjoEfTnA)
| TTS | A library for advanced Text-to-Speech generation, built on the latest research, was designed to achieve the best trade-off among ease-of-training, speed and quality |
- [Eren Gölge](https://github.com/erogol)
- [Aya-AlJafari](https://github.com/Aya-AlJafari)
- [Edresson Casanova](https://github.com/Edresson)
- [Josh Meyer](http://jrmeyer.github.io/) others
- [Kelly Davis](https://github.com/kdavis-coqui)
- [Reuben Morais](https://github.com/reuben)
- [blog post](https://coqui.ai/blog/tts/solving-attention-problems-of-tts-models-with-double-decoder-consistency)
- [](https://tts.readthedocs.io/en/latest/)
- [](https://github.com/coqui-ai/TTS-papers)
- [samples](https://erogol.github.io/ddc-samples/)
- [website](https://coqui.ai/)
- [](https://youtu.be/ADnBCz0Wd1U), [](https://youtu.be/Yglxf2WbkLU), [](https://youtu.be/alpI-DnVlO0)
| OpenCLIP | An open source implementation of CLIP |
- [Ross Wightman](https://rwightman.com/)
- [Cade Gordon](https://cadegordon.io/)
- [Vaishaal Shankar](http://vaishaal.com/)
- [](https://arxiv.org/abs/2109.01903), [](https://arxiv.org/abs/2103.00020), [](https://arxiv.org/abs/2111.02114), [](https://arxiv.org/abs/2107.04649), [](https://arxiv.org/abs/1902.10811), [](https://arxiv.org/abs/2107.04649)
- [data](https://ai.google.com/research/ConceptualCaptions/download), [data](https://laion.ai/blog/laion-5b/), [data](https://laion.ai/blog/laion-400-open-dataset/)
- [](https://github.com/mlfoundations/wise-ft), [](https://github.com/webdataset/webdataset), [](https://github.com/webdataset/tarp), [](https://github.com/google-research-datasets/conceptual-12m)
- [](https://huggingface.co/datasets/laion/laion2B-en), [](https://huggingface.co/laion/CLIP-ViT-B-32-laion2B-s34B-b79K), [](https://huggingface.co/laion/CLIP-ViT-L-14-laion2B-s32B-b82K), [](https://huggingface.co/laion/CLIP-ViT-H-14-laion2B-s32B-b79K), [](https://huggingface.co/laion/CLIP-ViT-g-14-laion2B-s12B-b42K)
| Stable Baselines3 | Set of reliable implementations of reinforcement learning algorithms in PyTorch |
- [Antonin Raffin](https://araffin.github.io/)
- [Ashley Hill](https://hill-a.me/)
- [Adam Gleave](https://www.gleave.me/)
- [Anssi Kanervisto](https://github.com/Miffyli) others
- [Maximilian Ernestus](https://github.com/ernestum)
- [Noah Dormann](https://github.com/ndormann)
- [](https://stable-baselines3.readthedocs)
- [](https://github.com/Stable-Baselines-Team/stable-baselines3-contrib), [](https://github.com/hill-a/stable-baselines), [](https://github.com/openai/gym/wiki/Environments)
- [paper](https://jmlr.org/papers/v22/20-1364.html)
- [](https://www.reddit.com/r/reinforcementlearning/)
- [](https://www.youtube.com/playlist?list=PLQVvvaa0QuDf0O2DWwLZBfJeYY-JOeZB1)
| RL Baselines3 Zoo | Training Framework for Stable Baselines3 Reinforcement Learning Agents | [Antonin Raffin](https://araffin.github.io/) | [](https://github.com/DLR-RM/rl-baselines3-zoo)
- [](https://arxiv.org/abs/2005.05719)
- [](https://stable-baselines3.readthedocs.io/en/master/)
- [](https://github.com/DLR-RM/rl-baselines3-zoo), [](https://github.com/openai/roboschool), [](https://github.com/Farama-Foundation/Minigrid)
- [](https://huggingface.co/sb3)
| Grounded-SAM | Marrying Grounding DINO with Segment Anything & Stable Diffusion & Recognize Anything - Automatically Detect, Segment and Generate Anything | [IDEA-Research](https://www.idea.edu.cn/) | [](https://github.com/IDEA-Research/Grounded-Segment-Anything)
- [](https://arxiv.org/abs/2304.02643), [](https://arxiv.org/abs/2303.05499)
- [](https://github.com/MasterBin-IIAU/UNINEXT), [](https://github.com/IDEA-Research/OSX), [](https://github.com/dvlab-research/VoxelNeXt), [](https://github.com/UX-Decoder/Semantic-SAM), [](https://github.com/UX-Decoder/Segment-Everything-Everywhere-All-At-Once), [](https://github.com/IDEA-Research/OpenSeeD), [](https://github.com/Computer-Vision-in-the-Wild/CVinW_Readings), [](https://github.com/sail-sg/EditAnything), [](https://github.com/feizc/IEA), [](https://github.com/Li-Qingyun/sam-mmrotate), [](https://github.com/VainF/Awesome-Anything), [](https://github.com/RockeyCoss/Prompt-Segment-Anything)
- [](https://youtu.be/oEQYStnF2l8), [](https://youtu.be/gKTYMfwPo4M), [](https://youtu.be/0Fpb8TBH0nM), [](https://youtu.be/GuEDDBWrN24)
| TFDS | Collection of ready-to-use datasets for use with TensorFlow, Jax, and other Machine Learning frameworks | [Google](https://www.tensorflow.org/) | [](https://github.com/tensorflow/datasets)
- [](https://towardsdatascience.com/youre-importing-data-wrong-c171f52eea00)
- [](https://www.tensorflow.org/datasets)
- [](https://youtu.be/YrMy-BAqk8k), [](https://youtu.be/6th3rahsw9Y), [](https://youtu.be/3HYy0SPd7TE), [](https://youtu.be/MvcK-MaXbHk)
| SentencePiece | An unsupervised text tokenizer and detokenizer mainly for Neural Network-based text generation systems where the vocabulary size is predetermined prior to the neural model training |
- [Taku Kudo](http://chasen.org/~taku/)
- [John Richardson](https://scholar.google.com/citations?user=PEvmYfgAAAAJ)
- [](https://arxiv.org/abs/1808.06226), [](https://arxiv.org/abs/1508.07909), [](https://arxiv.org/abs/1804.10959), [](https://arxiv.org/abs/1910.13267), [](https://arxiv.org/abs/1609.08144)
- [](https://github.com/moses-smt/mosesdecoder/blob/master/scripts/tokenizer/tokenizer.perl), [](https://github.com/rsennrich/subword-nmt), [](https://github.com/gperftools/gperftools), [](https://github.com/Microsoft/vcpkg)
- [](https://jacky2wong.medium.com/understanding-sentencepiece-under-standing-sentence-piece-ac8da59f6b08)
- [](https://youtu.be/U51ranzJBpY)
| Optimum | Extension of Transformers and Diffusers, providing a set of optimization tools enabling maximum efficiency to train and run models on targeted hardware, while keeping things easy to use | [Hugging Face](https://huggingface.co/) | [](https://github.com/huggingface/optimum)
- [](https://github.com/openvinotoolkit/nncf)
- [](https://huggingface.co/docs/optimum/index), [](https://huggingface.co/docs/transformers/main_classes/trainer)
- [](https://youtu.be/UJnfePM0Ur8), [](https://www.youtube.com/live/b1Gk9q9empA), [](https://youtu.be/_AKFDOnrZz8)
| TorchGeo | PyTorch domain library that provides datasets, transforms, samplers, and pre-trained models specific to geospatial data |
- [Adam Stewart](https://github.com/adamjstewart)
- [Caleb Robinson](https://calebrob.com/)
- [Isaac Corley](https://github.com/isaaccorley)
- [Anthony Ortiz](https://github.com/anthonymlortiz) others
- [Juan Lavista Ferres](https://www.microsoft.com/en-us/research/people/jlavista/)
- [Arindam Banerjee](https://arindam.cs.illinois.edu/)
- [NDBI](https://www.linkedin.com/pulse/ndvi-ndbi-ndwi-calculation-using-landsat-7-8-tek-bahadur-kshetri/)
- [NDVI](https://gisgeography.com/ndvi-normalized-difference-vegetation-index/)
- [NDWI](https://custom-scripts.sentinel-hub.com/custom-scripts/sentinel-2/ndwi/)
- [](https://arxiv.org/abs/2111.08872)
- [data](https://docs.sentinel-hub.com/api/latest/data/sentinel-2-l2a/), [data](https://www.cogeo.org/)
- [](https://github.com/davemlz/awesome-spectral-indices)
| LAVIS | Python deep learning library for LAnguage-and-VISion intelligence research and applications |
- [Dongxu Li](https://github.com/dxli94)
- [Junnan Li](https://github.com/LiJunnan1992)
- [Hung Le](https://sites.google.com/view/henryle2018/home)
- [Guangsen Wang](https://github.com/guangsen-wang) others
- [Silvio Savarese](https://scholar.google.com/citations?user=ImpbxLsAAAAJ)
- [Steven Hoi](https://sites.google.com/view/stevenhoi)
- [](https://arxiv.org/abs/2209.09019), [](https://arxiv.org/abs/2305.06500), [](https://arxiv.org/abs/2301.12597), [](https://arxiv.org/abs/2212.10846), [](https://arxiv.org/abs/2210.08773)
- [blog post](https://blog.salesforceairesearch.com/lavis-language-vision-library/)
- [](https://opensource.salesforce.com/LAVIS//latest/index.html)
- [](https://en.wikipedia.org/wiki/Merlion)
| AudioLM | Framework for high-quality audio generation with long-term consistency |
- [Phil Wang](https://lucidrains.github.io/)
- [Zalán Borsos](https://zalanborsos.com/)
- [Raphaël Marinier](https://github.com/RaphaelMarinier)
- [Damien Vincent](https://www.linkedin.com/in/damien-vincent-1958381) others
- [Eugene Kharitonov](https://eugene-kharitonov.github.io/)
- [Olivier Pietquin](https://research.google/people/105812)
- [Matt Sharifi](https://scholar.google.com/citations?user=GeQNBz0AAAAJ)
- [Olivier Teboul](https://scholar.google.com/citations?user=ep0OfyAAAAAJ)
- [David Grangier](http://david.grangier.info/)
- [Marco Tagliasacchi](https://scholar.google.com/citations?user=zwH1rZQAAAAJ)
- [Neil Zeghidour](https://github.com/lienz)
- [](https://arxiv.org/abs/2209.03143), [](https://arxiv.org/abs/2107.03312), [](https://arxiv.org/abs/2305.02765), [](https://arxiv.org/abs/2305.19466), [](https://arxiv.org/abs/2002.05202), [](https://arxiv.org/abs/1911.02150), [](https://arxiv.org/abs/2207.12598), [](https://arxiv.org/abs/2105.13290), [](https://arxiv.org/abs/2210.13432), [](https://arxiv.org/abs/2111.09883), [](https://arxiv.org/abs/2104.05707), [](https://arxiv.org/abs/2210.13438)
- [blog post](https://blog.research.google/2022/10/audiolm-language-modeling-approach-to.html)
- [](https://discord.gg/xBPBXfcFHd)
- [](https://github.com/facebookresearch/encodec), [](https://github.com/lucidrains/musiclm-pytorch)
- [project](https://google-research.github.io/seanet/audiolm/examples/)
- [](https://youtu.be/Vucewi_kPEU), [](https://youtu.be/behUbh0koZk), [](https://youtu.be/olNvmUCmY8o)
| Hello, many worlds | This tutorial shows how a classical neural network can learn to correct qubit calibration errors | [Michael Broughton](https://github.com/MichaelBroughton) |
- [](https://www.tensorflow.org/quantum/api_docs/python/tfq/layers), [](https://www.tensorflow.org/quantum/api_docs/python/tfq/get_expectation_op), [](https://www.tensorflow.org/guide/keras/functional)
- [](https://en.wikipedia.org/wiki/Pauli_matrices)
- [](https://youtu.be/-o9AhIz1uvo)
| Image segmentation | This tutorial focuses on the task of image segmentation, using a modified U-Net | [Billy Lamberta](https://github.com/lamberta) |
- [data](https://www.robots.ox.ac.uk/~vgg/data/pets/)
- [](https://www.kaggle.com/c/carvana-image-masking-challenge/overview)
- [u-net](https://lmb.informatik.uni-freiburg.de/people/ronneber/u-net/)
| Tzer | Coverage-Guided Tensor Compiler Fuzzing with Joint IR-Pass Mutation |
- [Jiawei Liu](https://jiawei-site.github.io/)
- [Yuxiang Wei](https://yuxiang.cs.illinois.edu/)
- [Sen Yang](https://github.com/syang-ng)
- [Yinlin Deng](https://dengyinlin.github.io/)
- [Lingming Zhang](http://lingming.cs.illinois.edu/)
- [](https://arxiv.org/abs/2202.09947)
- [](https://hub.docker.com/repository/docker/tzerbot/oopsla)
- [](https://tzer.readthedocs.io/en/latest/index.html)
- [](https://github.com/ganler/memcov)
| ArtLine | A Deep Learning based project for creating line art portraits | [Vijish Madhavan](https://github.com/vijishmadhavan) | [](https://github.com/vijishmadhavan/ArtLine)
- [](https://arxiv.org/abs/1805.08318), [](https://arxiv.org/abs/1710.10196), [](https://arxiv.org/abs/1707.02921), [](https://arxiv.org/abs/1603.08155)
- [data](https://cg.cs.tsinghua.edu.cn/people/~Yongjin/APDrawingDB.zip)
- [](https://github.com/yiranran/APDrawingGAN), [](https://github.com/jantic/DeOldify)
| Haiku | A library built on top of JAX designed to provide simple, composable abstractions for machine learning research |
- [Tom Hennigan](https://github.com/tomhennigan)
- [Trevor Cai](https://github.com/trevorcai)
- [Tamara Norman](https://github.com/tamaranorman)
- [Igor Babuschkin](https://www.babushk.in/)
- [](https://dm-haiku.readthedocs.io/en/latest/)
- [website](https://www.haiku-os.org/)
| SAHI | A lightweight vision library for performing large scale object detection & instance segmentation |
- [Fatih Cagatay Akyon](https://github.com/fcakyon)
- [Sinan Onur ALTINUÇ](https://github.com/sinanonur)
- [Alptekin Temizel](https://blog.metu.edu.tr/atemizel/)
- [Cemil Cengiz](https://scholar.google.com/citations?user=1Ull07EAAAAJ) others
- [Devrim Çavuşoğlu](https://github.com/devrimcavusoglu)
- [Kadir Şahin](https://github.com/ssahinnkadir)
- [Oğulcan Eryüksel](https://github.com/oulcan)
- [](https://arxiv.org/abs/2202.06934)
- [](https://github.com/fcakyon/small-object-detection-benchmark)
- [](https://huggingface.co/models?pipeline_tag=object-detection&sort=downloads)
- [](https://www.kaggle.com/remekkinas/sahi-slicing-aided-hyper-inference-yv5-and-yx)
- [](https://medium.com/codable/sahi-a-vision-library-for-performing-sliced-inference-on-large-images-small-objects-c8b086af3b80), [](https://medium.com/codable/convert-any-dataset-to-coco-object-detection-format-with-sahi-95349e1fe2b7)
| AmpliGraph | A suite of neural machine learning models for relational Learning, a branch of machine learning that deals with supervised learning on knowledge graphs |
- [Luca Costabello](https://luca.costabello.info/)
- [Adrianna Janik](https://github.com/adrijanik)
- [Chan Le Van](https://github.com/chanlevan)
- [Nicholas McCarthy](https://github.com/NicholasMcCarthy) others
- [Rory McGrath](http://www.rorymcgrath.ie/)
- [Sumit Pai](https://github.com/sumitpai)
- [](http://arxiv.org/abs/1702.05563), [](http://arxiv.org/abs/1705.10744), [](https://arxiv.org/abs/2105.08683), [](http://arxiv.org/abs/1612.03975), [](https://arxiv.org/abs/1912.10000), [](https://arxiv.org/abs/1412.6575)
- [](https://docs.ampligraph.org)
- [](https://papers.nips.cc/paper/2013/hash/1cecc7a77928ca8133fa24680a88d2f9-Abstract.html), [](https://papers.nips.cc/paper/2013/hash/b337e84de8752b27eda3a12363109e80-Abstract.html)
- [](https://youtu.be/gX_KHaU8ChI)
| NMT with attention | This notebook trains a seq2seq model for Spanish to English translation | [Billy Lamberta](https://github.com/lamberta) |
- [](https://arxiv.org/abs/1409.0473), [](https://arxiv.org/abs/1508.04025v5)
- [data](http://www.manythings.org/anki/)
- [](https://en.wikipedia.org/wiki/Neural_machine_translation)
| GLUE using BERT on TPU | This tutorial contains complete end-to-end code to train models on a TPU | [Google](https://www.tensorflow.org/) |
- [GLUE](https://gluebenchmark.com/)
- [](https://arxiv.org/abs/1810.04805)
- [](https://www.tensorflow.org/guide/tpu)
| TensorBoard | Suite of web applications for inspecting and understanding your TensorFlow runs and graphs | [Yuan Tang](https://terrytangyuan.github.io/) | [](https://github.com/tensorflow/tensorboard)
- [](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/training/supervisor.py)
- [](https://www.tensorflow.org/tensorboard/get_started), [](https://www.tensorflow.org/api_docs/python/tf/summary), [](https://www.tensorflow.org/api_docs/python/tf/linalg/matmul), [](https://www.tensorflow.org/api_docs/python/tf/nn/relu), [](https://www.tensorflow.org/api_docs/python/tf/compat/v1/train/summary_iterator)
- [website](https://tensorboard.dev/)
- [](https://en.wikipedia.org/wiki/Reservoir_sampling)
- [](https://youtu.be/eBbEDRsCmv4), [](https://youtu.be/BqgTU7_cBnk), [](https://youtu.be/qEQ-_EId-D0), [](https://youtu.be/3bownM3L5zM)
| High-performance Simulation with Kubernetes | This tutorial will describe how to set up high-performance simulation using a TFF runtime running on Kubernetes | [Jason Roselander](https://github.com/roselander) |
- [GKE](https://cloud.google.com/kubernetes-engine/)
- [](https://paperswithcode.com/task/federated-learning)
- [shell](https://cloud.google.com/shell/)
| Compel | Text prompt weighting and blending library for transformers-type text embedding systems | [Damian Stewart](http://damianstewart.com/) | [](https://github.com/damian0815/compel)
- [](https://github.com/invoke-ai/InvokeAI/issues/2832)
- [](https://huggingface.co/cactusfriend/nightmare-invokeai-prompts)
| DALL·E Flow | An interactive workflow for generating high-definition images from text prompt |
- [Han Xiao](https://hanxiao.io/)
- [Delgermurun Purevkhuu](https://delgermurun.com/)
- [Alex Cureton-Griffiths](http://blog.alexcg.net/)
- [](https://github.com/Jack000/glid-3-xl), [](https://github.com/jina-ai/docarray)
- [](https://huggingface.co/CompVis/stable-diffusion-v-1-4-original)
- [](https://www.youtube.com/playlist?list=PL3UBBWOUVhFYRUa_gpYYKBqEAkO4sxmne), [](https://www.youtube.com/c/jina-ai)
| Diffusers | Provides pretrained diffusion models across multiple modalities, such as vision and audio, and serves as a modular toolbox for inference and training of diffusion models | [Hugging Face](https://huggingface.co/) | [](https://github.com/huggingface/diffusers)
- [](https://arxiv.org/abs/2006.11239), [](https://arxiv.org/abs/2006.11239), [](https://arxiv.org/abs/2010.02502), [](https://arxiv.org/abs/2202.09778), [](https://arxiv.org/abs/2204.13902)
- [](https://github.com/hojonathanho/diffusion), [](https://github.com/pesser/pytorch_diffusion), [](https://github.com/ermongroup/ddim), [](https://github.com/heejkoo/Awesome-Diffusion-Models)
- [](https://huggingface.co/spaces/CompVis/text2img-latent-diffusion), [](https://huggingface.co/spaces/CompVis/celeba-latent-diffusion), [](https://huggingface.co/spaces/fusing/celeba-diffusion), [](https://huggingface.co/spaces/huggingface/diffuse-the-rest), [](https://huggingface.co/spaces/Shuang59/Composable-Diffusion)
- [](https://towardsdatascience.com/hugging-face-just-released-the-diffusers-library-846f32845e65)
- [](https://youtu.be/UzkdOg7wWmI)
| Sample Factory | One of the fastest RL libraries focused on very efficient synchronous and asynchronous implementations of policy gradients |
- [Aleksei Petrenko](https://alex-petrenko.github.io/)
- [Zhehui Huang](https://zhehui-huang.github.io/)
- [Tushar Kumar](https://github.com/tushartk)
- [Gaurav Sukhatme](http://robotics.usc.edu/~gaurav/)
- [Vladlen Koltun](http://vladlen.info/)
- [ICML](http://proceedings.mlr.press/v119/petrenko20a.html)
- [](https://arxiv.org/abs/2006.11751)
- [](https://www.samplefactory.dev/)
- [](https://github.com/alex-petrenko/faster-fifo)
- [](https://youtu.be/lLG17LKKSZc)
| Open-Assistant | Chat-based assistant that understands tasks, can interact with third-party systems, and retrieve information dynamically to do so |
- [Andreas Köpf](https://github.com/andreaskoepf)
- [Yannic Kilcher](https://github.com/yk)
- [Huu Nguyen](https://github.com/ontocord)
- [Christoph Schuhmann](http://christoph-schuhmann.de/) others
- [Keith Stevens](https://fozziethebeat.github.io/)
- [Abdullah Barhoum](https://github.com/AbdBarho)
- [Nguyen Minh Duc](https://github.com/notmd)
- [Oliver Stanley](https://olliestanley.github.io/)
- [James Melvin Ebenezer](https://github.com/melvinebenezer)
- [](https://arxiv.org/abs/2203.02155)
- [](https://projects.laion.ai/Open-Assistant/)
- [](https://huggingface.co/OpenAssistant)
- [](https://generativeai.pub/open-assistant-a-free-and-open-source-alternative-to-chatgpt-67d15229813)
- [website](https://open-assistant.io/)
- [](https://youtu.be/64Izfm24FKA), [](https://youtu.be/ddG2fM9i4Kk), [](https://youtu.be/FQIHLFLrTw0)
| NeMo | A conversational AI toolkit built for researchers working on automatic speech recognition, natural language processing, and text-to-speech synthesis |
- [Oleksii Kuchaiev](http://kuchaev.com/)
- [Jason Li](https://scholar.google.com/citations?user=V28bxDwAAAAJ)
- [Chip Huyen](https://huyenchip.com/)
- [Oleksii Hrinchuk](https://github.com/AlexGrinch) others
- [Ryan Leary](https://github.com/ryanleary)
- [Boris Ginsburg](https://github.com/borisgin)
- [Samuel Kriman](https://github.com/sam1373)
- [Stanislav Beliaev](https://github.com/stasbel)
- [Vitaly Lavrukhin](https://github.com/vsl9)
- [Jack Cook](https://jackcook.com/)
- [project](https://docs.nvidia.com/deeplearning/nemo/)
- [](https://youtu.be/wBgpMf_KQVw)
| BANMo | Given multiple casual videos capturing a deformable object, BANMo reconstructs an animatable 3D model, including an implicit canonical 3D shape, appearance, skinning weights, and time-varying articulations, without pre-defined shape templates or registered cameras |
- [Gengshan Yang](https://gengshan-y.github.io/)
- [Minh Vo](https://minhpvo.github.io/)
- [Natalia Neverova](https://nneverova.github.io/)
- [Deva Ramanan](http://www.cs.cmu.edu/~deva/) others
- [Andrea Vedaldi](https://www.robots.ox.ac.uk/~vedaldi/)
- [Hanbyul Joo](https://jhugestar.github.io/)
- [](https://arxiv.org/abs/2112.12761)
- [](https://github.com/kwea123/nerf_pl), [](https://github.com/gengshan-y/rigidmask), [](https://github.com/ShichenLiu/SoftRas), [](https://github.com/ThibaultGROUEIX/ChamferDistancePytorch)
- [project](https://banmo-www.github.io/)
- [](https://youtu.be/1NUa-yvFGA0), [](https://youtu.be/jDTy-liFoCQ)
| TPU | Reference models and tools for Cloud TPUs | [Google](https://cloud.google.com/) | [](https://github.com/tensorflow/tpu)
- [website](https://cloud.google.com/tpu/)
- [](https://en.wikipedia.org/wiki/Tensor_Processing_Unit)
- [](https://youtu.be/W7A-9MYvPwI), [](https://youtu.be/MXxN4fv01c8), [](https://youtu.be/FsxthdQ_sL4), [](https://youtu.be/zEOtG-ChmZE), [](https://youtu.be/kBjYK3K3P6M), [](https://youtu.be/8j1MWZGNoXM), [](https://youtu.be/hszd5UqnfLk)
| TF-Agents | A reliable, scalable and easy to use TensorFlow library for Contextual Bandits and Reinforcement Learning |
- [Sergio Guadarrama](https://github.com/sguada)
- [Anoop Korattikara](https://github.com/kbanoop)
- [Oscar Ramirez](https://github.com/oars)
- [Pablo Castro](https://psc-g.github.io/) others
- [Ethan Holly](https://github.com/eholly-g)
- [Sam Fishman](http://sam.fish/)
- [Ke Wang](https://scholar.google.com/citations?user=QRYX59sAAAAJ)
- [Ekaterina Gonina](https://github.com/egonina)
- [Neal Wu](https://twitter.com/WuNeal)
- [Efi Kokiopoulou](https://github.com/efiko)
- [Luciano Sbaiz](https://scholar.google.com/citations?user=fKBmhcUAAAAJ)
- [Jamie Smith](https://scholar.google.com/citations?user=jk17mo8AAAAJ)
- [Gábor Bartók](https://github.com/bartokg)
- [Jesse Berent](https://www.linkedin.com/in/jesse-berent-a1b6875)
- [Chris Harris](https://www.linkedin.com/in/charris)
- [Vincent Vanhoucke](https://vincent.vanhoucke.com/)
- [Eugene Brevdo](https://ebrevdo.github.io/)
- [](https://www.tensorflow.org/agents/api_docs/python/tf_agents)
- [](https://towardsdatascience.com/introduction-to-tf-agents-a-library-for-reinforcement-learning-in-tensorflow-68ab9add6ad6), [](https://medium.com/analytics-vidhya/tf-agents-a-flexible-reinforcement-learning-library-for-tensorflow-5f125420f64b)
- [](https://www.tensorflow.org/agents)
- [](https://youtu.be/2nKD6zFQ8xI), [](https://youtu.be/-TTziY7EmUA), [](https://youtu.be/52DTXidSVWc), [](https://youtu.be/U7g7-Jzj9qo), [](https://youtu.be/tAOApRQAgpc), [](https://youtu.be/X4eruXqNbDc), [](https://youtu.be/g0yDlAbi6Pc), [](https://youtu.be/VmZI_YkfPBM), [](https://youtu.be/7QFSziiAnxI)
| PyG | Library built upon PyTorch to easily write and train Graph Neural Networks for a wide range of applications related to structured data |
- [Matthias Fey](https://rusty1s.github.io/#/)
- [Jan Eric Lenssen](https://github.com/janericlenssen)
- [](https://arxiv.org/abs/1903.02428), [](https://arxiv.org/abs/1801.07829), [](https://arxiv.org/abs/1609.02907), [](https://arxiv.org/abs/2003.03123), [](https://arxiv.org/abs/1905.05178), [](https://arxiv.org/abs/1706.08566), [](https://arxiv.org/abs/1907.10903), [](https://arxiv.org/abs/1905.07953)
- [](https://pytorch-geometric.readthedocs.io/en/latest/)
- [](https://github.com/snap-stanford/ogb/tree/master/examples), [](https://github.com/pyg-team/pyg-lib), [](https://github.com/rusty1s/pytorch_scatter), [](https://github.com/rusty1s/pytorch_sparse), [](https://github.com/rusty1s/pytorch_cluster), [](https://github.com/AntonioLonga/PytorchGeometricTutorial)
- [](https://papers.nips.cc/paper/2018/hash/e77dbaf6759253c7c6d0efc5690369c7-Abstract.html), [](https://papers.nips.cc/paper/2017/hash/5dd9db5e033da9c6fb5ba83c7a7ebea9-Abstract.html), [](https://nips.cc/virtual/2020/public/poster_3fe230348e9a12c13120749e3f9fa4cd.html)
- [](https://pytorch.org/tutorials/beginner/basics/optimization_tutorial.html#full-implementation)
- [](https://www.youtube.com/playlist?list=PLGMXrbDNfqTzqxB1IGgimuhtfAhGd8lHF), [](https://www.youtube.com/playlist?list=PLGMXrbDNfqTwPxitLVHEbT9Pd6-oR_cud), [](https://youtu.be/-UjytpbqX4A)
| ruGPT3 | Example of inference of RuGPT3XL | [Anton Emelyanov](https://github.com/king-menin) | [](https://github.com/ai-forever/ru-gpts)
- [cristofari](https://sbercloud.ru/ru/christofari)
- [](https://github.com/microsoft/DeepSpeedExamples/tree/master/Megatron-LM)
- [](https://huggingface.co/transformers/main_classes/model.html#transformers.generation_utils.GenerationMixin.generate)
- [sparse attention](https://www.deepspeed.ai/tutorials/sparse-attention/)
| Stable Diffusion Videos | Create videos with Stable Diffusion by exploring the latent space and morphing between text prompts | [Nathan Raw](https://github.com/nateraw) | [](https://github.com/nateraw/stable-diffusion-videos)
- [](https://gist.github.com/karpathy/00103b0037c5aaea32fe1da1af553355), [](https://gist.github.com/nateraw/c989468b74c616ebbc6474aa8cdd9e53)
| PyTerrier | A Python framework for performing information retrieval experiments |
- [Craig Macdonald](https://www.dcs.gla.ac.uk/~craigm/)
- [Nicola Tonellotto](https://github.com/tonellotto)
- [](https://arxiv.org/abs/2007.14271)
- [](https://pyterrier.readthedocs.io)
- [](https://github.com/terrier-org/ecir2021tutorial), [](https://github.com/terrierteam/pyterrier_ance), [](https://github.com/terrierteam/pyterrier_colbert), [](https://github.com/terrierteam/pyterrier_pisa), [](https://github.com/terrierteam/pyterrier_t5), [](https://github.com/terrierteam/pyterrier_doc2query), [](https://github.com/terrierteam/pyterrier_deepct)
| DSP theory | Theory of digital signal processing: signals, filtration (IIR, FIR, CIC, MAF), transforms (FFT, DFT, Hilbert, Z-transform) etc |
- [Alexander Kapitanov](https://github.com/hukenovs)
- [Vladimir Fadeev](https://github.com/kirlf)
- [Karina Kvanchiani](https://github.com/karinakvanchiani)
- [Elizaveta Petrova](https://github.com/kleinsbotle)
- [Andrei Makhliarchuk](https://github.com/anotherhelloworld)
- [blog post](https://habr.com/ru/articles/460445/)
| Mubert | Prompt-based music generation via Mubert API | [Ilya Belikov](https://github.com/ferluht) | [](https://github.com/MubertAI/Mubert-Text-to-Music)
- [](https://mubert2.docs.apiary.io/)
- [project](https://mubert.com/)
- [](https://youtu.be/YJu0iXn-T_U), [](https://youtu.be/5UsaxJsFvAI), [](https://youtu.be/B0kkIpWifG4)
| Batch RL | Offline RL using the DQN replay dataset comprising the entire replay experience of a DQN agent on 60 Atari 2600 games |
- [Rishabh Agarwal](https://agarwl.github.io/)
- [Dale Schuurmans](https://webdocs.cs.ualberta.ca/~dale/)
- [Mohammad Norouzi](https://norouzi.github.io/)
- [DQN](https://www.nature.com/articles/nature14236?wm=book_wap_0005)
- [](https://arxiv.org/abs/1907.04543), [](https://arxiv.org/abs/1709.06009)
- [blog post](https://ai.googleblog.com/2020/04/an-optimistic-perspective-on-offline.html)
- [data](https://console.cloud.google.com/storage/browser/atari-replay-datasets), [data](https://research.google/resources/datasets/dqn-replay/)
- [](https://github.com/openai/atari-py/tree/0.2.5/atari_py/atari_roms), [](https://github.com/mgbellemare/Arcade-Learning-Environment), [](https://github.com/mila-iqia/SGI/blob/master/src/offline_dataset.py), [](https://github.com/kzl/decision-transformer/tree/master/atari)
- [project](https://offline-rl.github.io/)
- [slides](https://docs.google.com/presentation/d/1ROltXr6FIeYKrnGl0tKHGWI0pL4Zo8CnvAK2-cdpQyY)
- [talk](https://slideslive.com/38928373/an-optimistic-perspective-on-offline-deep-reinforcement-learning)
- [](https://www.tensorflow.org/install/install_linux)
| EfficientDet | New family of object detectors, called EfficientDet, which consistently achieve much better efficiency than prior art across a wide spectrum of resource constraints |
- [Mingxing Tan](https://scholar.google.com/citations?user=6POeyBoAAAAJ)
- [Ruoming Pang](https://scholar.google.com/citations?user=1fsmwB8AAAAJ)
- [Quoc Le](https://cs.stanford.edu/~quocle/)
- [](https://arxiv.org/abs/1911.09070), [](https://arxiv.org/abs/2103.13886), [](https://arxiv.org/abs/1905.11946), [](https://arxiv.org/abs/1804.02767)
- [blog post](https://ai.googleblog.com/2020/04/efficientdet-towards-scalable-and.html)
- [](https://medium.com/tensorflow/fitting-larger-networks-into-memory-583e3c758ff9)
- [](https://tfhub.dev/s?network-architecture=efficientdet)
- [tutorial](https://cloud.google.com/tpu/docs/tutorials/efficientnet)
- [](https://youtu.be/yJg1FX2goCo), [](https://youtu.be/OsA3zH5NKYc), [](https://youtu.be/qZobxWXlJ0g)
| ACME | A library of reinforcement learning components and agents |
- [Matt Hoffman](https://www.mwhoffman.com/)
- [Bobak Shahriari](https://github.com/bshahr)
- [John Aslanides](https://www.aslanides.io/)
- [Gabriel Barth-Maron](https://github.com/fastturtle) others
- [Feryal Behbahani](https://feryal.github.io/)
- [Tamara Norman](https://github.com/tamaranorman)
- [Abbas Abdolmaleki](https://scholar.google.com/citations?user=cCYTVWQAAAAJ)
- [Albin Cassirer](https://github.com/acassirer)
- [Fan Yang](https://github.com/ddmbr)
- [Kate Baumli](https://github.com/katebaumli)
- [Sarah Henderson](https://www.linkedin.com/in/sarah-henderson-agilecoach/)
- [Alex Novikov](https://scholar.google.ru/citations?user=jMUkLqwAAAAJ)
- [Sergio Gómez Colmenarejo](https://scholar.google.ru/citations?user=0Dkf68EAAAAJ)
- [Serkan Cabi](https://scholar.google.ru/citations?&user=l-HhJaUAAAAJ)
- [Caglar Gulcehre](https://www.caglarg.com/)
- [Tom Le Paine](http://tomlepaine.github.io/)
- [Andrew Cowie](https://scholar.google.ru/citations?&user=aTvi5mUAAAAJ)
- [Ziyu Wang](https://ziyuw.github.io/)
- [Bilal Piot](https://scholar.google.ru/citations?&user=fqxNUREAAAAJ)
- [Nando de Freitas](https://github.com/nandodf)
- [](https://arxiv.org/abs/2006.00979)
- [blog post](https://www.deepmind.com/publications/acme-a-new-framework-for-distributed-reinforcement-learning)
- [](https://dm-acme.readthedocs.io/en/latest/)
- [](https://github.com/deepmind/dm_env)
- [](https://youtu.be/NUwDr42bPOw), [](https://youtu.be/J1XCWjuyRaI), [](https://youtu.be/pFMuQWpHI5k)
| RWKV | Reinventing RNNs for the Transformer Era |
- [Bo Peng](https://github.com/BlinkDL)
- [Eric Alcaide](https://hypnopump.github.io/)
- [Quentin Anthony](https://quentin-anthony.github.io/)
- [Alon Albalak](https://alon-albalak.github.io/) others
- [Samuel Arcadinho](https://github.com/SSamDav)
- [Matteo Grella](http://www.matteogrella.com/)
- [Kranthi Kiran](https://kranthigv.github.io/)
- [Haowen Hou](https://github.com/howard-hou)
- [Przemyslaw Kazienko](https://kazienko.eu/en)
- [Jan Kocon](https://github.com/KoconJan)
- [Bartlomiej Koptyra](https://github.com/bkoptyra)
- [Ipsit Mantri](https://ipsitmantri.github.io/)
- [Ferdinand Mom](https://3outeille.github.io/)
- [Xiangru Tang](https://github.com/tangxiangru)
- [Johan Wind](https://johanwind.github.io/)
- [Stanisław Woźniak](https://www.researchgate.net/profile/Stanislaw-Wozniak-3)
- [Qihang Zhao](https://www.researchgate.net/profile/Qihang-Zhao-2)
- [Peng Zhou](https://pengzhou.sites.ucsc.edu/)
- [Jian Zhu](https://lingjzhu.github.io/)
- [Rui-Jie Zhu](https://scholar.google.com/citations?user=08ITzJsAAAAJ)
- [](https://arxiv.org/abs/2305.13048), [](https://arxiv.org/abs/2105.14103), [](https://arxiv.org/abs/2002.05202)
- [data](https://dldata-public.s3.us-east-2.amazonaws.com/simplebooks.zip)
- [demo](https://josephrocca.github.io/rwkv-v4-web/demo/)
- [](https://discord.gg/bDSBUMeFpc)
- [](https://github.com/saharNooby/rwkv.cpp), [](https://github.com/cgisky1980/ai00_rwkv_server), [](https://github.com/harrisonvanderbyl/rwkv-cpp-cuda), [](https://github.com/Blealtan/RWKV-LM-LoRA), [](https://github.com/TheRamU/Fay/blob/main/README_EN.md), [](https://github.com/ridgerchu/SpikeGPT), [](https://github.com/BlinkDL/RWKV-v2-RNN-Pile/tree/main/RWKV-v3), [](https://github.com/BlinkDL/SmallInitEmb), [](https://github.com/BlinkDL/RWKV-CUDA), [](https://github.com/BlinkDL/minGPT-tuned)
- [](https://huggingface.co/BlinkDL), [](https://huggingface.co/BlinkDL/clip-guided-binary-autoencoder)
- [](https://www.reddit.com/r/MachineLearning/comments/umq908/r_rwkvv2rnn_a_parallelizable_rnn_with/)
- [](https://twitter.com/BlinkDL_AI), [](https://twitter.com/HochreiterSepp/status/1524270961314484227)
- [website](https://www.rwkv.com/)
- [](https://youtu.be/x8pW19wKfXQ), [](https://youtu.be/B3Qa2rRsaXo), [](https://youtu.be/w-xydM6C6Qc)
| NetKet | Open-source project delivering cutting-edge methods for the study of many-body quantum systems with artificial neural networks and machine learning techniques |
- [Filippo Vicentini](https://filippovicentini.com/)
- [Damian Hofmann](https://github.com/femtobit)
- [Attila Szabó](https://github.com/attila-i-szabo)
- [Dian Wu](https://github.com/wdphy16) others
- [Christopher Roth](https://github.com/chrisrothUT)
- [Clemens Giuliani](https://github.com/inailuig)
- [Gabriel Pescia](https://github.com/gpescia)
- [Jannes Nys](https://github.com/jwnys)
- [Vladimir Vargas-Calderón](https://github.com/VolodyaCO)
- [Nikita Astrakhantsev](https://github.com/nikita-astronaut)
- [Giuseppe Carleo](https://github.com/gcarleo)
- [Kenny Choo](https://github.com/kchoo1118)
- [James Smith](https://jamesetsmith.github.io/)
- [Tom Westerhout](https://github.com/twesterhout)
- [Fabien Alet](https://github.com/fabienalet)
- [Emily Davis](https://github.com/emilyjd)
- [Stavros Efthymiou](https://github.com/stavros11)
- [Ivan Glasser](https://www.researchgate.net/profile/Ivan-Glasser)
- [Sheng-Hsuan Lin](https://shhslin.github.io/)
- [Marta Mauri](https://github.com/martamau)
- [Mazzola Guglielmo](https://www.ics.uzh.ch/en/research/research-groups/Guglielmo-Mazzola0.html)
- [Christian Mendl](http://christian.mendl.net/)
- [Evert Nieuwenburg](https://evert.info/)
- [Ossian O'Reilly](https://github.com/ooreilly)
- [Hugo Théveniaut](https://github.com/theveniaut)
- [Giacomo Torlai](https://github.com/GTorlai)
- [Alexander Wietek](https://awietek.github.io/)
- [](https://arxiv.org/abs/2112.10526)
- [](https://netket.readthedocs.io/en/latest/index.html)
- [](https://github.com/mpi4jax/mpi4jax), [](https://github.com/cloudhan/jax-windows-builder)
- [website](https://www.netket.org/)
- [](https://youtu.be/Ryz-o71tuy8)
| pymdp | Package for simulating Active Inference agents in Markov Decision Process environments |
- [Conor Heins](https://github.com/conorheins)
- [Alec Tschantz](https://github.com/alec-tschantz)
- [Beren Millidge](https://www.beren.io/)
- [Brennan Klein](https://github.com/jkbren) others
- [Arun Niranjan](https://github.com/Arun-Niranjan)
- [Daphne Demekas](https://github.com/daphnedemekas)
- [](https://arxiv.org/abs/2201.03904)
- [](https://pymdp-rtd.readthedocs.io/en/stable/)
| Stable Diffusion | A latent text-to-image diffusion model |
- [Robin Rombach](https://github.com/rromb)
- [Andreas Blattmann](https://github.com/ablattmann)
- [Dominik Lorenz](https://github.com/qp-qp)
- [Patrick Esser](https://github.com/pesser)
- [Björn Ommer](https://ommer-lab.com/people/ommer/)
- [](https://arxiv.org/abs/2205.11487), [](https://arxiv.org/abs/2207.12598), [](https://arxiv.org/abs/2202.09778), [](https://arxiv.org/abs/2108.01073)
- [](https://arxiv.org/abs/2112.10752), [](https://github.com/christophschuhmann/improved-aesthetic-predictor), [](https://github.com/ShieldMnt/invisible-watermark), [](https://github.com/openai/guided-diffusion), [](https://github.com/lucidrains/denoising-diffusion-pytorch), [](https://github.com/lucidrains/x-transformers)
- [](https://huggingface.co/CompVis), [](https://huggingface.co/datasets/laion/laion2B-en), [](https://huggingface.co/datasets/laion/laion-high-resolution)
| Deep-MAC | Welcome to the Novel class segmentation demo | [Vighnesh Birodkar](http://vighneshbirodkar.github.io/) |
- [](https://arxiv.org/abs/2104.00613)
- [](https://paperswithcode.com/method/deep-mac)
| NL-Augmenter | A collaborative effort intended to add transformations of datasets dealing with natural language |
- [Aadesh Gupta](https://github.com/aadesh11)
- [Timothy Sum Hon Mun](https://github.com/timothy22000)
- [Aditya Srivatsa](https://github.com/kvadityasrivatsa)
- [Xudong Shen](https://github.com/XudongOliverShen) others
- [Juan Diego Rodriguez](https://github.com/juand-r)
- [Ashish Shrivastava](https://github.com/ashish3586)
- [Nagender Aneja](https://researchid.co/naneja)
- [Zijie Wang](https://zijie.wang/)
- [Yiwen Shi](https://github.com/Yiwen-Shi)
- [Afnan Mir](https://github.com/afnanmmir)
- [William Soto](https://github.com/sotwi)
- [Chandan Singh](https://csinva.io/)
- [Claude Roux](https://github.com/ClaudeRoux)
- [Abinaya Mahendiran](https://github.com/AbinayaM02)
- [Anna Shvets](https://github.com/asnota)
- [Kaustubh Dhole](https://github.com/kaustubhdhole)
- [Bryan Wilie](https://github.com/bryanwilie)
- [Jamie Simon](https://james-simon.github.io/)
- [Mukund Varma](https://github.com/MukundVarmaT)
- [Sang Han](https://github.com/jjangsangy)
- [Denis Kleyko](https://github.com/denkle)
- [Samuel Cahyawijaya](https://github.com/SamuelCahyawijaya)
- [Filip Cornell](https://github.com/Filco306)
- [Tanay Dixit](https://tanay2001.github.io/)
- [Connor Boyle](https://github.com/boyleconnor)
- [Genta Indra Winata](https://gentawinata.com/)
- [Seungjae Ryan Lee](https://github.com/seungjaeryanlee)
- [Marcin Namysl](https://github.com/mnamysl)
- [Roman Sitelew](https://github.com/RomanPlusPlus)
- [Zhenhao Li](https://zhenhaoli.net/)
- [Fiona Tan](https://tanfiona.github.io/)
- [](https://arxiv.org/abs/2112.02721)
- [website](https://gem-benchmark.com/nl_augmenter)
| Accelerate | A simple way to train and use PyTorch models with multi-GPU, TPU, mixed-precision | [Hugging Face](https://huggingface.co/) | [](https://github.com/huggingface/accelerate)
- [](https://huggingface.co/docs/accelerate/index)
| YOLOv5 on Custom Objects | This notebook shows training on your own custom objects | [Jacob Solawetz](https://blog.roboflow.com/author/jacob/) |
- [blog post](https://blog.roboflow.com/how-to-train-yolov5-on-a-custom-dataset/)
- [data](https://public.roboflow.ai/object-detection/bccd)
| MindsEye | Graphical user interface built to run multimodal ai art models for free from a Google Colab, without needing edit a single line of code or know any programming |
- [multimodal.art](https://multimodal.art/)
- [João Paulo Apolinário Passos](http://www.apolinariopassos.com.br/portfolio/)
- [](https://github.com/openai/guided-diffusion)
- [project](https://multimodal.art/mindseye)
| py-irt | Fitting Item Response Theory models using variational inference |
- [John Lalor](https://jplalor.github.io/)
- [Hong Yu](https://scholar.google.com/citations?user=TyXe64wAAAAJ)
- [Pedro Rodriguez](https://www.pedro.ai/)
- [Joe Barrow](https://jbarrow.ai/) others
- [Alexander Hoyle](https://alexanderhoyle.com/)
- [Robin Jia](https://robinjia.github.io/)
- [Jordan Boyd-Graber](https://github.com/ezubaric)
- [](https://arxiv.org/abs/1908.11421)
- [paper](https://www.frontiersin.org/articles/10.3389/fpsyg.2016.01422/full)
- [](https://youtu.be/akUxtt21Mlc)
| SberSwap | A new face swap method for image and video domains |
- [Daniil Chesakov](https://github.com/Danyache)
- [Anastasia Maltseva](https://github.com/NastyaMittseva)
- [Alexander Groshev](https://github.com/AlexanderGroshev)
- [Andrey Kuznetsov](https://www.linkedin.com/in/andrey-kuznetsov-70ab12127)
- [Denis Dimitrov](https://github.com/denndimitrov)
- [](https://arxiv.org/abs/2202.03046), [](https://arxiv.org/abs/1912.13457), [](https://arxiv.org/abs/1901.08971), [](https://arxiv.org/abs/2106.06340), [](https://arxiv.org/abs/2005.05005), [](https://arxiv.org/abs/2106.09965)
- [blog post](https://habr.com/ru/company/sberbank/blog/645919/)
- [data](https://www.robots.ox.ac.uk/~vgg/data/vgg_face/)
| BIG-bench | A collaborative benchmark intended to probe large language models and extrapolate their future capabilities |
- [Jaehoon Lee](https://jaehlee.github.io/)
- [Jascha Sohl-Dickstein](http://www.sohldickstein.com/)
- [Vinay Ramasesh](https://ramasesh.github.io/)
- [Sajant Anand](https://github.com/sajantanand) others
- [Alicia Parrish](https://aliciaparrish.com/)
- [Ethan Dyer](https://github.com/ethansdyer)
- [Liam Dugan](http://liamdugan.com/)
- [Dieuwke Hupkes](https://github.com/dieuwkehupkes)
- [Daniel Freeman](https://github.com/cdfreeman-google)
- [Guy Gur-Ari](https://github.com/guygurari)
- [Aitor Lewkowycz](https://github.com/lewkowycz)
- [API](https://google.github.io/BIG-bench/docs/html/bigbench/index.html)
- [](https://arxiv.org/abs/2206.04615)
- [](https://github.com/google/seqio)
| HuggingArtists | Choose your favorite Artist and train a language model to write new lyrics based on their unique voice | [Aleksey Korshuk](https://github.com/AlekseyKorshuk) | [](https://github.com/AlekseyKorshuk/huggingartists)
- [](https://huggingface.co/spaces/AlekseyKorshuk/huggingartists), [](https://huggingface.co/huggingartists)
| Introduction to the TensorFlow Models NLP library | You will learn how to build transformer-based models for common NLP tasks including pretraining, span labelling and classification using the building blocks from NLP modeling library | [Chen Chen](https://github.com/chenGitHuber) | [](https://github.com/tensorflow/models/tree/master/official/nlp/modeling)
- [](https://arxiv.org/abs/1810.04805)
| Cirq | A python framework for creating, editing, and invoking Noisy Intermediate Scale Quantum circuits |
- [Balint Pato](https://refactorium.com/)
- [Matthew Harrigan](https://mpharrigan.com/)
- [Animesh Sinha](https://github.com/AnimeshSinha1309)
- [Matthew Neeley](https://github.com/maffoo) others
- [Dave Bacon](https://dabacon.org/)
- [Matteo Pompili](https://github.com/matpompili)
- [Michael Broughton](https://github.com/MichaelBroughton)
- [](https://en.wikipedia.org/wiki/Quantum_logic_gate#Hadamard_gate)
- [](https://youtu.be/16ZfkPRVf2w)
| CLIP-as-service | A low-latency high-scalability service for embedding images and text | [Han Xiao](https://hanxiao.io/) | [](https://github.com/jina-ai/clip-as-service)
- [data](https://sites.google.com/view/totally-looks-like-dataset)
- [](https://github.com/jina-ai/docarray)
- [website](https://clip-as-service.jina.ai/)
- [](https://www.youtube.com/playlist?list=PL3UBBWOUVhFYRUa_gpYYKBqEAkO4sxmne), [](https://www.youtube.com/c/jina-ai)
| Jina | MLOps framework that empowers anyone to build cross-modal and multi-modal applications on the cloud | [Han Xiao](https://hanxiao.io/) | [](https://github.com/jina-ai/jina)
- [data](https://sites.google.com/view/totally-looks-like-dataset)
- [](https://docs.jina.ai/)
- [](https://github.com/jina-ai/example-grafana-prometheus/blob/main/grafana-dashboards/flow.json)
- [hub](https://hub.jina.ai/)
- [](https://www.youtube.com/playlist?list=PL3UBBWOUVhFYRUa_gpYYKBqEAkO4sxmne), [](https://www.youtube.com/c/jina-ai)
| Flashlight | Fast, flexible machine learning library written entirely in C++ |
- [Jacob Kahn](https://jacobkahn.me/)
- [Vineel Pratap](https://github.com/vineelpratap)
- [Tatiana Likhomanenko](https://github.com/tlikhomanenko)
- [Qiantong Xu](https://github.com/xuqiantong) others
- [Awni Hannun](https://awnihannun.com/)
- [Jeff Cai](https://ieeexplore.ieee.org/author/37086866180)
- [Paden Tomasello](https://github.com/padentomasello)
- [Ann Lee](https://scholar.google.com/citations?user=Am6PakYAAAAJ)
- [Edouard Grave](https://github.com/EdouardGrave)
- [Gilad Avidov](https://github.com/avidov)
- [Benoit Steiner](http://bsteiner.info/)
- [Vitaliy Liptchinsky](https://scholar.google.com/citations?user=zl4dA-gAAAAJ)
- [Gabriel Synnaeve](https://syhw.github.io/)
- [Ronan Collobert](https://ronan.collobert.com/)
- [](https://arxiv.org/abs/2201.12465)
- [](https://hub.docker.com/r/flml/flashlight/tags?page=1&ordering=last_updated&name=cuda-latest)
- [](https://fl.readthedocs.io/en/latest/)
- [](https://github.com/arrayfire/arrayfire), [](https://github.com/microsoft/vcpkg), [](https://github.com/arrayfire/arrayfire-ml/), [](https://github.com/nvidia/cub), [](https://github.com/USCiLab/cereal), [](https://github.com/nothings/stb), [](https://github.com/facebookincubator/gloo), [](https://github.com/oneapi-src/oneDNN), [](https://github.com/google/glog), [](https://github.com/gflags/gflags), [](https://github.com/flashlight/text)
| Evidently | An open-source framework to evaluate, test and monitor ML models in production |
- [Elena Samuylova](https://github.com/elenasamuylova)
- [Emeli Dral](https://github.com/emeli-dral)
- [Olga Filippova](https://github.com/0lgaF)
- [](https://docs.evidentlyai.com/)
- [](https://github.com/0lgaF/my_tab_with_evidently)
- [website](https://evidentlyai.com/)
- [](https://www.youtube.com/c/EvidentlyAI), [](https://youtu.be/L4Pv6ExBQPM)
| RL Unplugged | Suite of benchmarks for offline reinforcement learning |
- [Caglar Gulcehre](https://www.caglarg.com/)
- [Ziyu Wang](https://ziyuw.github.io/)
- [Alexander Novikov](https://scholar.google.com/citations?user=jMUkLqwAAAAJ)
- [Tom Le Paine](http://tomlepaine.github.io/) others
- [Sergio Gómez Colmenarejo](https://scholar.google.com/citations?user=0Dkf68EAAAAJ)
- [Konrad Żołna](https://github.com/kondiz)
- [Rishabh Agarwal](https://agarwl.github.io/)
- [Josh Merel](https://sites.google.com/site/jsmerel/)
- [Daniel Mankowitz](https://danielmankowitz.wixsite.com/danielm)
- [Cosmin Paduraru](https://scholar.google.com/citations?user=oz4Ca9AAAAAJ)
- [Gabriel Dulac-Arnold](http://gabe.squirrelsoup.net/)
- [Jerry Li](https://github.com/jerryli27)
- [Mohammad Norouzi](https://norouzi.github.io/)
- [Matt Hoffman](https://www.mwhoffman.com/)
- [Ofir Nachum](https://scholar.google.com/citations?user=C-ZlBWMAAAAJ)
- [George Tucker](https://sites.google.com/view/gjt)
- [Nicolas Heess](https://scholar.google.com/citations?user=79k7bGEAAAAJ)
- [Nando de Freitas](https://github.com/nandodf)
- [](https://arxiv.org/abs/2006.13888), [](https://arxiv.org/abs/1907.04543), [](https://arxiv.org/abs/1709.06009), [](https://arxiv.org/abs/1811.09656), [](https://arxiv.org/abs/1811.11711), [](https://arxiv.org/abs/1909.12238), [](https://arxiv.org/abs/1911.09451), [](https://arxiv.org/abs/1801.00690), [](https://arxiv.org/abs/2003.11881), [](https://arxiv.org/abs/2103.09575)
- [data](https://console.cloud.google.com/storage/browser/rl_unplugged)
- [](https://github.com/deepmind/lab), [](https://github.com/google-research/realworldrl_suite#installation)
- [](https://youtu.be/n8yNYzbUMJ0)
| Scenic | Codebase with a focus on research around attention-based models for computer vision |
- [Mostafa Dehghani](https://www.mostafadehghani.com/)
- [Alexey Gritsenko](https://github.com/AlexeyG)
- [Anurag Arnab](https://github.com/anuragarnab)
- [Matthias Minderer](https://matthias.minderer.net/)
- [Yi Tay](https://vanzytay.github.io/)
- [](https://arxiv.org/abs/2110.11403)
- [](https://medium.com/syncedreview/google-open-sources-scenic-a-jax-library-for-rapid-computer-vision-model-prototyping-and-894dbdeddbae)
- [](https://www.reddit.com/r/deeplearning/comments/qgyjck/r_google_opensources_scenic_a_jax_library_for/)
| Text generation with RNN | This tutorial demonstrates how to generate text using a character-based RNN | [Billy Lamberta](https://github.com/lamberta) |
- [link](http://karpathy.github.io/2015/05/21/rnn-effectiveness/)
- [](https://paperswithcode.com/task/text-generation)
| CLIPDraw | Synthesize drawings to match a text prompt |
- [Kevin Frans](https://www.kvfrans.com/)
- [Lisa Soros](https://scholar.google.com/citations?user=iUkpvMUAAAAJ)
- [Olaf Witkowski](https://olafwitkowski.com/)
- [](https://arxiv.org/abs/2106.14843), [](https://arxiv.org/abs/1508.06576), [](https://arxiv.org/abs/2105.00162)
- [blog post](https://kvfrans.com/clipdraw-exploring-text-to-drawing-synthesis/)
- [](https://github.com/BachiLi/diffvg/blob/master/apps/painterly_rendering.py)
| CodeGen | Family of open-source model for program synthesis |
- [Erik Nijkamp](https://eriknijkamp.com/)
- [Bo Pang](https://scholar.google.com/citations?user=s9fNEVEAAAAJ)
- [Hiroaki Hayashi](https://hiroakih.me/)
- [Lifu Tu](https://lifu-tu.github.io/) others
- [Huan Wang](https://huan-december.github.io/)
- [Yingbo Zhou](https://scholar.google.com/citations?user=H_6RQ7oAAAAJ)
- [Silvio Savarese](https://cvgl.stanford.edu/silvio/)
- [Caiming Xiong](http://cmxiong.com/)
- [](https://arxiv.org/abs/2203.13474), [](https://arxiv.org/abs/2305.02309)
- [](https://github.com/salesforce/jaxformer)
- [](https://huggingface.co/models?search=salesforce+codegen)
| deep-significance | Easy-to-use package containing different significance tests and utility functions specifically tailored towards research needs and usability |
- [Dennis Ulmer](http://dennisulmer.eu/)
- [Christian Hardmeier](https://christianhardmeier.rax.ch/)
- [Jes Frellsen](https://frellsen.org/)
- [](https://arxiv.org/abs/2204.06815)
- [blog post](https://machinelearningmastery.com/statistical-hypothesis-tests/)
- [](https://deep-significance.readthedocs.io/en/latest/)
- [](https://github.com/rtmdrr/replicability-analysis-NLP), [](https://github.com/rtmdrr/testSignificanceNLP), [](https://github.com/rtmdrr/DeepComparison)
- [](https://en.wikipedia.org/wiki/Multiple_comparisons_problem)
| Text classification with RNN | This text classification tutorial trains a recurrent neural network on the IMDB large movie review dataset for sentiment analysis | [Billy Lamberta](https://github.com/lamberta) |
- [data](http://ai.stanford.edu/~amaas/data/sentiment/)
- [link](https://developers.google.com/machine-learning/glossary/#recurrent_neural_network)
- [](https://paperswithcode.com/task/text-classification)
| RLDS | Reinforcement Learning Datasets and it is an ecosystem of tools to store, retrieve and manipulate episodic data in the context of Sequential Decision Making including RL, Learning for Demonstrations, Offline RL or Imitation Learning |
- [Sabela Ramos](https://github.com/sabelaraga)
- [Sertan Girgin](https://sites.google.com/site/girgint/home)
- [Léonard Hussenot](https://leonardhussenot.github.io/)
- [Damien Vincent](https://www.linkedin.com/in/damien-vincent-1958381) others
- [Hanna Yakubovich](https://github.com/yakubanna)
- [Daniel Toyama](https://github.com/kenjitoyama)
- [Anita Gergely](https://www.linkedin.com/in/anita-g-318064b2/)
- [Piotr Stanczyk](https://scholar.google.com/citations?user=fKVK0dYAAAAJ)
- [Raphaël Marinier](https://github.com/RaphaelMarinier)
- [Jeremiah Harmsen](https://github.com/jharmsen)
- [Olivier Pietquin](https://research.google/people/105812/)
- [Nikola Momchev](https://scholar.google.com/citations?user=PbWgaswAAAAJ)
- [](https://arxiv.org/abs/2111.02767)
- [blog post](https://ai.googleblog.com/2021/12/rlds-ecosystem-to-generate-share-and.html)
- [](https://github.com/deepmind/envlogger), [](https://github.com/google-research/rlds-creator), [](https://github.com/Farama-Foundation/D4RL), [](https://github.com/deepmind/dm_env/blob/master/docs/index.md)
- [](http://www.tensorflow.org/datasets/catalog/overview), [](https://www.tensorflow.org/datasets/catalog/robosuite_panda_pick_place_can), [](https://www.tensorflow.org/datasets/catalog/locomotion), [](https://www.tensorflow.org/datasets/catalog/mt_opt), [](https://www.tensorflow.org/datasets/external_tfrecord?hl=en#load_dataset_with_tfds), [](https://www.tensorflow.org/api_docs/python/tf/data), [](https://www.tensorflow.org/guide/data_performance#optimize_performance), [](https://www.tensorflow.org/api_docs/python/tf/data/Dataset#shuffle), [](https://www.tensorflow.org/datasets/splits), [](https://www.tensorflow.org/datasets/api_docs/python/tfds/load)
| Real-Time Voice Cloning | SV2TTS with a vocoder that works in real-time |
- [Corentin Jemine](https://github.com/CorentinJ)
- [Erdene-Ochir Tuguldur](https://github.com/tugstugi)
- [](https://arxiv.org/abs/1806.04558), [](https://arxiv.org/abs/1802.08435), [](https://arxiv.org/abs/1703.10135), [](https://arxiv.org/abs/1710.10467)
- [](https://github.com/fatchord/WaveRNN), [](https://github.com/coqui-ai/tts), [](https://github.com/resemble-ai/Resemblyzer)
- [](https://youtu.be/-O_hYhToKoA)
| BLIP | VLP framework which transfers flexibly to both vision-language understanding and generation tasks |
- [Junnan Li](https://github.com/LiJunnan1992)
- [Dongxu Li](https://sites.google.com/view/dongxu-li/home)
- [Caiming Xiong](http://cmxiong.com/)
- [Steven Hoi](https://sites.google.com/view/stevenhoi)
- [](https://arxiv.org/abs/2201.12086)
- [blog post](https://blog.salesforceairesearch.com/blip-bootstrapping-language-image-pretraining/)
- [](https://github.com/facebookresearch/fairscale), [](https://github.com/salesforce/ALPRO), [](https://github.com/dmlc/decord), [](https://github.com/salesforce/ALBEF), [](https://github.com/rwightman/pytorch-image-models/tree/main/timm)
- [](https://youtu.be/X2k7n4FuI7c)
| VideoGPT | A conceptually simple architecture for scaling likelihood based generative modeling to natural videos |
- [Wilson Yan](https://wilson1yan.github.io/)
- [Yunzhi Zhang](https://zzyunzhi.github.io/)
- [Pieter Abbeel](https://people.eecs.berkeley.edu/~pabbeel/)
- [Aravind Srinivas](https://people.eecs.berkeley.edu/~aravind/)
- [](https://arxiv.org/abs/2104.10157), [](https://arxiv.org/abs/1904.10509)
- [data](https://www.crcv.ucf.edu/data/UCF101.php)
- [](https://huggingface.co/spaces/akhaliq/VideoGPT)
- [project](https://wilson1yan.github.io/videogpt/index.html)
| Silero Models | Pre-trained speech-to-text, text-to-speech and text-enhancement models made embarrassingly simple | [Silero team](https://www.silero.ai/about/) | [](https://github.com/snakers4/silero-models)
- [STT](https://thegradient.pub/towards-an-imagenet-moment-for-speech-to-text/), [STT](https://thegradient.pub/a-speech-to-text-practitioners-criticisms-of-industry-and-academia/), [STT](https://habr.com/ru/post/519562/)
- [TTS](https://habr.com/ru/post/660571/), [TTS](https://habr.com/ru/post/549482/)
- [Text Enhancement](https://habr.com/ru/post/581960/)
- [VAD](https://thegradient.pub/one-voice-detector-to-rule-them-all/), [VAD](https://habr.com/ru/post/537276/)
- [website](https://www.silero.ai/)
| ArcaneGAN | Process video in the style of the Arcane animated series | [Alexander Spirin](https://github.com/Sxela) | [](https://github.com/Sxela/ArcaneGAN)
- [](https://github.com/Sxela/stylegan3_blending)
- [](https://youtu.be/Fi199uFW6jE), [](https://youtu.be/AJG4X7IokG8)
| textlesslib | A library aimed to facilitate research in Textless NLP |
- [Eugene Kharitonov](https://eugene-kharitonov.github.io/)
- [Jade Copet](https://scholar.google.com/citations?user=GRMLwjAAAAAJ)
- [Kushal Lakhotia](https://about.me/hikushalhere)
- [Nguyễn Tú Anh](https://tuanh208.github.io/) others
- [Paden Tomasello](https://scholar.google.com/citations?user=sBtWMGYAAAAJ)
- [Ann Lee](https://ai.facebook.com/people/ann-lee)
- [Ali Elkahky](https://scholar.google.com/citations?user=KB3S8RoAAAAJ)
- [Wei-Ning Hsu](https://wnhsu.github.io/)
- [Abdelrahman Mohamed](https://ai.facebook.com/people/abdelrahman-mohamed/)
- [Emmanuel Dupoux](http://www.lscp.net/persons/dupoux/)
- [Yossi Adi](https://www.cs.huji.ac.il/~adiyoss/)
- [](https://arxiv.org/abs/2202.07359)
- [](https://github.com/NVIDIA/waveglow), [](https://github.com/keithito/tacotron), [](https://github.com/NVIDIA/tacotron2), [](https://github.com/pseeth/torch-stft)
- [](https://paperswithcode.com/dataset/librispeech)
| AV-HuBERT | Self-supervised representation learning framework for audio-visual speech |
- [Bowen Shi](https://home.ttic.edu/~bshi/)
- [Wei-Ning Hsu](http://people.csail.mit.edu/wnhsu/)
- [Kushal Lakhotia](https://about.me/hikushalhere)
- [Abdelrahman Mohamed](http://www.cs.toronto.edu/~asamir/)
- [](https://arxiv.org/abs/2201.02184), [](https://arxiv.org/abs/2201.01763), [](https://arxiv.org/abs/1810.04805), [](https://arxiv.org/abs/1911.04890)
- [blog post](https://ai.facebook.com/blog/ai-that-understands-speech-by-looking-as-well-as-hearing/)
| Lingvo | Framework for building neural networks in Tensorflow, particularly sequence models |
- [Jonathan Shen](https://github.com/jonathanasdf)
- [Patrick Nguyen](https://scholar.google.com/citations?user=38fqeIYAAAAJ)
- [Yonghui Wu](https://scholar.google.com/citations?user=55FnA9wAAAAJ)
- [Zhifeng Chen](https://github.com/zffchen78)
- [](https://arxiv.org/abs/1902.08295), [](https://arxiv.org/abs/1508.01211), [](https://arxiv.org/abs/1412.1602), [](https://arxiv.org/abs/1602.02410), [](https://arxiv.org/abs/2006.16668), [](https://arxiv.org/abs/2106.04060)
- [](https://github.com/tensorflow/lingvo/blob/master/docker/dev.Dockerfile), [](https://github.com/tensorflow/lingvo/blob/master/docker/lib.dockerfile)
- [](https://tensorflow.github.io/lingvo/)
| RuDOLPH | A fast and light text-image-text transformer designed for a quick and easy fine-tuning setup for the solution of various tasks: from generating images by text description and image classification to visual question answering and more |
- [Alex Shonenkov](https://github.com/shonenkov)
- [Misha Konstantinov](https://github.com/zeroshot-ai)
- [](https://arxiv.org/abs/2005.14165), [](https://arxiv.org/abs/2102.12092), [](https://arxiv.org/abs/2103.00020)
| DeepDream | This tutorial contains a minimal implementation of DeepDream: an experiment that visualizes the patterns learned by a neural network | [Billy Lamberta](https://github.com/lamberta) |
- [](https://arxiv.org/abs/1409.4842)
- [blog post](https://ai.googleblog.com/2015/06/inceptionism-going-deeper-into-neural.html)
- [](https://en.wikipedia.org/wiki/Inception)
| MLP | The most basic neural network architectures, a multilayer perceptron, also known as a feedforward network | [Ben Trevett](https://bentrevett.com/) |
- [NN and DL](http://neuralnetworksanddeeplearning.com/)
- [](https://arxiv.org/abs/1702.03118), [](https://arxiv.org/abs/2108.12943), [](https://arxiv.org/abs/2111.04020)
- [optimization](https://ruder.io/optimizing-gradient-descent/)
- [](https://pytorch.org/vision/stable/transforms.html#transforms-on-pil-image-only), [](https://pytorch.org/vision/stable/transforms.html#transforms-on-torch-tensor-only)
- [](https://en.wikipedia.org/wiki/Multilayer_perceptron)
| AlexNet | A neural network model that uses convolutional neural network layers and was designed for the ImageNet challenge | [Ben Trevett](https://bentrevett.com/) | [](https://github.com/davidtvs/pytorch-lr-finder)
- [ILSVRC](https://image-net.org/challenges/LSVRC/)
- [LR](https://sgugger.github.io/how-do-you-find-a-good-learning-rate.html)
- [PMLR](https://proceedings.mlr.press/v9/glorot10a.html)
- [](https://arxiv.org/abs/1409.0575)
- [cifar-10](https://www.cs.toronto.edu/~kriz/cifar.html)
- [dropout](https://sebastianraschka.com/faq/docs/dropout-activation.html)
- [](https://papers.nips.cc/paper/2012/hash/c399862d3b9d6b76c8436e924a68c45b-Abstract.html)
- [](https://pytorch.org/vision/stable/models.html)
- [](https://paperswithcode.com/method/alexnet)
- [](https://en.wikipedia.org/wiki/Regularization_(mathematics)), [](https://en.wikipedia.org/wiki/AlexNet)
| VGG | Very Deep Convolutional Networks for Large-Scale Image Recognition | [Ben Trevett](https://bentrevett.com/) | [](https://github.com/pytorch/vision/blob/main/torchvision/models/vgg.py#L47)
- [ILSVRC](https://image-net.org/challenges/LSVRC/)
- [](https://arxiv.org/abs/1409.1556), [](https://arxiv.org/abs/1506.01186), [](https://arxiv.org/abs/1801.06146), [](https://arxiv.org/abs/1502.03167), [](https://arxiv.org/abs/1805.11604)
- [cifar-10](https://www.cs.toronto.edu/~kriz/cifar.html)
- [](https://pytorch.org/vision/stable/models.html)
- [](https://paperswithcode.com/method/vgg)
- [](https://youtu.be/HR0lt1hlR6U?t=5900), [](https://youtu.be/j1jIoHN3m0s), [](https://youtu.be/RNnKtNrsrmg)
| LeNet | A neural network model that uses convolutional neural network layers and was designed for classifying handwritten characters | [Ben Trevett](https://bentrevett.com/) |
- [CNN](https://cs231n.github.io/convolutional-networks/)
- [LeNet-5](http://yann.lecun.com/exdb/lenet/)
- [guide](https://adeshpande3.github.io/A-Beginner%27s-Guide-To-Understanding-Convolutional-Neural-Networks/)
- [paper](http://yann.lecun.com/exdb/publis/pdf/lecun-01a.pdf)
- [](https://paperswithcode.com/method/lenet)
- [](https://en.wikipedia.org/wiki/Convolution), [](https://en.wikipedia.org/wiki/Sobel_operator), [](https://en.wikipedia.org/wiki/Gaussian_blur)
| Music Composer | Synthesizing symbolic music in MIDI format using the Music Transformer model | [bazanovvanya](https://github.com/bazanovvanya) | [](https://github.com/ai-forever/music-composer)
- [](https://arxiv.org/abs/1909.05858)
- [blog post](https://habr.com/ru/company/sberbank/blog/583592/)
- [data](https://magenta.tensorflow.org/datasets/maestro), [data](https://colinraffel.com//projects/lmd/)
- [](https://github.com/gwinndr/MusicTransformer-Pytorch), [](https://github.com/bytedance/GiantMIDI-Piano), [](https://github.com/mdeff/fma)
| FLAML | Lightweight Python library that finds accurate machine learning models automatically, efficiently and economically |
- [Chi Wang](https://github.com/sonichi)
- [Qingyun Wu](https://qingyun-wu.github.io/)
- [](https://arxiv.org/abs/2106.04815), [](https://arxiv.org/abs/2005.01571)
- [](https://microsoft.github.io/FLAML/)
- [paper](https://www.microsoft.com/en-us/research/publication/flaml-a-fast-and-lightweight-automl-library/)
- [](https://www.youtube.com/channel/UCfU0zfFXHXdAd5x-WvFBk5A), [](https://youtu.be/euXpDYGgkGM)
| CompilerGym | A reinforcement learning toolkit for compiler optimizations |
- [Chris Cummins](https://chriscummins.cc/)
- [Bram Wasti](https://github.com/bwasti)
- [Jiadong Guo](https://jd-eth.github.io/)
- [Brandon Cui](https://www.linkedin.com/in/bcui19/) others
- [Jason Ansel](https://jasonansel.com/)
- [Sahir Gomez](https://github.com/sahirgomez1)
- [Olivier Teytaud](https://github.com/teytaud)
- [Benoit Steiner](http://bsteiner.info/)
- [Yuandong Tian](http://yuandong-tian.com/)
- [Hugh Leather](https://github.com/hughleat)
- [](https://arxiv.org/abs/2109.08267)
- [](https://facebookresearch.github.io/CompilerGym/)
| Reformer | Performs on par with Transformer models while being much more memory-efficient and much faster on long sequences |
- [Phil Wang](https://lucidrains.github.io/)
- [Nikita Kitaev](https://kitaev.com/)
- [Łukasz Kaiser](https://scholar.google.com/citations?user=JWmiQR0AAAAJ)
- [Anselm Levskaya](https://anselmlevskaya.com/)
- [](https://arxiv.org/abs/2001.04451), [](https://arxiv.org/abs/1907.01470), [](https://arxiv.org/abs/1910.05895), [](https://arxiv.org/abs/1909.11556), [](https://arxiv.org/abs/1911.02150), [](https://arxiv.org/abs/2002.05202), [](https://arxiv.org/abs/2003.05997), [](https://arxiv.org/abs/2003.04887), [](https://arxiv.org/abs/2002.07028), [](https://arxiv.org/abs/2103.03404), [](https://arxiv.org/abs/2104.09864)
- [blog post](https://ai.googleblog.com/2020/01/reformer-efficient-transformer.html)
- [](https://github.com/lucidrains/routing-transformer), [](https://github.com/lucidrains/sinkhorn-transformer), [](https://github.com/lucidrains/performer-pytorch), [](https://github.com/lucidrains/linear-attention-transformer/), [](https://github.com/lucidrains/compressive-transformer-pytorch)
- [](https://proceedings.neurips.cc/paper/2019/hash/9d8df73a3cfbf3c5b47bc9b50f214aff-Abstract.html), [](https://proceedings.neurips.cc/paper_files/paper/2017/hash/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html)
- [](https://youtu.be/i4H0kjxrias), [](https://youtu.be/Kf3x3lqf9cQ), [](https://youtu.be/0eTULzrOztQ)
| ruDALL·E | Generate images from texts in Russian | [Alex Shonenkov](https://github.com/shonenkov) | [](https://github.com/ai-forever/ru-dalle)
- [](https://github.com/bes-dev/vqvae_dwt_distiller.pytorch), [](https://github.com/boomb0om/Real-ESRGAN-colab)
- [](https://huggingface.co/spaces/multimodalart/rudalle)
- [project](https://rudalle.ru/)
| DeepStyle | The Neural Style algorithm synthesizes a pastiche by separating and combining the content of one image with the style of another image using convolutional neural networks |
- [Cameron Smith](https://github.com/cysmith)
- [Alexander Spirin](https://github.com/Sxela)
- [](https://arxiv.org/abs/1604.08610), [](https://arxiv.org/abs/1606.05897), [](https://arxiv.org/abs/1508.06576)
- [cvpr](https://www.cv-foundation.org/openaccess/content_cvpr_2016/papers/Gatys_Image_Style_Transfer_CVPR_2016_paper.pdf)
- [](https://en.wikipedia.org/wiki/Pastiche), [](https://en.wikipedia.org/wiki/The_Starry_Night), [](https://en.wikipedia.org/wiki/YUV), [](https://en.wikipedia.org/wiki/Lab_color_space), [](https://en.wikipedia.org/wiki/YCbCr), [](https://en.wikipedia.org/wiki/CIELUV), [](https://en.wikipedia.org/wiki/Pareidolia)
| Text2Animation | Generate images from text phrases with VQGAN and CLIP with animation and keyframes |
- [Katherine Crowson](https://kath.io/)
- [Ryan Murdock](https://twitter.com/advadnoun)
- [Chigozie Nri](https://github.com/chigozienri)
- [Denis Malimonov](https://github.com/tg-bomze)
- [](https://arxiv.org/abs/2012.09841), [](https://arxiv.org/abs/2103.00020)
- [](https://www.youtube.com/channel/UCToztRy9FSTIhEen_1x4FAw)
| EfficientNetV2 | A family of image classification models, which achieve better parameter efficiency and faster training speed than prior arts |
- [Mingxing Tan](https://scholar.google.com/citations?user=6POeyBoAAAAJ)
- [Quoc Le](https://cs.stanford.edu/~quocle/)
- [](https://arxiv.org/abs/2104.00298), [](https://arxiv.org/abs/1905.11946)
- [](https://github.com/NVIDIA/TensorRT/tree/master/samples/python/efficientnet)
| Droidlet | A modular embodied agent architecture and platform for building embodied agents |
- [Anurag Pratik](https://github.com/anuragprat1k)
- [Soumith Chintala](https://soumith.ch/)
- [Kavya Srinet](https://github.com/kavyasrinet)
- [Dhiraj Gandhi](https://dhiraj100892.github.io/) others
- [Rebecca Qian](https://github.com/Rebecca-Qian)
- [Yuxuan Sun](https://github.com/snyxan)
- [Ryan Drew](https://rdrew.dev/)
- [Sara Elkafrawy](https://github.com/saraEbrahim)
- [Anoushka Tiwari](https://www.linkedin.com/in/anoushka-tiwari)
- [Tucker Hart](https://www.linkedin.com/in/tucker-hart-05a638133)
- [Mary Williamson](https://scholar.google.com/citations?user=Ys4xB-QAAAAJ)
- [Abhinav Gupta](http://www.cs.cmu.edu/~abhinavg/)
- [Arthur Szlam](https://scholar.google.com/citations?user=u3-FxUgAAAAJ)
- [](https://arxiv.org/abs/2101.10384), [](https://arxiv.org/abs/1907.08584)
- [](https://facebookresearch.github.io/droidlet/)
| GPT-J-6B | A 6 billion parameter, autoregressive text generation model trained on The Pile |
- [Ben Wang](https://benwang.dev/)
- [Aran Komatsuzaki](https://arankomatsuzaki.wordpress.com/about-me/)
- [Janko Prester](https://www.jankoprester.com/)
- [The Pile](https://pile.eleuther.ai/)
- [blog post](https://arankomatsuzaki.wordpress.com/2021/06/04/gpt-j/)
- [](https://github.com/EleutherAI/gpt-neox), [](https://github.com/microsoft/DeepSpeed)
- [web demo](https://6b.eleuther.ai/)
| Machine learning course | This course is broad and shallow, but author will provide additional links so that you can deepen your understanding of the ML method you need | [Тимчишин Віталій](https://github.com/fbeilstein) | [](https://github.com/fbeilstein/machine_learning)
- [blog post](https://vas3k.com/blog/machine_learning/)
- [](https://www.youtube.com/playlist?list=PLkDeTjsoxDVgnb2lIYo9-1l4XYhrIyS6A), [](https://youtu.be/-RdOwhmqP5s), [](https://youtu.be/R13BD8qKeTg), [](https://youtu.be/ZkjP5RJLQF4), [](https://youtu.be/J4Wdy0Wc_xQ), [](https://youtu.be/mBcLRGuAFUk), [](https://youtu.be/YIGtalP1mv0), [](https://youtu.be/Yz5pySyEtsU), [](https://youtu.be/x5zLaWT5KPs), [](https://youtu.be/yBwpo-L80Mc), [](https://www.youtube.com/playlist?list=PL3FW7Lu3i5JvHM8ljYj-zLfQRF3EO8sYv)
| Lucid Sonic Dreams | Syncs GAN-generated visuals to music | [Mikael Alafriz](https://github.com/mikaelalafriz) | [](https://github.com/mikaelalafriz/lucid-sonic-dreams)
- [](https://github.com/NVlabs/stylegan2), [](https://github.com/justinpinkney/awesome-pretrained-stylegan2)
- [](https://towardsdatascience.com/introducing-lucid-sonic-dreams-sync-gan-art-to-music-with-a-few-lines-of-python-code-b04f88722de1)
- [](https://youtu.be/l-nGC-ve7sI)
| textgenrnn | Generate text using a pretrained neural network with a few lines of code, or easily train your own text-generating neural network of any size and complexity | [Max Woolf](https://minimaxir.com/) | [](https://github.com/minimaxir/textgenrnn)
- [blog post](http://minimaxir.com/2018/05/text-neural-networks/)
- [](https://www.youtube.com/watch?v=RW7mP6BfZuY)
| TensorRT | SDK for high-performance deep learning inference, includes a deep learning inference optimizer and runtime that delivers low latency and high throughput for inference applications | [nvidia](https://developer.nvidia.com/) | [](https://github.com/NVIDIA/TensorRT)
- [blog post](https://developer.nvidia.com/blog/speeding-up-deep-learning-inference-using-tensorrt-updated/)
- [](https://docs.nvidia.com/deeplearning/tensorrt/)
- [forum](https://forums.developer.nvidia.com/c/ai-data-science/deep-learning/tensorrt)
- [website](https://developer.nvidia.com/tensorrt)
- [](https://youtu.be/TU5BMU6iYZ0), [](https://youtu.be/6rZNLaS775w), [](https://youtu.be/G_KhUFCUSsY), [](https://youtu.be/7kJ-jph9gCw)
| BasicSR | Open Source Image and Video Restoration Toolbox for Super-resolution, Denoise, Deblurring, etc. |
- [Xintao Wang](https://xinntao.github.io/)
- [Liangbin Xie](https://liangbinxie.github.io/)
- [Ke Yu](https://github.com/yuke93)
- [Kelvin Chan](https://ckkelvinchan.github.io/) others
- [Chen Change Loy](https://www.mmlab-ntu.com/person/ccloy/)
- [Chao Dong](https://scholar.google.com/citations?user=OSDCB0UAAAAJ)
- [](https://arxiv.org/abs/2012.02181)
- [](https://basicsr.readthedocs.io/en/latest/)
- [](https://github.com/xinntao/ESRGAN), [](https://github.com/xindongzhang/ECBSR), [](https://github.com/Lotayou/Face-Renovation), [](https://github.com/csxmli2016/DFDNet), [](https://github.com/rosinality/stylegan2-pytorch), [](https://github.com/xinntao/facexlib), [](https://github.com/xinntao/HandyView), [](https://github.com/xinntao/HandyFigure), [](https://github.com/xinntao/SFTGAN), [](https://github.com/xinntao/DNI), [](https://github.com/xinntao/HandyCrawler), [](https://github.com/xinntao/HandyWriting)
- [](https://youtu.be/KaMYsxWkmww)
| Hyperopt | Python library for serial and parallel optimization over awkward search spaces, which may include real-valued, discrete, and conditional dimensions |
- [James Bergstra](https://github.com/jaberg)
- [Dan Yamins](https://github.com/yamins81)
- [David Cox](https://scholar.google.com/citations?user=6S-WgLkAAAAJ)
- [ICML](https://proceedings.mlr.press/v28/bergstra13.html)
- [](http://hyperopt.github.io/hyperopt/)
- [](https://github.com/hyperopt/hyperopt-sklearn), [](https://github.com/hyperopt/hyperopt-nnet), [](https://github.com/hyperopt/hyperopt-nnet), [](https://github.com/hyperopt/hyperopt-convnet), [](https://github.com/hyperopt/hyperopt-gpsmbo)
- [](https://papers.nips.cc/paper/2011/hash/86e8f7ab32cfd12577bc2619bc635690-Abstract.html)
- [](https://youtu.be/Mp1xnPfE4PY), [](https://youtu.be/tdwgR1AqQ8Y), [](https://youtu.be/tteE_Vtmrv4)
| CNN | This tutorial demonstrates training a simple Convolutional Neural Network to classify CIFAR images | [Billy Lamberta](https://github.com/lamberta) |
- [cifar](https://www.cs.toronto.edu/~kriz/cifar.html)
- [link](https://developers.google.com/machine-learning/glossary/#convolutional_neural_network)
| Custom GPT-2 + Tokenizer | Train a custom GPT-2 model for free on a GPU using aitextgen! | [Max Woolf](https://minimaxir.com/) | [](https://github.com/minimaxir/aitextgen)
- [data](https://raw.githubusercontent.com/karpathy/char-rnn/master/data/tinyshakespeare/input.txt)
- [](https://docs.aitextgen.io/)
| Train a GPT-2 Text-Generating Model | Retrain an advanced text generating neural network on any text dataset for free on a GPU using Colaboratory using aitextgen! | [Max Woolf](https://minimaxir.com/) | [](https://github.com/minimaxir/aitextgen)
- [data](https://raw.githubusercontent.com/karpathy/char-rnn/master/data/tinyshakespeare/input.txt)
- [](https://docs.aitextgen.io/)
- [](https://paperswithcode.com/task/text-generation)
| EasyNMT | Easy to use, state-of-the-art machine translation for more than 100+ languages | [Nils Reimers](https://www.nils-reimers.de/) | [](https://github.com/UKPLab/EasyNMT)
- [](https://arxiv.org/abs/2008.00401), [](https://arxiv.org/abs/2010.11125)
- [demo](http://easynmt.net/demo/)
- [](https://github.com/Helsinki-NLP/Opus-MT), [](https://github.com/facebookresearch/fairseq/tree/main/examples/multilingual)
| SkinDeep | Remove Body Tattoo Using Deep Learning | [Vijish Madhavan](https://github.com/vijishmadhavan) | [](https://github.com/vijishmadhavan/SkinDeep)
- [](https://arxiv.org/abs/1805.08318), [](https://arxiv.org/abs/1710.10196), [](https://arxiv.org/abs/1707.02921), [](https://arxiv.org/abs/1603.08155)
- [](https://github.com/jantic/DeOldify)
| PaddleHub | Pre-trained models toolkit based on PaddlePaddle: 400+ models including Image, Text, Audio, Video and Cross-Modal with Easy Inference & Serving |
- [Zeyu Chen](https://github.com/ZeyuChen)
- [Zewu Wu](https://github.com/nepeplwu)
- [Bin Long](https://github.com/sjtubinlong)
- [Xuefei Zhang](https://github.com/Steffy-zxf) others
- [Jinxuan Qiu](https://github.com/kinghuin)
- [Yuhan Shen](https://github.com/ShenYuhan)
- [Yuying Hao](https://github.com/haoyuying)
- [Xiaojie Chen](https://github.com/KPatr1ck)
- [](https://paddlehub.readthedocs.io/en)
- [](https://github.com/PaddlePaddle/PaddleOCR), [](https://github.com/PaddlePaddle/PaddleDetection), [](https://github.com/PaddlePaddle/PaddleGAN), [](https://github.com/CMU-Perceptual-Computing-Lab/openpose), [](https://github.com/PaddlePaddle/PaddleSeg), [](https://github.com/PaddlePaddle/PaddleClas), [](https://github.com/PaddlePaddle/ERNIE), [](https://github.com/baidu/LAC), [](https://github.com/baidu/DDParser), [](https://github.com/PaddlePaddle/PaddleSpeech)
- [](https://huggingface.co/PaddlePaddle)
- [](https://medium.com/analytics-vidhya/paddlehub-fdd1ec75a07b)
- [website](https://www.paddlepaddle.org.cn/en)
- [](https://youtu.be/9adXuF_lTSg)
| OCTIS | Framework for training, analyzing, and comparing Topic Models, whose optimal hyper-parameters are estimated using a Bayesian Optimization approach |
- [Silvia Terragni](https://silviatti.github.io/)
- [Elisabetta Fersini](https://www.unimib.it/elisabetta-fersini)
- [Antonio Candelieri](https://www.unimib.it/antonio-candelieri)
- [Pietro Tropeano](https://github.com/pietrotrope) others
- [Bruno Galuzzi](https://github.com/brunoG89)
- [Lorenzo Famiglini](https://github.com/lorenzofamiglini)
- [Davide Pietrasanta](https://github.com/davidepietrasanta)
- [](https://arxiv.org/abs/1703.01488)
- [data](https://www.dbpedia.org/resources/ontology/), [data](https://www.statmt.org/europarl/)
- [](https://github.com/estebandito22/PyTorchAVITM)
- [](https://towardsdatascience.com/a-beginners-guide-to-octis-optimizing-and-comparing-topic-models-is-simple-590554ec9ba6), [](https://towardsdatascience.com/a-beginners-guide-to-octis-vol-2-optimizing-topic-models-1214e58be1e5)
- [](https://papers.nips.cc/paper/2000/hash/f9d1152547c0bde01830b7e8bd60024c-Abstract.html)
- [paper](https://aclanthology.org/2021.eacl-demos.31/)
- [](https://paperswithcode.com/dataset/20-newsgroups)
- [](https://youtu.be/nPmiWBFFJ8E)
| PyTorchVideo | Deeplearning library with a focus on video understanding work |
- [Haoqi Fan](https://haoqifan.github.io/)
- [Tullie Murrell](https://github.com/tullie)
- [Heng Wang](https://hengcv.github.io/)
- [Kalyan Vasudev Alwala](https://github.com/kalyanvasudev) others
- [Yanghao Li](https://github.com/lyttonhao)
- [Yilei Li](https://liyilui.github.io/personal_page/)
- [Bo Xiong](https://github.com/bxiong1202)
- [Nikhila Ravi](https://nikhilaravi.com/)
- [Meng Li](https://mengli.me/)
- [Haichuan Yang](https://hyang1990.github.io/)
- [Jitendra Malik](https://scholar.google.com/citations?user=oY9R5YQAAAAJ)
- [Ross Girshick](https://github.com/rbgirshick)
- [Matt Feiszli](https://scholar.google.com/citations?user=A-wA73gAAAAJ)
- [Aaron Adcock](https://scholar.google.com/citations?&user=oa78zHUAAAAJ)
- [Wan-Yen Lo](https://github.com/wanyenlo)
- [Christoph Feichtenhofer](http://feichtenhofer.github.io/)
- [](https://arxiv.org/abs/2111.09887), [](https://arxiv.org/abs/2104.11227)
- [blog post](https://ai.facebook.com/blog/pytorchvideo-a-deep-learning-library-for-video-understanding/)
- [](https://pytorchvideo.readthedocs.io/en/latest/index.html)
- [website](https://github.com/facebookresearch/pytorchvideo)
- [](https://youtu.be/b7-gnpqz9Qg)
| GPT Neo | An implementation of model & data parallel GPT2 & GPT3 -like models, with the ability to scale up to full GPT3 sizes (and possibly more!), using the mesh-tensorflow library | [EleutherAI](https://www.eleuther.ai/) | [](https://github.com/EleutherAI/gpt-neo)
- [GPT-2](https://openai.com/blog/better-language-models/)
- [](https://arxiv.org/abs/2005.14165), [](https://arxiv.org/abs/2004.05150), [](https://arxiv.org/abs/1701.06538)
- [](https://github.com/tensorflow/mesh), [](https://github.com/EleutherAI/gpt-neox/)
- [pretrained](https://the-eye.eu/public/AI/gptneo-release/)
| CVAE | This notebook demonstrates how train a Variational Autoencoder on the MNIST dataset | [Billy Lamberta](https://github.com/lamberta) |
- [](https://arxiv.org/abs/1312.6114), [](https://arxiv.org/abs/1401.4082)
| Big Sleep | Text to image generation, using OpenAI's CLIP and a BigGAN | [Phil Wang](https://lucidrains.github.io/) | [](https://github.com/lucidrains/big-sleep)
- [](https://arxiv.org/abs/2103.00020), [](https://arxiv.org/abs/1809.11096)
- [](https://github.com/openai/CLIP)
- [](https://www.reddit.com/r/bigsleep/comments/lxawb4/how_to_use_some_of_the_newer_features_of/), [](https://www.reddit.com/r/bigsleep/)
| Deep Daze | Text to image generation using OpenAI's CLIP and Siren | [Phil Wang](https://lucidrains.github.io/) | [](https://github.com/lucidrains/deep-daze)
- [](https://arxiv.org/abs/2103.00020), [](https://arxiv.org/abs/2006.09661)
- [](https://github.com/openai/CLIP)
- [](https://www.reddit.com/r/deepdaze/)
| DCGAN | This tutorial demonstrates how to generate images of handwritten digits using a Deep Convolutional Generative Adversarial Network | [Billy Lamberta](https://github.com/lamberta) |
- [](https://arxiv.org/abs/1511.06434), [](https://arxiv.org/abs/1701.00160)
- [](https://www.kaggle.com/jessicali9530/celeba-dataset)
| Adversarial FGSM | This tutorial creates an adversarial example using the Fast Gradient Signed Method attack. This was one of the first and most popular attacks to fool a neural network. | [Billy Lamberta](https://github.com/lamberta) |
- [](https://arxiv.org/abs/1412.6572)
- [imagenet](http://www.image-net.org/)
- [](https://www.tensorflow.org/versions/r2.0/api_docs/python/tf/keras/applications/MobileNetV2)
| GAN steerability | We will navigate in GAN latent space to simulate various camera transformations |
- [Ali Jahanian](http://people.csail.mit.edu/jahanian/)
- [Lucy Chai](http://people.csail.mit.edu/lrchai/)
- [Phillip Isola](http://web.mit.edu/phillipi/)
- [](https://arxiv.org/abs/1907.07171), [](https://arxiv.org/abs/1809.11096)
- [project](https://ali-design.github.io/gan_steerability/)
- [](https://youtu.be/nS0V64sF7Cw)
| Trax | End-to-end library for deep learning that focuses on clear code and speed | [Google](https://research.google/teams/brain/) | [](https://github.com/google/trax)
- [](https://arxiv.org/abs/1910.00177)
- [discuss](https://groups.google.com/u/1/g/trax-discuss)
- [](https://trax-ml.readthedocs.io/en/latest/)
- [](https://www.kaggle.com/abhinavwalia95/entity-annotated-corpus), [](https://www.kaggle.com/code/dschettler8845/exploration-of-trax-framework)
- [](https://towardsdatascience.com/get-started-with-google-trax-for-nlp-ff8dcd3119cf), [](https://medium.com/analytics-vidhya/brief-view-of-googles-trax-library-b78eae008cb6)
- [](https://www.tensorflow.org/datasets/catalog/overview), [](https://tensorflow.org/guide/tf_numpy)
- [](https://youtu.be/qlTsaHAtJBY)
| bsuite | A collection of carefully-designed experiments that investigate core capabilities of an RL agent with two main objectives |
- [Ian Osband](http://iosband.github.io/)
- [Yotam Doron](http://www.yotamdoron.com/)
- [Matteo Hessel](https://github.com/mtthss)
- [John Aslanides](https://www.aslanides.io/) others
- [Eren Sezener](http://erensezener.com/)
- [Andre Saraiva](https://andresnds.wordpress.com/)
- [Katrina McKinney](https://medium.com/@katrinamckinney)
- [Tor Lattimore](http://tor-lattimore.com/)
- [Csaba Szepesvari](https://sites.ualberta.ca/~szepesva/)
- [Satinder Singh](http://web.eecs.umich.edu/~baveja/)
- [Benjamin Van Roy](https://web.stanford.edu/~bvr/)
- [Richard Sutton](http://www.incompleteideas.net/)
- [David Silver](https://www.davidsilver.uk/)
- [Hado Van Hasselt](https://hadovanhasselt.com/)
- [](https://github.com/openai/gym)
- [paper](https://openreview.net/forum?id=rygf-kSYwH)
- [](https://youtu.be/Wcv4eU_qtZU)
| TF-Ranking | End-to-end walkthrough of training a TensorFlow Ranking neural network model which incorporates sparse textual features | [Rama Kumar](https://github.com/ramakumar1729) | [](https://github.com/tensorflow/ranking)
- [](https://arxiv.org/abs/1910.09676), [](https://arxiv.org/abs/1812.00073), [](https://arxiv.org/abs/1905.08957), [](https://arxiv.org/abs/1811.04415)
- [data](http://hamedz.ir/resources/)
- [](https://github.com/tensorflow/serving/blob/master/tensorflow_serving/apis/input.proto#L72)
- [](https://en.wikipedia.org/wiki/Mean_reciprocal_rank), [](https://en.wikipedia.org/wiki/Discounted_cumulative_gain)
| Toon-Me | A fun project to toon portrait images | [Vijish Madhavan](https://github.com/vijishmadhavan) | [](https://github.com/vijishmadhavan/Toon-Me)
- [](https://arxiv.org/abs/1710.10196), [](https://arxiv.org/abs/1707.02921), [](https://arxiv.org/abs/1603.08155)
| TensorNetwork | A library for easy and efficient manipulation of tensor networks | [Chase Roberts](http://thenerdstation.github.io/) | [](https://github.com/google/TensorNetwork)
- [](https://arxiv.org/abs/1708.00006), [](https://arxiv.org/abs/1306.2164)
- [](https://tensornetwork.readthedocs.io/)
- [](https://www.youtube.com/watch?v=YN2YBB0viKo)
| Spleeter | Deezer source separation library including pretrained models |
- [Romain Hennequin](http://romain-hennequin.fr/)
- [Anis Khlif](https://github.com/alreadytaikeune)
- [Félix Voituret](https://github.com/Faylixe)
- [Manuel Moussallam](https://mmoussallam.github.io/)
- [blog post](https://deezer.io/releasing-spleeter-deezer-r-d-source-separation-engine-2b88985e797e)
- [data](https://sigsep.github.io/datasets/musdb.html)
- [project](https://research.deezer.com/projects/spleeter.html)
| Bullet Physics SDK | Real-time collision detection and multi-physics simulation for VR, games, visual effects, robotics, machine learning etc |
- [Erwin Coumans](https://github.com/erwincoumans)
- [Yunfei Bai](https://github.com/YunfeiBai)
- [](https://docs.google.com/document/d/10sXEhzFRSnvFcl3XxNGhnD4N2SedqwdAvK3dsihxVUA/edit#heading=h.2ye70wns7io3)
- [](https://github.com/Microsoft/vcpkg)
- [website](https://pybullet.org)
- [](https://www.youtube.com/playlist?list=PLinBNdD-7nkNCfoEKap4z3qadLVj8QB4a), [](https://youtu.be/9p0O941opGc), [](https://youtu.be/kZxPaGdoSJY), [](https://www.youtube.com/playlist?list=PL9LUFPiB6N3YrS0O7XM_1sBVWRnSRB643)
| Person Remover | Project that combines Pix2Pix and YOLO arhitectures in order to remove people or other objects from photos |
- [Javier Gamazo](https://www.javiergamazo.com/)
- [Daryl Autar](https://github.com/Daryl149)
- [](https://github.com/javirk/Person-remover-partial-convolutions), [](https://github.com/zzh8829/yolov3-tf2)
- [](https://www.youtube.com/watch?v=_dRjY9gMcxE)
| Semantic Segmentation | Pytorch implementation for Semantic Segmentation/Scene Parsing on MIT ADE20K dataset |
- [Bolei Zhou](https://boleizhou.github.io/)
- [Hang Zhao](https://hangzhaomit.github.io/)
- [Xavier Puig](https://people.csail.mit.edu/xavierpuig/)
- [Sanja Fidler](http://www.cs.toronto.edu/~fidler/index.html)
- [Antonio Torralba](https://groups.csail.mit.edu/vision/torralbalab/)
- [](https://arxiv.org/abs/1608.05442), [](https://arxiv.org/abs/1612.01105), [](https://arxiv.org/abs/1807.10221), [](https://arxiv.org/abs/1904.04514)
- [](https://github.com/CSAILVision/sceneparsing), [](https://github.com/vacancy/Synchronized-BatchNorm-PyTorch), [](https://github.com/hszhao/semseg)
- [project](http://sceneparsing.csail.mit.edu/)
| Gin Config | Lightweight configuration framework for Python, based on dependency injection |
- [Dan Holtmann-Rice](https://github.com/dhr)
- [Sergio Guadarrama](https://github.com/sguada)
- [Nathan Silberman](http://nsilberman.com/)
- [](https://towardsdatascience.com/stop-worrying-about-configs-with-gin-218562dd5c91)
| Dopamine | Research framework for fast prototyping of reinforcement learning algorithms |
- [Pablo Castro](https://psc-g.github.io/)
- [Subhodeep Moitra](http://www.deepmoitra.com/)
- [Carles Gelada](https://github.com/cgel)
- [Saurabh Kumar](https://scholar.google.com/citations?user=Rkr2uT8AAAAJ)
- [Marc Bellemare](http://www.marcgbellemare.info/)
- [](https://arxiv.org/abs/1812.06110), [](https://arxiv.org/abs/1511.05952), [](https://arxiv.org/abs/1812.05905), [](https://arxiv.org/abs/1806.06923)
- [baselines](https://google.github.io/dopamine/baselines/)
- [blog post](https://opensource.googleblog.com/2019/02/dopamine-2.0.html)
- [](https://google.github.io/dopamine/docker/)
- [](https://google.github.io/dopamine/docs/)
- [](https://github.com/openai/atari-py#roms), [](https://github.com/openai/mujoco-py#install-mujoco)
- [](https://medium.com/the-21st-century/google-dopamine-new-rl-framework-f84a35b7fb3f)
- [](https://www.youtube.com/live/FWFoyFjeAaM?feature=share), [](https://youtu.be/bd4CsDp00RA)
| Analyzing Tennis Serve | We'll use the Video Intelligence API to analyze a tennis serve, including the angle of the arms and legs during the serve | [Dale Markowitz](https://daleonai.com/) | [](https://github.com/google/making_with_ml/tree/master/sports_ai)
- [blog post](https://daleonai.com/machine-learning-for-sports)
- [](https://manivannan-ai.medium.com/find-the-angle-between-three-points-from-2d-using-python-348c513e2cd)
- [](https://www.youtube.com/watch?v=yLrOy2Xedgk)
| YOLOv4 | This tutorial will help you build YOLOv4 easily in the cloud with GPU enabled so that you can run object detections in milliseconds! | [Alexey Bochkovskiy](http://www.alexeyab.com/) | [](https://github.com/AlexeyAB/darknet)
- [](https://arxiv.org/abs/2004.10934), [](https://arxiv.org/abs/2011.08036)
- [](https://alexeyab84.medium.com/yolov4-the-most-accurate-real-time-neural-network-on-ms-coco-dataset-73adfd3602fe), [](https://alexeyab84.medium.com/scaled-yolo-v4-is-the-best-neural-network-for-object-detection-on-ms-coco-dataset-39dfa22fa982)
- [project](https://pjreddie.com/darknet/)
- [](https://www.reddit.com/r/MachineLearning/comments/gydxzd/p_yolov4_the_most_accurate_realtime_neural/)
- [](https://youtu.be/1_SiUOYUoOI), [](https://youtu.be/YDFf-TqJOFE)
| TensorFlow Graphics | Differentiable computer graphics in tensorflow |
- [Julien Valentin](https://github.com/julienvalentin)
- [Cem Keskin](https://github.com/cem-keskin)
- [Pavel Pidlypenskyi](https://github.com/podlipensky)
- [Ameesh Makadia](https://github.com/amakadia) others
- [Avneesh Sud](https://github.com/avneesh-g)
- [Sofien Bouaziz](http://sofienbouaziz.com/)
- [](https://medium.com/syncedreview/computer-graphics-computer-vision-tensorflow-graphics-110e955e26bb)
- [](https://www.tensorflow.org/graphic)
- [](https://twitter.com/_TFGraphics_)
- [](https://youtu.be/Un0JDL3i5Hg)
| GAN Dissection | Visualizing and Understanding Generative Adversarial Networks |
- [David Bau](https://people.csail.mit.edu/davidbau/home/)
- [Jun-Yan Zhu](https://www.cs.cmu.edu/~junyanz/)
- [Hendrik Strobelt](http://hendrik.strobelt.com/)
- [Bolei Zhou](https://boleizhou.github.io/) others
- [Joshua Tenenbaum](https://mitibmwatsonailab.mit.edu/people/joshua-tenenbaum/)
- [William Freeman](https://billf.mit.edu/)
- [Antonio Torralba](https://groups.csail.mit.edu/vision/torralbalab/)
- [](https://arxiv.org/abs/1811.10597), [](https://arxiv.org/abs/1901.09887), [](https://arxiv.org/abs/1807.10221)
- [demo](http://gandissect.res.ibm.com/ganpaint.html)
- [](https://github.com/CSAILVision/NetDissect), [](https://github.com/junyanz/iGAN)
- [project](https://gandissect.csail.mit.edu/)
- [](https://www.youtube.com/watch?v=yVCgUYe4JTM)
| Sonnet | Library built on top of TensorFlow 2 designed to provide simple, composable abstractions for machine learning research |
- [Malcolm Reynolds](https://github.com/malcolmreynolds)
- [Jack Rae](https://github.com/dm-jrae)
- [Andreas Fidjeland](https://github.com/akfidjeland)
- [Fabio Viola](https://github.com/fabioviola) others
- [Adrià Puigdomènech](https://github.com/adria-p)
- [Frederic Besse](https://github.com/fbesse)
- [Tim Green](http://tfgg.me/)
- [Sébastien Racanière](https://scholar.google.com/citations?user=o-h0vrQAAAAJ)
- [Gabriel Barth-Maron](https://github.com/fastturtle)
- [Diego Casas](https://github.com/diegolascasas)
- [blog post](https://www.deepmind.com/blog/open-sourcing-sonnet-a-new-library-for-constructing-neural-networks)
- [](https://sonnet.readthedocs.io/en/latest/index.html)
- [](https://papers.nips.cc/paper/2016/hash/fb87582825f9d28a8d42c5e5e5e8b23d-Abstract.html)
- [](https://www.tensorflow.org/guide/checkpoint), [](https://www.tensorflow.org/guide/saved_model)
- [](https://youtu.be/rlpQjnUvoKw)
| Classification of chest vs. adominal X-rays | The goal of this tutorial is to build a deep learning classifier to accurately differentiate between chest and abdominal X-rays | [tmoneyx01](https://github.com/tmoneyx01) | [](https://github.com/mdai/mdai-client-py)
- [annotator](https://public.md.ai/annotator/project/PVq9raBJ)
- [](https://docs.md.ai/)
| Earth Engine Python API and Folium Interactive Mapping | This notebook demonstrates how to setup the Earth Engine and provides several examples for visualizing Earth Engine processed data interactively using the folium library | [Qiusheng Wu](https://wetlands.io/) | [](https://github.com/python-visualization/folium)
- [api](https://developers.google.com/earth-engine/python_install)
| Tensor2Tensor | Library for deep learning models that is well-suited for neural machine translation and includes the reference implementation of the state-of-the-art Transformer model |
- [Ashish Vaswani](https://scholar.google.com/citations?user=oR9sCGYAAAAJ)
- [Samy Bengio](https://scholar.google.com/citations?user=Vs-MdPcAAAAJ)
- [Eugene Brevdo](https://ebrevdo.github.io/)
- [François Chollet](https://fchollet.com/) others
- [Aidan Gomez](https://gom.ai/)
- [Stephan Gouws](https://scholar.google.com/citations?user=lLTdYUYAAAAJ)
- [Llion Jones](https://www.linkedin.com/in/llion-jones-9ab3064b)
- [Łukasz Kaiser](https://scholar.google.com/citations?user=JWmiQR0AAAAJ)
- [Nal Kalchbrenner](https://www.nal.ai/)
- [Niki Parmar](https://github.com/nikiparmar)
- [Ryan Sepassi](https://ryansepassi.com/)
- [Noam Shazeer](https://github.com/nshazeer)
- [Jakob Uszkoreit](https://scholar.google.com/citations?user=mOG0bwsAAAAJ)
- [](https://arxiv.org/abs/1803.07416), [](https://arxiv.org/abs/1812.02825), [](https://arxiv.org/abs/1706.03762), [](https://arxiv.org/abs/1706.03059), [](https://arxiv.org/abs/1706.05137), [](https://arxiv.org/abs/1801.09797)
- [blog post](https://ai.googleblog.com/2017/06/accelerating-deep-learning-research.html)
- [data](https://research.fb.com/downloads/babi/)
- [](https://towardsdatascience.com/tensor2tensor-and-one-model-to-learn-them-all-7ef3f9b61ba4)
- [](https://tensorflow.github.io/tensor2tensor/cloud_mlengine.html), [](https://tensorflow.github.io/tensor2tensor/cloud_tpu.html)
- [](https://youtu.be/O2UvKxaOH7c), [](https://youtu.be/VYQ8n3Besrw), [](https://youtu.be/cS2UZKHq4i4)
| Traffic counting | Making Road Traffic Counting App based on Computer Vision and OpenCV | [Andrey Nikishaev](https://github.com/creotiv) | [](https://github.com/creotiv/object_detection_projects/tree/master/opencv_traffic_counting)
- [](https://medium.com/machine-learning-world/tutorial-making-road-traffic-counting-app-based-on-computer-vision-and-opencv-166937911660)
- [](https://www.youtube.com/watch?v=_o5iLbRHKao)
| Imagededup | This package provides functionality to make use of hashing algorithms that are particularly good at finding exact duplicates as well as convolutional neural networks which are also adept at finding near duplicates |
- [Tanuj Jain](https://github.com/tanujjain)
- [Christopher Lennan](https://github.com/clennan)
- [Dat Tran](https://dat-tran.com/)
- [](https://arxiv.org/abs/1704.04861)
- [](https://fullstackml.com/wavelet-image-hash-in-python-3504fdd282b5)
- [project](https://idealo.github.io/imagededup/)
# Best of the best
| authors | repositories | papers |
|---|---|---|
|
- [Billy Lamberta](https://github.com/lamberta)
- [Ziwei Liu](https://liuziwei7.github.io/)
- [Daniel Cohen-Or](https://danielcohenor.com/)
- [Adam Roberts](https://github.com/adarob)
- [Jesse Engel](https://github.com/jesseengel)
- [Google](https://www.tensorflow.org/)
- [Chen Change Loy](https://www.mmlab-ntu.com/person/ccloy/)
- [Curtis Hawthorne](https://github.com/cghawthorne)
- [Eli Shechtman](https://research.adobe.com/person/eli-shechtman/)
- [Björn Ommer](https://ommer-lab.com/people/ommer/)
- [Yuval Alaluf](https://yuval-alaluf.github.io/)
- [Xintao Wang](https://xinntao.github.io/)
- [Ying Shan](https://scholar.google.com/citations?user=4oXBp9UAAAAJ)
- [Patrick Esser](https://github.com/pesser)
- [Robin Rombach](https://github.com/rromb)
- [Or Patashnik](https://orpatashnik.github.io/)
- [Antonio Torralba](https://groups.csail.mit.edu/vision/torralbalab/)
- [Bolei Zhou](https://boleizhou.github.io/)
- [Krzysztof Ostrowski](https://github.com/krzys-ostrowski)
- [Max Woolf](https://minimaxir.com/)
- [Jon Barron](https://jonbarron.info/)
- [Xiaohua Zhai](https://github.com/xiaohuazhai)
- [Ishan Misra](https://imisra.github.io/)
- [Nikhila Ravi](https://nikhilaravi.com/)
- [Yossi Adi](https://www.cs.huji.ac.il/~adiyoss/)
- [Gabriel Synnaeve](https://syhw.github.io/)
- [Jia-Bin Huang](https://jbhuang0604.github.io/)
- [Karen Simonyan](https://scholar.google.com/citations?user=L7lMQkQAAAAJ)
- [Amit Bermano](https://www.cs.tau.ac.il/~amberman/)
- [Jun-Yan Zhu](https://www.cs.cmu.edu/~junyanz/)
- [Phil Wang](https://lucidrains.github.io/)
- [Ben Trevett](https://bentrevett.com/)
- tensorflow/models [](https://github.com/tensorflow/models)
- CompVis/stable-diffusion [](https://github.com/CompVis/stable-diffusion)
- openai/whisper [](https://github.com/openai/whisper)
- CorentinJ/Real-Time-Voice-Cloning [](https://github.com/CorentinJ/Real-Time-Voice-Cloning)
- ultralytics/yolov5 [](https://github.com/ultralytics/yolov5)
- KillianLucas/open-interpreter [](https://github.com/KillianLucas/open-interpreter)
- iperov/DeepFaceLab [](https://github.com/iperov/DeepFaceLab)
- facebookresearch/segment-anything [](https://github.com/facebookresearch/segment-anything)
- jakevdp/PythonDataScienceHandbook [](https://github.com/jakevdp/PythonDataScienceHandbook)
- LAION-AI/Open-Assistant [](https://github.com/LAION-AI/Open-Assistant)
- Stability-AI/stablediffusion [](https://github.com/Stability-AI/stablediffusion)
- XingangPan/DragGAN [](https://github.com/XingangPan/DragGAN)
- microsoft/visual-chatgpt [](https://github.com/microsoft/visual-chatgpt)
- TencentARC/GFPGAN [](https://github.com/TencentARC/GFPGAN)
- lllyasviel/Fooocus [](https://github.com/lllyasviel/Fooocus)
- google-research/google-research [](https://github.com/google-research/google-research)
- suno-ai/bark [](https://github.com/suno-ai/bark)
- ray-project/ray [](https://github.com/ray-project/ray)
- comfyanonymous/ComfyUI [](https://github.com/comfyanonymous/ComfyUI)
- facebookresearch/fairseq [](https://github.com/facebookresearch/fairseq)
- coqui-ai/TTS [](https://github.com/coqui-ai/TTS)
- facebookresearch/detectron2 [](https://github.com/facebookresearch/detectron2)
- google/jax [](https://github.com/google/jax)
- xinntao/Real-ESRGAN [](https://github.com/xinntao/Real-ESRGAN)
- deezer/spleeter [](https://github.com/deezer/spleeter)
- microsoft/autogen [](https://github.com/microsoft/autogen)
- svc-develop-team/so-vits-svc [](https://github.com/svc-develop-team/so-vits-svc)
- huggingface/diffusers [](https://github.com/huggingface/diffusers)
- openai/CLIP [](https://github.com/openai/CLIP)
- ultralytics/ultralytics [](https://github.com/ultralytics/ultralytics)
- AlexeyAB/darknet [](https://github.com/AlexeyAB/darknet)
- openai/gpt-2 [](https://github.com/openai/gpt-2)
- AlphaFold [](https://doi.org/10.1038/s41586-021-03819-2)
- MoCo [](https://doi.org/10.1109/CVPR42600.2020.00975)
- EfficientDet [](https://doi.org/10.1109/CVPR42600.2020.01079)
- DeepLabCut [](https://doi.org/10.1038/s41593-018-0209-y)
- StyleGAN 2 [](https://doi.org/10.1109/CVPR42600.2020.00813)
- Fine-tuning a BERT [](https://doi.org/10.18653/v1/N19-1423)
- ConvNeXt [](https://doi.org/10.1109/CVPR52688.2022.01167)
- LDM [](https://doi.org/10.1109/CVPR52688.2022.01042)
- Neural Style Transfer [](https://doi.org/10.1167/16.12.326)
- PIFu [](https://doi.org/10.1109/ICCV.2019.00239)
- Taming Transformers for High-Resolution Image Synthesis [](https://doi.org/10.1109/CVPR46437.2021.01268)
- VIBE [](https://doi.org/10.1109/CVPR42600.2020.00530)
- InterFaceGAN [](https://doi.org/10.1109/CVPR42600.2020.00926)
- Pixel2Style2Pixel [](https://doi.org/10.1109/CVPR46437.2021.00232)
- Mask2Former [](https://doi.org/10.1109/CVPR52688.2022.00135)
- ByteTrack [](https://doi.org/10.1007/978-3-031-20047-2_1)
- PIFuHD [](https://doi.org/10.1109/CVPR42600.2020.00016)
- Nerfies [](https://doi.org/10.1109/ICCV48922.2021.00581)
- Skillful Precipitation Nowcasting Using Deep Generative Models of Radar [](https://doi.org/10.1038/s41586-021-03854-z)
- Parallel WaveGAN [](https://doi.org/10.1109/ICASSP40776.2020.9053795)
- BiT [](https://doi.org/10.1007/978-3-030-58558-7_29)
- encoder4editing [](https://doi.org/10.1145/3450626.3459838)
- Wav2Lip [](https://doi.org/10.1145/3394171.3413532)
- SeFa [](https://doi.org/10.1109/CVPR46437.2021.00158)
- Cleanlab [](https://doi.org/10.1613/jair.1.12125)
- CartoonGAN [](https://doi.org/10.1109/CVPR.2018.00986)
- NAFNet [](https://doi.org/10.1007/978-3-031-20071-7_2)
- TediGAN [](https://doi.org/10.1109/CVPR46437.2021.00229)
- ReStyle [](https://doi.org/10.1109/ICCV48922.2021.00664)
- StyleGAN-NADA [](https://doi.org/10.1145/3528223.3530164)
- 3D Photo Inpainting [](https://doi.org/10.1109/CVPR42600.2020.00805)
- AlphaTensor [](https://doi.org/10.1038/s41586-022-05172-4)
[](https://starchart.cc/amrzv/awesome-colab-notebooks)
(generated by [generate_markdown.py](generate_markdown.py) based on [research.json](data/research.json) and [tutorials.json](data/tutorials.json))