Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/foundationvision/var
[GPT beats diffusion🔥] [scaling laws in visual generation📈] Official impl. of "Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction"
https://github.com/foundationvision/var
auto-regressive-model diffusion-models image-generation transformers
Last synced: 3 months ago
JSON representation
[GPT beats diffusion🔥] [scaling laws in visual generation📈] Official impl. of "Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction"
- Host: GitHub
- URL: https://github.com/foundationvision/var
- Owner: FoundationVision
- License: mit
- Created: 2024-04-01T16:53:18.000Z (3 months ago)
- Default Branch: main
- Last Pushed: 2024-04-10T14:53:33.000Z (3 months ago)
- Last Synced: 2024-04-10T17:17:25.214Z (3 months ago)
- Topics: auto-regressive-model, diffusion-models, image-generation, transformers
- Language: Python
- Homepage:
- Size: 60.5 KB
- Stars: 1,714
- Watchers: 45
- Forks: 161
- Open Issues: 10
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Lists
- awesome-ChatGPT-repositories - VAR - [GPT beats diffusion🔥] [scaling laws in visual generation📈] Official impl. of "Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction" (Others)
README
# VAR: a new visual generation method elevates GPT-style models beyond diffusion🚀 & Scaling laws observed📈
[](https://var.vision/demo)
[](https://arxiv.org/abs/2404.02905)
[](https://huggingface.co/FoundationVision/var)
[](https://paperswithcode.com/sota/image-generation-on-imagenet-256x256?tag_filter=485&p=visual-autoregressive-modeling-scalable-image)
Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction
## 🕹️ Try and Play with VAR!
We provide a [demo website](https://var.vision/demo) for you to play with VAR models and generate images interactively. Enjoy the fun of visual autoregressive modeling!
We also provide [demo_sample.ipynb](demo_sample.ipynb) for you to see more technical details about VAR.
[//]: # (
)
[//]: # (
)
## What's New?
### 🔥 Introducing VAR: a new paradigm in autoregressive visual generation✨:
Visual Autoregressive Modeling (VAR) redefines the autoregressive learning on images as coarse-to-fine "next-scale prediction" or "next-resolution prediction", diverging from the standard raster-scan "next-token prediction".
### 🔥 For the first time, GPT-style autoregressive models surpass diffusion models🚀:
### 🔥 Discovering power-law Scaling Laws in VAR transformers📈:
### 🔥 Zero-shot generalizability🛠️:
#### For a deep dive into our analyses, discussions, and evaluations, check out our [paper](https://arxiv.org/abs/2404.02905).
## VAR zoo
We provide VAR models for you to play with, which are on 
or can be downloaded from the following links:
| model | reso. | FID | rel. cost | #params | HF weights🤗 |
|:----------:|:-----:|:--------:|:---------:|:-------:|:------------------------------------------------------------------------------------|
| VAR-d16 | 256 | 3.55 | 0.4 | 310M | [var_d16.pth](https://huggingface.co/FoundationVision/var/resolve/main/var_d16.pth) |
| VAR-d20 | 256 | 2.95 | 0.5 | 600M | [var_d20.pth](https://huggingface.co/FoundationVision/var/resolve/main/var_d20.pth) |
| VAR-d24 | 256 | 2.33 | 0.6 | 1.0B | [var_d24.pth](https://huggingface.co/FoundationVision/var/resolve/main/var_d24.pth) |
| VAR-d30 | 256 | 1.97 | 1 | 2.0B | [var_d30.pth](https://huggingface.co/FoundationVision/var/resolve/main/var_d30.pth) |
| VAR-d30-re | 256 | **1.80** | 1 | 2.0B | [var_d30.pth](https://huggingface.co/FoundationVision/var/resolve/main/var_d30.pth) |
You can load these models to generate images via the codes in [demo_sample.ipynb](demo_sample.ipynb). Note: you need to download [vae_ch160v4096z32.pth](https://huggingface.co/FoundationVision/var/resolve/main/vae_ch160v4096z32.pth) first.
## Installation
## License
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
## Citation
If our work assists your research, feel free to give us a star ⭐ or cite us using:
```
@Article{VAR,
title={Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction},
author={Keyu Tian and Yi Jiang and Zehuan Yuan and Bingyue Peng and Liwei Wang},
year={2024},
eprint={2404.02905},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
```