{"id":13958456,"url":"https://github.com/lucidrains/nuwa-pytorch","last_synced_at":"2025-04-12T16:42:26.599Z","repository":{"id":38319529,"uuid":"432768606","full_name":"lucidrains/nuwa-pytorch","owner":"lucidrains","description":"Implementation of NÜWA, state of the art attention network for text to video synthesis, in Pytorch","archived":false,"fork":false,"pushed_at":"2023-01-17T17:56:10.000Z","size":1866,"stargazers_count":546,"open_issues_count":8,"forks_count":56,"subscribers_count":22,"default_branch":"main","last_synced_at":"2025-04-03T22:06:56.574Z","etag":null,"topics":["artificial-intelligence","attention-mechanism","deep-learning","text-to-audio","text-to-video","transformers"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lucidrains.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-11-28T16:44:48.000Z","updated_at":"2025-03-16T13:18:55.000Z","dependencies_parsed_at":"2023-02-10T11:00:31.214Z","dependency_job_id":null,"html_url":"https://github.com/lucidrains/nuwa-pytorch","commit_stats":null,"previous_names":[],"tags_count":111,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lucidrains%2Fnuwa-pytorch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lucidrains%2Fnuwa-pytorch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lucidrains%2Fnuwa-pytorch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lucidrains%2Fnuwa-pytorch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lucidrains","download_url":"https://codeload.github.com/lucidrains/nuwa-pytorch/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248599294,"owners_count":21131257,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["artificial-intelligence","attention-mechanism","deep-learning","text-to-audio","text-to-video","transformers"],"created_at":"2024-08-08T13:01:36.462Z","updated_at":"2025-04-12T16:42:26.557Z","avatar_url":"https://github.com/lucidrains.png","language":"Python","funding_links":[],"categories":["其他_机器视觉","Python"],"sub_categories":["网络服务_其他"],"readme":"\u003cimg src=\"./nuwa.png\" width=\"400px\"\u003e\u003c/img\u003e\n\n## NÜWA - Pytorch\n\n\u003ca href=\"https://discord.gg/xBPBXfcFHd\"\u003e\u003cimg alt=\"Join us on Discord\" src=\"https://img.shields.io/discord/823813159592001537?color=5865F2\u0026logo=discord\u0026logoColor=white\"\u003e\u003c/a\u003e\u003c/br\u003e\n\nImplementation of \u003ca href=\"https://arxiv.org/abs/2111.12417\"\u003eNÜWA\u003c/a\u003e, state of the art attention network for text to video synthesis, in Pytorch. It also contain an extension into video and audio generation, using a dual decoder approach.\n\n\u003ca href=\"https://www.youtube.com/watch?v=InhMx1h0N40\"\u003eYannic Kilcher\u003c/a\u003e\n\n\u003ca href=\"https://www.youtube.com/watch?v=C9CTnZJ9ZE0\"\u003eDeepReader\u003c/a\u003e\n\n## Status\n\n- March 2022 - seeing signs of life with a difficult version of \u003ca href=\"https://github.com/lucidrains/nuwa-pytorch/discussions/2#discussioncomment-2375973\"\u003emoving mnist\u003c/a\u003e\n\n- April 2022 - It seems as though a \u003ca href=\"https://github.com/lucidrains/video-diffusion-pytorch\"\u003ediffusion based method\u003c/a\u003e has taken the new throne for SOTA. However, I will continue on with NUWA, extending it to use multi-headed codes + \u003ca href=\"https://github.com/lucidrains/RQ-Transformer/blob/main/rq_transformer/hierarchical_causal_transformer.py\"\u003ehierarchical causal transformer\u003c/a\u003e. I think that direction is untapped for improving on this line of work.\n\n\n## Install\n\n```bash\n$ pip install nuwa-pytorch\n```\n\n## Usage\n\nFirst train the VAE\n\n```python\nimport torch\nfrom nuwa_pytorch import VQGanVAE\n\nvae = VQGanVAE(\n    dim = 512,\n    channels = 3,               # default is 3, but can be changed to any value for the training of the segmentation masks (sketches)\n    image_size = 256,           # image size\n    num_layers = 4,             # number of downsampling layers\n    num_resnet_blocks = 2,      # number of resnet blocks\n    vq_codebook_size = 8192,    # codebook size\n    vq_decay = 0.8              # codebook exponential decay\n)\n\nimgs = torch.randn(10, 3, 256, 256)\n\n# alternate learning for autoencoder ...\n\nloss = vae(imgs, return_loss = True)\nloss.backward()\n\n# and the discriminator ...\n\ndiscr_loss = vae(imgs, return_discr_loss = True)\ndiscr_loss.backward()\n\n# do above for many steps\n\n# return reconstructed images and make sure they look ok\n\nrecon_imgs = vae(imgs)\n\n```\n\nThen, with your learned VAE\n\n```python\nimport torch\nfrom nuwa_pytorch import NUWA, VQGanVAE\n\n# autoencoder\n\nvae = VQGanVAE(\n    dim = 64,\n    num_layers = 4,\n    image_size = 256,\n    num_conv_blocks = 2,\n    vq_codebook_size = 8192\n)\n\n# NUWA transformer\n\nnuwa = NUWA(\n    vae = vae,\n    dim = 512,\n    text_num_tokens = 20000,                # number of text tokens\n    text_enc_depth = 12,                    # text encoder depth\n    text_enc_heads = 8,                     # number of attention heads for encoder\n    text_max_seq_len = 256,                 # max sequence length of text conditioning tokens (keep at 256 as in paper, or shorter, if your text is not that long)\n    max_video_frames = 10,                  # number of video frames\n    image_size = 256,                       # size of each frame of video\n    dec_depth = 64,                         # video decoder depth\n    dec_heads = 8,                          # number of attention heads in decoder\n    dec_reversible = True,                  # reversible networks - from reformer, decoupling memory usage from depth\n    enc_reversible = True,                  # reversible encoders, if you need it\n    attn_dropout = 0.05,                    # dropout for attention\n    ff_dropout = 0.05,                      # dropout for feedforward\n    sparse_3dna_kernel_size = (5, 3, 3),    # kernel size of the sparse 3dna attention. can be a single value for frame, height, width, or different values (to simulate axial attention, etc)\n    sparse_3dna_dilation = (1, 2, 4),       # cycle dilation of 3d conv attention in decoder, for more range\n    shift_video_tokens = True               # cheap relative positions for sparse 3dna transformer, by shifting along spatial dimensions by one\n).cuda()\n\n# data\n\ntext = torch.randint(0, 20000, (1, 256)).cuda()\nvideo = torch.randn(1, 10, 3, 256, 256).cuda() # (batch, frames, channels, height, width)\n\nloss = nuwa(\n    text = text,\n    video = video,\n    return_loss = True  # set this to True, only for training, to return cross entropy loss\n)\n\nloss.backward()\n\n# do above with as much data as possible\n\n# then you can generate a video from text\n\nvideo = nuwa.generate(text = text, num_frames = 5) # (1, 5, 3, 256, 256)\n\n```\n\n## Conditioning on Sketches\n\nIn the paper, they also present a way to condition the video generation based on segmentation mask(s). You can easily do this as well, given you train a `VQGanVAE` on the sketches before hand.\n\nThen, you will use `NUWASketch` instead of `NUWA`, which can accept the sketch VAE as a reference\n\nex.\n\n```python\nimport torch\nfrom nuwa_pytorch import NUWASketch, VQGanVAE\n\n# autoencoder, one for main video, the other for the sketch\n\nvae = VQGanVAE(\n    dim = 64,\n    num_layers = 4,\n    image_size = 256,\n    num_conv_blocks = 2,\n    vq_codebook_size = 8192\n)\n\nsketch_vae = VQGanVAE(\n    dim = 512,\n    channels = 5,                # say the sketch has 5 classes\n    num_layers = 4,\n    image_size = 256,\n    num_conv_blocks = 2,\n    vq_codebook_size = 8192\n)\n\n# NUWA transformer for conditioning with sketches\n\nnuwa = NUWASketch(\n    vae = vae,\n    sketch_vae = sketch_vae,\n    dim = 512,                              # model dimensions\n    sketch_enc_depth = 12,                  # sketch encoder depth\n    sketch_enc_heads = 8,                   # number of attention heads for sketch encoder\n    sketch_max_video_frames = 3,            # max number of frames for sketches\n    sketch_enc_use_sparse_3dna = True,      # whether to use 3d-nearby attention (of full attention if False) for sketch encoding transformer\n    max_video_frames = 10,                  # number of video frames\n    image_size = 256,                       # size of each frame of video\n    dec_depth = 64,                         # video decoder depth\n    dec_heads = 8,                          # number of attention heads in decoder\n    dec_reversible = True,                  # reversible networks - from reformer, decoupling memory usage from depth\n    enc_reversible = True,                  # reversible encoders, if you need it\n    attn_dropout = 0.05,                    # dropout for attention\n    ff_dropout = 0.05,                      # dropout for feedforward\n    sparse_3dna_kernel_size = (5, 3, 3),    # kernel size of the sparse 3dna attention. can be a single value for frame, height, width, or different values (to simulate axial attention, etc)\n    sparse_3dna_dilation = (1, 2, 4),       # cycle dilation of 3d conv attention in decoder, for more range\n    cross_2dna_kernel_size = 5,             # 2d kernel size of spatial grouping of attention from video frames to sketches\n    cross_2dna_dilation = 1,                # 2d dilation of spatial attention from video frames to sketches\n    shift_video_tokens = True               # cheap relative positions for sparse 3dna transformer, by shifting along spatial dimensions by one\n).cuda()\n\n# data\n\nsketch = torch.randn(2, 2, 5, 256, 256).cuda() # (batch, frames, segmentation classes, height, width)\nsketch_mask = torch.ones(2, 2).bool().cuda()   # (batch, frames) [Optional]\nvideo = torch.randn(2, 10, 3, 256, 256).cuda() # (batch, frames, channels, height, width)\n\nloss = nuwa(\n    sketch = sketch,\n    sketch_mask =sketch_mask,\n    video = video,\n    return_loss = True  # set this to True, only for training, to return cross entropy loss\n)\n\nloss.backward()\n\n# do above with as much data as possible\n\n# then you can generate a video from sketch(es)\n\nvideo = nuwa.generate(sketch = sketch, num_frames = 5) # (1, 5, 3, 256, 256)\n\n```\n\n## Text to Video and Audio\n\nThis repository will also offer a variant of NUWA that can produce both video and audio. For now, the audio will need to be encoded manually.\n\n```python\nimport torch\nfrom nuwa_pytorch import NUWAVideoAudio, VQGanVAE\n\n# autoencoder\n\nvae = VQGanVAE(\n    dim = 64,\n    num_layers = 4,\n    image_size = 256,\n    num_conv_blocks = 2,\n    vq_codebook_size = 100\n)\n\n# NUWA transformer\n\nnuwa = NUWAVideoAudio(\n    vae = vae,\n    dim = 512,\n    num_audio_tokens = 2048,                # codebook size for audio tokens\n    num_audio_tokens_per_video_frame = 32,  # number of audio tokens per video frame\n    cross_modality_attn_every = 3,          # cross modality attention every N layers\n    text_num_tokens = 20000,                # number of text tokens\n    text_enc_depth = 1,                     # text encoder depth\n    text_enc_heads = 8,                     # number of attention heads for encoder\n    text_max_seq_len = 256,                 # max sequence length of text conditioning tokens (keep at 256 as in paper, or shorter, if your text is not that long)\n    max_video_frames = 10,                  # number of video frames\n    image_size = 256,                       # size of each frame of video\n    dec_depth = 4,                          # video decoder depth\n    dec_heads = 8,                          # number of attention heads in decoder\n    enc_reversible = True,                  # reversible encoders, if you need it\n    dec_reversible = True,                  # quad-branched reversible network, for making depth of twin video / audio decoder independent of network depth. recommended to be turned on unless you have a ton of memory at your disposal\n    attn_dropout = 0.05,                    # dropout for attention\n    ff_dropout = 0.05,                      # dropout for feedforward\n    sparse_3dna_kernel_size = (5, 3, 3),    # kernel size of the sparse 3dna attention. can be a single value for frame, height, width, or different values (to simulate axial attention, etc)\n    sparse_3dna_dilation = (1, 2, 4),       # cycle dilation of 3d conv attention in decoder, for more range\n    shift_video_tokens = True               # cheap relative positions for sparse 3dna transformer, by shifting along spatial dimensions by one\n).cuda()\n\n# data\n\ntext = torch.randint(0, 20000, (1, 256)).cuda()\naudio = torch.randint(0, 2048, (1, 32 * 10)).cuda() # (batch, audio tokens per frame * max video frames)\nvideo = torch.randn(1, 10, 3, 256, 256).cuda() # (batch, frames, channels, height, width)\n\nloss = nuwa(\n    text = text,\n    video = video,\n    audio = audio,\n    return_loss = True  # set this to True, only for training, to return cross entropy loss\n)\n\nloss.backward()\n\n# do above with as much data as possible\n\n# then you can generate a video from text\n\nvideo, audio = nuwa.generate(text = text, num_frames = 5) # (1, 5, 3, 256, 256), (1, 32 * 5 == 160)\n\n```\n\n## Trainers\n\nThis library will offer some utilities to make training easier. For starters, you can use the `VQGanVAETrainer` class to take care of training the `VQGanVAE`. Simply wrap the model and also pass in the image folder path as well as the various training hyperparameters.\n\n```python\nimport torch\nfrom nuwa_pytorch import VQGanVAE, VQGanVAETrainer\n\nvae = VQGanVAE(\n    dim = 64,\n    image_size = 256,\n    num_layers = 5,\n    vq_codebook_size = 1024,\n    vq_use_cosine_sim = True,\n    vq_codebook_dim = 32,\n    vq_orthogonal_reg_weight = 10,\n    vq_orthogonal_reg_max_codes = 128,\n).cuda()\n\ntrainer = VQGanVAETrainer(\n    vae,                           # VAE defined above\n    folder ='/path/to/images',     # path to images\n    lr = 3e-4,                     # learning rate\n    num_train_steps = 100000,      # number of training steps\n    batch_size = 8,                # batch size\n    grad_accum_every = 4           # gradient accumulation (effective batch size is (batch_size x grad_accum_every))\n)\n\ntrainer.train()\n\n# results and model checkpoints will be saved periodically to ./results\n```\n\nTo train NUWA, first you need to organize a folder of `.gif` files with corresponding `.txt` files containing its caption. It should be organized as such.\n\nex.\n\n```\n📂video-and-text-data\n ┣ 📜cat.gif\n ┣ 📜cat.txt\n ┣ 📜dog.gif\n ┣ 📜dog.txt\n ┣ 📜turtle.gif\n ┗ 📜turtle.txt\n```\n\nThen you will load your previously trained VQGan-VAE and train NUWA with the `GifVideoDataset` and `NUWATrainer` classes.\n\n```python\nimport torch\nfrom nuwa_pytorch import NUWA, VQGanVAE\nfrom nuwa_pytorch.train_nuwa import GifVideoDataset, NUWATrainer\n\n# dataset\n\nds = GifVideoDataset(\n    folder = './path/to/videos/',\n    channels = 1\n)\n\n# autoencoder\n\nvae = VQGanVAE(\n    dim = 64,\n    image_size = 256,\n    num_layers = 5,\n    num_resnet_blocks = 2,\n    vq_codebook_size = 512,\n    attn_dropout = 0.1\n)\n\nvae.load_state_dict(torch.load('./path/to/trained/vae.pt'))\n\n# NUWA transformer\n\nnuwa = NUWA(\n    vae = vae,\n    dim = 512,\n    text_enc_depth = 6,\n    text_max_seq_len = 256,\n    max_video_frames = 10,\n    dec_depth = 12,\n    dec_reversible = True,\n    enc_reversible = True,\n    attn_dropout = 0.05,\n    ff_dropout = 0.05,\n    sparse_3dna_kernel_size = (5, 3, 3),\n    sparse_3dna_dilation = (1, 2, 4),\n    shift_video_tokens = True\n).cuda()\n\n# data\n\ntrainer = NUWATrainer(\n    nuwa = nuwa,                 # NUWA transformer\n    dataset = dataset,           # video dataset class\n    num_train_steps = 1000000,   # number of training steps\n    lr = 3e-4,                   # learning rate\n    wd = 0.01,                   # weight decay\n    batch_size = 8,              # batch size\n    grad_accum_every = 4,        # gradient accumulation\n    max_grad_norm = 0.5,         # gradient clipping\n    num_sampled_frames = 10,     # number of frames to sample\n    results_folder = './results' # folder to store checkpoints and samples\n)\n\ntrainer.train()\n```\n\n## VQ improvements\n\nThis library depends on this \u003ca href=\"https://github.com/lucidrains/vector-quantize-pytorch\"\u003evector quantization\u003c/a\u003e library, which comes with a number of improvements (improved vqgan, orthogonal codebook regularization, etc). To use any of these improvements, you can configure the vector quantizer keyword params by prepending `vq_` on `VQGanVAE` initialization.\n\nex. cosine sim proposed in \u003ca href=\"https://arxiv.org/abs/2110.04627\"\u003eimproved vqgan\u003c/a\u003e\n\n```python\nfrom nuwa_pytorch import VQGanVAE\n\nvae = VQGanVAE(\n    dim = 64,\n    image_size = 256,\n    num_layers = 4,\n    vq_use_cosine_sim = True\n    # VectorQuantize will be initialized with use_cosine_sim = True\n    # https://github.com/lucidrains/vector-quantize-pytorch#cosine-similarity\n).cuda()\n```\n\n## Todo\n\n- [x] complete 3dna causal attention in decoder\n- [x] write up easy generation functions\n- [x] make sure GAN portion of VQGan is correct, reread paper\n- [x] make sure adaptive weight in vqgan is correctly built\n- [x] offer new vqvae improvements (orthogonal reg and smaller codebook dimensions)\n- [x] batch video tokens -\u003e vae during video generation, to prevent oom\n- [x] query chunking in 3dna attention, to put a cap on peak memory\n- [x] flesh out VAE resnet blocks, offer some choices\n- [x] add all stability tricks from cogview paper by default\n- [x] make VQGan able to accept custom VGG for LPAPs loss (audio)\n- [x] add feedforward chunking\n- [x] add shift token in decoder for cheap powerful RPE\n- [x] add reversible networks, to save on memory on depth\n- [x] support kernel sizes different along each dimension for sparse 3dna\n- [x] add some autotrainer that takes care of the alternating updates of discriminator and VQVAE generator\n- [x] segmentation mask encoder, make sure embeddings can undergo 3dna attention with decoder during cross attention\n- [x] finish 2d-nearby cross attention for sketches\n- [x] able to add convnext blocks to other layers in vqgan vae\n- [x] offer vqvae training script\n- [x] handle variable lengthed sketches, accept a mask on the sketch frames dimension\n- [x] take care of audio transformer and cross modality attention\n- [x] add audio transformer, and build audio / video nearby cross attention\n- [x] make dual decoder reversible\n- [x] rotary embeddings for encoder\n- [x] add cycle dilation to audio\n- [x] omit vgg from VAE state dict\n- [x] add cosine sim attention from swinv2 as an option\n- [x] add axial positional embedding to audio\n- [ ] Triton kernel for 3dna attention\n- [ ] offer a colab with moving mnist example, conditioned on present digits\n- [ ] build NUWA controller class that can accept text or sketch\n- [ ] key masking for 3dna attention - for variable sketch length masking\n- [ ] figure out spec vqgan and fit it into the framework, take care of audio encoding / decoding automatically\n- [ ] turn into CLI tool, like stylegan2-pytorch\n- [ ] look into integrating https://github.com/lucidrains/RQ-Transformer for both video and audio\n- [ ] inference caching\n\n## Citations\n\n```bibtex\n@misc{wu2021nuwa,\n    title   = {N\\\"UWA: Visual Synthesis Pre-training for Neural visUal World creAtion}, \n    author  = {Chenfei Wu and Jian Liang and Lei Ji and Fan Yang and Yuejian Fang and Daxin Jiang and Nan Duan},\n    year    = {2021},\n    eprint  = {2111.12417},\n    archivePrefix = {arXiv},\n    primaryClass = {cs.CV}\n}\n```\n\n```bibtex\n@misc{esser2021taming,\n    title   = {Taming Transformers for High-Resolution Image Synthesis},\n    author  = {Patrick Esser and Robin Rombach and Björn Ommer},\n    year    = {2021},\n    eprint  = {2012.09841},\n    archivePrefix = {arXiv},\n    primaryClass = {cs.CV}\n}\n```\n\n```bibtex\n@misc{iashin2021taming,\n    title   = {Taming Visually Guided Sound Generation},\n    author  = {Vladimir Iashin and Esa Rahtu},\n    year    = {2021},\n    eprint  = {2110.08791},\n    archivePrefix = {arXiv},\n    primaryClass = {cs.CV}\n}\n```\n\n```bibtex\n@misc{ding2021cogview,\n    title   = {CogView: Mastering Text-to-Image Generation via Transformers},\n    author  = {Ming Ding and Zhuoyi Yang and Wenyi Hong and Wendi Zheng and Chang Zhou and Da Yin and Junyang Lin and Xu Zou and Zhou Shao and Hongxia Yang and Jie Tang},\n    year    = {2021},\n    eprint  = {2105.13290},\n    archivePrefix = {arXiv},\n    primaryClass = {cs.CV}\n}\n```\n\n```bibtex\n@misc{kitaev2020reformer,\n    title   = {Reformer: The Efficient Transformer},\n    author  = {Nikita Kitaev and Łukasz Kaiser and Anselm Levskaya},\n    year    = {2020},\n    eprint  = {2001.04451},\n    archivePrefix = {arXiv},\n    primaryClass = {cs.LG}\n}\n```\n\n```bibtex\n@misc{shazeer2020talkingheads,\n    title   = {Talking-Heads Attention}, \n    author  = {Noam Shazeer and Zhenzhong Lan and Youlong Cheng and Nan Ding and Le Hou},\n    year    = {2020},\n    eprint  = {2003.02436},\n    archivePrefix = {arXiv},\n    primaryClass = {cs.LG}\n}\n```\n\n```bibtex\n@misc{shazeer2020glu,\n    title   = {GLU Variants Improve Transformer},\n    author  = {Noam Shazeer},\n    year    = {2020},\n    url     = {https://arxiv.org/abs/2002.05202}    \n}\n```\n\n```bibtex\n@misc{su2021roformer,\n    title   = {RoFormer: Enhanced Transformer with Rotary Position Embedding},\n    author  = {Jianlin Su and Yu Lu and Shengfeng Pan and Bo Wen and Yunfeng Liu},\n    year    = {2021},\n    eprint  = {2104.09864},\n    archivePrefix = {arXiv},\n    primaryClass = {cs.CL}\n}\n```\n\n```bibtex\n@inproceedings{ho2021classifierfree,\n    title   = {Classifier-Free Diffusion Guidance},\n    author  = {Jonathan Ho and Tim Salimans},\n    booktitle = {NeurIPS 2021 Workshop on Deep Generative Models and Downstream Applications},\n    year    = {2021},\n    url     = {https://openreview.net/forum?id=qw8AKxfYbI}\n}\n```\n\n```bibtex\n@misc{liu2021swin,\n    title   = {Swin Transformer V2: Scaling Up Capacity and Resolution},\n    author  = {Ze Liu and Han Hu and Yutong Lin and Zhuliang Yao and Zhenda Xie and Yixuan Wei and Jia Ning and Yue Cao and Zheng Zhang and Li Dong and Furu Wei and Baining Guo},\n    year    = {2021},\n    eprint  = {2111.09883},\n    archivePrefix = {arXiv},\n    primaryClass = {cs.CV}\n}\n```\n\n```bibtex\n@misc{crowson2022,\n    author  = {Katherine Crowson},\n    url     = {https://twitter.com/RiversHaveWings/status/1478093658716966912}\n}\n```\n\n*Attention is the rarest and purest form of generosity.* - Simone Weil\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flucidrains%2Fnuwa-pytorch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flucidrains%2Fnuwa-pytorch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flucidrains%2Fnuwa-pytorch/lists"}