Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/xiaoqian-shen/MoStGAN-V
[CVPR 2023] Official PyTorch implementation of MoStGAN-V
https://github.com/xiaoqian-shen/MoStGAN-V
Last synced: about 2 months ago
JSON representation
[CVPR 2023] Official PyTorch implementation of MoStGAN-V
- Host: GitHub
- URL: https://github.com/xiaoqian-shen/MoStGAN-V
- Owner: xiaoqian-shen
- Created: 2022-11-13T07:03:57.000Z (about 2 years ago)
- Default Branch: main
- Last Pushed: 2023-06-15T14:02:18.000Z (over 1 year ago)
- Last Synced: 2024-08-03T08:02:32.007Z (5 months ago)
- Language: Python
- Homepage: https://xiaoqian-shen.github.io/MoStGAN-V
- Size: 77.4 MB
- Stars: 24
- Watchers: 1
- Forks: 2
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
- awesome-face-related-list - MoStGAN-V - shen.github.io/MoStGAN-V/). [arXiv](https://arxiv.org/abs/2304.02777). (Generative Model / Generative Adversarial Network (GAN))
README
# MoStGAN-V
Official PyTorch implementation for the paper:
> **MoStGAN-V: Video Generation with Temporal Motion Styles**, ***CVPR 2023***.
>
> Xiaoqian Shen, Xiang Li, Mohamed Elhoseiny
>
> 
## Installation
```
conda env create -f environment.yaml
```
And also make sure [StyleGAN2-ADA](https://github.com/NVlabs/stylegan2-ada-pytorch#requirements) is runnable.
## System requirements
4 32GB V100s are required, training time is approximately 2 days
## Data
+ [CelebV-HQ](https://celebv-hq.github.io)
+ [FaceForensics](https://github.com/ondyari/FaceForensics)
+ [SkyTimelapse](https://github.com/weixiong-ur/mdgan)
+ [RainbowJelly](https://www.youtube.com/watch?v=P8Bit37hlsQ)
+ [UCF101](https://www.crcv.ucf.edu/data/UCF101.php)
We follow the same procedure as [StyleGAN-V](https://github.com/universome/stylegan-v) to process all datasets
```
convert_videos_to_frames.py -s /path/to/source -t /path/to/target --video_ext mp4 --target_size 256
```
FaceForensics was preprocessed with `src/scripts/preprocess_ffs.py` to extract face crops, (result in a little bit unstable).
## Training
```
python src/infra/launch.py hydra.run.dir=. exp_suffix=my_experiment_name env=local dataset=ffs dataset.resolution=256 num_gpus=4
```
## Inference
+ evaluation
```
src/scripts/calc_metrics.py
```
+ generation
```
python src/scripts/generate.py --network_pkl /path/to/network-snapshot.pkl --num_videos 25 --as_grids true --save_as_mp4 true --fps 25 --video_len 128 --batch_size 25 --outdir /path/to/output/dir --truncation_psi 0.9
```
You can find the checkpoints from [here](https://drive.google.com/drive/folders/1ZlGmjRmjV4_ZzcU2t2RN0RdvFfeTniAW?usp=sharing)
## Reference
This code is mainly built upon [StyleGAN2-ADA](https://github.com/NVlabs/stylegan2-ada-pytorch) and [StyleGAN-V](https://github.com/universome/stylegan-v) repositories.
Baseline codes are from [MoCoGAN-HD](https://github.com/snap-research/MoCoGAN-HD), [VideoGPT](https://github.com/wilson1yan/VideoGPT), [DIGAN](https://github.com/sihyun-yu/digan), [StyleGAN-V](https://github.com/universome/stylegan-v)
## Bibtex
```
@article{shen2023mostganv,
author = {Xiaoqian Shen and Xiang Li and Mohamed Elhoseiny},
title = {MoStGAN-V: Video Generation with Temporal Motion Styles},
journal = {arXiv preprint arXiv:2304.02777},
year = {2023},
}
```