https://github.com/devzhk/implicit-competitive-regularization
Code for: Implicit Competitive Regularization in GANs
https://github.com/devzhk/implicit-competitive-regularization
Last synced: 8 months ago
JSON representation
Code for: Implicit Competitive Regularization in GANs
- Host: GitHub
- URL: https://github.com/devzhk/implicit-competitive-regularization
- Owner: devzhk
- License: apache-2.0
- Created: 2019-10-08T04:45:27.000Z (about 6 years ago)
- Default Branch: master
- Last Pushed: 2022-01-14T01:55:14.000Z (almost 4 years ago)
- Last Synced: 2025-04-30T05:43:49.347Z (8 months ago)
- Language: Python
- Homepage:
- Size: 1.32 MB
- Stars: 114
- Watchers: 10
- Forks: 19
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
Implicit competitive regularization in GANs
This code contains experiments for our ICML paper: [Implicit competitive regularization in GANs](http://proceedings.mlr.press/v119/schaefer20a.html).
## For competitive optimization
Optimizers in this package are for competitive optimization problems, given by
$$
\min_{x}\max_{y} f(x,y)
$$
### Install through pip
`pip install CGDs`
See details at CGDs package: [CGDs · PyPI](https://pypi.org/project/CGDs/).
You can also directly copy the folder 'optims' to your workspace.
## How to use
The package contains the original Compeititive Gradient Descent (BCGD), and the Adaptive Competitive Gradient Descent (ACGD).
Quickstart with notebook: [Examples of using ACGD](https://colab.research.google.com/drive/1-52aReaBAPNBtq2NcHxKkVIbdVXdyqtH?usp=sharing).
**It's important to force cudnn to benchmark and pick the best algo.**
Check more details at [cgds-package: Package for CGD and ACGD optimizers ](https://github.com/devzhk/cgds-package).
```python
import torch
torch.backends.cudnn.benchmark = True
from CGDs import ACGD
device = torch.device('cuda:0')
lr = 0.0001
G = Generator()
D = Discriminator()
optimizer = ACGD(max_params=G.parameters(), min_params=D.parameters(), lr_max=lr, lr_min=lr, device=device)
# max_parems is maximizing the objective function while the min_params is trying to minimizing it.
# BCGD(max_params=G.parameters(), min_params=D.parameters(), lr_max=lr, lr_min=lr, device=device)
# ACGD: Adaptive CGD;
for img in dataloader:
d_real = D(img)
z = torch.randn((batch_size, z_dim), device=device)
d_fake = D(G(z))
loss = criterion(d_real, d_fake)
optimizer.zero_grad()
optimizer.step(loss=loss)
```
==**Warning**==:
1. **zero sum** game setting only. This implementation uses conjugate gradient method to solve matrix inversion efficiently, which requires the matrix to be positive definite. If you are using competitive gradient descent (CGD) algorithm for non-zero sum games, please check more details in CGD paper https://arxiv.org/abs/1905.12103. For example, GMRES (the generalized minimal residual) algorithm can be a solver for non-zero sum setting.
2. This implementation doesn't work with torch.nn.parallel.DistributedDataParallel module because we need autograd.grad() to compute Hessian vector product. See details at [DDP doc]([DistributedDataParallel — PyTorch 1.7.0 documentation](https://pytorch.org/docs/stable/generated/torch.nn.parallel.DistributedDataParallel.html)) .
## Citation
Please cite the following paper if you find this code useful. Thanks!
```
@misc{schfer2019implicit,
title={Implicit competitive regularization in GANs},
author={Florian Schäfer and Hongkai Zheng and Anima Anandkumar},
year={2019},
eprint={1910.05852},
archivePrefix={arXiv},
primaryClass={cs.LG}
}
```