https://github.com/rasbt/cyclemoid-pytorch
Cyclemoid implementation for PyTorch
https://github.com/rasbt/cyclemoid-pytorch
Last synced: 6 months ago
JSON representation
Cyclemoid implementation for PyTorch
- Host: GitHub
- URL: https://github.com/rasbt/cyclemoid-pytorch
- Owner: rasbt
- License: mit
- Created: 2022-03-31T02:08:28.000Z (almost 4 years ago)
- Default Branch: main
- Last Pushed: 2022-04-02T14:17:36.000Z (almost 4 years ago)
- Last Synced: 2025-07-22T10:52:05.308Z (6 months ago)
- Language: Python
- Size: 1.86 MB
- Stars: 90
- Watchers: 3
- Forks: 7
- Open Issues: 5
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
This is an implementation of the cyclemoid activation function for PyTorch.
The cyclemoid function achieved state-of-the-art results in a recent benchmark with other popular activation functions as shown below:
**Note that this is a figure from the paper submitted on April 1st, 2022. An arxiv preprint will be uploaded soon.**
## Installation
You can install the cyclemoid-pytorch package via
```python
pip install cyclemoid_pytorch
```
## Usage
This package implements a `CycleMoid` class and a `cyclemoid` function. You can use these are drop-in replacements for any activation in PyTorch. For example
```python
from cyclemoid_pytorch import CycleMoid
torch.nn.Sequential(
# ...,
CycleMoid(), # instead of torch.nn.ReLU()
# ...
)
```
or
```python
from cyclemoid_pytorch import cyclemoid
# ...
def forward(self, x):
# ...
x = cyclemoid(x) # instead of torch.sigmoid(x)
```
## Visualization
```python
import matplotlib.pyplot as plt
import torch
from cyclemoid_pytorch import cyclemoid
x = torch.arange(-5, 5, 0.01)
y = cyclemoid(x)
plt.plot(x, y)
```
## Demo
For a concrete usage, check out the [demo notebook](docs/demo.ipynb).
## Appendix
You can now also use the cyclemoid activation in Keras.
```python
import tensorflow as tf
from cyclemoid_pytorch.easteregg import CycleMoid
tf.keras.utils.get_custom_objects()['cyclemoid'] = CycleMoid
model = tf.keras.Sequential(
[
tf.keras.Input(...),
tf.keras.layers.Conv2D(..., activation="cyclemoid"),
# ...
]
)
```