https://github.com/willguimont/learnable_fourier_positional_encoding
Learnable Fourier Features for Multi-Dimensional Spatial Positional Encoding
https://github.com/willguimont/learnable_fourier_positional_encoding
attention-is-all-you-need deep-learning deep-neural-networks deeplearning positional-encoding pytorch transformer
Last synced: about 1 month ago
JSON representation
Learnable Fourier Features for Multi-Dimensional Spatial Positional Encoding
- Host: GitHub
- URL: https://github.com/willguimont/learnable_fourier_positional_encoding
- Owner: willGuimont
- License: mit
- Created: 2021-12-14T17:42:45.000Z (over 3 years ago)
- Default Branch: master
- Last Pushed: 2024-09-30T00:06:43.000Z (7 months ago)
- Last Synced: 2025-04-02T11:56:53.728Z (about 1 month ago)
- Topics: attention-is-all-you-need, deep-learning, deep-neural-networks, deeplearning, positional-encoding, pytorch, transformer
- Language: Python
- Homepage:
- Size: 6.84 KB
- Stars: 45
- Watchers: 2
- Forks: 9
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Learnable Fourier Features for Multi-Dimensional Spatial Positional Encoding
Implementation of [Learnable Fourier Features for Multi-Dimensional Spatial Positional Encoding](https://arxiv.org/abs/2106.02795) by Li, Si, Li, Hsieh and Bengio.
## Installation
```bash
pip install learnable_fourier_positional_encoding
```
## Usage
```python
import torch
from learnable_fourier_positional_encoding import LearnableFourierPositionalEncoding
G = 3
M = 17
x = torch.randn((97, G, M))
enc = LearnableFourierPositionalEncoding(G, M, 768, 32, 768, 10)
pex = enc(x)
print(pex.shape)
```