https://github.com/willguimont/learnable_fourier_positional_encoding

Learnable Fourier Features for Multi-Dimensional Spatial Positional Encoding
https://github.com/willguimont/learnable_fourier_positional_encoding

attention-is-all-you-need deep-learning deep-neural-networks deeplearning positional-encoding pytorch transformer

Last synced: 4 days ago
JSON representation

Learnable Fourier Features for Multi-Dimensional Spatial Positional Encoding

• Host: GitHub
• URL: https://github.com/willguimont/learnable_fourier_positional_encoding
• Owner: willGuimont
• License: mit
• Created: 2021-12-14T17:42:45.000Z (about 3 years ago)
• Default Branch: master
• Last Pushed: 2024-09-30T00:06:43.000Z (4 months ago)
• Last Synced: 2025-01-23T19:11:13.669Z (18 days ago)
• Topics: attention-is-all-you-need, deep-learning, deep-neural-networks, deeplearning, positional-encoding, pytorch, transformer
• Language: Python
• Homepage:
• Size: 6.84 KB
• Stars: 43
• Watchers: 3
• Forks: 9
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# Learnable Fourier Features for Multi-Dimensional Spatial Positional Encoding

Implementation of [Learnable Fourier Features for Multi-Dimensional Spatial Positional Encoding](https://arxiv.org/abs/2106.02795) by Li, Si, Li, Hsieh and Bengio.

## Installation

```bash

pip install learnable_fourier_positional_encoding

```

## Usage

```python

import torch

from learnable_fourier_positional_encoding import LearnableFourierPositionalEncoding

G = 3

M = 17

x = torch.randn((97, G, M))

enc = LearnableFourierPositionalEncoding(G, M, 768, 32, 768, 10)

pex = enc(x)

print(pex.shape)

```