Projects in Awesome Lists tagged with positional-encoding

https://github.com/lucidrains/rotary-embedding-torch

Implementation of Rotary Embeddings, from the Roformer paper, in Pytorch

artificial-intelligence deep-learning positional-encoding

Last synced: 14 May 2025

https://github.com/liruilong940607/prope

Cameras as Relative Positional Encoding

multi-view positional-encoding transformer

Last synced: 07 Aug 2025

https://github.com/vijaydwivedi75/gnn-lspe

Source code for GNN-LSPE (Graph Neural Networks with Learnable Structural and Positional Representations), ICLR 2022

attention geometric-deep-learning gnn gnn-lspe graph-deep-learning graph-neural-networks graph-representation-learning graph-transformer graphs lspe message-passing molecules positional-encoding representation-learning transformer-networks transformers

Last synced: 03 Apr 2025

https://github.com/willguimont/learnable_fourier_positional_encoding

Learnable Fourier Features for Multi-Dimensional Spatial Positional Encoding

attention-is-all-you-need deep-learning deep-neural-networks deeplearning positional-encoding pytorch transformer

Last synced: 08 Oct 2025

https://github.com/vita-group/ms-poe

"Found in the Middle: How Language Models Use Long Contexts Better via Plug-and-Play Positional Encoding" Zhenyu Zhang, Runjin Chen, Shiwei Liu, Zhewei Yao, Olatunji Ruwase, Beidi Chen, Xiaoxia Wu, Zhangyang Wang.

large-language-models long-context lost-in-the-middle positional-encoding

Last synced: 19 Apr 2025

https://github.com/VITA-Group/Ms-PoE

"Found in the Middle: How Language Models Use Long Contexts Better via Plug-and-Play Positional Encoding" Zhenyu Zhang, Runjin Chen, Shiwei Liu, Zhewei Yao, Olatunji Ruwase, Beidi Chen, Xiaoxia Wu, Zhangyang Wang.

large-language-models long-context lost-in-the-middle positional-encoding

Last synced: 16 May 2025

https://github.com/aryaaftab/rotary-embedding-tensorflow

Implementation of Rotary Embeddings, from the Roformer paper, in Tensorflow

deeplearning positional-encoding tensorflow2

Last synced: 26 Aug 2025

https://github.com/liaoyanqing666/transformer_pytorch

完整的原版transformer程序，complete origin transformer program

beginner multi-head-attention positional-encoding python pytorch transformer

Last synced: 10 Oct 2025

https://github.com/Ave-Sergeev/Tictonix

Crate for `Embedings` and `Positional Encoding` (Rust) 2025

deep-learning embeddings machine-learning math positional-encoding rust

Last synced: 08 May 2025

https://github.com/rhoadesscholar/rose

PyTorch implementation of Rotary Spatial Embeddings

attention positional-encoding pytorch rotary-position-embedding rotary-position-encoding transformer

Last synced: 07 Sep 2025

https://github.com/dcarpintero/transformer101

Annotated vanilla implementation in PyTorch of the Transformer model introduced in 'Attention Is All You Need'.

attention-is-all-you-need dot-product-attention dropout-layers encoder-decoder-architecture feedforward-neural-network gelu linear-layers multihead-attention normalization-layers positional-encoding pytorch self-attention softmax transfomer

Last synced: 07 May 2026

https://github.com/jhaayush2004/my-transformer

Code implementation of Transformer Model in "Attention is All You Need" in PyTorch.

cross-attention encoder-decoder feedforward-neural-network positional-encoding python pytorch self-attention transformer-architecture

Last synced: 13 Oct 2025

https://github.com/ave-sergeev/tictonix

Crate for `Embedings` and `Positional Encoding` (Rust) 2025

embeddings positional-encoding rust

Last synced: 11 Jun 2025

https://github.com/joymohanty8999/transformer-translate-cpp

This repository contains a custom implementation of the Transformer architecture from scratch in C++. Inspired by the “Attention Is All You Need” paper, the project includes all key components of the Transformer, such as multi-head attention, positional encoding, and encoder-decoder layers.

attention-mechanism cmakelists cpp17 deep-learning positional-encoding transformers

Last synced: 08 Aug 2025

https://github.com/tillbeemelmanns/learnable_fourier_positional_encoding

Learnable Fourier Features for Multi-Dimensional Spatial Positional Encoding - Tensorflow

positional-encoding transfomer vision-transformer

Last synced: 19 May 2026

https://github.com/khushirajurkar/vision-transformer-image-classification

A Vision Transformer (ViT) implementation for image classification using CIFAR-10 dataset, leveraging HuggingFace's Trainer API for computational efficiency

cifar-10 computer-vision data-augmentation deep-learning huggingface image-classification machine-learning model-evaluation neural-networks patch-encoding positional-encoding self-attention trainer-api transfer-learning transformer vision-transformer