Projects in Awesome Lists tagged with positional-encoding

https://github.com/lucidrains/rotary-embedding-torch

Implementation of Rotary Embeddings, from the Roformer paper, in Pytorch

artificial-intelligence deep-learning positional-encoding

Last synced: 19 Dec 2024

https://github.com/vijaydwivedi75/gnn-lspe

Source code for GNN-LSPE (Graph Neural Networks with Learnable Structural and Positional Representations), ICLR 2022

attention geometric-deep-learning gnn gnn-lspe graph-deep-learning graph-neural-networks graph-representation-learning graph-transformer graphs lspe message-passing molecules positional-encoding representation-learning transformer-networks transformers

Last synced: 20 Dec 2024

https://github.com/VITA-Group/Ms-PoE

"Found in the Middle: How Language Models Use Long Contexts Better via Plug-and-Play Positional Encoding" Zhenyu Zhang, Runjin Chen, Shiwei Liu, Zhewei Yao, Olatunji Ruwase, Beidi Chen, Xiaoxia Wu, Zhangyang Wang.

large-language-models long-context lost-in-the-middle positional-encoding

Last synced: 19 Nov 2024

https://github.com/vita-group/ms-poe

"Found in the Middle: How Language Models Use Long Contexts Better via Plug-and-Play Positional Encoding" Zhenyu Zhang, Runjin Chen, Shiwei Liu, Zhewei Yao, Olatunji Ruwase, Beidi Chen, Xiaoxia Wu, Zhangyang Wang.

large-language-models long-context lost-in-the-middle positional-encoding

Last synced: 16 Nov 2024

https://github.com/aryaaftab/rotary-embedding-tensorflow

Implementation of Rotary Embeddings, from the Roformer paper, in Tensorflow

deeplearning positional-encoding tensorflow2

Last synced: 10 Nov 2024

https://github.com/dcarpintero/transformer101

Annotated vanilla implementation in PyTorch of the Transformer model introduced in 'Attention Is All You Need'

attention-is-all-you-need dot-product-attention dropout-layers encoder-decoder-architecture feedforward-neural-network gelu linear-layers multihead-attention normalization-layers positional-encoding pytorch self-attention softmax transfomer

Last synced: 20 Nov 2024

https://github.com/tillbeemelmanns/learnable_fourier_positional_encoding

Learnable Fourier Features for Multi-Dimensional Spatial Positional Encoding - Tensorflow

positional-encoding transfomer vision-transformer

Last synced: 25 Nov 2024