https://github.com/hazdzz/scalenorm

The PyTorch implementation of ScaleNorm.
https://github.com/hazdzz/scalenorm

normalization pytorch

Last synced: 12 days ago
JSON representation

The PyTorch implementation of ScaleNorm.

• Host: GitHub
• URL: https://github.com/hazdzz/scalenorm
• Owner: hazdzz
• License: mit
• Created: 2024-08-22T05:20:25.000Z (3 months ago)
• Default Branch: main
• Last Pushed: 2024-09-03T12:58:07.000Z (2 months ago)
• Last Synced: 2024-10-17T12:37:41.144Z (29 days ago)
• Topics: normalization, pytorch
• Language: Python
• Homepage:
• Size: 6.84 KB
• Stars: 0
• Watchers: 2
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# ScaleNorm

## About

The PyTorch implementation of ScaleNorm from the paper *Transformers without Tears: Improving the Normalization of Self-Attention*.

## Paper

https://arxiv.org/abs/1910.05895

## Citation

```

@inproceedings{nguyen-salazar-2019-transformers,

  title     = {Transformers without Tears: Improving the Normalization of Self-Attention},

  author    = {Nguyen, Toan Q. and Salazar, Julian},

  booktitle = {Proceedings of the 16th International Conference on Spoken Language Translation},

  month     = {11},

  year      = {2019},

  publisher = {Association for Computational Linguistics}

}

```