https://github.com/BlinkDL/SmallInitEmb

LayerNorm(SmallInit(Embedding)) in a Transformer to improve convergence
https://github.com/BlinkDL/SmallInitEmb

Last synced: 3 months ago
JSON representation

LayerNorm(SmallInit(Embedding)) in a Transformer to improve convergence

• Host: GitHub
• URL: https://github.com/BlinkDL/SmallInitEmb
• Owner: BlinkDL
• Created: 2022-02-17T19:26:01.000Z (over 2 years ago)
• Default Branch: main
• Last Pushed: 2022-02-21T10:45:18.000Z (over 2 years ago)
• Last Synced: 2024-07-18T22:20:51.441Z (4 months ago)
• Language: Python
• Homepage:
• Size: 29.8 MB
• Stars: 45
• Watchers: 3
• Forks: 3
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project