https://github.com/rasbt/try-lion-optimizer

https://github.com/rasbt/try-lion-optimizer

Last synced: about 1 year ago
JSON representation

• Host: GitHub
• URL: https://github.com/rasbt/try-lion-optimizer
• Owner: rasbt
• License: apache-2.0
• Created: 2023-03-09T20:31:09.000Z (over 3 years ago)
• Default Branch: main
• Last Pushed: 2023-03-09T21:00:44.000Z (over 3 years ago)
• Last Synced: 2025-05-07T16:04:50.035Z (about 1 year ago)
• Language: Jupyter Notebook
• Size: 831 KB
• Stars: 14
• Watchers: 2
• Forks: 1
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

          
# try-lion-optimizer

Simple comparison between the [Mar 2023 Lion optimizer](https://github.com/lucidrains/lion-pytorch) and Adam and SGD for finetuning DistilBert.

![lion-optimizer](fig/lion-optimizer.png)