Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/nanowell/ademamix-optimizer-pytorch

The AdEMAMix Optimizer: Better, Faster, Older.
https://github.com/nanowell/ademamix-optimizer-pytorch

ademamix artificial-intelligence deep-neural-networks machine-learning multimodal optimizer pytorch

Last synced: about 7 hours ago
JSON representation

The AdEMAMix Optimizer: Better, Faster, Older.

Host: GitHub
URL: https://github.com/nanowell/ademamix-optimizer-pytorch
Owner: nanowell
License: mit
Created: 2024-09-06T13:52:38.000Z (5 months ago)
Default Branch: main
Last Pushed: 2024-09-12T22:26:01.000Z (5 months ago)
Last Synced: 2024-10-06T02:06:08.915Z (4 months ago)
Topics: ademamix, artificial-intelligence, deep-neural-networks, machine-learning, multimodal, optimizer, pytorch
Language: Python
Homepage:
Size: 13.7 KB
Stars: 159
Watchers: 3
Forks: 9
Open Issues: 1
Metadata Files:
- Readme: README.md
- License: LICENSE

Awesome Lists containing this project

README

**THE ADEMAMIX OPTIMIZER: BETTER, FASTER, OLDER**

# Algo:

![image](https://github.com/user-attachments/assets/a37a9760-7b8d-4aab-b514-24918ffe2484)

# Experiments from paper:

![image](https://github.com/user-attachments/assets/d8a21fa5-6ff2-4f1b-a08e-165bb8f83d0b)
#
![image](https://github.com/user-attachments/assets/7a45788a-a71e-445b-85a0-a8f2403798b9)
#
![image](https://github.com/user-attachments/assets/dc426f87-cdca-45cb-9569-210544a4222e)

### Reference

Pagliardini, M., Ablin, P., & Grangier, D. (2024). The AdEMAMix Optimizer: Better, Faster, Older. arXiv preprint arXiv:2409.03137. [https://arxiv.org/abs/2409.03137](https://arxiv.org/abs/2409.03137)

## Support
If you find this project valuable, please consider starring it on GitHub. Your support is greatly appreciated, and we welcome any feedback or suggestions through GitHub issues.

For those interested in contributing further, we have a [GitHub Sponsors](https://github.com/sponsors/nanowell) page. Your sponsorship will significantly aid us in maintaining and enhancing this project. Thank you for your support!