https://github.com/joelburget/moe-sae

https://github.com/joelburget/moe-sae

Last synced: 3 months ago
JSON representation

• Host: GitHub
• URL: https://github.com/joelburget/moe-sae
• Owner: joelburget
• License: mit
• Created: 2024-08-11T17:12:20.000Z (11 months ago)
• Default Branch: main
• Last Pushed: 2024-12-28T00:22:55.000Z (6 months ago)
• Last Synced: 2025-03-20T14:52:55.286Z (3 months ago)
• Language: Python
• Size: 17.6 KB
• Stars: 1
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
I'm now focused on Olmoe, though `train_mixtral.py` still exists.

Sweep:

```

wandb sweep --project moe-sae olmoe-config.yaml

wandb agent 

```

Train:

```

python3 train_olmoe.py

```