https://github.com/antonio-f/mixture-of-experts-from-scratch

Mixture of Experts from scratch
https://github.com/antonio-f/mixture-of-experts-from-scratch

artificial-intelligence conditional-computation easy from-scratch gating hands-on jupyter-notebook machine-learning mixture-of-experts nlp python pytorch routing simple small-dataset tutorial

Last synced: about 4 hours ago
JSON representation

Mixture of Experts from scratch

• Host: GitHub
• URL: https://github.com/antonio-f/mixture-of-experts-from-scratch
• Owner: antonio-f
• Created: 2024-02-04T09:56:56.000Z (about 1 year ago)
• Default Branch: main
• Last Pushed: 2024-04-12T21:58:15.000Z (10 months ago)
• Last Synced: 2024-04-13T05:51:00.985Z (10 months ago)
• Topics: artificial-intelligence, conditional-computation, easy, from-scratch, gating, hands-on, jupyter-notebook, machine-learning, mixture-of-experts, nlp, python, pytorch, routing, simple, small-dataset, tutorial
• Language: Jupyter Notebook
• Homepage:
• Size: 234 KB
• Stars: 1
• Watchers: 2
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# Mixture of Experts from scratch

Simple implementation of Mixture of Experts technique.

Also available on [Substack](https://monads.substack.com/p/mixture-of-experts-from-scratch).