Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
awesome-adaptive-computation
A curated reading list of research in Adaptive Computation, Dynamic Compute & Mixture of Experts (MoE).
https://github.com/koayon/awesome-adaptive-computation
Last synced: 5 days ago
JSON representation
-
About
- System 2 - ->
-
Mixture of Experts (Sparse MoE)
- Mixtral-8x7B
- PyTorch code
- DeMix pdf
- Task-MoE pdf
- ELMForest - Branch, Train, Merge (BTM)
- c-BTM
- official PyTorch code
- review paper
- PyTorch code
- model
- One Wide Feedforward paper
- code
- code
- official code
- code
- official Jax code
- HydraMoE
- official PyTorch code
- c-BTM code
- models
- MoE-Infinity - Infinity)
- Branch, Train, Mix (BTX)
- JetMoE
- Branch, Train, Mix (BTX)
- Multi-gate
- Gemini 1.5 Pro - dbrx-new-state-art-open-llm) is another powerful MoE model and it seems that MoE is now the go-to architecture for large models.
- here
- code
- Dynamic Routing in MoEs
- Task-MoE pdf
- c-BTM
- MoE-Infinity - Infinity)
- DeMix pdf
- review paper
- One Wide Feedforward paper
-
Other Modular Architectures
-
Early Exit: End-to-End Adaptive Computation
-
Adaptive Computation for Black-box models
- Reflexion
- Debate
- Chain of Thought
- Tree of Thought
- Chain of Verification
- pdf2
- pdf2
- blog
- Online Speculative Decoding
- inverse scaling
- PyTorch code
- pdf2
- PyTorch code
- Online Speculative Decoding
- Recurrent Drafter
- large n-gram models
- model mapping
- here
- blog
- Recurrent Drafter
- large n-gram models
- REST - on tokens from the web for the speculative decoding head.
- pytorch code
- Accelerated Speculative Sampling (ASpS) with Tree Monte Carlo
- blog
- PyTorch blog
- Debate
- Tree of Thought
- Chain of Thought
- pdf2
- inverse scaling
-
Continual Learning
-
Tools & Agents
-
Games
-
Pre-cursors to Adaptive Computation
-
Open Source Libraries
-
AI Safety
-
Other
-
Scaling Laws
-
More Compute Per Output Token
Programming Languages
Categories
Mixture of Experts (Sparse MoE)
73
Games
52
Adaptive Computation for Black-box models
49
Early Exit: End-to-End Adaptive Computation
20
Other Modular Architectures
19
Open Source Libraries
15
Scaling Laws
13
Continual Learning
13
Tools & Agents
12
Pre-cursors to Adaptive Computation
8
Other
7
More Compute Per Output Token
5
AI Safety
4
About
1
Sub Categories
Keywords
ai
3
mixture-of-experts
2
python
2
openai
2
gpt-4
2
language-model
2
llm
2
large-language-models
1
text
1
diffusion-models
1
quantization
1
pytorch
1
offloading
1
google-colab
1
deep-learning
1
colab-notebook
1
multi-modal
1
moe
1
large-vision-language-model
1
llm-inference
1
speculative-decoding
1
agent
1
agent-based-model
1
developer-tools
1
lms
1
artificial-intelligence
1
autonomous-agents
1
autonomous-agent
1
code-generation
1
codebase-generation
1
codegen
1
coding-assistant
1
gpt-engineer
1