https://github.com/goliaro/moe_inference

https://github.com/goliaro/moe_inference

Last synced: about 1 month ago
JSON representation

• Host: GitHub
• URL: https://github.com/goliaro/moe_inference
• Owner: goliaro
• Created: 2022-09-23T20:56:09.000Z (over 2 years ago)
• Default Branch: main
• Last Pushed: 2023-04-21T22:12:12.000Z (over 1 year ago)
• Last Synced: 2024-10-16T14:12:31.688Z (3 months ago)
• Language: Jupyter Notebook
• Homepage: https://moe-inference.readthedocs.io
• Size: 46.9 MB
• Stars: 1
• Watchers: 3
• Forks: 1
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# MoE inference

This repository contains our work on the design and implementation of a scalable inference system for MoE.

For now, we are familiarizing with existing solutions and benchmarking them. In particular, you can find scripts/instructions/containers/raw results for the following platforms:

- Fairseq by Meta: [benchmark/fairseq](./benchmark/fairseq) folder

- DeepSpeed-MoE by Microsoft: [benchmark/deepspeed_moe](./benchmark/deepspeed_moe) folder 

- FasterTransformer by NVIDIA: [benchmark/faster_transformer](./benchmark/faster_transformer) folder

Please check the [docs](./docs) folder for additional info.