https://github.com/moeru-ai/demodel
🚀🛸 Easily boost the speed of pulling your models and datasets from various of inference runtimes. (e.g. 🤗 HuggingFace, 🐫 Ollama, vLLM, and more!)
https://github.com/moeru-ai/demodel
dataset huggingface modelscope ollama p2p transformers transformers-js vllm
Last synced: 11 months ago
JSON representation
🚀🛸 Easily boost the speed of pulling your models and datasets from various of inference runtimes. (e.g. 🤗 HuggingFace, 🐫 Ollama, vLLM, and more!)
- Host: GitHub
- URL: https://github.com/moeru-ai/demodel
- Owner: moeru-ai
- License: mit
- Created: 2025-02-18T08:20:06.000Z (about 1 year ago)
- Default Branch: main
- Last Pushed: 2025-03-08T13:16:57.000Z (about 1 year ago)
- Last Synced: 2025-06-10T00:11:20.747Z (11 months ago)
- Topics: dataset, huggingface, modelscope, ollama, p2p, transformers, transformers-js, vllm
- Language: Rust
- Homepage:
- Size: 47.9 KB
- Stars: 5
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
Awesome Lists containing this project
README
# `demodel`
Easily boost the speed of pulling your models and datasets from various of inference runtimes. (e.g. [🤗 HuggingFace](https://huggingface.co/), [🐫 Ollama](https://ollama.com/), [vLLM](https://vllm.ai/), and more!)
- Out of the mind when dealing with the slow speed from the internet when pulling models and datasets?
- Already downloaded the model or dataset in another cluster or node, maybe Homelab server, but cannot share them easily?
- You got poor connection to HuggingFace or Ollama but got friends locally with models already?
- You want to serve your models and datasets to your friends locally?
`demodel` here to rescue!
## Features
Out of the box support for:
- [🤗 `huggingface-cli`](https://huggingface.co/docs/huggingface_hub/cli)
- [🤗 `transformers`](https://huggingface.co/docs/transformers/en/index)
- [Ollama](https://ollama.com/)
- [🤗 `transformers.js`](https://huggingface.co/docs/transformers.js/en/index) (both Browser and Node.js)
- [vLLM](https://github.com/vllm-project/vllm)
- [SGLang](https://github.com/sgl-project/sglang)
## Acknowledgement
- [`boring`](https://github.com/cloudflare/boring)
- [Why BoringSSL?](https://github.com/sfackler/rust-openssl/issues/1519#issue-984198410)
- [Oxy is Cloudflare's Rust-based next generation proxy framework](https://blog.cloudflare.com/introducing-oxy/)