https://github.com/huggingface/ratchet

A cross-platform browser ML framework.
https://github.com/huggingface/ratchet

Last synced: 5 days ago
JSON representation

A cross-platform browser ML framework.

• Host: GitHub
• URL: https://github.com/huggingface/ratchet
• Owner: huggingface
• License: mit
• Created: 2024-01-11T15:00:17.000Z (10 months ago)
• Default Branch: master
• Last Pushed: 2024-10-29T10:37:18.000Z (5 days ago)
• Last Synced: 2024-10-29T10:56:28.836Z (5 days ago)
• Language: Rust
• Homepage: https://ratchet.sh
• Size: 14 MB
• Stars: 613
• Watchers: 8
• Forks: 31
• Open Issues: 31
• Metadata Files:
  • Readme: README.md
  • Contributing: CONTRIBUTING.md
  • License: LICENSE

Awesome Lists containing this project

README

        




Demo Site | Discord | Roadmap



A web-first, cross-platform ML developer toolkit








We are on a mission to bring fast, cross platform GPU accelerated inference on native + browser.

> [!NOTE]  

> Ratchet is currently in active development. We are working on the engine, adding more models and improving compatibility. Please, reach out if you'd like to help!

## Getting Started

The easiest way to experience Ratchet is to check out our [Hugging Face spaces](https://huggingface.co/FL33TW00D-HF):

- [Whisper](https://huggingface.co/spaces/FL33TW00D-HF/ratchet-whisper)

- [Phi](https://huggingface.co/spaces/FL33TW00D-HF/ratchet-phi)

To dig deeper, check out the [examples](https://github.com/FL33TW00D/ratchet/tree/master/examples)

We welcome contributions from the community. If you have any ideas or suggestions, please feel free to open an issue or pull request.

### Javascript

```javascript

// Asynchronous loading & caching with IndexedDB

let model = await Model.load(AvailableModels.WHISPER_TINY, Quantization.Q8, (p: number) => setProgress(p))

let result = await model.run({ input });

```

### Rust

Rust crate & CLI coming soon...

## Philosophy

We want a toolkit for developers to make integrating performant AI functionality into existing production applications easy.

The following principles will help us accomplish this:

1. **Inference only**

2. **WebGPU/CPU only**

3. First class quantization support

4. Lazy computation

5. Inplace by default

## Supported Models

- Whisper

- Phi 2 & 3

- Moondream

## Upcoming Models

- Gemini 2 2B