https://github.com/iceber/wasmcloud-wasi-nn

https://github.com/iceber/wasmcloud-wasi-nn

Last synced: over 1 year ago
JSON representation

• Host: GitHub
• URL: https://github.com/iceber/wasmcloud-wasi-nn
• Owner: Iceber
• Created: 2024-01-22T09:09:14.000Z (over 2 years ago)
• Default Branch: main
• Last Pushed: 2024-02-08T05:23:58.000Z (over 2 years ago)
• Last Synced: 2025-01-26T03:28:22.635Z (over 1 year ago)
• Language: Rust
• Size: 217 KB
• Stars: 1
• Watchers: 2
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

          
# [wasmCloud](https://github.com/wasmCloud/wasmCloud) & [wasi-nn](https://github.com/WebAssembly/wasi-nn)

wasmCloud-wasi-nn provides wasmCloud with [wasi-nn](https://github.com/WebAssembly/wasi-nn) compatibility.

The implementation of the wasi-nn capability provider is very flexible, and currently two types of providers have been implemented

* **Native OpenVINO**: This implementation is basically the same as the wasmtime implementation of [wasi-nn/openvino](https://github.com/bytecodealliance/wasmtime/blob/main/crates/wasi-nn/src/backend/openvino.rs)..

A simple demo can be seen [here](https://twitter.com/IceberGu/status/1742483365960601925), and a [more complete video](https://youtu.be/fK3DcnRe0Do) 

* **Candle based implementation**: [Candle](https://github.com/huggingface/candle) provides support for loading [many types of models](https://github.com/huggingface/candle?tab=readme-ov-file#features), e.g., **LLMs**, **Text to image Models**, **Image to text Models**, etc.

You can see a demo of Candle-based LLMs [here](https://twitter.com/IceberGu/status/1746782269870936177)

![candle provider arch](./candle-provider.png)

The code for providers hasn't been pushed yet, feel free to discuss with me in Slack or Issue!