Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/dawi369/oxide
🦀 A barebones inferencing server in Rust, using Hyper for HTTP logic and ORT for inferencing models. Containerized and easily hostable. Complete control over type of model and inferencing behavior!
https://github.com/dawi369/oxide
container docker http hyper inferencing model ort rust server
Last synced: about 1 month ago
JSON representation
🦀 A barebones inferencing server in Rust, using Hyper for HTTP logic and ORT for inferencing models. Containerized and easily hostable. Complete control over type of model and inferencing behavior!
- Host: GitHub
- URL: https://github.com/dawi369/oxide
- Owner: dawi369
- License: mit
- Created: 2024-11-19T10:52:37.000Z (about 2 months ago)
- Default Branch: main
- Last Pushed: 2024-11-19T12:49:59.000Z (about 1 month ago)
- Last Synced: 2024-11-19T13:30:58.667Z (about 1 month ago)
- Topics: container, docker, http, hyper, inferencing, model, ort, rust, server
- Language: Rust
- Homepage:
- Size: 1.12 MB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Oxide
Oxide is a lightweight containerized model inferencing server built without unnecessary frameworks! 🦀
---
*Oxide Home Page*
---
## Features
- Inferences custom models on any hardware you want.
- Choose between custom tokenizers, full control over your model.
- incredibly lightweight
## Installation
Installing Oxide can be done from source.
### From Source
Clone this repository,
```sh
git clone https://github.com/dawi369/Oxide.git
```
To render run server, navigate to the root of the repo directory, and run
```sh
cargo run --release
```
## Usage
```
Oxide: Inference your custom models on the cloud!
Usage:
curl -X POST http://>/inference \
-H "Content-Type: text/plain" \
-d "Please generate a warm welcome."
```
## Author
(c) [David Erwin](https://www.daviderwin.me/)