Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/underneathall/pinferencia
Python + Inference - Model Deployment library in Python. Simplest model inference server ever.
https://github.com/underneathall/pinferencia
ai artificial-intelligence computer-vision data-science deep-learning huggingface inference inference-server machine-learning model-deployment model-serving modelserver nlp paddlepaddle predict python pytorch serving tensorflow transformers
Last synced: 10 days ago
JSON representation
Python + Inference - Model Deployment library in Python. Simplest model inference server ever.
- Host: GitHub
- URL: https://github.com/underneathall/pinferencia
- Owner: underneathall
- License: apache-2.0
- Created: 2022-04-04T15:50:47.000Z (over 2 years ago)
- Default Branch: main
- Last Pushed: 2023-02-14T22:50:48.000Z (over 1 year ago)
- Last Synced: 2024-10-24T21:22:23.524Z (17 days ago)
- Topics: ai, artificial-intelligence, computer-vision, data-science, deep-learning, huggingface, inference, inference-server, machine-learning, model-deployment, model-serving, modelserver, nlp, paddlepaddle, predict, python, pytorch, serving, tensorflow, transformers
- Language: Python
- Homepage: https://pinferencia.underneathall.app
- Size: 9.57 MB
- Stars: 562
- Watchers: 41
- Forks: 87
- Open Issues: 17
-
Metadata Files:
- Readme: Readme.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
Awesome Lists containing this project
README
Simple, but Powerful.
---
English Doc
|
中文文档
|
中文Readme
Help wanted. Translation, rap lyrics, all wanted. Feel free to create an issue.
---
**Pinferencia** tries to be the simplest machine learning inference server ever!
**Three extra lines and your model goes online**.
Serving a model with GUI and REST API has never been so easy.
If you want to
- give your model a **GUI** and **REST API**
- find a **simple but robust** way to serve your model
- write **minimal** codes while maintain controls over you service
- **avoid** any **heavy-weight** solutions
- **compatible** with other tools/platforms
You're at the right place.
## Features
**Pinferencia** features include:
- **Fast to code, fast to go alive**. Minimal codes needed, minimal transformation needed. Just based on what you have.
- **100% Test Coverage**: Both statement and branch coverages, no kidding. Have you ever known any model serving tool so seriously tested?
- **Easy to use, easy to understand**.
- **A pretty and clean GUI** out of box.
- **Automatic API documentation page**. All API explained in details with online try-out feature.
- **Serve any model**, even a single function can be served.
- **Support Kserve API**, compatible with Kubeflow, TF Serving, Triton and TorchServe. There is no pain switching to or from them, and **Pinferencia** is much faster for prototyping!
## Install
### Recommend
```bash
pip install "pinferencia[streamlit]"
```
### Backend Only
```bash
pip install "pinferencia"
```
## Quick Start
**Serve Any Model**
```python title="app.py"
from pinferencia import Server
class MyModel:
def predict(self, data):
return sum(data)
model = MyModel()
service = Server()
service.register(model_name="mymodel", model=model, entrypoint="predict")
```
Just run:
```
pinfer app:service
```
Hooray, your service is alive. Go to http://127.0.0.1:8501/ and have fun.
**Any Deep Learning Models?** Just as easy. Simple train or load your model, and register it with the service. Go alive immediately.
**Hugging Face**
Details: [HuggingFace Pipeline - Vision](https://pinferencia.underneathall.app/ml/huggingface/pipeline/vision/)
```python title="app.py" linenums="1"
from transformers import pipeline
from pinferencia import Server
vision_classifier = pipeline(task="image-classification")
def predict(data):
return vision_classifier(images=data)
service = Server()
service.register(model_name="vision", model=predict)
```
**Pytorch**
```python title="app.py"
import torch
from pinferencia import Server
# train your models
model = "..."
# or load your models (1)
# from state_dict
model = TheModelClass(*args, **kwargs)
model.load_state_dict(torch.load(PATH))
# entire model
model = torch.load(PATH)
# torchscript
model = torch.jit.load('model_scripted.pt')
model.eval()
service = Server()
service.register(model_name="mymodel", model=model)
```
**Tensorflow**
```python title="app.py"
import tensorflow as tf
from pinferencia import Server
# train your models
model = "..."
# or load your models (1)
# saved_model
model = tf.keras.models.load_model('saved_model/model')
# HDF5
model = tf.keras.models.load_model('model.h5')
# from weights
model = create_model()
model.load_weights('./checkpoints/my_checkpoint')
loss, acc = model.evaluate(test_images, test_labels, verbose=2)
service = Server()
service.register(model_name="mymodel", model=model, entrypoint="predict")
```
Any model of any framework will just work the same way. Now run `uvicorn app:service --reload` and enjoy!
## Contributing
If you'd like to contribute, details are [here](./CONTRIBUTING.md)