{"id":13784214,"url":"https://github.com/underneathall/pinferencia","last_synced_at":"2025-10-08T17:11:34.824Z","repository":{"id":37160476,"uuid":"477769513","full_name":"underneathall/pinferencia","owner":"underneathall","description":"Python + Inference - Model Deployment library in Python. Simplest model inference server ever.","archived":false,"fork":false,"pushed_at":"2023-02-14T22:50:48.000Z","size":10034,"stargazers_count":551,"open_issues_count":17,"forks_count":83,"subscribers_count":38,"default_branch":"main","last_synced_at":"2025-09-30T19:06:23.049Z","etag":null,"topics":["ai","artificial-intelligence","computer-vision","data-science","deep-learning","huggingface","inference","inference-server","machine-learning","model-deployment","model-serving","modelserver","nlp","paddlepaddle","predict","python","pytorch","serving","tensorflow","transformers"],"latest_commit_sha":null,"homepage":"https://pinferencia.underneathall.app","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/underneathall.png","metadata":{"files":{"readme":"Readme.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-04-04T15:50:47.000Z","updated_at":"2025-09-30T13:17:02.000Z","dependencies_parsed_at":"2024-08-03T19:02:58.611Z","dependency_job_id":"87640b6d-aa09-44e4-9d3b-9c4d7b9eaaea","html_url":"https://github.com/underneathall/pinferencia","commit_stats":null,"previous_names":[],"tags_count":6,"template":false,"template_full_name":null,"purl":"pkg:github/underneathall/pinferencia","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/underneathall%2Fpinferencia","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/underneathall%2Fpinferencia/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/underneathall%2Fpinferencia/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/underneathall%2Fpinferencia/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/underneathall","download_url":"https://codeload.github.com/underneathall/pinferencia/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/underneathall%2Fpinferencia/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":278981518,"owners_count":26079640,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-08T02:00:06.501Z","response_time":56,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","artificial-intelligence","computer-vision","data-science","deep-learning","huggingface","inference","inference-server","machine-learning","model-deployment","model-serving","modelserver","nlp","paddlepaddle","predict","python","pytorch","serving","tensorflow","transformers"],"created_at":"2024-08-03T19:00:37.446Z","updated_at":"2025-10-08T17:11:34.807Z","avatar_url":"https://github.com/underneathall.png","language":"Python","funding_links":[],"categories":["*Ops for AI","Frameworks/Servers for Serving"],"sub_categories":["Model Serving \u0026 Inference"],"readme":"![Pinferencia](/docs/assets/images/logo_header.png)\n\n\u003cp align=\"center\"\u003e\n    \u003cem\u003eSimple, but Powerful.\u003c/em\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n    \u003ca href=\"https://lgtm.com/projects/g/underneathall/pinferencia/context:python\"\u003e\n        \u003cimg alt=\"Language grade: Python\" src=\"https://img.shields.io/lgtm/grade/python/g/underneathall/pinferencia.svg?logo=lgtm\u0026logoWidth=18\"/\u003e\n    \u003c/a\u003e\n    \u003ca href=\"https://codecov.io/gh/underneathall/pinferencia\"\u003e\n        \u003cimg src=\"https://codecov.io/gh/underneathall/pinferencia/branch/main/graph/badge.svg?token=M7J77E4IWC\"/\u003e\n    \u003c/a\u003e\n    \u003ca href=\"https://opensource.org/licenses/Apache-2.0\"\u003e\n        \u003cimg src=\"https://img.shields.io/badge/License-Apache_2.0-blue.svg\"/\u003e\n    \u003c/a\u003e\n    \u003ca href=\"https://pypi.org/project/pinferencia/\"\u003e\n        \u003cimg alt=\"PyPI\" src=\"https://img.shields.io/pypi/v/pinferencia?color=green\"\u003e\n    \u003c/a\u003e\n    \u003cimg alt=\"PyPI - Python Version\" src=\"https://img.shields.io/pypi/pyversions/pinferencia\"\u003e\n\u003c/p\u003e\n\n---\n\n\u003cp align=\"center\"\u003e\n\u003ca href=\"https://pinferencia.underneathall.app\" target=\"_blank\"\u003e\n    English Doc\n\u003c/a\u003e |\n\u003ca href=\"https://pinferencia.underneathall.app/0.2.1/zh/\" target=\"_blank\"\u003e\n    中文文档\n\u003c/a\u003e|\n\u003ca href=\"./Readme.zh.md\" target=\"_blank\"\u003e\n    中文Readme\n\u003c/a\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n    \u003cem\u003eHelp wanted. Translation, rap lyrics, all wanted. Feel free to create an issue.\u003c/em\u003e\n\u003c/p\u003e\n\n---\n\n**Pinferencia** tries to be the simplest machine learning inference server ever!\n\n**Three extra lines and your model goes online**.\n\nServing a model with GUI and REST API has never been so easy.\n\n![Pinferencia-GUI](/docs/assets/images/examples/translation-gui.png)\n\n![Pinferencia-REST API](/docs/assets/images/examples/translate-app.png)\n\nIf you want to\n\n- give your model a **GUI** and **REST API**\n- find a **simple but robust** way to serve your model\n- write **minimal** codes while maintain controls over you service\n- **avoid** any **heavy-weight** solutions\n- **compatible** with other tools/platforms\n\nYou're at the right place.\n\n## Features\n\n**Pinferencia** features include:\n\n- **Fast to code, fast to go alive**. Minimal codes needed, minimal transformation needed. Just based on what you have.\n- **100% Test Coverage**: Both statement and branch coverages, no kidding. Have you ever known any model serving tool so seriously tested?\n- **Easy to use, easy to understand**.\n- **A pretty and clean GUI** out of box.\n- **Automatic API documentation page**. All API explained in details with online try-out feature.\n- **Serve any model**, even a single function can be served.\n- **Support Kserve API**, compatible with Kubeflow, TF Serving, Triton and TorchServe. There is no pain switching to or from them, and **Pinferencia** is much faster for prototyping!\n\n## Install\n\n### Recommend\n\n```bash\npip install \"pinferencia[streamlit]\"\n```\n\n### Backend Only\n\n```bash\npip install \"pinferencia\"\n```\n\n## Quick Start\n\n**Serve Any Model**\n\n```python title=\"app.py\"\nfrom pinferencia import Server\n\n\nclass MyModel:\n    def predict(self, data):\n        return sum(data)\n\n\nmodel = MyModel()\n\nservice = Server()\nservice.register(model_name=\"mymodel\", model=model, entrypoint=\"predict\")\n```\n\nJust run:\n\n```\npinfer app:service\n```\n\nHooray, your service is alive. Go to http://127.0.0.1:8501/ and have fun.\n\n**Any Deep Learning Models?** Just as easy. Simple train or load your model, and register it with the service. Go alive immediately.\n\n**Hugging Face**\n\nDetails: [HuggingFace Pipeline - Vision](https://pinferencia.underneathall.app/ml/huggingface/pipeline/vision/)\n\n```python title=\"app.py\" linenums=\"1\"\nfrom transformers import pipeline\n\nfrom pinferencia import Server\n\nvision_classifier = pipeline(task=\"image-classification\")\n\n\ndef predict(data):\n    return vision_classifier(images=data)\n\n\nservice = Server()\nservice.register(model_name=\"vision\", model=predict)\n\n```\n\n**Pytorch**\n\n```python title=\"app.py\"\nimport torch\n\nfrom pinferencia import Server\n\n\n# train your models\nmodel = \"...\"\n\n# or load your models (1)\n# from state_dict\nmodel = TheModelClass(*args, **kwargs)\nmodel.load_state_dict(torch.load(PATH))\n\n# entire model\nmodel = torch.load(PATH)\n\n# torchscript\nmodel = torch.jit.load('model_scripted.pt')\n\nmodel.eval()\n\nservice = Server()\nservice.register(model_name=\"mymodel\", model=model)\n```\n\n**Tensorflow**\n\n```python title=\"app.py\"\nimport tensorflow as tf\n\nfrom pinferencia import Server\n\n\n# train your models\nmodel = \"...\"\n\n# or load your models (1)\n# saved_model\nmodel = tf.keras.models.load_model('saved_model/model')\n\n# HDF5\nmodel = tf.keras.models.load_model('model.h5')\n\n# from weights\nmodel = create_model()\nmodel.load_weights('./checkpoints/my_checkpoint')\nloss, acc = model.evaluate(test_images, test_labels, verbose=2)\n\nservice = Server()\nservice.register(model_name=\"mymodel\", model=model, entrypoint=\"predict\")\n```\n\nAny model of any framework will just work the same way. Now run `uvicorn app:service --reload` and enjoy!\n\n\n## Contributing\n\nIf you'd like to contribute, details are [here](./CONTRIBUTING.md)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Funderneathall%2Fpinferencia","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Funderneathall%2Fpinferencia","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Funderneathall%2Fpinferencia/lists"}