{"id":13699546,"url":"https://github.com/uber/neuropod","last_synced_at":"2025-06-11T17:40:02.900Z","repository":{"id":38209927,"uuid":"167240425","full_name":"uber/neuropod","owner":"uber","description":"A uniform interface to run deep learning models from multiple frameworks","archived":false,"fork":false,"pushed_at":"2024-01-03T06:09:01.000Z","size":3344,"stargazers_count":935,"open_issues_count":53,"forks_count":77,"subscribers_count":23,"default_branch":"master","last_synced_at":"2025-05-04T16:42:28.203Z","etag":null,"topics":["deep-learning","deeplearning","incubation","inference","keras","machine-learning","machinelearning","pytorch","tensorflow"],"latest_commit_sha":null,"homepage":"https://neuropod.ai","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/uber.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":"AUTHORS","dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-01-23T19:22:22.000Z","updated_at":"2025-04-11T09:13:59.000Z","dependencies_parsed_at":"2024-11-13T09:17:02.819Z","dependency_job_id":null,"html_url":"https://github.com/uber/neuropod","commit_stats":null,"previous_names":[],"tags_count":15,"template":false,"template_full_name":null,"purl":"pkg:github/uber/neuropod","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/uber%2Fneuropod","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/uber%2Fneuropod/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/uber%2Fneuropod/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/uber%2Fneuropod/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/uber","download_url":"https://codeload.github.com/uber/neuropod/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/uber%2Fneuropod/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":259308162,"owners_count":22837974,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","deeplearning","incubation","inference","keras","machine-learning","machinelearning","pytorch","tensorflow"],"created_at":"2024-08-02T20:00:36.035Z","updated_at":"2025-06-11T17:40:02.847Z","avatar_url":"https://github.com/uber.png","language":"C++","readme":"# Neuropod\n\n## What is Neuropod?\n\n[Neuropod](https://github.com/uber/neuropod) is a library that provides a uniform interface to run deep learning models from multiple frameworks in C++ and Python. Neuropod makes it easy for researchers to build models in a framework of their choosing while also simplifying productionization of these models.\n\nIt currently supports TensorFlow, PyTorch, TorchScript, Keras and [Ludwig](http://ludwig.ai).\n\nFor more information:\n\n - [Uber Engineering blog post introducing Neuropod](https://eng.uber.com/introducing-neuropod/)\n - [Talk at NVIDIA GTC Spring 2021](https://www.nvidia.com/en-us/on-demand/session/gtcspring21-s31643/)\n\n## Why use Neuropod?\n\n#### Run models from any supported framework using one API\n\nRunning a TensorFlow model looks exactly like running a PyTorch model.\n\n```py\nx = np.array([1, 2, 3, 4])\ny = np.array([5, 6, 7, 8])\n\nfor model_path in [TF_ADDITION_MODEL_PATH, PYTORCH_ADDITION_MODEL_PATH]:\n    # Load the model\n    neuropod = load_neuropod(model_path)\n\n    # Run inference\n    results = neuropod.infer({\"x\": x, \"y\": y})\n\n    # array([6, 8, 10, 12])\n    print results[\"out\"]\n```\n\nSee the [tutorial](https://neuropod.ai/tutorial/), [Python guide](https://neuropod.ai/pyguide/), or [C++ guide](https://neuropod.ai/cppguide/) for more examples.\n\nSome benefits of this include:\n\n- All of your inference code is framework agnostic.\n- You can easily switch between deep learning frameworks if necessary without changing runtime code.\n- Avoid the learning curve of using the C++ libtorch API and the C/C++ TF API\n\nAny Neuropod model can be run from both C++ and Python (even PyTorch models that have not been converted to TorchScript).\n\n#### Define a Problem API\n\nThis lets you focus more on the problem you're solving rather than the framework you're using to solve it.\n\nFor example, if you define a problem API for 2d object detection, any model that implements it can reuse all the existing inference code and infrastructure for that problem.\n\n```py\nINPUT_SPEC = [\n    # BGR image\n    {\"name\": \"image\", \"dtype\": \"uint8\", \"shape\": (1200, 1920, 3)},\n]\n\nOUTPUT_SPEC = [\n    # shape: (num_detections, 4): (xmin, ymin, xmax, ymax)\n    # These values are in units of pixels. The origin is the top left corner\n    # with positive X to the right and positive Y towards the bottom of the image\n    {\"name\": \"boxes\", \"dtype\": \"float32\", \"shape\": (\"num_detections\", 4)},\n\n    # The list of classes that the network can output\n    # This must be some subset of ['vehicle', 'person', 'motorcycle', 'bicycle']\n    {\"name\": \"supported_object_classes\", \"dtype\": \"string\", \"shape\": (\"num_classes\",)},\n\n    # The probability of each class for each detection\n    # These should all be floats between 0 and 1\n    {\"name\": \"object_class_probability\", \"dtype\": \"float32\", \"shape\": (\"num_detections\", \"num_classes\")},\n]\n```\n\nThis lets you\n\n- Build a single metrics pipeline for a problem\n- Easily compare models solving the same problem (even if they're in different frameworks)\n- Build optimized inference code that can run any model that solves a particular problem\n- Swap out models that solve the same problem at runtime with no code change (even if the models are from different frameworks)\n- Run fast experiments\n\nSee the [tutorial](https://neuropod.ai/tutorial/) for more details.\n\n#### Build generic tools and pipelines\n\nIf you have several models that take in a similar set of inputs, you can build and optimize one framework-agnostic input generation pipeline and share it across models.\n\n#### Other benefits\n\n- Fully self-contained models (including custom ops)\n- [Efficient zero-copy operations](https://neuropod.ai/advanced/efficient_tensor_creation/)\n- [Tested on](https://neuropod.ai/developing/#build-matrix) platforms including\n    - Mac, Linux, Linux (GPU)\n    - Four or five versions of each supported framework\n    - Five versions of Python\n\n- Model isolation with [out-of-process execution](https://neuropod.ai/advanced/ope/)\n    - Use multiple different versions of frameworks in the same application\n        - Ex: Experimental models using Torch nightly along with models using Torch 1.1.0\n- Switch from running in-process to running out-of-process with [one line of code](https://neuropod.ai/advanced/ope/)\n\n## Getting started\n\nSee the [basic introduction tutorial](https://neuropod.ai/tutorial/) for an overview of how to get started with Neuropod.\n\nThe [Python guide](https://neuropod.ai/pyguide/) and [C++ guide](https://neuropod.ai/cppguide/) go into more detail on running Neuropod models.\n","funding_links":[],"categories":["Sensor Processing","C++","Frameworks/Servers for Serving"],"sub_categories":["Machine Learning"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fuber%2Fneuropod","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fuber%2Fneuropod","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fuber%2Fneuropod/lists"}