{"id":13438671,"url":"https://github.com/openvinotoolkit/openvino","last_synced_at":"2025-05-12T22:35:46.340Z","repository":{"id":36954406,"uuid":"153097643","full_name":"openvinotoolkit/openvino","owner":"openvinotoolkit","description":"OpenVINO™ is an open source toolkit for optimizing and deploying AI inference","archived":false,"fork":false,"pushed_at":"2025-05-12T17:22:12.000Z","size":886823,"stargazers_count":8275,"open_issues_count":583,"forks_count":2594,"subscribers_count":197,"default_branch":"master","last_synced_at":"2025-05-12T17:56:55.051Z","etag":null,"topics":["ai","computer-vision","deep-learning","deploy-ai","diffusion-models","generative-ai","good-first-issue","inference","llm-inference","natural-language-processing","nlp","openvino","optimize-ai","performance-boost","recommendation-system","speech-recognition","stable-diffusion","transformers","yolo"],"latest_commit_sha":null,"homepage":"https://docs.openvino.ai","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/openvinotoolkit.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2018-10-15T10:54:40.000Z","updated_at":"2025-05-12T17:22:16.000Z","dependencies_parsed_at":"2025-05-05T16:15:31.721Z","dependency_job_id":"0284b0f2-7b60-42e8-b078-02c3f9756fdc","html_url":"https://github.com/openvinotoolkit/openvino","commit_stats":{"total_commits":10840,"total_committers":470,"mean_commits":23.06382978723404,"dds":0.9291512915129151,"last_synced_commit":"de30d8523df3adf7c0bbcba52991d46b95d25b6b"},"previous_names":[],"tags_count":71,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openvinotoolkit%2Fopenvino","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openvinotoolkit%2Fopenvino/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openvinotoolkit%2Fopenvino/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openvinotoolkit%2Fopenvino/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/openvinotoolkit","download_url":"https://codeload.github.com/openvinotoolkit/openvino/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253801178,"owners_count":21966607,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","computer-vision","deep-learning","deploy-ai","diffusion-models","generative-ai","good-first-issue","inference","llm-inference","natural-language-processing","nlp","openvino","optimize-ai","performance-boost","recommendation-system","speech-recognition","stable-diffusion","transformers","yolo"],"created_at":"2024-07-31T03:01:07.401Z","updated_at":"2025-05-12T22:35:41.317Z","avatar_url":"https://github.com/openvinotoolkit.png","language":"C++","funding_links":[],"categories":["C++","Further resources:","🎯 Tool Categories","📦 Deployment \u0026 Optimization","Deep Learning Framework","Deployment and Serving","其他_机器学习与深度学习","Real-World Projects","Research \u0026 Data Analysis","Tooling \u0026 Infrastructure","Lighter and Deployment Frameworks","Repos","**Tools and Frameworks**","Inference","Libraries","11. Specialized Domains","2. **Production Tools**","Inference Engines \u0026 Backends (22)","Local Inference and Serving"],"sub_categories":["🎯 Optimization Frameworks","🔧 Model Optimization Tools","Deployment \u0026 Distribution","AI / Machine Learning","Deployment \u0026 Optimization","Inference Engine","Inference Framework","Run locally"],"readme":"\u003cdiv align=\"center\"\u003e\n\u003cimg src=\"docs/dev/assets/openvino-logo-purple-black.svg\" width=\"400px\"\u003e\n\n\u003ch3 align=\"center\"\u003e\nOpen-source software toolkit for optimizing and deploying deep learning models.\n\u003c/h3\u003e\n\n\u003cp align=\"center\"\u003e\n \u003ca href=\"https://docs.openvino.ai/2025/index.html\"\u003e\u003cb\u003eDocumentation\u003c/b\u003e\u003c/a\u003e • \u003ca href=\"https://blog.openvino.ai\"\u003e\u003cb\u003eBlog\u003c/b\u003e\u003c/a\u003e • \u003ca href=\"https://docs.openvino.ai/2025/about-openvino/key-features.html\"\u003e\u003cb\u003eKey Features\u003c/b\u003e\u003c/a\u003e • \u003ca href=\"https://docs.openvino.ai/2025/get-started/learn-openvino.html\"\u003e\u003cb\u003eTutorials\u003c/b\u003e\u003c/a\u003e • \u003ca href=\"https://docs.openvino.ai/2025/documentation/openvino-ecosystem.html\"\u003e\u003cb\u003eIntegrations\u003c/b\u003e\u003c/a\u003e • \u003ca href=\"https://docs.openvino.ai/2025/about-openvino/performance-benchmarks.html\"\u003e\u003cb\u003eBenchmarks\u003c/b\u003e\u003c/a\u003e • \u003ca href=\"https://github.com/openvinotoolkit/openvino.genai\"\u003e\u003cb\u003eGenerative AI\u003c/b\u003e\u003c/a\u003e\n\u003c/p\u003e\n\n[![PyPI Status](https://badge.fury.io/py/openvino.svg)](https://badge.fury.io/py/openvino)\n[![Anaconda Status](https://anaconda.org/conda-forge/openvino/badges/version.svg)](https://anaconda.org/conda-forge/openvino)\n[![brew Status](https://img.shields.io/homebrew/v/openvino)](https://formulae.brew.sh/formula/openvino)\n\n[![PyPI Downloads](https://static.pepy.tech/badge/openvino)](https://pepy.tech/project/openvino)\n[![Anaconda Downloads](https://anaconda.org/conda-forge/libopenvino/badges/downloads.svg)](https://anaconda.org/conda-forge/openvino/files)\n[![brew Downloads](https://img.shields.io/homebrew/installs/dy/openvino)](https://formulae.brew.sh/formula/openvino)\n \u003c/div\u003e\n\n\n- **Inference Optimization**: Boost deep learning performance in computer vision, automatic speech recognition, generative AI, natural language processing with large and small language models, and many other common tasks.\n- **Flexible Model Support**: Use models trained with popular frameworks such as PyTorch, TensorFlow, ONNX, Keras, PaddlePaddle, and JAX/Flax. Directly integrate models built with transformers and diffusers from the Hugging Face Hub using Optimum Intel. Convert and deploy models without original frameworks.\n- **Broad Platform Compatibility**: Reduce resource demands and efficiently deploy on a range of platforms from edge to cloud. OpenVINO™ supports inference on CPU (x86, ARM), GPU (Intel integrated \u0026 discrete GPU) and AI accelerators (Intel NPU).\n- **Community and Ecosystem**: Join an active community contributing to the enhancement of deep learning performance across various domains.\n\nCheck out the [OpenVINO Cheat Sheet](https://docs.openvino.ai/2025/_static/download/OpenVINO_Quick_Start_Guide.pdf) and [Key Features](https://docs.openvino.ai/2025/about-openvino/key-features.html) for a quick reference.\n\n\n## Installation\n\n[Get your preferred distribution of OpenVINO](https://docs.openvino.ai/2025/get-started/install-openvino.html) or use this command for quick installation:\n\n```sh\npip install -U openvino\n```\n\nCheck [system requirements](https://docs.openvino.ai/2025/about-openvino/release-notes-openvino/system-requirements.html) and [supported devices](https://docs.openvino.ai/2025/documentation/compatibility-and-support/supported-devices.html) for detailed information.\n\n## Tutorials and Examples\n\n[OpenVINO Quickstart example](https://docs.openvino.ai/2025/get-started.html) will walk you through the basics of deploying your first model.\n\nLearn how to optimize and deploy popular models with the [OpenVINO Notebooks](https://github.com/openvinotoolkit/openvino_notebooks)📚:\n- [Create an LLM-powered Chatbot using OpenVINO](https://github.com/openvinotoolkit/openvino_notebooks/blob/latest/notebooks/llm-chatbot/llm-chatbot-generate-api.ipynb)\n- [YOLOv11 Optimization](https://github.com/openvinotoolkit/openvino_notebooks/blob/latest/notebooks/yolov11-optimization/yolov11-object-detection.ipynb)\n- [Text-to-Image Generation](https://github.com/openvinotoolkit/openvino_notebooks/blob/latest/notebooks/text-to-image-genai/text-to-image-genai.ipynb)\n- [Multimodal assistant with LLaVa and OpenVINO](https://github.com/openvinotoolkit/openvino_notebooks/blob/latest/notebooks/llava-multimodal-chatbot/llava-multimodal-chatbot-genai.ipynb)\n- [Automatic speech recognition using Whisper and OpenVINO](https://github.com/openvinotoolkit/openvino_notebooks/blob/latest/notebooks/whisper-asr-genai/whisper-asr-genai.ipynb)\n\nDiscover more examples in the [OpenVINO Samples (Python \u0026 C++)](https://docs.openvino.ai/2025/get-started/learn-openvino/openvino-samples.html) and [Notebooks (Python)](https://docs.openvino.ai/2025/get-started/learn-openvino/interactive-tutorials-python.html).\n\nHere are easy-to-follow code examples demonstrating how to run PyTorch and TensorFlow model inference using OpenVINO:\n\n**PyTorch Model**\n\n```python\nimport openvino as ov\nimport torch\nimport torchvision\n\n# load PyTorch model into memory\nmodel = torch.hub.load(\"pytorch/vision\", \"shufflenet_v2_x1_0\", weights=\"DEFAULT\")\n\n# convert the model into OpenVINO model\nexample = torch.randn(1, 3, 224, 224)\nov_model = ov.convert_model(model, example_input=(example,))\n\n# compile the model for CPU device\ncore = ov.Core()\ncompiled_model = core.compile_model(ov_model, 'CPU')\n\n# infer the model on random data\noutput = compiled_model({0: example.numpy()})\n```\n\n**TensorFlow Model**\n\n```python\nimport numpy as np\nimport openvino as ov\nimport tensorflow as tf\n\n# load TensorFlow model into memory\nmodel = tf.keras.applications.MobileNetV2(weights='imagenet')\n\n# convert the model into OpenVINO model\nov_model = ov.convert_model(model)\n\n# compile the model for CPU device\ncore = ov.Core()\ncompiled_model = core.compile_model(ov_model, 'CPU')\n\n# infer the model on random data\ndata = np.random.rand(1, 224, 224, 3)\noutput = compiled_model({0: data})\n```\n\nOpenVINO supports the CPU, GPU, and NPU [devices](https://docs.openvino.ai/2025/openvino-workflow/running-inference/inference-devices-and-modes.html) and works with models from PyTorch, TensorFlow, ONNX, TensorFlow Lite, PaddlePaddle, and JAX/Flax [frameworks](https://docs.openvino.ai/2025/openvino-workflow/model-preparation.html). It includes [APIs](https://docs.openvino.ai/2025/api/api_reference.html) in C++, Python, C, NodeJS, and offers the GenAI API for optimized model pipelines and performance.\n\n## Generative AI with OpenVINO\n\nGet started with the OpenVINO GenAI [installation](https://docs.openvino.ai/2025/get-started/install-openvino/install-openvino-genai.html) and refer to the [detailed guide](https://docs.openvino.ai/2025/openvino-workflow-generative/generative-inference.html) to explore the capabilities of Generative AI using OpenVINO.\n\nLearn how to run LLMs and GenAI with [Samples](https://github.com/openvinotoolkit/openvino.genai/tree/master/samples) in the [OpenVINO™ GenAI repo](https://github.com/openvinotoolkit/openvino.genai). See GenAI in action with Jupyter notebooks: [LLM-powered Chatbot](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/llm-chatbot) and [LLM Instruction-following pipeline](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/llm-question-answering).\n\n## Documentation\n\n[User documentation](https://docs.openvino.ai/) contains detailed information about OpenVINO and guides you from installation through optimizing and deploying models for your AI applications.\n\n[Developer documentation](./docs/dev/index.md) focuses on the OpenVINO architecture and describes [building](./docs/dev/build.md)  and [contributing](./CONTRIBUTING.md) processes.\n\n## OpenVINO Ecosystem\n\n### OpenVINO Tools\n\n-   [Neural Network Compression Framework (NNCF)](https://github.com/openvinotoolkit/nncf) - advanced model optimization techniques including quantization, filter pruning, binarization, and sparsity.\n-   [GenAI Repository](https://github.com/openvinotoolkit/openvino.genai) and [OpenVINO Tokenizers](https://github.com/openvinotoolkit/openvino_tokenizers) - resources and tools for developing and optimizing Generative AI applications.\n-   [OpenVINO™ Model Server (OVMS)](https://github.com/openvinotoolkit/model_server) - a scalable, high-performance solution for serving models optimized for Intel architectures.\n-   [Intel® Geti™](https://geti.intel.com/) - an interactive video and image annotation tool for computer vision use cases.\n\n### Integrations\n\n-   [🤗Optimum Intel](https://github.com/huggingface/optimum-intel) - grab and use models leveraging OpenVINO within the Hugging Face API.\n-   [Torch.compile](https://docs.openvino.ai/2025/openvino-workflow/torch-compile.html) - use OpenVINO for Python-native applications by JIT-compiling code into optimized kernels.\n-   [OpenVINO LLMs inference and serving with vLLM​](https://docs.vllm.ai/en/stable/getting_started/openvino-installation.html) - enhance vLLM's fast and easy model serving with the OpenVINO backend.\n-   [OpenVINO Execution Provider for ONNX Runtime](https://onnxruntime.ai/docs/execution-providers/OpenVINO-ExecutionProvider.html) - use OpenVINO as a backend with your existing ONNX Runtime code.\n-   [LlamaIndex](https://docs.llamaindex.ai/en/stable/examples/llm/openvino/) - build context-augmented GenAI applications with the LlamaIndex framework and enhance runtime performance with OpenVINO.\n-   [LangChain](https://python.langchain.com/docs/integrations/llms/openvino/) - integrate OpenVINO with the LangChain framework to enhance runtime performance for GenAI applications.\n-   [Keras 3](https://github.com/keras-team/keras) - Keras 3 is a multi-backend deep learning framework. Users can switch model inference to the OpenVINO backend using the Keras API.\n\nCheck out the [Awesome OpenVINO](https://github.com/openvinotoolkit/awesome-openvino) repository to discover a collection of community-made AI projects based on OpenVINO!\n\n## Performance\n\nExplore [OpenVINO Performance Benchmarks](https://docs.openvino.ai/2025/about-openvino/performance-benchmarks.html) to discover the optimal hardware configurations and plan your AI deployment based on verified data.\n\n## Contribution and Support\n\nCheck out [Contribution Guidelines](./CONTRIBUTING.md) for more details.\nRead the [Good First Issues section](./CONTRIBUTING.md#3-start-working-on-your-good-first-issue), if you're looking for a place to start contributing. We welcome contributions of all kinds!\n\nYou can ask questions and get support on:\n\n* [GitHub Issues](https://github.com/openvinotoolkit/openvino/issues).\n* OpenVINO channels on the [Intel DevHub Discord server](https://discord.gg/7pVRxUwdWG).\n* The [`openvino`](https://stackoverflow.com/questions/tagged/openvino) tag on Stack Overflow\\*.\n\n\n## Resources\n\n* [Release Notes](https://docs.openvino.ai/2025/about-openvino/release-notes-openvino.html)\n* [OpenVINO Blog](https://blog.openvino.ai/)\n* [OpenVINO™ toolkit on Medium](https://medium.com/@openvino)\n\n\n## Telemetry\n\nOpenVINO™ collects software performance and usage data for the purpose of improving OpenVINO™ tools.\nThis data is collected directly by OpenVINO™ or through the use of Google Analytics 4.\nYou can opt-out at any time by running the command:\n\n``` bash\nopt_in_out --opt_out\n```\n\nMore Information is available at [OpenVINO™ Telemetry](https://docs.openvino.ai/2025/about-openvino/additional-resources/telemetry.html).\n\n## License\n\nOpenVINO™ Toolkit is licensed under [Apache License Version 2.0](LICENSE).\nBy contributing to the project, you agree to the license and copyright terms therein and release your contribution under these terms.\n\n---\n\\* Other names and brands may be claimed as the property of others.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopenvinotoolkit%2Fopenvino","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fopenvinotoolkit%2Fopenvino","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopenvinotoolkit%2Fopenvino/lists"}