{"id":29182344,"url":"https://github.com/openvinotoolkit/openvino.genai","last_synced_at":"2025-07-01T20:05:26.999Z","repository":{"id":203999422,"uuid":"705678696","full_name":"openvinotoolkit/openvino.genai","owner":"openvinotoolkit","description":"Run Generative AI models with simple C++/Python API and using OpenVINO Runtime","archived":false,"fork":false,"pushed_at":"2025-06-27T08:19:13.000Z","size":34433,"stargazers_count":295,"open_issues_count":74,"forks_count":249,"subscribers_count":28,"default_branch":"master","last_synced_at":"2025-06-27T08:23:56.652Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://openvinotoolkit.github.io/openvino.genai/","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/openvinotoolkit.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":"SUPPORTED_MODELS.md","governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-10-16T13:38:16.000Z","updated_at":"2025-06-27T07:08:58.000Z","dependencies_parsed_at":null,"dependency_job_id":"b64ca9d9-bb27-4a8b-bdfd-24d03218837e","html_url":"https://github.com/openvinotoolkit/openvino.genai","commit_stats":null,"previous_names":["openvinotoolkit/openvino.genai"],"tags_count":11,"template":false,"template_full_name":null,"purl":"pkg:github/openvinotoolkit/openvino.genai","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openvinotoolkit%2Fopenvino.genai","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openvinotoolkit%2Fopenvino.genai/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openvinotoolkit%2Fopenvino.genai/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openvinotoolkit%2Fopenvino.genai/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/openvinotoolkit","download_url":"https://codeload.github.com/openvinotoolkit/openvino.genai/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openvinotoolkit%2Fopenvino.genai/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":263029208,"owners_count":23402354,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-07-01T20:03:52.452Z","updated_at":"2025-07-01T20:05:26.892Z","avatar_url":"https://github.com/openvinotoolkit.png","language":"C++","funding_links":[],"categories":["Table of content"],"sub_categories":["Generative AI"],"readme":"# OpenVINO™ GenAI\n\n![](src/docs/openvino_genai.svg)\n\nOpenVINO™ GenAI is a library of the most popular Generative AI model pipelines, optimized execution methods, and samples that run on top of highly performant [OpenVINO Runtime](https://github.com/openvinotoolkit/openvino).\n\nThis library is friendly to PC and laptop execution, and optimized for resource consumption. It requires no external dependencies to run generative models as it already includes all the core functionality (e.g. tokenization via openvino-tokenizers).\n\n![Text generation using LLaMa 3.2 model running on Intel ARC770 dGPU](./samples/generation.gif)\n\n## Getting Started\n\nPlease follow the following blogs to setup your first hands-on experience with C++ and Python samples.\n\n* [How to Build OpenVINO™ GenAI APP in C++](https://medium.com/openvino-toolkit/how-to-build-openvino-genai-app-in-c-32dcbe42fa67)\n* [How to run Llama 3.2 locally with OpenVINO™](https://medium.com/openvino-toolkit/how-to-run-llama-3-2-locally-with-openvino-60a0f3674549)\n\n\n## Supported Generative AI scenarios\n\nOpenVINO™ GenAI library provides very lightweight C++ and Python APIs to run following Generative Scenarios:\n - Text generation using Large Language Models. For example, chat with local LLaMa model\n - Image generation using Diffuser models, for example, generation using Stable Diffusion models\n - Speech recognition using Whisper family models\n - Text generation using Large Visual Models, for instance, Image analysis using LLaVa or miniCPM models family\n - Text-to-speech generation using SpeechT5 TTS models\n - Text embedding for Retrieval-Augmented Generation (RAG). For example, compute embeddings for documents and queries to enable efficient retrieval in RAG workflows.\n\nLibrary efficiently supports LoRA adapters for Text and Image generation scenarios:\n- Load multiple adapters per model\n- Select active adapters for every generation\n- Mix multiple adapters with coefficients via alpha blending\n\nAll scenarios are run on top of OpenVINO Runtime that supports inference on CPU, GPU and NPU. See [here](https://docs.openvino.ai/2025/about-openvino/release-notes-openvino/system-requirements.html) for platform support matrix.\n\n## Supported Generative AI optimization methods\n\nOpenVINO™ GenAI library provides a transparent way to use state-of-the-art generation optimizations:\n- Speculative decoding that employs two models of different sizes and uses the large model to periodically correct the results of the small model. See [here](https://pytorch.org/blog/hitchhikers-guide-speculative-decoding/) for more detailed overview\n- KVCache token eviction algorithm that reduces the size of the KVCache by pruning less impacting tokens.\n\nAdditionally, OpenVINO™ GenAI library implements a continuous batching approach to use OpenVINO within LLM serving. Continuous batching library could be used in LLM serving frameworks and supports the following features:\n- Prefix caching that caches fragments of previous generation requests and corresponding KVCache entries internally and uses them in case of repeated query. See [here](https://google.com) for more detailed overview\n\nContinuous batching functionality is used within OpenVINO Model Server (OVMS) to serve LLMs, see [here](https://docs.openvino.ai/2025/openvino-workflow/model-server/ovms_what_is_openvino_model_server.html) for more details.\n\n## Installing OpenVINO GenAI\n\n```sh\n    # Installing OpenVINO GenAI via pip\n    pip install openvino-genai\n\n    # Install optimum-intel to be able to download, convert and optimize LLMs from Hugging Face\n    # Optimum is not required to run models, only to convert and compress\n    pip install optimum-intel@git+https://github.com/huggingface/optimum-intel.git\n\n    # (Optional) Install (TBD) to be able to download models from Model Scope\n```\n\n## Performing text generation \n\u003cdetails\u003e\n\nFor more examples check out our [Generative AI workflow](https://docs.openvino.ai/2025/openvino-workflow-generative.html)\n\n### Converting and compressing text generation model from Hugging Face library\n\n```sh\n#(Basic) download and convert to OpenVINO TinyLlama-Chat-v1.0 model\noptimum-cli export openvino --model \"TinyLlama/TinyLlama-1.1B-Chat-v1.0\" --weight-format fp16 --trust-remote-code \"TinyLlama-1.1B-Chat-v1.0\"\n\n#(Recommended) download, convert to OpenVINO and compress to int4 TinyLlama-Chat-v1.0 model\noptimum-cli export openvino --model \"TinyLlama/TinyLlama-1.1B-Chat-v1.0\" --weight-format int4 --trust-remote-code \"TinyLlama-1.1B-Chat-v1.0\"\n```\n\n### Run generation using LLMPipeline API in Python\n\n```python\nimport openvino_genai\n#Will run model on CPU, GPU or NPU are possible options\npipe = openvino_genai.LLMPipeline(\"./TinyLlama-1.1B-Chat-v1.0/\", \"CPU\")\nprint(pipe.generate(\"The Sun is yellow because\", max_new_tokens=100))\n```\n\n### Run generation using LLMPipeline in C++\n\nCode below requires installation of C++ compatible package (see [here](https://docs.openvino.ai/2025/get-started/install-openvino/install-openvino-genai.html#archive-installation) for more details)\n\n```cpp\n#include \"openvino/genai/llm_pipeline.hpp\"\n#include \u003ciostream\u003e\n\nint main(int argc, char* argv[]) {\n    std::string models_path = argv[1];\n    ov::genai::LLMPipeline pipe(models_path, \"CPU\");\n    std::cout \u003c\u003c pipe.generate(\"The Sun is yellow because\", ov::genai::max_new_tokens(100)) \u003c\u003c '\\n';\n}\n```\n\n### Sample notebooks using this API\n\nSee [here](https://openvinotoolkit.github.io/openvino_notebooks/?search=Create+an+LLM-powered+Chatbot+using+OpenVINO+Generate+API)\n\n\u003c/details\u003e\n\n## Performing visual language text generation\n\u003cdetails\u003e\n\nFor more examples check out our [Generative AI workflow](https://docs.openvino.ai/2025/openvino-workflow-generative.html)\n\n### Converting and compressing the model from Hugging Face library\n\nTo convert the [OpenGVLab/InternVL2-1B](https://huggingface.co/OpenGVLab/InternVL2-1B) model, `timm` and `einops` are required: `pip install timm einops`.\n\n```sh\n# Download and convert the OpenGVLab/InternVL2-1B model to OpenVINO with int4 weight-compression for the language model\n# Other components are compressed to int8\noptimum-cli export openvino -m OpenGVLab/InternVL2-1B --trust-remote-code --weight-format int4 InternVL2-1B\n```\n\n### Run generation using VLMPipeline API in Python\n\nSee [Visual Language Chat](https://github.com/openvinotoolkit/openvino.genai/tree/master/samples/python/visual_language_chat) for a demo application.\n\nRun the following command to download a sample image:\n\n```sh\ncurl -O \"https://storage.openvinotoolkit.org/test_data/images/dog.jpg\"\n```\n\n```python\nimport numpy as np\nimport openvino as ov\nimport openvino_genai\nfrom PIL import Image\n\n# Choose GPU instead of CPU in the line below to run the model on Intel integrated or discrete GPU\npipe = openvino_genai.VLMPipeline(\"./InternVL2-1B\", \"CPU\")\n\nimage = Image.open(\"dog.jpg\")\nimage_data = np.array(image)\nimage_data = ov.Tensor(image_data)\n\nprompt = \"Can you describe the image?\"\nresult = pipe.generate(prompt, image=image_data, max_new_tokens=100)\nprint(result.texts[0])\n```\n\n### Run generation using VLMPipeline in C++\n\nCode below requires installation of C++ compatible package (see [here](https://docs.openvino.ai/2025/get-started/install-openvino/install-openvino-genai.html#archive-installation) for more details). See [Visual Language Chat](https://github.com/openvinotoolkit/openvino.genai/tree/master/samples/cpp/visual_language_chat) for a demo application.\n\n```cpp\n#include \"openvino/genai/visual_language/pipeline.hpp\"\n#include \"load_image.hpp\"\n#include \u003ciostream\u003e\n\nint main(int argc, char* argv[]) {\n    std::string models_path = argv[1];\n    ov::genai::VLMPipeline pipe(models_path, \"CPU\");\n    ov::Tensor rgb = utils::load_image(argv[2]);\n    std::cout \u003c\u003c pipe.generate(\n        prompt,\n        ov::genai::image(rgb),\n        ov::genai::max_new_tokens(100)\n    ) \u003c\u003c '\\n';\n}\n```\n\n### Sample notebooks using this API\n\nSee [here](https://openvinotoolkit.github.io/openvino_notebooks/?search=Visual-language+assistant+with+MiniCPM-V2+and+OpenVINO)\n\n\u003c/details\u003e\n\n## Performing image generation\n\n\u003cdetails\u003e\n\nFor more examples check out our [Generative AI workflow](https://docs.openvino.ai/2025/openvino-workflow-generative.html)\n\n### Converting and compressing image generation model from Hugging Face library\n\n```sh\n#Download and convert to OpenVINO dreamlike-anime-1.0 model\noptimum-cli export openvino --model dreamlike-art/dreamlike-anime-1.0 --weight-format fp16 dreamlike_anime_1_0_ov/FP16\n\n#You can also use INT8 hybrid quantization to further optimize the model and reduce inference latency\noptimum-cli export openvino --model dreamlike-art/dreamlike-anime-1.0 --weight-format int8 --dataset conceptual_captions dreamlike_anime_1_0_ov/INT8\n```\n\n### Run generation using Text2Image API in Python\n\n```python\nimport argparse\nfrom PIL import Image\nimport openvino_genai\n\ndevice = 'CPU'  # GPU can be used as well\npipe = openvino_genai.Text2ImagePipeline(\"./dreamlike_anime_1_0_ov/INT8\", device)\nimage_tensor = pipe.generate(\"cyberpunk cityscape like Tokyo New York with tall buildings at dusk golden hour cinematic lighting\")\n\nimage = Image.fromarray(image_tensor.data[0])\nimage.save(\"image.bmp\")\n```\n\n### Run generation using Text2Image API in C++\n\nCode below requires installation of C++ compatible package (see [here](https://docs.openvino.ai/2025/get-started/install-openvino/install-openvino-genai.html#archive-installation) for additional setup details, or this blog for full instruction [How to Build OpenVINO™ GenAI APP in C++](https://medium.com/openvino-toolkit/how-to-build-openvino-genai-app-in-c-32dcbe42fa67)\n\n```cpp\n#include \"openvino/genai/image_generation/text2image_pipeline.hpp\"\n#include \"imwrite.hpp\"\n\nint main(int argc, char* argv[]) {\n   const std::string models_path = argv[1], prompt = argv[2];\n   const std::string device = \"CPU\";  // GPU can be used as well\n\n   ov::genai::Text2ImagePipeline pipe(models_path, device);\n   ov::Tensor image = pipe.generate(prompt);\n\n   imwrite(\"image.bmp\", image, true);\n}\n```\n\n### Run generation using Image2Image API in Python\n\n```python\nimport argparse\nfrom PIL import Image\nimport openvino_genai\nimport openvino as ov\n\ndevice = 'CPU'  # GPU can be used as well\npipe = openvino_genai.Image2ImagePipeline(\"./dreamlike_anime_1_0_ov/INT8\", device)\n\nimage = Image.open(\"small_city.jpg\")\nimage_data = np.array(image)[None]\nimage_data = ov.Tensor(image_data)\n\nimage_tensor = pipe.generate(\n    \"cyberpunk cityscape like Tokyo New York with tall buildings at dusk golden hour cinematic lighting\",\n    image=image_data,\n    strength=0.8\n)\n\nimage = Image.fromarray(image_tensor.data[0])\nimage.save(\"image.bmp\")\n```\n\n### Run generation using Image2Image API in C++\n\nCode below requires installation of C++ compatible package (see [here](https://docs.openvino.ai/2025/get-started/install-openvino/install-openvino-genai.html#archive-installation) for additional setup details, or this blog for full instruction [How to Build OpenVINO™ GenAI APP in C++](https://medium.com/openvino-toolkit/how-to-build-openvino-genai-app-in-c-32dcbe42fa67)\n\n```cpp\n#include \"openvino/genai/image_generation/image2image_pipeline.hpp\"\n#include \"load_image.hpp\"\n#include \"imwrite.hpp\"\n\nint main(int argc, char* argv[]) {\n   const std::string models_path = argv[1], prompt = argv[2], image_path = argv[3];\n   const std::string device = \"CPU\";  // GPU can be used as well\n\n   ov::Tensor image = utils::load_image(image_path);\n\n   ov::genai::Image2ImagePipeline pipe(models_path, device);\n   ov::Tensor generated_image = pipe.generate(prompt, image, ov::genai::strength(0.8f));\n\n   imwrite(\"image.bmp\", generated_image, true);\n}\n```\n\n### Run generation using Inpainting API in Python\n\n```python\nimport argparse\nfrom PIL import Image\nimport openvino_genai\nimport openvino as ov\n\ndef read_image(path: str) -\u003e openvino.Tensor:\n    pic = Image.open(path).convert(\"RGB\")\n    image_data = np.array(pic)[None]\n    return openvino.Tensor(image_data)\n\ndevice = 'CPU'  # GPU can be used as well\npipe = openvino_genai.InpaintingPipeline(args.model_dir, device)\n\nimage = read_image(\"image.jpg\")\nmask_image = read_image(\"mask.jpg\")\n\nimage_tensor = pipe.generate(\n    \"Face of a yellow cat, high resolution, sitting on a park bench\",\n    image=image,\n    mask_image=mask_image\n)\n\nimage = Image.fromarray(image_tensor.data[0])\nimage.save(\"image.bmp\")\n```\n\n### Run generation using Inpainting API in C++\n\nCode below requires installation of C++ compatible package (see [here](https://docs.openvino.ai/2025/get-started/install-openvino/install-openvino-genai.html#archive-installation) for additional setup details, or this blog for full instruction [How to Build OpenVINO™ GenAI APP in C++](https://medium.com/openvino-toolkit/how-to-build-openvino-genai-app-in-c-32dcbe42fa67)\n\n```cpp\n#include \"openvino/genai/image_generation/inpainting_pipeline.hpp\"\n#include \"load_image.hpp\"\n#include \"imwrite.hpp\"\n\nint main(int argc, char* argv[]) {\n   const std::string models_path = argv[1], prompt = argv[2];\n   const std::string device = \"CPU\";  // GPU can be used as well\n\n   ov::Tensor image = utils::load_image(argv[3]);\n   ov::Tensor mask_image = utils::load_image(argv[4]);\n\n   ov::genai::InpaintingPipeline pipe(models_path, device);\n   ov::Tensor generated_image = pipe.generate(prompt, image, mask_image);\n\n   imwrite(\"image.bmp\", generated_image, true);\n}\n```\n\n### Sample notebooks using this API\n\nSee [here](https://openvinotoolkit.github.io/openvino_notebooks/?search=Text+to+Image+pipeline+and+OpenVINO+with+Generate+API)\n\n\u003c/details\u003e\n\n## Speech-to-text processing using Whisper Pipeline\n\u003cdetails\u003e\n\nFor more examples check out our [Generative AI workflow](https://docs.openvino.ai/2025/openvino-workflow-generative.html)\n\nNOTE: Whisper Pipeline requires preprocessing of audio input (to adjust sampling rate and normalize)\n \n ### Converting and quantizing speech-to-text model from Hugging Face library\n```sh\n#Download and convert to OpenVINO whisper-base model\noptimum-cli export openvino --model openai/whisper-base whisper-base\n\n#Download, convert and apply int8 static quantization to whisper-base model\noptimum-cli export openvino --model openai/whisper-base \\\n--quant-mode int8 --dataset librispeech --num-samples 32 whisper-base-int8\n```\n\n### Run generation using Whisper Pipeline API in Python\n\nNOTE: This sample is a simplified version of the full sample that is available [here](./samples/python/whisper_speech_recognition/whisper_speech_recognition.py)\n\n```python\nimport openvino_genai\nimport librosa\n\ndef read_wav(filepath):\n    raw_speech, samplerate = librosa.load(filepath, sr=16000)\n    return raw_speech.tolist()\n\ndevice = \"CPU\" # GPU can be used as well\npipe = openvino_genai.WhisperPipeline(\"whisper-base\", device)\nraw_speech = read_wav(\"sample.wav\")\nprint(pipe.generate(raw_speech))\n```\n\n \n### Run generation using Whisper Pipeline API in C++\n\nNOTE: This sample is a simplified version of the full sample that is available [here](./samples/cpp/whisper_speech_recognition/whisper_speech_recognition.cpp)\n\n```cpp\n#include \u003ciostream\u003e\n\n#include \"audio_utils.hpp\"\n#include \"openvino/genai/whisper_pipeline.hpp\"\n\nint main(int argc, char* argv[]) {\n    std::filesystem::path models_path = argv[1];\n    std::string wav_file_path = argv[2];\n    std::string device = \"CPU\"; // GPU can be used as well\n\n    ov::genai::WhisperPipeline pipeline(models_path, device);\n\n    ov::genai::RawSpeechInput raw_speech = utils::audio::read_wav(wav_file_path);\n\n    std::cout \u003c\u003c pipeline.generate(raw_speech, ov::genai::max_new_tokens(100)) \u003c\u003c '\\n';\n}\n```\n\n ### Sample notebooks using this API\n\nSee [here](https://openvinotoolkit.github.io/openvino_notebooks/?search=Automatic+speech+recognition+using+Whisper+and+OpenVINO+with+Generate+API)\n\n\u003c/details\u003e\n\n## Performing text-to-speech generation\n\u003cdetails\u003e\n\nFor more examples check out our [Generative AI workflow](https://docs.openvino.ai/2025/openvino-workflow-generative.html)\n\nNOTE: Currently, text-to-speech in OpenVINO GenAI supports the SpeechT5 TTS model. The generated audio signal is a single-channel (mono) waveform with a sampling rate of 16 kHz.\n \n### Converting text-to-speech model from Hugging Face library\n```sh\n# Download and convert to OpenVINO\noptimum-cli export openvino --model microsoft/speecht5_tts --model-kwargs \"{\\\"vocoder\\\": \\\"microsoft/speecht5_hifigan\\\"}\" ov_speecht5_tts\n```\n\n### Run generation using Text-to-speech API in Python\n\nNOTE: This sample is a simplified version of the full sample that is available [here](./samples/python/speech_generation/text2speech.py)\n\n```python\nimport openvino_genai\nimport soundfile as sf\n\npipe = openvino_genai.Text2SpeechPipeline(\"ov_speecht5_tts\", \"CPU\")\n\n# additionally, a speaker embedding can be specified as the target voice input to the generate method\nresult = pipe.generate(\"Hello OpenVINO GenAI\")\nspeech = result.speeches[0]\nsf.write(\"output_audio.wav\", speech.data[0], samplerate=16000)\n```\n\n \n### Run generation using Text-to-speech API in C++\n\nNOTE: This sample is a simplified version of the full sample that is available [here](./samples/cpp/speech_generation/text2speech.cpp)\n\n```cpp\n#include \"audio_utils.hpp\"\n#include \"openvino/genai/speech_generation/text2speech_pipeline.hpp\"\n\nint main(int argc, char* argv[]) {\n    ov::genai::Text2SpeechPipeline pipe(\"ov_speecht5_tts\", \"CPU\");\n\n    // additionally, a speaker embedding can be specified as the target voice input to the generate method\n    auto gen_speech = pipe.generate(\"Hello OpenVINO GenAI\");\n\n    auto waveform_size = gen_speech.speeches[0].get_size();\n    auto waveform_ptr = gen_speech.speeches[0].data\u003cconst float\u003e();\n    auto bits_per_sample = gen_speech.speeches[0].get_element_type().bitwidth();\n    utils::audio::save_to_wav(waveform_ptr, waveform_size, \"output_audio.wav\", bits_per_sample);\n\n    return 0;\n}\n```\n\n\u003c/details\u003e\n\n## Text Embeddings\n\u003cdetails\u003e\n\n### Converting and preparing a text embedding model from Hugging Face library\n\n```sh\n# Download and convert the BAAI/bge-small-en-v1.5 model to OpenVINO format\noptimum-cli export openvino --trust-remote-code --model BAAI/bge-small-en-v1.5 BAAI/bge-small-en-v1.5\n```\n\n### Compute embeddings using TextEmbeddingPipeline API in Python\n\n```python\nimport openvino_genai\n\npipeline = openvino_genai.TextEmbeddingPipeline(\"./BAAI/bge-small-en-v1.5\", \"CPU\")\n\ndocuments = [\"Document 1\", \"Document 2\"]\nembeddings = pipeline.embed_documents(documents)\n\nquery = \"The Sun is yellow because\"\nquery_embedding = pipeline.embed_query(query)\n```\n\n### Compute embeddings using TextEmbeddingPipeline API in C++\n\n```cpp\n#include \"openvino/genai/rag/text_embedding_pipeline.hpp\"\n#include \u003ciostream\u003e\n\nint main(int argc, char* argv[]) {\n    std::string models_path = argv[1];\n    std::vector\u003cstd::string\u003e documents(argv + 2, argv + argc);\n    std::string device = \"CPU\";  // GPU can be used as well\n\n    ov::genai::TextEmbeddingPipeline pipeline(models_path, device);\n\n    const ov::genai::EmbeddingResults embeddings = pipeline.embed_documents(documents);\n}\n```\n\u003c/details\u003e\n\n## Additional materials\n\n- [List of supported models](https://github.com/openvinotoolkit/openvino.genai/blob/master/SUPPORTED_MODELS.md) (NOTE: models can work, but were not tried yet)\n- [OpenVINO Generative AI workflow](https://docs.openvino.ai/2025/openvino-workflow-generative.html)\n- [Optimum-intel and OpenVINO](https://huggingface.co/docs/optimum/intel/openvino/export)\n\n## License\n\nThe OpenVINO™ GenAI repository is licensed under [Apache License Version 2.0](LICENSE).\nBy contributing to the project, you agree to the license and copyright terms therein and release\nyour contribution under these terms.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopenvinotoolkit%2Fopenvino.genai","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fopenvinotoolkit%2Fopenvino.genai","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopenvinotoolkit%2Fopenvino.genai/lists"}