{"id":21220805,"url":"https://github.com/rupeshs/ovllm_node_addon","last_synced_at":"2025-07-10T12:31:14.112Z","repository":{"id":247245655,"uuid":"825355318","full_name":"rupeshs/ovllm_node_addon","owner":"rupeshs","description":"OpenVINO LLM Node.js C++ addon","archived":false,"fork":false,"pushed_at":"2024-07-13T13:43:57.000Z","size":1181,"stargazers_count":4,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-07-13T14:52:39.184Z","etag":null,"topics":["addon","cpp","genai","llm","node-gyp","nodejs","openvino","python","tinyllama","tinyllamachat"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rupeshs.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-07-07T14:58:14.000Z","updated_at":"2024-07-13T13:44:01.000Z","dependencies_parsed_at":"2024-07-13T14:48:57.439Z","dependency_job_id":null,"html_url":"https://github.com/rupeshs/ovllm_node_addon","commit_stats":null,"previous_names":["rupeshs/ovllm_node_addon"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rupeshs%2Fovllm_node_addon","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rupeshs%2Fovllm_node_addon/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rupeshs%2Fovllm_node_addon/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rupeshs%2Fovllm_node_addon/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rupeshs","download_url":"https://codeload.github.com/rupeshs/ovllm_node_addon/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":225637111,"owners_count":17500365,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["addon","cpp","genai","llm","node-gyp","nodejs","openvino","python","tinyllama","tinyllamachat"],"created_at":"2024-11-20T22:14:09.096Z","updated_at":"2024-11-20T22:14:09.801Z","avatar_url":"https://github.com/rupeshs.png","language":"C++","readme":"# Node.js OpenVINO LLM C++ addon\n\nThis is a Node.js addon for [OpenVINO GenAI](https://github.com/openvinotoolkit/openvino.genai/) LLM.\nTested using TinyLLama chat 1.1 OpenVINO int4 model on Windows 11 (Intel Core i7 CPU).\n\nWatch below YouTube video for demo :\n\n[![IMAGE ALT TEXT HERE](https://img.youtube.com/vi/dAk8rlFE3QE/0.jpg)](https://www.youtube.com/watch?v=dAk8rlFE3QE)\n\n## Build\n\n- Visual Studio 2022 (C++)\n- Node 20.11 or higher\n- node-gyp\n- Python 3.11\n- Used [openvino_genai_windows_2024.2.0.0_x86_64](https://docs.openvino.ai/2024/get-started/install-openvino.html?PACKAGE=OPENVINO_BASE\u0026VERSION=v_2024_2_0\u0026OP_SYSTEM=WINDOWS\u0026DISTRIBUTION=ARCHIVE) release\n\nRun the following commands to build:\n\n```\nnpm install\nnode-gyp configure\nnode-gyp build\n```\n\n## Run\n\nTo test the Node.js OpenVINO LLM addon run the `index.js` script.\n\n`node index.js D:/demo/TinyLlama-1.1B-Chat-v1.0-openvino-int4`\n\nDisable streaming\n\n`node index.js D:/demo/TinyLlama-1.1B-Chat-v1.0-openvino-int4 nostream`\n\n## Supported models\n\nSupported models are [here](https://github.com/openvinotoolkit/openvino.genai/blob/releases/2024/2/src/docs/SUPPORTED_MODELS.md)\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frupeshs%2Fovllm_node_addon","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frupeshs%2Fovllm_node_addon","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frupeshs%2Fovllm_node_addon/lists"}