{"id":43570803,"url":"https://github.com/crabnebula-dev/tauri-plugin-llm","last_synced_at":"2026-02-03T22:14:11.168Z","repository":{"id":334887480,"uuid":"1108338433","full_name":"crabnebula-dev/tauri-plugin-llm","owner":"crabnebula-dev","description":"Tauri plugin to load and interact with most LLMs","archived":false,"fork":false,"pushed_at":"2026-01-27T10:31:56.000Z","size":545,"stargazers_count":1,"open_issues_count":4,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-01-27T22:38:34.477Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/crabnebula-dev.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE_POLYFORM-NONCOM","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-12-02T10:27:56.000Z","updated_at":"2026-01-27T10:22:29.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/crabnebula-dev/tauri-plugin-llm","commit_stats":null,"previous_names":["crabnebula-dev/tauri-plugin-llm"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/crabnebula-dev/tauri-plugin-llm","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/crabnebula-dev%2Ftauri-plugin-llm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/crabnebula-dev%2Ftauri-plugin-llm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/crabnebula-dev%2Ftauri-plugin-llm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/crabnebula-dev%2Ftauri-plugin-llm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/crabnebula-dev","download_url":"https://codeload.github.com/crabnebula-dev/tauri-plugin-llm/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/crabnebula-dev%2Ftauri-plugin-llm/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29059427,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-03T20:13:53.544Z","status":"ssl_error","status_checked_at":"2026-02-03T20:13:40.507Z","response_time":96,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-02-03T22:14:10.468Z","updated_at":"2026-02-03T22:14:11.159Z","avatar_url":"https://github.com/crabnebula-dev.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Tauri Plugin LLM\n\nThis Tauri plugin allows loading and running inference on various large language models.\n\n| Platform | Supported |\n| -------- | --------- |\n| Linux    | ✓        |\n| Windows  | ✓        |\n| macOS    | ✓         |\n| Android  | ?         |\n| iOS      | ?         |\n\n# Requirements\n\n- Rust \u003e= 1.77\n- Go \u003e= 1.20 \n\n# Install\n\n# Usage\n\nThe plugin is not bundled with any LLM. The LLM must be shipped separately. In order to load an LLM, a specific configuration must be provided for the plugin. The JSON example configuration sets `Qwen3-4B-GGUF` as model to be loaded. \n\nFile `tauri.conf.json` :\n```json\n{\n  \"build\": { ... },\n  \"tauri\": { ... },\n  \"plugins\": {\n    \"tauri-plugin-llm\": {\n        \"tokenizer_file\": \"./models/Qwen3-4B-Instruct-2507-FP8/tokenizer.json\",\n        \"tokenizer_config_file\": \"./models/Qwen3-4B-Instruct-2507-FP8/tokenizer_config.json\",\n        \"model_config_file\": \"./models/Qwen3-4B-Instruct-2507-FP8/config.json\",\n        \"model_index_file\": \"\",\n        \"model_file\": \"./models/Qwen3-4B-GGUF/Qwen3-4B-Q4_K_M.gguf\",\n        \"model_dir\": \"./models/Qwen3-4B-GGUF/\",\n        \"model_config\": {\n            \"top_k\": 20,\n            \"top_p\": 0.8,\n            \"temperature\": 0.4,\n            \"name\": \"Qwen3-4B-GGUF\",\n            \"file_type\": \"GGUF\",\n            \"penalty\": 1.0,\n            \"seed\": \"Random\",\n            \"thinking\": false,\n            \"streaming\": true\n        },\n        \"verbose\": true\n    }\n  }\n}\n\n```\n\n\u003e **Note**: _The example above shows a model present at some folder called `model`. The model itself will not be shipped with the plugin._\n\n\nThe `LLMRuntime` will try to load an LLM and defer the initialization process to the actual loader. The Tauri Plugin LLM does not support all available models, yet. Internally, the model will be run in a dedicated thread to avoid blocking the main thread. \n\n```Rust\n// For demonstration purposes we load the configuration from the tests. \n// Normally you would load the tauri config\nlet config = LLMRuntimeConfig::from_path(\"tests/fixtures/test_runtime_qwen3.config.json\")?;\n\n// Initialize the Runtime and detect available specific runtimes\nlet mut runtime = LLMRuntime::from_config(config)?;\n\n// Starting the runtime. This will internally initialize the model and \n// then run inference on the model on the incoming message.\nruntime.run();\n\nif let Err(_) = runtime.send(\n\n    // We will send a simple text prompt and instruct the model to just return anything we wrote\n    Query::Prompt {\n    messages: vec![QueryMessage {\n        role: \"user\".to_string(),\n        content: \"Hello, World\".to_string() },\n        QueryMessage {\n        role: \"system\".to_string(),\n        content: \"You are a helpful assistant. Your task is to echo the incoming message. Do not describe anything. \".to_string()},\n    ],\n    tools: vec![],\n    config: Some(QueryConfig::default()),\n}) {\n    loop {\n        // We can continously check, if a message from the model is available\n        if let Ok(message) = runtime.retry_recv() {\n            tracing::info!(\"Received Message : {:?}\", message);\n            break;\n        }\n    }\n}\n\nOk(())\n```\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcrabnebula-dev%2Ftauri-plugin-llm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcrabnebula-dev%2Ftauri-plugin-llm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcrabnebula-dev%2Ftauri-plugin-llm/lists"}