{"id":46168218,"url":"https://github.com/rustic-ai/uni-xervo","last_synced_at":"2026-04-15T06:04:27.866Z","repository":{"id":339341406,"uuid":"1161465604","full_name":"rustic-ai/uni-xervo","owner":"rustic-ai","description":null,"archived":false,"fork":false,"pushed_at":"2026-03-13T08:36:03.000Z","size":542,"stargazers_count":1,"open_issues_count":3,"forks_count":1,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-03-13T23:38:41.385Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://rustic-ai.github.io/uni-xervo/","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rustic-ai.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":"SUPPORT.md","governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-02-19T06:13:25.000Z","updated_at":"2026-03-13T08:08:14.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/rustic-ai/uni-xervo","commit_stats":null,"previous_names":["rustic-ai/uni-xervo"],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/rustic-ai/uni-xervo","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rustic-ai%2Funi-xervo","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rustic-ai%2Funi-xervo/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rustic-ai%2Funi-xervo/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rustic-ai%2Funi-xervo/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rustic-ai","download_url":"https://codeload.github.com/rustic-ai/uni-xervo/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rustic-ai%2Funi-xervo/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31828533,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-14T18:05:02.291Z","status":"online","status_checked_at":"2026-04-15T02:00:06.175Z","response_time":63,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-03-02T13:47:40.187Z","updated_at":"2026-04-15T06:04:27.852Z","avatar_url":"https://github.com/rustic-ai.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Uni-Xervo\n\nUnified Rust runtime for embedding, reranking, and generation across local and remote model providers.\n\n`uni-xervo` gives you one runtime and one API surface for mixed model stacks, so application code stays stable while you swap providers, models, and execution modes.\n\n## Overview\n\nUni-Xervo is built around three core ideas:\n\n- Model aliases: your app requests models by stable names like `embed/default` or `generate/llm`.\n- Provider abstraction: local and remote providers implement the same task traits.\n- Runtime deduplication: equivalent model specs share one loaded instance.\n\nCore tasks:\n\n- `embed` for vector embeddings\n- `rerank` for relevance scoring\n- `generate` for text generation, vision, image generation, and speech synthesis\n\n## Why Uni-Xervo?\n\n- Keep product code provider-agnostic.\n- Mix local and remote models in one runtime.\n- Multimodal generation: text, vision, diffusion (image gen), and speech pipelines.\n- Enforce config correctness with schema-backed option validation.\n- Control startup behavior with lazy, eager, or background warmup.\n- Add retries/timeouts per model alias instead of hard-coding behavior.\n\n## Provider Support\n\n| Provider ID | Tasks | Cargo Feature |\n| --- | --- | --- |\n| `local/candle` | `embed` | `provider-candle` |\n| `local/fastembed` | `embed` | `provider-fastembed` |\n| `local/mistralrs` | `embed`, `generate` (text, vision, diffusion, speech) | `provider-mistralrs` |\n| `remote/openai` | `embed`, `generate` | `provider-openai` |\n| `remote/gemini` | `embed`, `generate` | `provider-gemini` |\n| `remote/vertexai` | `embed`, `generate` | `provider-vertexai` |\n| `remote/mistral` | `embed`, `generate` | `provider-mistral` |\n| `remote/anthropic` | `generate` | `provider-anthropic` |\n| `remote/voyageai` | `embed`, `rerank` | `provider-voyageai` |\n| `remote/cohere` | `embed`, `rerank`, `generate` | `provider-cohere` |\n| `remote/azure-openai` | `embed`, `generate` | `provider-azure-openai` |\n\n## Installation\n\nUse only the features you need.\n\n```toml\n[dependencies]\nuni-xervo = { version = \"0.2.0\", default-features = false, features = [\"provider-candle\"] }\ntokio = { version = \"1\", features = [\"full\"] }\n```\n\nDefault feature set:\n\n- `provider-candle`\n\nIf you want local embeddings + OpenAI generation:\n\n```toml\n[dependencies]\nuni-xervo = { version = \"0.2.0\", default-features = false, features = [\"provider-candle\", \"provider-openai\"] }\ntokio = { version = \"1\", features = [\"full\"] }\n```\n\nGPU acceleration flag:\n\n- `gpu-cuda` for CUDA-enabled builds.\n\n## Quick Start (Rust)\n\n```rust\nuse uni_xervo::api::{ModelAliasSpec, ModelTask};\nuse uni_xervo::provider::candle::LocalCandleProvider;\nuse uni_xervo::runtime::ModelRuntime;\n\n#[tokio::main]\nasync fn main() -\u003e Result\u003c(), Box\u003cdyn std::error::Error\u003e\u003e {\n    let spec = ModelAliasSpec {\n        alias: \"embed/local\".to_string(),\n        task: ModelTask::Embed,\n        provider_id: \"local/candle\".to_string(),\n        model_id: \"sentence-transformers/all-MiniLM-L6-v2\".to_string(),\n        revision: None,\n        warmup: Default::default(),\n        required: true,\n        timeout: None,\n        load_timeout: None,\n        retry: None,\n        options: serde_json::Value::Null,\n    };\n\n    let runtime = ModelRuntime::builder()\n        .register_provider(LocalCandleProvider::new())\n        .catalog(vec![spec])\n        .build()\n        .await?;\n\n    let embedder = runtime.embedding(\"embed/local\").await?;\n    let vectors = embedder.embed(vec![\"hello world\"]).await?;\n    println!(\"vector dims = {}\", vectors[0].len());\n\n    Ok(())\n}\n```\n\n## JSON Config Example (`generate/llm`)\n\nModel catalogs are JSON arrays of `ModelAliasSpec`.\n\n`model-catalog.json`:\n\n```json\n[\n  {\n    \"alias\": \"embed/default\",\n    \"task\": \"embed\",\n    \"provider_id\": \"local/candle\",\n    \"model_id\": \"sentence-transformers/all-MiniLM-L6-v2\",\n    \"warmup\": \"lazy\",\n    \"required\": true,\n    \"options\": null\n  },\n  {\n    \"alias\": \"generate/llm\",\n    \"task\": \"generate\",\n    \"provider_id\": \"remote/openai\",\n    \"model_id\": \"gpt-4o-mini\",\n    \"warmup\": \"lazy\",\n    \"timeout\": 30,\n    \"retry\": {\n      \"max_attempts\": 3,\n      \"initial_backoff_ms\": 200\n    },\n    \"options\": {\n      \"api_key_env\": \"OPENAI_API_KEY\"\n    }\n  }\n]\n```\n\n## Load JSON Config and Run Generation\n\n```rust\nuse uni_xervo::provider::{LocalCandleProvider, RemoteOpenAIProvider};\nuse uni_xervo::runtime::ModelRuntime;\nuse uni_xervo::traits::{GenerationOptions, Message};\n\n#[tokio::main]\nasync fn main() -\u003e Result\u003c(), Box\u003cdyn std::error::Error\u003e\u003e {\n    let runtime = ModelRuntime::builder()\n        .register_provider(LocalCandleProvider::new())\n        .register_provider(RemoteOpenAIProvider::new())\n        .catalog_from_file(\"model-catalog.json\")?\n        .build()\n        .await?;\n\n    let llm = runtime.generator(\"generate/llm\").await?;\n    let result = llm\n        .generate(\n            \u0026[\n                Message::user(\"You are a concise assistant.\"),\n                Message::assistant(\"Understood.\"),\n                Message::user(\"Explain what embeddings are in one paragraph.\"),\n            ],\n            GenerationOptions {\n                max_tokens: Some(200),\n                temperature: Some(0.3),\n                top_p: Some(0.9),\n                ..Default::default()\n            },\n        )\n        .await?;\n\n    println!(\"{}\", result.text);\n    Ok(())\n}\n```\n\n## Configuration and Validation\n\n- Catalog schema: `schemas/model-catalog.schema.json`\n- Provider option schemas: `schemas/provider-options/*.schema.json`\n- Unknown keys or wrong value types fail fast during runtime build/register.\n\nDefault remote credential env vars:\n\n| Provider ID | Default credential env var | Extra required options |\n| --- | --- | --- |\n| `remote/openai` | `OPENAI_API_KEY` | None |\n| `remote/gemini` | `GEMINI_API_KEY` | None |\n| `remote/vertexai` | `VERTEX_AI_TOKEN` | `project_id` option or `VERTEX_AI_PROJECT` |\n| `remote/mistral` | `MISTRAL_API_KEY` | None |\n| `remote/anthropic` | `ANTHROPIC_API_KEY` | None |\n| `remote/voyageai` | `VOYAGE_API_KEY` | None |\n| `remote/cohere` | `CO_API_KEY` | None |\n| `remote/azure-openai` | `AZURE_OPENAI_API_KEY` | `resource_name` option |\n\n## CLI Prefetch Utility\n\nThe repository includes a prefetch CLI target (`src/bin/prefetch.rs`) to pre-download local model artifacts:\n\n```bash\ncargo run --bin prefetch -- model-catalog.json --dry-run\ncargo run --bin prefetch -- model-catalog.json\n```\n\nRemote providers are skipped by design because they do not cache local weights.\n\n## Development\n\n```bash\n# Build\n./scripts/build.sh\n\n# Format + check + test\n./scripts/test.sh\n\n# Ignored integration tests (real providers)\n./scripts/test-integration.sh\n```\n\nIntegration tests for real providers are gated by `EXPENSIVE_TESTS=1` and relevant API credentials.\n\n## Docs\n\n- Contributing guide: `CONTRIBUTING.md`\n- Development guide: `DEVELOPMENT.md`\n- Community guidelines: `COMMUNITY.md`\n- Code of conduct: `CODE_OF_CONDUCT.md`\n- Support guide: `SUPPORT.md`\n- Security policy: `SECURITY.md`\n- User guide: `docs/USER_GUIDE.md`\n- Testing guide: `TESTING.md`\n- Website docs: `website/`\n\n## License\n\nApache-2.0 (`LICENSE`).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frustic-ai%2Funi-xervo","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frustic-ai%2Funi-xervo","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frustic-ai%2Funi-xervo/lists"}