{"id":24861454,"url":"https://github.com/ztgx/topkio.rs","last_synced_at":"2026-03-04T12:31:55.299Z","repository":{"id":274119730,"uuid":"920116860","full_name":"zTgx/topkio.rs","owner":"zTgx","description":"Not all AI agent frameworks are called topkio.","archived":false,"fork":false,"pushed_at":"2025-01-25T02:02:13.000Z","size":106,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-01-25T02:27:28.215Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://topkio.rs","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/zTgx.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-01-21T15:39:55.000Z","updated_at":"2025-01-25T02:02:17.000Z","dependencies_parsed_at":"2025-01-25T02:27:35.619Z","dependency_job_id":"d6e4c294-842d-42b3-ac01-b964941300f1","html_url":"https://github.com/zTgx/topkio.rs","commit_stats":null,"previous_names":["ztgx/topkio.rs"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zTgx%2Ftopkio.rs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zTgx%2Ftopkio.rs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zTgx%2Ftopkio.rs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zTgx%2Ftopkio.rs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/zTgx","download_url":"https://codeload.github.com/zTgx/topkio.rs/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":236589566,"owners_count":19173429,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-01-31T22:05:28.224Z","updated_at":"2025-06-13T03:01:32.162Z","avatar_url":"https://github.com/zTgx.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Topkio - Open Source LLM Gateway\n\nTopkio is an open-source LLM gateway built with Rust and Axum. It provides a unified API for interacting with multiple language models, including OpenAI, Gemini, and Ollama. Topkio simplifies the integration of different LLM providers into your applications, offering a consistent interface and high performance.\n\n## Features\n\n*   **Unified API**: Consistent API calls for various LLM providers (OpenAI, Gemini, Ollama).\n*   **High Performance**: Built with Rust for speed and efficiency.\n*   **Modular Design**: Easy to extend with new providers and models.\n*   **Configuration**: Uses a `topkio.toml` file for easy configuration of providers and models.\n*   **Health Checks**: Basic health check functionality for backends.\n*   **Model Identifier**: Parses model identifiers to route requests to the correct backend.\n*   **Graceful Shutdown**: Implements graceful shutdown using Tokio signals.\n\n## Current Status - Work in Progress (WIP)\n\nTopkio is under active development. Key areas currently being worked on:\n\n*   Implementation of provider APIs (Gemini, Ollama).\n*   Dynamic model configuration and management.\n*   Comprehensive unit tests and documentation.\n*   Improved error handling and validation.\n\n**How to Contribute**\n\n*   **Users**: Report bugs, request features, and provide feedback via [GitHub Issues](https://github.com/zTgx/topkio.rs/issues). Test the OpenAI integration, which is the most stable.\n*   **Contributors**: Look for issues labeled `help wanted` or `good first issue`. Focus on adding tests, improving provider implementations, or enhancing documentation.\n\n## Quick Start\n\n1.  **Clone the repository:**\n\n    ```bash\n    git clone https://github.com/zTgx/topkio.rs.git\n    ```\n2.  **Configure `topkio.toml`:**\n\n    Create a `topkio.toml` file in the `config` directory based on the example provided.  Configure your desired LLM providers (Ollama, Gemini, etc.) with their respective API keys and base URLs.\n\n3.  **Run the application:**\n\n    ```bash\n    cargo run\n    ```\n\n4.  **Send API requests:**\n\nExample `curl` requests:\n\n```bash\ncurl -X POST http://localhost:3000/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\n    \"model\": \"ollama:llama3.2\",\n    \"messages\": [{\"role\": \"user\", \"content\": \"Explain AI in 10 words\"}],\n    \"stream\": false\n  }'\n```\n\n```bash\ncurl -X POST http://localhost:3000/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\n    \"model\": \"gemini:gemini-2.0-flash\",\n    \"messages\": [\n      {\"role\": \"user\", \"content\": \"Explain AI in 10 words\"}\n    ],\n    \"stream\": false\n  }'\n```\n\n    Rust example using `reqwest`:\n\n```rust\nuse reqwest::Client;\nuse serde_json::json;\n\n#[tokio::main]\nasync fn main() -\u003e Result\u003c(), Box\u003cdyn std::error::Error\u003e\u003e {\n    let client = Client::new();\n    let response = client\n        .post(\"http://localhost:3000/chat/completions\")\n        .header(\"Content-Type\", \"application/json\")\n        .body(json!({\n            \"model\": \"ollama:llama3.2\",\n            \"messages\": [{\"role\": \"user\", \"content\": \"Explain AI in 10 words\"}],\n            \"stream\": false\n        }).to_string())\n        .send()\n        .await?;\n\n    println!(\"Response: {:?}\", response);\n    Ok(())\n}\n```\n\n## Configuration\n\nThe `topkio.toml` file is used to configure the gateway.  Here's an example:\n\n```toml\n[server]\nhost = \"0.0.0.0\"\nport = 3000\n\n[providers.ollama]\nenabled = true\nurl = \"http://localhost:11434\"\n\n[providers.gemini]\nenabled = true\napi_key = \"YOUR_GEMINI_API_KEY\"\nurl = \"https://generativelanguage.googleapis.com\"\n```\n\n## License\nApache License 2.0","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fztgx%2Ftopkio.rs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fztgx%2Ftopkio.rs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fztgx%2Ftopkio.rs/lists"}