{"id":15282771,"url":"https://github.com/guywaldman/orch","last_synced_at":"2025-04-04T20:13:19.469Z","repository":{"id":248787098,"uuid":"647024564","full_name":"guywaldman/orch","owner":"guywaldman","description":"Rust framework for LLM orchestration","archived":false,"fork":false,"pushed_at":"2024-07-27T21:01:25.000Z","size":188,"stargazers_count":202,"open_issues_count":2,"forks_count":4,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-03-28T19:08:23.865Z","etag":null,"topics":["ai","llm","rust"],"latest_commit_sha":null,"homepage":"https://crates.io/crates/orch","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/guywaldman.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-05-29T22:37:55.000Z","updated_at":"2025-02-19T16:11:39.000Z","dependencies_parsed_at":"2024-10-23T01:16:03.201Z","dependency_job_id":null,"html_url":"https://github.com/guywaldman/orch","commit_stats":null,"previous_names":["guywaldman/janus","guywaldman/orch"],"tags_count":13,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/guywaldman%2Forch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/guywaldman%2Forch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/guywaldman%2Forch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/guywaldman%2Forch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/guywaldman","download_url":"https://codeload.github.com/guywaldman/orch/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247242680,"owners_count":20907134,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","llm","rust"],"created_at":"2024-09-30T14:39:28.584Z","updated_at":"2025-04-04T20:13:19.443Z","avatar_url":"https://github.com/guywaldman.png","language":"Rust","funding_links":[],"categories":["Rust"],"sub_categories":[],"readme":"# orch\n\n![Crates.io Version](https://img.shields.io/crates/v/orch?link=https%3A%2F%2Fcrates.io%2Fcrates%2Forch)\n![Crates.io Total Downloads](https://img.shields.io/crates/d/orch?link=https%3A%2F%2Fcrates.io%2Fcrates%2Forch)\n\n`orch` is a library for building language model powered applications and agents for the Rust programming language.\nIt was primarily built for usage in [magic-cli](https://github.com/guywaldman/magic-cli), but can be used in other contexts as well.\n\n\u003e [!NOTE]\n\u003e\n\u003e If the project gains traction, this can be compiled as an addon to other languages such as Python or a standalone WebAssembly module.\n\n# Installation\n\n```shell\ncargo add orch\ncargo add orch_response\n```\n\nAlternatively, add `orch` as a dependency to your `Cargo.toml` file:\n\n```toml\n[dependencies]\norch = \"*\" # Substitute with the latest version\norch_response = \"*\" # Substitute with the latest version\n```\n\n# Basic Usage\n\n## Simple Text Generation\n\n```rust no_run\nuse orch::execution::*;\nuse orch::lm::*;\n\n#[tokio::main]\nasync fn main() {\n  let lm = OllamaBuilder::new().try_build().unwrap();\n  let executor = TextExecutorBuilder::new().with_lm(\u0026lm).try_build().unwrap();\n  let response = executor.execute(\"What is 2+2?\").await.expect(\"Execution failed\");\n  println!(\"{}\", response.content);\n}\n```\n\n## Streaming Text Generation\n\n```rust no_run\nuse orch::execution::*;\nuse orch::lm::*;\nuse tokio_stream::StreamExt;\n\n#[tokio::main]\nasync fn main() {\n  let lm = OllamaBuilder::new().try_build().unwrap();\n  let executor = TextExecutorBuilder::new().with_lm(\u0026lm).try_build().unwrap();\n  let mut response = executor.execute_stream(\"What is 2+2?\").await.expect(\"Execution failed\");\n  while let Some(chunk) = response.stream.next().await {\n    match chunk {\n      Ok(chunk) =\u003e print!(\"{chunk}\"),\n      Err(e) =\u003e {\n        println!(\"Error: {e}\");\n        break;\n      }\n    }\n  }\n  println!();\n}\n```\n\n## Structured Data Generation\n\n```rust no_run\nuse orch::execution::*;\nuse orch::lm::*;\nuse orch::response::*;\n\n#[derive(Variants, serde::Deserialize)]\npub enum ResponseVariants {\n    Answer(AnswerResponseVariant),\n    Fail(FailResponseVariant),\n}\n\n#[derive(Variant, serde::Deserialize)]\n#[variant(\n    variant = \"Answer\",\n    scenario = \"You know the capital city of the country\",\n    description = \"Capital city of the country\"\n)]\npub struct AnswerResponseVariant {\n    #[schema(\n        description = \"Capital city of the received country\",\n        example = \"London\"\n    )]\n    pub capital: String,\n}\n\n#[derive(Variant, serde::Deserialize)]\n#[variant(\n    variant = \"Fail\",\n    scenario = \"You don't know the capital city of the country\",\n    description = \"Reason why the capital city is not known\"\n)]\npub struct FailResponseVariant {\n    #[schema(\n        description = \"Reason why the capital city is not known\",\n        example = \"Country 'foobar' does not exist\"\n    )]\n    pub reason: String,\n}\n\n#[tokio::main]\nasync fn main() {\n    let lm = OllamaBuilder::new().try_build().unwrap();\n    let executor = StructuredExecutorBuilder::new()\n    .with_lm(\u0026lm)\n    .with_preamble(\"You are a geography expert who helps users understand the capital city of countries around the world.\")\n\t\t.with_options(Box::new(variants!(ResponseVariants)))\n    .try_build()\n    .unwrap();\n    let response = executor\n        .execute(\"What is the capital of Fooland?\")\n        .await\n        .expect(\"Execution failed\");\n\n    println!(\"Response:\");\n    match response.content {\n        ResponseVariants::Answer(answer) =\u003e {\n            println!(\"Capital city: {}\", answer.capital);\n        }\n        ResponseVariants::Fail(fail) =\u003e {\n            println!(\"Model failed to generate a response: {}\", fail.reason);\n        }\n    }\n}\n```\n\n## Embedding Generation\n\n```rust no_run\nuse orch::execution::*;\nuse orch::lm::*;\n\n#[tokio::main]\nasync fn main() {\n  let lm = OllamaBuilder::new().try_build().unwrap();\n  let executor = TextExecutorBuilder::new()\n    .with_lm(\u0026lm)\n    .try_build()\n    .unwrap();\n  let embedding = executor\n    .generate_embedding(\"Phrase to generate an embedding for\")\n    .await\n    .expect(\"Execution failed\");\n\n  println!(\"Embedding:\");\n  println!(\"{:?}\", embedding);\n}\n```\n\n## More Examples\n\nSee the [examples](https://github.com/guywaldman/orch/tree/main/core/examples) directory for usage examples.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fguywaldman%2Forch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fguywaldman%2Forch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fguywaldman%2Forch/lists"}