https://github.com/Abraxas-365/langchain-rust
🦜️🔗LangChain for Rust, the easiest way to write LLM-based programs in Rust
https://github.com/Abraxas-365/langchain-rust
langchain llm llms openai rust
Last synced: over 1 year ago
JSON representation
🦜️🔗LangChain for Rust, the easiest way to write LLM-based programs in Rust
- Host: GitHub
- URL: https://github.com/Abraxas-365/langchain-rust
- Owner: Abraxas-365
- License: mit
- Created: 2024-01-31T14:40:00.000Z (over 2 years ago)
- Default Branch: main
- Last Pushed: 2024-05-01T18:22:13.000Z (about 2 years ago)
- Last Synced: 2024-05-02T06:17:02.511Z (about 2 years ago)
- Topics: langchain, llm, llms, openai, rust
- Language: Rust
- Homepage:
- Size: 656 KB
- Stars: 198
- Watchers: 8
- Forks: 24
- Open Issues: 17
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
Awesome Lists containing this project
- awesome-llm-and-aigc - Abraxas-365/langchain-rust - 365/langchain-rust?style=social"/> : 🦜️🔗LangChain for Rust, the easiest way to write LLM-based programs in Rust. (Summary)
- awesome-rust-list - Abraxas-365/langchain-rust - 365/langchain-rust?style=social"/> : 🦜️🔗LangChain for Rust, the easiest way to write LLM-based programs in Rust. (Machine Learning)
- awesome-langchain - Langchain-rust - 365/langchain-rust?style=social) (Ports to other languages)
- awesome-rust-ai-libraries - LangChain-rust - LangChain-rust is a Rust implementation of the LangChain framework for building LLM-powered applications, featuring chain composition, tool integration, memory management, prompt templating, and support for multiple LLM providers including OpenAI and Anthropic. It enables developers to build AI agents, construct RAG pipelines, and create conversational AI systems with Rust's performance and safety guarantees. The project is available at https://github.com/Abraxas-365/langchain-rust. ([Read more](/details/langchain-rust.md)) `Langchain` `Chains` `Agents` (RAG Tools)
README
# 🦜️🔗LangChain Rust
[![Latest Version]][crates.io]
[Latest Version]: https://img.shields.io/crates/v/langchain-rust.svg
[crates.io]: https://crates.io/crates/langchain-rust
⚡ Building applications with LLMs through composability, with Rust! ⚡
[](https://discord.gg/JJFcTFbanu)
[](https://langchain-rust.sellie.tech/get-started/quickstart)
## 🤔 What is this?
This is the Rust language implementation of [LangChain](https://github.com/langchain-ai/langchain).
## Current Features
- LLMs
- [x] [OpenAi](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/llm_openai.rs)
- [x] [Azure OpenAi](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/llm_azure_open_ai.rs)
- [x] [Ollama](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/llm_ollama.rs)
- [x] [Anthropic Claude](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/llm_anthropic_claude.rs)
- Embeddings
- [x] [OpenAi](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/embedding_openai.rs)
- [x] [Azure OpenAi](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/embedding_azure_open_ai.rs)
- [x] [Ollama](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/embedding_ollama.rs)
- [x] [Local FastEmbed](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/embedding_fastembed.rs)
- [x] [MistralAI](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/embedding_mistralai.rs)
- VectorStores
- [x] [OpenSearch](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/vector_store_opensearch.rs)
- [x] [Postgres](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/vector_store_postgres.rs)
- [x] [Qdrant](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/vector_store_qdrant.rs)
- [x] [Sqlite](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/vector_store_sqlite_vss.rs)
- [x] [SurrealDB](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/vector_store_surrealdb/src/main.rs)
- Chain
- [x] [LLM Chain](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/llm_chain.rs)
- [x] [Conversational Chain](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/conversational_chain.rs)
- [x] [Conversational Retriever Simple](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/conversational_retriever_simple_chain.rs)
- [x] [Conversational Retriever With Vector Store](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/conversational_retriever_chain_with_vector_store.rs)
- [x] [Sequential Chain](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/sequential_chain.rs)
- [x] [Q&A Chain](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/qa_chain.rs)
- [x] [SQL Chain](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/sql_chain.rs)
- Agents
- [x] [Chat Agent with Tools](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/agent.rs)
- [x] [Open AI Compatible Tools Agent](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/open_ai_tools_agent.rs)
- Tools
- [x] Serpapi/Google
- [x] DuckDuckGo Search
- [x] [Wolfram/Math](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/wolfram_tool.rs)
- [x] Command line
- [x] [Text2Speech](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/speech2text_openai.rs)
- Semantic Routing
- [x] [Static Routing](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/semantic_routes.rs)
- [x] [Dynamic Routing](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/dynamic_semantic_routes.rs)
- Document Loaders
- [x] PDF
```rust
use futures_util::StreamExt;
async fn main() {
let path = "./src/document_loaders/test_data/sample.pdf";
let loader = PdfExtractLoader::from_path(path).expect("Failed to create PdfExtractLoader");
// let loader = LoPdfLoader::from_path(path).expect("Failed to create LoPdfLoader");
let docs = loader
.load()
.await
.unwrap()
.map(|d| d.unwrap())
.collect::>()
.await;
}
```
- [x] Pandoc
```rust
use futures_util::StreamExt;
async fn main() {
let path = "./src/document_loaders/test_data/sample.docx";
let loader = PandocLoader::from_path(InputFormat::Docx.to_string(), path)
.await
.expect("Failed to create PandocLoader");
let docs = loader
.load()
.await
.unwrap()
.map(|d| d.unwrap())
.collect::>()
.await;
}
```
- [x] HTML
```rust
use futures_util::StreamExt;
use url::Url;
async fn main() {
let path = "./src/document_loaders/test_data/example.html";
let html_loader = HtmlLoader::from_path(path, Url::parse("https://example.com/").unwrap())
.expect("Failed to create html loader");
let documents = html_loader
.load()
.await
.unwrap()
.map(|x| x.unwrap())
.collect::>()
.await;
}
```
- [x] HTML To Markdown
```rust
use futures_util::StreamExt;
use url::Url;
async fn main() {
let path = "./src/document_loaders/test_data/example.html";
let html_to_markdown_loader = HtmlToMarkdownLoader::from_path(path, Url::parse("https://example.com/").unwrap(), HtmlToMarkdownOptions::default().with_skip_tags(vec!["figure".to_string()]))
.expect("Failed to create html to markdown loader");
let documents = html_to_markdown_loader
.load()
.await
.unwrap()
.map(|x| x.unwrap())
.collect::>()
.await;
}
```
- [x] CSV
```rust
use futures_util::StreamExt;
async fn main() {
let path = "./src/document_loaders/test_data/test.csv";
let columns = vec![
"name".to_string(),
"age".to_string(),
"city".to_string(),
"country".to_string(),
];
let csv_loader = CsvLoader::from_path(path, columns).expect("Failed to create csv loader");
let documents = csv_loader
.load()
.await
.unwrap()
.map(|x| x.unwrap())
.collect::>()
.await;
}
```
- [x] Git commits
```rust
use futures_util::StreamExt;
async fn main() {
let path = "/path/to/git/repo";
let git_commit_loader = GitCommitLoader::from_path(path).expect("Failed to create git commit loader");
let documents = csv_loader
.load()
.await
.unwrap()
.map(|x| x.unwrap())
.collect::>()
.await;
}
```
- [x] Source code
```rust
let loader_with_dir =
SourceCodeLoader::from_path("./src/document_loaders/test_data".to_string())
.with_dir_loader_options(DirLoaderOptions {
glob: None,
suffixes: Some(vec!["rs".to_string()]),
exclude: None,
});
let stream = loader_with_dir.load().await.unwrap();
let documents = stream.map(|x| x.unwrap()).collect::>().await;
```
## Installation
This library heavily relies on `serde_json` for its operation.
### Step 1: Add `serde_json`
First, ensure `serde_json` is added to your Rust project.
```bash
cargo add serde_json
```
### Step 2: Add `langchain-rust`
Then, you can add `langchain-rust` to your Rust project.
#### Simple install
```bash
cargo add langchain-rust
```
#### With Sqlite
##### sqlite-vss
Download additional sqlite_vss libraries from
```bash
cargo add langchain-rust --features sqlite-vss
```
##### sqlite-vec
Download additional sqlite_vec libraries from
```bash
cargo add langchain-rust --features sqlite-vec
```
#### With Postgres
```bash
cargo add langchain-rust --features postgres
```
#### With SurrialDB
```bash
cargo add langchain-rust --features surrealdb
```
#### With Qdrant
```bash
cargo add langchain-rust --features qdrant
```
Please remember to replace the feature flags `sqlite`, `postgres` or `surrealdb` based on your
specific use case.
This will add both `serde_json` and `langchain-rust` as dependencies in your `Cargo.toml`
file. Now, when you build your project, both dependencies will be fetched and compiled, and will be available for use in your project.
Remember, `serde_json` is a necessary dependencies, and `sqlite`, `postgres` and `surrealdb`
are optional features that may be added according to project needs.
### Quick Start Conversational Chain
```rust
use langchain_rust::{
chain::{Chain, LLMChainBuilder},
fmt_message, fmt_placeholder, fmt_template,
language_models::llm::LLM,
llm::openai::{OpenAI, OpenAIModel},
message_formatter,
prompt::HumanMessagePromptTemplate,
prompt_args,
schemas::messages::Message,
template_fstring,
};
#[tokio::main]
async fn main() {
//We can then initialize the model:
// If you'd prefer not to set an environment variable you can pass the key in directly via the `openai_api_key` named parameter when initiating the OpenAI LLM class:
// let open_ai = OpenAI::default()
// .with_config(
// OpenAIConfig::default()
// .with_api_key(""),
// ).with_model(OpenAIModel::Gpt4oMini.to_string());
let open_ai = OpenAI::default().with_model(OpenAIModel::Gpt4oMini.to_string());
//Once you've installed and initialized the LLM of your choice, we can try using it! Let's ask it what LangSmith is - this is something that wasn't present in the training data so it shouldn't have a very good response.
let resp = open_ai.invoke("What is rust").await.unwrap();
println!("{}", resp);
// We can also guide it's response with a prompt template. Prompt templates are used to convert raw user input to a better input to the LLM.
let prompt = message_formatter![
fmt_message!(Message::new_system_message(
"You are world class technical documentation writer."
)),
fmt_template!(HumanMessagePromptTemplate::new(template_fstring!(
"{input}", "input"
)))
];
//We can now combine these into a simple LLM chain:
let chain = LLMChainBuilder::new()
.prompt(prompt)
.llm(open_ai.clone())
.build()
.unwrap();
//We can now invoke it and ask the same question. It still won't know the answer, but it should respond in a more proper tone for a technical writer!
match chain
.invoke(prompt_args! {
"input" => "Quien es el escritor de 20000 millas de viaje submarino",
})
.await
{
Ok(result) => {
println!("Result: {:?}", result);
}
Err(e) => panic!("Error invoking LLMChain: {:?}", e),
}
//If you want to prompt to have a list of messages you could use the `fmt_placeholder` macro
let prompt = message_formatter![
fmt_message!(Message::new_system_message(
"You are world class technical documentation writer."
)),
fmt_placeholder!("history"),
fmt_template!(HumanMessagePromptTemplate::new(template_fstring!(
"{input}", "input"
))),
];
let chain = LLMChainBuilder::new()
.prompt(prompt)
.llm(open_ai)
.build()
.unwrap();
match chain
.invoke(prompt_args! {
"input" => "Who is the writer of 20,000 Leagues Under the Sea, and what is my name?",
"history" => vec![
Message::new_human_message("My name is: luis"),
Message::new_ai_message("Hi luis"),
],
})
.await
{
Ok(result) => {
println!("Result: {:?}", result);
}
Err(e) => panic!("Error invoking LLMChain: {:?}", e),
}
}
```