Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/Abraxas-365/langchain-rust

🦜️🔗LangChain for Rust, the easiest way to write LLM-based programs in Rust
https://github.com/Abraxas-365/langchain-rust

langchain llm llms openai rust

Last synced: 12 days ago
JSON representation

🦜️🔗LangChain for Rust, the easiest way to write LLM-based programs in Rust

Awesome Lists containing this project

README

        

# 🦜️🔗LangChain Rust

[![Latest Version]][crates.io]

[Latest Version]: https://img.shields.io/crates/v/langchain-rust.svg
[crates.io]: https://crates.io/crates/langchain-rust

⚡ Building applications with LLMs through composability, with Rust! ⚡

[![Discord](https://dcbadge.vercel.app/api/server/JJFcTFbanu?style=for-the-badge)](https://discord.gg/JJFcTFbanu)
[![Docs: Tutorial](https://img.shields.io/badge/docs-tutorial-success?style=for-the-badge&logo=appveyor)](https://langchain-rust.sellie.tech/get-started/quickstart)

## 🤔 What is this?

This is the Rust language implementation of [LangChain](https://github.com/langchain-ai/langchain).

## Current Features

- LLMs

- [x] [OpenAi](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/llm_openai.rs)
- [x] [Azure OpenAi](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/llm_azure_open_ai.rs)
- [x] [Ollama](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/llm_ollama.rs)
- [x] [Anthropic Claude](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/llm_anthropic_claude.rs)

- Embeddings

- [x] [OpenAi](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/embedding_openai.rs)
- [x] [Azure OpenAi](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/embedding_azure_open_ai.rs)
- [x] [Ollama](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/embedding_ollama.rs)
- [x] [Local FastEmbed](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/embedding_fastembed.rs)
- [x] [MistralAI](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/embedding_mistralai.rs)

- VectorStores

- [x] [OpenSearch](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/vector_store_opensearch.rs)
- [x] [Postgres](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/vector_store_postgres.rs)
- [x] [Qdrant](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/vector_store_qdrant.rs)
- [x] [Sqlite](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/vector_store_sqlite_vss.rs)
- [x] [SurrealDB](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/vector_store_surrealdb/src/main.rs)

- Chain

- [x] [LLM Chain](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/llm_chain.rs)
- [x] [Conversational Chain](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/conversational_chain.rs)
- [x] [Conversational Retriever Simple](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/conversational_retriever_simple_chain.rs)
- [x] [Conversational Retriever With Vector Store](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/conversational_retriever_chain_with_vector_store.rs)
- [x] [Sequential Chain](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/sequential_chain.rs)
- [x] [Q&A Chain](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/qa_chain.rs)
- [x] [SQL Chain](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/sql_chain.rs)

- Agents

- [x] [Chat Agent with Tools](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/agent.rs)
- [x] [Open AI Compatible Tools Agent](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/open_ai_tools_agent.rs)

- Tools

- [x] Serpapi/Google
- [x] DuckDuckGo Search
- [x] [Wolfram/Math](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/wolfram_tool.rs)
- [x] Command line
- [x] [Text2Speech](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/speech2text_openai.rs)

- Semantic Routing

- [x] [Static Routing](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/semantic_routes.rs)
- [x] [Dynamic Routing](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/dynamic_semantic_routes.rs)

- Document Loaders

- [x] PDF

```rust
use futures_util::StreamExt;

async fn main() {
let path = "./src/document_loaders/test_data/sample.pdf";

let loader = PdfExtractLoader::from_path(path).expect("Failed to create PdfExtractLoader");
// let loader = LoPdfLoader::from_path(path).expect("Failed to create LoPdfLoader");

let docs = loader
.load()
.await
.unwrap()
.map(|d| d.unwrap())
.collect::>()
.await;

}
```

- [x] Pandoc

```rust
use futures_util::StreamExt;

async fn main() {

let path = "./src/document_loaders/test_data/sample.docx";

let loader = PandocLoader::from_path(InputFormat::Docx.to_string(), path)
.await
.expect("Failed to create PandocLoader");

let docs = loader
.load()
.await
.unwrap()
.map(|d| d.unwrap())
.collect::>()
.await;
}
```

- [x] HTML

```rust
use futures_util::StreamExt;
use url::Url;

async fn main() {
let path = "./src/document_loaders/test_data/example.html";
let html_loader = HtmlLoader::from_path(path, Url::parse("https://example.com/").unwrap())
.expect("Failed to create html loader");

let documents = html_loader
.load()
.await
.unwrap()
.map(|x| x.unwrap())
.collect::>()
.await;
}
```

- [x] CSV

```rust
use futures_util::StreamExt;

async fn main() {
let path = "./src/document_loaders/test_data/test.csv";
let columns = vec![
"name".to_string(),
"age".to_string(),
"city".to_string(),
"country".to_string(),
];
let csv_loader = CsvLoader::from_path(path, columns).expect("Failed to create csv loader");

let documents = csv_loader
.load()
.await
.unwrap()
.map(|x| x.unwrap())
.collect::>()
.await;
}
```

- [x] Git commits

```rust
use futures_util::StreamExt;

async fn main() {
let path = "/path/to/git/repo";
let git_commit_loader = GitCommitLoader::from_path(path).expect("Failed to create git commit loader");

let documents = csv_loader
.load()
.await
.unwrap()
.map(|x| x.unwrap())
.collect::>()
.await;
}
```

- [x] Source code

```rust

let loader_with_dir =
SourceCodeLoader::from_path("./src/document_loaders/test_data".to_string())
.with_dir_loader_options(DirLoaderOptions {
glob: None,
suffixes: Some(vec!["rs".to_string()]),
exclude: None,
});

let stream = loader_with_dir.load().await.unwrap();
let documents = stream.map(|x| x.unwrap()).collect::>().await;
```

## Installation

This library heavily relies on `serde_json` for its operation.

### Step 1: Add `serde_json`

First, ensure `serde_json` is added to your Rust project.

```bash
cargo add serde_json
```

### Step 2: Add `langchain-rust`

Then, you can add `langchain-rust` to your Rust project.

#### Simple install

```bash
cargo add langchain-rust
```

#### With Sqlite

##### sqlite-vss

Download additional sqlite_vss libraries from

```bash
cargo add langchain-rust --features sqlite-vss
```

##### sqlite-vec

Download additional sqlite_vec libraries from

```bash
cargo add langchain-rust --features sqlite-vec
```

#### With Postgres

```bash
cargo add langchain-rust --features postgres
```

#### With SurrialDB

```bash
cargo add langchain-rust --features surrealdb
```

#### With Qdrant

```bash
cargo add langchain-rust --features qdrant
```

Please remember to replace the feature flags `sqlite`, `postgres` or `surrealdb` based on your
specific use case.

This will add both `serde_json` and `langchain-rust` as dependencies in your `Cargo.toml`
file. Now, when you build your project, both dependencies will be fetched and compiled, and will be available for use in your project.

Remember, `serde_json` is a necessary dependencies, and `sqlite`, `postgres` and `surrealdb`
are optional features that may be added according to project needs.

### Quick Start Conversational Chain

```rust
use langchain_rust::{
chain::{Chain, LLMChainBuilder},
fmt_message, fmt_placeholder, fmt_template,
language_models::llm::LLM,
llm::openai::{OpenAI, OpenAIModel},
message_formatter,
prompt::HumanMessagePromptTemplate,
prompt_args,
schemas::messages::Message,
template_fstring,
};

#[tokio::main]
async fn main() {
//We can then initialize the model:
// If you'd prefer not to set an environment variable you can pass the key in directly via the `openai_api_key` named parameter when initiating the OpenAI LLM class:
// let open_ai = OpenAI::default()
// .with_config(
// OpenAIConfig::default()
// .with_api_key(""),
// ).with_model(OpenAIModel::Gpt4oMini.to_string());
let open_ai = OpenAI::default().with_model(OpenAIModel::Gpt4oMini.to_string());

//Once you've installed and initialized the LLM of your choice, we can try using it! Let's ask it what LangSmith is - this is something that wasn't present in the training data so it shouldn't have a very good response.
let resp = open_ai.invoke("What is rust").await.unwrap();
println!("{}", resp);

// We can also guide it's response with a prompt template. Prompt templates are used to convert raw user input to a better input to the LLM.
let prompt = message_formatter![
fmt_message!(Message::new_system_message(
"You are world class technical documentation writer."
)),
fmt_template!(HumanMessagePromptTemplate::new(template_fstring!(
"{input}", "input"
)))
];

//We can now combine these into a simple LLM chain:

let chain = LLMChainBuilder::new()
.prompt(prompt)
.llm(open_ai.clone())
.build()
.unwrap();

//We can now invoke it and ask the same question. It still won't know the answer, but it should respond in a more proper tone for a technical writer!

match chain
.invoke(prompt_args! {
"input" => "Quien es el escritor de 20000 millas de viaje submarino",
})
.await
{
Ok(result) => {
println!("Result: {:?}", result);
}
Err(e) => panic!("Error invoking LLMChain: {:?}", e),
}

//If you want to prompt to have a list of messages you could use the `fmt_placeholder` macro

let prompt = message_formatter![
fmt_message!(Message::new_system_message(
"You are world class technical documentation writer."
)),
fmt_placeholder!("history"),
fmt_template!(HumanMessagePromptTemplate::new(template_fstring!(
"{input}", "input"
))),
];

let chain = LLMChainBuilder::new()
.prompt(prompt)
.llm(open_ai)
.build()
.unwrap();
match chain
.invoke(prompt_args! {
"input" => "Who is the writer of 20,000 Leagues Under the Sea, and what is my name?",
"history" => vec![
Message::new_human_message("My name is: luis"),
Message::new_ai_message("Hi luis"),
],

})
.await
{
Ok(result) => {
println!("Result: {:?}", result);
}
Err(e) => panic!("Error invoking LLMChain: {:?}", e),
}
}
```