Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/Abraxas-365/langchain-rust

🦜️🔗LangChain for Rust, the easiest way to write LLM-based programs in Rust
https://github.com/Abraxas-365/langchain-rust

langchain llm llms openai rust

Last synced: about 1 month ago
JSON representation

🦜️🔗LangChain for Rust, the easiest way to write LLM-based programs in Rust

Awesome Lists containing this project

README 

        
# 🦜️🔗LangChain Rust



[![Latest Version]][crates.io]



[Latest Version]: https://img.shields.io/crates/v/langchain-rust.svg

[crates.io]: https://crates.io/crates/langchain-rust



⚡ Building applications with LLMs through composability, with Rust! ⚡



[![Discord](https://dcbadge.vercel.app/api/server/JJFcTFbanu?style=for-the-badge)](https://discord.gg/JJFcTFbanu)

[![Docs: Tutorial](https://img.shields.io/badge/docs-tutorial-success?style=for-the-badge&logo=appveyor)](https://langchain-rust.sellie.tech/get-started/quickstart)



## 🤔 What is this?



This is the Rust language implementation of [LangChain](https://github.com/langchain-ai/langchain).



## Current Features



- LLMs



  - [x] [OpenAi](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/llm_openai.rs)

  - [x] [Azure OpenAi](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/llm_azure_open_ai.rs)

  - [x] [Ollama](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/llm_ollama.rs)

  - [x] [Anthropic Claude](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/llm_anthropic_claude.rs)



- Embeddings



  - [x] [OpenAi](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/embedding_openai.rs)

  - [x] [Azure OpenAi](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/embedding_azure_open_ai.rs)

  - [x] [Ollama](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/embedding_ollama.rs)

  - [x] [Local FastEmbed](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/embedding_fastembed.rs)

  - [x] [MistralAI](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/embedding_mistralai.rs)



- VectorStores



  - [x] [OpenSearch](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/vector_store_opensearch.rs)

  - [x] [Postgres](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/vector_store_postgres.rs)

  - [x] [Qdrant](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/vector_store_qdrant.rs)

  - [x] [Sqlite](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/vector_store_sqlite_vss.rs)

  - [x] [SurrealDB](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/vector_store_surrealdb/src/main.rs)



- Chain



  - [x] [LLM Chain](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/llm_chain.rs)

  - [x] [Conversational Chain](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/conversational_chain.rs)

  - [x] [Conversational Retriever Simple](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/conversational_retriever_simple_chain.rs)

  - [x] [Conversational Retriever With Vector Store](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/conversational_retriever_chain_with_vector_store.rs)

  - [x] [Sequential Chain](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/sequential_chain.rs)

  - [x] [Q&A Chain](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/qa_chain.rs)

  - [x] [SQL Chain](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/sql_chain.rs)



- Agents



  - [x] [Chat Agent with Tools](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/agent.rs)

  - [x] [Open AI Compatible Tools Agent](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/open_ai_tools_agent.rs)



- Tools



  - [x] Serpapi/Google

  - [x] DuckDuckGo Search

  - [x] [Wolfram/Math](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/wolfram_tool.rs)

  - [x] Command line

  - [x] [Text2Speech](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/speech2text_openai.rs)



- Semantic Routing



  - [x] [Static Routing](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/semantic_routes.rs)

  - [x] [Dynamic Routing](https://github.com/Abraxas-365/langchain-rust/blob/main/examples/dynamic_semantic_routes.rs)



- Document Loaders



  - [x] PDF



    ```rust

    use futures_util::StreamExt;



    async fn main() {

        let path = "./src/document_loaders/test_data/sample.pdf";



        let loader = PdfExtractLoader::from_path(path).expect("Failed to create PdfExtractLoader");

        // let loader = LoPdfLoader::from_path(path).expect("Failed to create LoPdfLoader");



        let docs = loader

            .load()

            .await

            .unwrap()

            .map(|d| d.unwrap())

            .collect::>()

            .await;



    }

    ```



  - [x] Pandoc



    ```rust

    use futures_util::StreamExt;



    async fn main() {



        let path = "./src/document_loaders/test_data/sample.docx";



        let loader = PandocLoader::from_path(InputFormat::Docx.to_string(), path)

            .await

            .expect("Failed to create PandocLoader");



        let docs = loader

            .load()

            .await

            .unwrap()

            .map(|d| d.unwrap())

            .collect::>()

            .await;

    }

    ```



  - [x] HTML



    ```rust

    use futures_util::StreamExt;

    use url::Url;



    async fn main() {

        let path = "./src/document_loaders/test_data/example.html";

        let html_loader = HtmlLoader::from_path(path, Url::parse("https://example.com/").unwrap())

            .expect("Failed to create html loader");



        let documents = html_loader

            .load()

            .await

            .unwrap()

            .map(|x| x.unwrap())

            .collect::>()

            .await;

    }

    ```



  - [x] CSV



    ```rust

    use futures_util::StreamExt;



    async fn main() {

        let path = "./src/document_loaders/test_data/test.csv";

        let columns = vec![

            "name".to_string(),

            "age".to_string(),

            "city".to_string(),

            "country".to_string(),

        ];

        let csv_loader = CsvLoader::from_path(path, columns).expect("Failed to create csv loader");



        let documents = csv_loader

            .load()

            .await

            .unwrap()

            .map(|x| x.unwrap())

            .collect::>()

            .await;

    }

    ```



  - [x] Git commits



    ```rust

    use futures_util::StreamExt;



    async fn main() {

        let path = "/path/to/git/repo";

        let git_commit_loader = GitCommitLoader::from_path(path).expect("Failed to create git commit loader");



        let documents = csv_loader

            .load()

            .await

            .unwrap()

            .map(|x| x.unwrap())

            .collect::>()

            .await;

    }

    ```



  - [x] Source code



    ```rust



    let loader_with_dir =

    SourceCodeLoader::from_path("./src/document_loaders/test_data".to_string())

    .with_dir_loader_options(DirLoaderOptions {

    glob: None,

    suffixes: Some(vec!["rs".to_string()]),

    exclude: None,

    });



    let stream = loader_with_dir.load().await.unwrap();

    let documents = stream.map(|x| x.unwrap()).collect::>().await;

    ```



## Installation



This library heavily relies on `serde_json` for its operation.



### Step 1: Add `serde_json`



First, ensure `serde_json` is added to your Rust project.



```bash

cargo add serde_json

```



### Step 2: Add `langchain-rust`



Then, you can add `langchain-rust` to your Rust project.



#### Simple install



```bash

cargo add langchain-rust

```



#### With Sqlite



##### sqlite-vss



Download additional sqlite_vss libraries from 



```bash

cargo add langchain-rust --features sqlite-vss

```



##### sqlite-vec



Download additional sqlite_vec libraries from 



```bash

cargo add langchain-rust --features sqlite-vec

```



#### With Postgres



```bash

cargo add langchain-rust --features postgres

```



#### With SurrialDB



```bash

cargo add langchain-rust --features surrealdb

```



#### With Qdrant



```bash

cargo add langchain-rust --features qdrant

```



Please remember to replace the feature flags `sqlite`, `postgres` or `surrealdb` based on your

specific use case.



This will add both `serde_json` and `langchain-rust` as dependencies in your `Cargo.toml`

file. Now, when you build your project, both dependencies will be fetched and compiled, and will be available for use in your project.



Remember, `serde_json` is a necessary dependencies, and `sqlite`, `postgres` and `surrealdb`

are optional features that may be added according to project needs.



### Quick Start Conversational Chain



```rust

use langchain_rust::{

    chain::{Chain, LLMChainBuilder},

    fmt_message, fmt_placeholder, fmt_template,

    language_models::llm::LLM,

    llm::openai::{OpenAI, OpenAIModel},

    message_formatter,

    prompt::HumanMessagePromptTemplate,

    prompt_args,

    schemas::messages::Message,

    template_fstring,

};



#[tokio::main]

async fn main() {

    //We can then initialize the model:

    // If you'd prefer not to set an environment variable you can pass the key in directly via the `openai_api_key` named parameter when initiating the OpenAI LLM class:

    // let open_ai = OpenAI::default()

    //     .with_config(

    //         OpenAIConfig::default()

    //             .with_api_key(""),

    //     ).with_model(OpenAIModel::Gpt4oMini.to_string());

    let open_ai = OpenAI::default().with_model(OpenAIModel::Gpt4oMini.to_string());



    //Once you've installed and initialized the LLM of your choice, we can try using it! Let's ask it what LangSmith is - this is something that wasn't present in the training data so it shouldn't have a very good response.

    let resp = open_ai.invoke("What is rust").await.unwrap();

    println!("{}", resp);



    // We can also guide it's response with a prompt template. Prompt templates are used to convert raw user input to a better input to the LLM.

    let prompt = message_formatter![

        fmt_message!(Message::new_system_message(

            "You are world class technical documentation writer."

        )),

        fmt_template!(HumanMessagePromptTemplate::new(template_fstring!(

            "{input}", "input"

        )))

    ];



    //We can now combine these into a simple LLM chain:



    let chain = LLMChainBuilder::new()

        .prompt(prompt)

        .llm(open_ai.clone())

        .build()

        .unwrap();



    //We can now invoke it and ask the same question. It still won't know the answer, but it should respond in a more proper tone for a technical writer!



    match chain

        .invoke(prompt_args! {

        "input" => "Quien es el escritor de 20000 millas de viaje submarino",

           })

        .await

    {

        Ok(result) => {

            println!("Result: {:?}", result);

        }

        Err(e) => panic!("Error invoking LLMChain: {:?}", e),

    }



    //If you want to prompt to have a list of messages you could use the `fmt_placeholder` macro



    let prompt = message_formatter![

        fmt_message!(Message::new_system_message(

            "You are world class technical documentation writer."

        )),

        fmt_placeholder!("history"),

        fmt_template!(HumanMessagePromptTemplate::new(template_fstring!(

            "{input}", "input"

        ))),

    ];



    let chain = LLMChainBuilder::new()

        .prompt(prompt)

        .llm(open_ai)

        .build()

        .unwrap();

    match chain

        .invoke(prompt_args! {

        "input" => "Who is the writer of 20,000 Leagues Under the Sea, and what is my name?",

        "history" => vec![

                Message::new_human_message("My name is: luis"),

                Message::new_ai_message("Hi luis"),

                ],



        })

        .await

    {

        Ok(result) => {

            println!("Result: {:?}", result);

        }

        Err(e) => panic!("Error invoking LLMChain: {:?}", e),

    }

}

```