Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

awesome-rust-llm

🦀 A curated list of Rust tools, libraries, and frameworks for working with LLMs, GPT, AI
https://github.com/jondot/awesome-rust-llm

Last synced: 5 days ago
JSON representation

Models & Inference
- llm - a Rust library for running inference from a number of supported LLMs, loads [ggml](https://github.com/ggerganov/ggml) based models
- rust-bert - all in one pipelines for for transformer-based models (BERT, DistilBERT, GPT2,...). Good for local embedding, port of `transformers` (python)
- llm-chain - chaining LLMs in Rust
- smartgpt - use LLMs with the ability to complete complex tasks using plugins
- diffusers - Stable Diffusion using Rust, 45% faster than Pytorch
- postgresml - an amazing Postgres extension to do model fetching, inference all with SQL as part of your Postgres instance
Projects
- aichat - a pure Rust CLI implementing AI chat, with advanced features such as real-time streaming, text highlighting and more
- browser-agent - a headless browser driven by GPT-4. Sends off a simplified page representation and receives & executes instructions from GPT using a custom message format
- tenere - TUI interface for LLMs
- ajeto - LLM personal assistant
- shortgpt - Ask shortgpt for instant and concise answers
- autorust - macros that generate AI driven rust code in compile time
- clerk - LLM based file organizer
- gptcommit - prepare commit message with GPT
- browser-agent - a headless browser driven by GPT-4. Sends off a simplified page representation and receives & executes instructions from GPT using a custom message format
LLM Memory
- memex - Super-simple, fully Rust powered "memory" (doc store + semantic search) for LLM projects, semantic search, etc.
- motorhead - a memory and information retrieval server for LLMs.
- js example - py)
Core Libraries
- tiktoken - tiktoken is a Python library with a Rust core implementing a fast [BPE](https://en.wikipedia.org/wiki/Byte_pair_encoding) tokeniser for use with OpenAI's models
- tiktoken-rs - a Rust focused library based on the `tiktoken` core with additional enhancements for use in Rust code. (Pyton parts in `tiktoken` done in pure Rust)
- polars - a faster, pure Rust pandas alternative
- rllama - a pure Rust implemenation of LLaMa inference. Great for embedding into other apps or wrapping for a scripting language.
- whatlang - Rust library using a multiclass logistic regression model to detect languages
- OpenAI API - a strongly typed Rust client for the OpenAI API
Tools
- spider - crawler / spider written in Rust for when you need a whole-website dump. Unlike Scrapy, focuses on dumping data but the post-processing is done later.
Services
- dust - a full service for workflow running with composable blocks. Core is in Rust, various frontends in Typescript.
Vector Stores
- pgvecto.rs - Vector database plugin for Postgres, written in Rust, specifically designed for LLM. 20x faster than pgvector
- qdrant - Qdrant - Vector Database for the next generation of AI applications

Programming Languages

Rust 23 Python 2 TypeScript 1

Ecosyste.ms: Awesome

awesome-rust-llm

Models & Inference

Projects

LLM Memory

Core Libraries

Tools

Services

Vector Stores