{"id":22122099,"url":"https://github.com/starlightsearch/embedanything","last_synced_at":"2025-12-28T04:33:21.993Z","repository":{"id":231546988,"uuid":"780174294","full_name":"StarlightSearch/EmbedAnything","owner":"StarlightSearch","description":"Production-ready Inference, Ingestion and Indexing built in Rust 🦀","archived":false,"fork":false,"pushed_at":"2025-05-06T08:13:11.000Z","size":38397,"stargazers_count":555,"open_issues_count":18,"forks_count":49,"subscribers_count":8,"default_branch":"main","last_synced_at":"2025-05-14T17:05:06.323Z","etag":null,"topics":["colpali","embedding-models","index","indexing","information-retrieval","ingestion","jina","large-language-models","late-interaction","machine-learning","modernbert","onnx","onnxruntime","openai","rag","rust","rust-lang","splade","vector-database","vision-language-model"],"latest_commit_sha":null,"homepage":"https://embed-anything.com/","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/StarlightSearch.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":"docs/roadmap/contribution.md","authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-03-31T22:37:47.000Z","updated_at":"2025-05-13T10:35:57.000Z","dependencies_parsed_at":"2024-04-15T12:26:01.448Z","dependency_job_id":"0bc769b6-cf15-47c0-8bb2-bf7904e6a08e","html_url":"https://github.com/StarlightSearch/EmbedAnything","commit_stats":null,"previous_names":["starlightsearch/embedanything"],"tags_count":34,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/StarlightSearch%2FEmbedAnything","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/StarlightSearch%2FEmbedAnything/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/StarlightSearch%2FEmbedAnything/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/StarlightSearch%2FEmbedAnything/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/StarlightSearch","download_url":"https://codeload.github.com/StarlightSearch/EmbedAnything/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254190396,"owners_count":22029632,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["colpali","embedding-models","index","indexing","information-retrieval","ingestion","jina","large-language-models","late-interaction","machine-learning","modernbert","onnx","onnxruntime","openai","rag","rust","rust-lang","splade","vector-database","vision-language-model"],"created_at":"2024-12-01T15:17:22.642Z","updated_at":"2025-12-28T04:33:21.984Z","avatar_url":"https://github.com/StarlightSearch.png","language":"Rust","funding_links":[],"categories":["NLP"],"sub_categories":[],"readme":"\n\n\u003cp align =\"center\"\u003e\n\u003cimg width=400 src = \"https://res.cloudinary.com/dltwftrgc/image/upload/v1712504276/Projects/EmbedAnything_500_x_200_px_a4l8xu.png\"\u003e\n\u003c/p\u003e\n\n\n\n\u003cdiv align=\"center\"\u003e\n\n[![Downloads](https://static.pepy.tech/badge/embed-anything)](https://pepy.tech/project/embed-anything)\n[![gpu](https://static.pepy.tech/badge/embed-anything-gpu)](https://www.pepy.tech/projects/embed-anything-gpu)\n[![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1CowJrqZxDDYJzkclI-rbHaZHgL9C6K3p?usp=sharing)\n[![roadmap](https://img.shields.io/badge/Discord-%235865F2.svg?style=flat\u0026logo=discord\u0026logoColor=white)](https://discord.gg/juETVTMdZu)\n[![MkDocs](https://img.shields.io/badge/Blogs-F38020?.svg?logoColor=fff)](https://embed-anything.com/blog/)\n\n\u003c/div\u003e\n\n\n\u003cdiv align=\"center\"\u003e\n\n  \u003cp align=\"center\"\u003e\n    \u003cb\u003e Highly Performant, Modular and Memory Safe\u003c/b\u003e\n    \u003cbr /\u003e\n    \u003cb\u003e Ingestion, Inference and Indexing in Rust 🦀\u003c/b\u003e\n    \u003cbr /\u003e\n    \u003ca href=\"https://embed-anything.com/references/\"\u003ePython docs »\u003c/a\u003e\n    \u003cbr /\u003e\n    \u003ca href=\"https://docs.rs/embed_anything/latest/embed_anything/\"\u003eRust docs »\u003c/a\u003e\n    \u003cbr /\u003e\n    \u003ca href=\"https://github.com/StarlightSearch/EmbedAnything?tab=readme-ov-file#benchmarks\"\u003e\u003cstrong\u003eBenchmarks\u003c/strong\u003e\u003c/a\u003e\n    ·\n    \u003ca href=\"https://github.com/StarlightSearch/EmbedAnything?tab=readme-ov-file#%EF%B8%8Ffaq\"\u003e\u003cstrong\u003eFAQ\u003c/strong\u003e\u003c/a\u003e\n    ·\n    \u003ca href=\"https://github.com/StarlightSearch/EmbedAnything/tree/main/examples/adapters\"\u003e\u003cstrong\u003eAdapters\u003c/strong\u003e\u003c/a\u003e\n    .\n    \u003ca href=\"https://github.com/StarlightSearch/EmbedAnything?tab=readme-ov-file#-our-past-collaborations\"\u003e\u003cstrong\u003eCollaborations\u003c/strong\u003e\u003c/a\u003e\n    .\n     \u003ca href=\"https://github.com/StarlightSearch/EmbedAnything?tab=readme-ov-file#-notebooks\"\u003e\u003cstrong\u003eNotebooks\u003c/strong\u003e\u003c/a\u003e\n\n\n    \n  \u003c/p\u003e\n\u003c/div\u003e\n\n\nEmbedAnything is a minimalist, yet highly performant, modular, lightning-fast, lightweight, multisource, multimodal, and local embedding pipeline built in Rust. Whether you're working with text, images, audio, PDFs, websites, or other media, EmbedAnything streamlines the process of generating embeddings from various sources and seamlessly streaming (memory-efficient-indexing) them to a vector database. It supports dense, sparse, ONNX, model2vec and late-interaction embeddings, offering flexibility for a wide range of use cases.\n\n\u003cp align =\"center\"\u003e\n\u003cimg width=400 src = \"https://res.cloudinary.com/dogbbs77y/image/upload/v1766251819/streaming_popagm.png\"\u003e\n\u003c/p\u003e\n\n\u003c!-- TABLE OF CONTENTS --\u003e\n\u003cdetails\u003e\n  \u003csummary\u003eTable of Contents\u003c/summary\u003e\n  \u003col\u003e\n    \u003cli\u003e\n      \u003ca href=\"#about-the-project\"\u003eAbout The Project\u003c/a\u003e\n      \u003cul\u003e\n        \u003cli\u003e\u003ca href=\"https://github.com/StarlightSearch/EmbedAnything?tab=readme-ov-file#the-benefit-of-rust-for-speed\"\u003eBuilt With Rust\u003c/a\u003e\u003c/li\u003e\n        \u003cli\u003e\u003ca href=\"https://github.com/StarlightSearch/EmbedAnything?tab=readme-ov-file#why-candle\"\u003eWhy Candle?\u003c/a\u003e\u003c/li\u003e\n      \u003c/ul\u003e\n    \u003c/li\u003e\n    \u003cli\u003e\n      \u003ca href=\"https://github.com/StarlightSearch/EmbedAnything?tab=readme-ov-file#-getting-started\"\u003eGetting Started\u003c/a\u003e\n      \u003cul\u003e\n        \u003cli\u003e\u003ca href=\"https://github.com/StarlightSearch/EmbedAnything?tab=readme-ov-file#-installation\"\u003eInstallation\u003c/a\u003e\u003c/li\u003e\n      \u003c/ul\u003e\n    \u003c/li\u003e\n    \u003cli\u003e\u003ca href=\"https://github.com/StarlightSearch/EmbedAnything?tab=readme-ov-file#-getting-started\"\u003eUsage\u003c/a\u003e\u003c/li\u003e\n    \u003cli\u003e\u003ca href=\"https://github.com/StarlightSearch/EmbedAnything?tab=readme-ov-file#roadmap\"\u003eRoadmap\u003c/a\u003e\u003c/li\u003e\n    \u003cli\u003e\u003ca href=\"https://github.com/StarlightSearch/EmbedAnything?tab=readme-ov-file#quick-start\"\u003eContributing\u003c/a\u003e\u003c/li\u003e\n    \u003cli\u003e\u003ca href=\"https://github.com/StarlightSearch/EmbedAnything?tab=readme-ov-file#Supported-Models\"\u003eHow to add custom model and chunk size\u003c/a\u003e\u003c/li\u003e\n    \n  \u003c/ol\u003e\n\u003c/details\u003e\n\n\n## 🚀 Key Features\n\n\n- **No Dependency on Pytorch**: Easy to deploy on cloud, comes with low memory footprint.\n- **Highly Modular** : Choose any vectorDB adapter for RAG, with ~~1 line~~ 1 word of code\n- **Candle Backend** : Supports BERT, Jina, ColPali, Splade, ModernBERT, Reranker, Qwen\n- **ONNX Backend**: Supports BERT, Jina, ColPali, ColBERT Splade, Reranker, ModernBERT, Qwen\n- **Cloud Embedding Models:**: Supports OpenAI, Cohere, and Gemini.\n- **MultiModality** : Works with text sources like PDFs, txt, md, Images JPG and Audio, .WAV\n- **GPU support** : Hardware acceleration on GPU as well.\n- **Chunking** : In-built chunking methods like semantic, late-chunking\n- **Vector Streaming:** Separate file processing, Indexing and Inferencing on different threads, reduces latency.\n\n## 💡What is Vector Streaming\n\n Embedding models are computationally expensive and time-consuming. By separating document preprocessing from model inference, you can significantly reduce pipeline latency and improve throughput.\n\nVector streaming transforms a sequential bottleneck into an efficient, concurrent workflow.\n\nThe embedding process happens separetly from the main process, so as to maintain high performance enabled by rust MPSC, and no memory leak as embeddings are directly saved to vector database. Find our [blog](https://starlight-search.com/blog/2025/02/25/vector%20database/).\n\n[![EmbedAnythingXWeaviate](https://res.cloudinary.com/dltwftrgc/image/upload/v1731166897/demo_o8auu4.gif)](https://www.youtube.com/watch?v=OJRWPLQ44Dw)\n\n## 🦀 Why Embed Anything \n\n➡️Faster execution. \u003cbr /\u003e\n➡️No Pytorch Dependency, thus low-memory footprint and easy to deploy on cloud. \u003cbr /\u003e\n➡️True multithreading \u003cbr /\u003e\n➡️Running embedding models locally and efficiently \u003cbr /\u003e\n➡️In-built chunking methods like semantic, late-chunking \u003cbr/\u003e\n➡️Supports range of models, Dense, Sparse, Late-interaction, ReRanker, ModernBert.\u003cbr /\u003e\n➡️Memory Management: Rust enforces memory management simultaneously, preventing memory leaks and crashes that can plague other languages \u003cbr /\u003e\n\n## 🍓 Our Past Collaborations:\n\nWe have collaborated with reputed enterprise like\n[Elastic](https://www.youtube.com/live/OzQopxkxHyY?si=l6KasNNuCNOKky6f), [Weaviate](https://www.linkedin.com/posts/sonam-pankaj_machinelearning-data-ai-activity-7238832243622768644-gB8c?utm_source=share\u0026utm_medium=member_desktop\u0026rcm=ACoAABlF_IAB4Y74d5JJwj0CUwpTkhuskE0PAt4), [SingleStore](https://www.linkedin.com/events/buildingdomain-specificragappli7295319309566775297/theater/), [Milvus](https://milvus.io/docs/build_RAG_with_milvus_and_embedAnything.md) \nand [Analytics Vidya Datahours](https://community.analyticsvidhya.com/c/datahour/multimodal-embeddings-and-search-with-embed-anything-6adba0)\n\nYou can get in touch with us for further collaborations.\n\n## Benchmarks\n\n### Inference Speed benchmarks.\nOnly measures embedding model inference speed, on onnx-runtime. [Code](https://colab.research.google.com/drive/1nXvd25hDYO-j7QGOIIC0M7MDpovuPCaD?usp=sharing)\n\n\u003cimg src=\"https://res.cloudinary.com/dltwftrgc/image/upload/v1730405688/embed_time_zusmua.png\" width=\"500\"\u003e\n\n\nBenchmarks with other fromeworks coming soon!! 🚀\n# ⭐ Supported Models\n\nWe support any hugging-face models on Candle. And We also support ONNX runtime for BERT and ColPali.\n\n## How to add custom model on candle: from_pretrained_hf\n\n```python\nfrom embed_anything import EmbeddingModel, WhichModel, TextEmbedConfig\nimport embed_anything\n\n# Load a custom BERT model from Hugging Face\nmodel = EmbeddingModel.from_pretrained_hf(\n    model_id=\"sentence-transformers/all-MiniLM-L12-v2\"\n)\n\n# Configure embedding parameters\nconfig = TextEmbedConfig(\n    chunk_size=1000,      # Maximum characters per chunk\n    batch_size=32,        # Number of chunks to process in parallel\n    splitting_strategy=\"sentence\"  # How to split text: \"sentence\", \"word\", or \"semantic\"\n)\n\n# Embed a file (supports PDF, TXT, MD, etc.)\ndata = embed_anything.embed_file(\"path/to/your/file.pdf\", embedder=model, config=config)\n\n# Access the embeddings and text\nfor item in data:\n    print(f\"Text: {item.text[:100]}...\")  # First 100 characters\n    print(f\"Embedding shape: {len(item.embedding)}\")\n    print(f\"Metadata: {item.metadata}\")\n    print(\"---\" * 20)\n```\n\n\n| Model  | HF link |\n| ------------- | ------------- | \n| Jina  | [Jina Models](https://huggingface.co/collections/jinaai/jina-embeddings-v2-65708e3ec4993b8fb968e744) | \n| Bert | All Bert based models |\n| CLIP | openai/clip-* | \n| Whisper| [OpenAI Whisper models](https://huggingface.co/collections/openai/whisper-release-6501bba2cf999715fd953013)|\n| ColPali | starlight-ai/colpali-v1.2-merged-onnx|\n| Colbert | answerdotai/answerai-colbert-small-v1, jinaai/jina-colbert-v2 and more |\n| Splade | [Splade Models](https://huggingface.co/collections/naver/splade-667eb6df02c2f3b0c39bd248) and other Splade like models |\n| Model2Vec | model2vec, minishlab/potion-base-8M |\n| Qwen3-Embedding | Qwen/Qwen3-Embedding-0.6B |\n| Reranker | [Jina Reranker Models](https://huggingface.co/jinaai/jina-reranker-v2-base-multilingual), Xenova/bge-reranker, Qwen/Qwen3-Reranker-4B |\n\n\n\n\n## Splade Models (Sparse Embeddings)\n\nSparse embeddings are useful for keyword-based retrieval and hybrid search scenarios.\n\n```python\nfrom embed_anything import EmbeddingModel, WhichModel, TextEmbedConfig\nimport embed_anything\n\n# Load a SPLADE model for sparse embeddings\nmodel = EmbeddingModel.from_pretrained_hf(\n    model_id=\"prithivida/Splade_PP_en_v1\"\n)\n\n# Configure the embedding process\nconfig = TextEmbedConfig(chunk_size=1000, batch_size=32)\n\n# Embed text files\ndata = embed_anything.embed_file(\"test_files/document.txt\", embedder=model, config=config)\n\n# Sparse embeddings are useful for hybrid search (combining dense and sparse)\nfor item in data:\n    print(f\"Text: {item.text}\")\n    print(f\"Sparse embedding (non-zero values): {sum(1 for x in item.embedding if x != 0)}\")\n```\n\n## ONNX-Runtime: from_pretrained_onnx\n\nONNX models provide faster inference and lower memory usage. Use the `ONNXModel` enum for pre-configured models or provide a custom model path.\n\n### BERT Models\n\n```python\nfrom embed_anything import EmbeddingModel, WhichModel, ONNXModel, Dtype, TextEmbedConfig\nimport embed_anything\n\n# Option 1: Use a pre-configured ONNX model (recommended)\nmodel = EmbeddingModel.from_pretrained_onnx(\n    WhichModel.Bert, \n    model_id=ONNXModel.BGESmallENV15Q  # Quantized BGE model for faster inference\n)\n\n# Option 2: Use a custom ONNX model from Hugging Face\nmodel = EmbeddingModel.from_pretrained_onnx(\n    WhichModel.Bert, \n    model_id=\"onnx_model_link\",\n    dtype=Dtype.F16  # Use half precision for faster inference\n)\n\n# Embed files with ONNX model\nconfig = TextEmbedConfig(chunk_size=1000, batch_size=32)\ndata = embed_anything.embed_file(\"test_files/document.pdf\", embedder=model, config=config)\n```\n\n### ModernBERT (Quantized)\n\nModernBERT is a state-of-the-art BERT variant optimized for efficiency.\n\n```python\nfrom embed_anything import EmbeddingModel, WhichModel, ONNXModel, Dtype\n\n# Load quantized ModernBERT for maximum efficiency\nmodel = EmbeddingModel.from_pretrained_onnx(\n    WhichModel.Bert, \n    model_id=ONNXModel.ModernBERTBase, \n    dtype=Dtype.Q4F16  # 4-bit quantized for minimal memory usage\n)\n\n# Use it like any other model\ndata = embed_anything.embed_file(\"test_files/document.pdf\", embedder=model)\n```\n\n### ColPali (Document Embedding)\n\nColPali is optimized for document and image-text embedding tasks.\n\n```python\nfrom embed_anything import ColpaliModel\nimport numpy as np\n\n# Load ColPali ONNX model\nmodel = ColpaliModel.from_pretrained_onnx(\n    \"starlight-ai/colpali-v1.2-merged-onnx\", \n    None\n)\n\n# Embed a PDF file (ColPali processes pages as images)\ndata = model.embed_file(\"test_files/document.pdf\", batch_size=1)\n\n# Query the embedded document\nquery = \"What is the main topic?\"\nquery_embedding = model.embed_query(query)\n\n# Calculate similarity scores\nfile_embeddings = np.array([e.embedding for e in data])\nquery_emb = np.array([e.embedding for e in query_embedding])\n\n# Find most relevant pages\nscores = np.einsum(\"bnd,csd-\u003ebcns\", query_emb, file_embeddings).max(axis=3).sum(axis=2).squeeze()\ntop_pages = np.argsort(scores)[::-1][:5]\n\nfor page_idx in top_pages:\n    print(f\"Page {data[page_idx].metadata['page_number']}: {data[page_idx].text[:200]}\")\n```\n\n### ColBERT (Late-Interaction Embeddings)\n\nColBERT provides token-level embeddings for fine-grained semantic matching.\n\n```python\nfrom embed_anything import ColbertModel\nimport numpy as np\n\n# Load ColBERT ONNX model\nmodel = ColbertModel.from_pretrained_onnx(\n    \"jinaai/jina-colbert-v2\", \n    path_in_repo=\"onnx/model.onnx\"\n)\n\n# Embed sentences\nsentences = [\n    \"The quick brown fox jumps over the lazy dog\", \n    \"The cat is sleeping on the mat\", \n    \"The dog is barking at the moon\", \n    \"I love pizza\", \n    \"The dog is sitting in the park\"\n]\n\n# ColBERT returns token-level embeddings\nembeddings = model.embed(sentences, batch_size=2)\n\n# Each embedding is a matrix: [num_tokens, embedding_dim]\nfor i, emb in enumerate(embeddings):\n    print(f\"Sentence {i+1}: {sentences[i]}\")\n    print(f\"Embedding shape: {emb.shape}\")  # Shape: (num_tokens, embedding_dim)\n```\n\n### ReRankers\n\nRerankers improve retrieval quality by re-scoring candidate documents.\n\n```python\nfrom embed_anything import Reranker, Dtype, RerankerResult, DocumentRank\n\n# Load a reranker model\nreranker = Reranker.from_pretrained(\n    \"jinaai/jina-reranker-v1-turbo-en\", \n    dtype=Dtype.F16\n)\n\n# Query and candidate documents\nquery = \"What is the capital of France?\"\ncandidates = [\n    \"France is a country in Europe.\", \n    \"Paris is the capital of France.\",\n    \"The Eiffel Tower is in Paris.\"\n]\n\n# Rerank documents (returns top-k results)\nresults: list[RerankerResult] = reranker.rerank(\n    [query], \n    candidates, \n    top_k=2  # Return top 2 results\n)\n\n# Access reranked results\nfor result in results:\n    documents: list[DocumentRank] = result.documents\n    for doc in documents:\n        print(f\"Score: {doc.score:.4f} | Text: {doc.text}\")\n```\n\n### Cloud Embedding Models (Cohere Embed v4)\n\nUse cloud models for high-quality embeddings without local model deployment.\n\n```python\nfrom embed_anything import EmbeddingModel, WhichModel\nimport os\n\n# Set your API key\nos.environ[\"COHERE_API_KEY\"] = \"your-api-key-here\"\n\n# Initialize the cloud model\nmodel = EmbeddingModel.from_pretrained_cloud(\n    WhichModel.CohereVision, \n    model_id=\"embed-v4.0\"\n)\n\n# Use it like any other model\ndata = embed_anything.embed_file(\"test_files/document.pdf\", embedder=model)\n```\n\n### Qwen 3 - Embedding\n\nQwen3 supports over 100 languages including various programming languages.\n\n```python\nfrom embed_anything import EmbeddingModel, WhichModel, TextEmbedConfig, Dtype\nimport numpy as np\n\n# Initialize Qwen3 embedding model\nmodel = EmbeddingModel.from_pretrained_hf(\n    WhichModel.Qwen3, \n    model_id=\"Qwen/Qwen3-Embedding-0.6B\",\n    dtype=Dtype.F32\n)\n\n# Configure embedding\nconfig = TextEmbedConfig(\n    chunk_size=1000,\n    batch_size=2,\n    splitting_strategy=\"sentence\"\n)\n\n# Embed a file\ndata = model.embed_file(\"test_files/document.pdf\", config=config)\n\n# Query embedding\nquery = \"Which GPU is used for training\"\nquery_embedding = np.array(model.embed_query([query])[0].embedding)\n\n# Calculate similarities\nembedding_array = np.array([e.embedding for e in data])\nsimilarities = np.matmul(query_embedding, embedding_array.T)\n\n# Get top results\ntop_5_indices = np.argsort(similarities)[-5:][::-1]\nfor idx in top_5_indices:\n    print(f\"Score: {similarities[idx]:.4f} | {data[idx].text[:200]}\")\n```\n\n\n## For Semantic Chunking\n\nSemantic chunking preserves meaning by splitting text at semantically meaningful boundaries rather than fixed sizes.\n\n```python\nfrom embed_anything import EmbeddingModel, WhichModel, TextEmbedConfig\nimport embed_anything\n\n# Main embedding model for generating final embeddings\nmodel = EmbeddingModel.from_pretrained_hf(\n    WhichModel.Bert, \n    model_id=\"sentence-transformers/all-MiniLM-L12-v2\"\n)\n\n# Semantic encoder for determining chunk boundaries\n# This model analyzes text to find natural semantic breaks\nsemantic_encoder = EmbeddingModel.from_pretrained_hf(\n    model_id=\"jinaai/jina-embeddings-v2-small-en\"\n)\n\n# Configure semantic chunking\nconfig = TextEmbedConfig(\n    chunk_size=1000,                    # Target chunk size\n    batch_size=32,                      # Batch processing size\n    splitting_strategy=\"semantic\",      # Use semantic splitting\n    semantic_encoder=semantic_encoder    # Model for semantic analysis\n)\n\n# Embed with semantic chunking\ndata = embed_anything.embed_file(\"test_files/document.pdf\", embedder=model, config=config)\n\n# Chunks will be split at semantically meaningful boundaries\nfor item in data:\n    print(f\"Chunk: {item.text[:200]}...\")\n    print(\"---\" * 20)\n```\n\n## For Late-Chunking\n\nLate-chunking splits text into smaller units first, then combines them during embedding for better context preservation.\n\n```python\nfrom embed_anything import EmbeddingModel, WhichModel, TextEmbedConfig, EmbedData\n\n# Load your embedding model\nmodel = EmbeddingModel.from_pretrained_hf(\n    model_id=\"sentence-transformers/all-MiniLM-L12-v2\"\n)\n\n# Configure late-chunking\nconfig = TextEmbedConfig(\n    chunk_size=1000,              # Maximum chunk size\n    batch_size=8,                 # Batch size for processing\n    splitting_strategy=\"sentence\", # Split by sentences first\n    late_chunking=True,           # Enable late-chunking\n)\n\n# Embed a file with late-chunking\ndata: list[EmbedData] = model.embed_file(\"test_files/attention.pdf\", config=config)\n\n# Late-chunking helps preserve context across sentence boundaries\nfor item in data:\n    print(f\"Text: {item.text}\")\n    print(f\"Embedding dimension: {len(item.embedding)}\")\n    print(\"---\" * 20)\n```\n\n# 🧑‍🚀 Getting Started\n\n## 💚 Installation\n\n`\npip install embed-anything\n`\u003cbr/\u003e\n\nFor GPUs and using special models like ColPali \u003cbr/\u003e\n\n`\npip install embed-anything-gpu\n`\n\n🚧❌ If it shows cuda error while running on windowns, run the following command:\n\n```\nos.add_dll_directory(\"C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.6/bin\")\n```\n## 📒 Notebooks\n\n\n|   |   \n| ------------- | \n| [End-to-End Retrieval and Reranking using VectorDB Adapters](https://colab.research.google.com/drive/1gct0lEplyW8VWGPXUgpLcQuMQeZDl6D5?usp=sharing)  | \n| [ColPali-Onnx](https://colab.research.google.com/drive/1yCVbpkoe53ymiCxG8ttJNbRhECy1Q-Du?usp=sharing)  | \n| [Adapters](https://github.com/StarlightSearch/EmbedAnything/tree/main/examples/adapters) |  |\n| [Qwen3- Embedings](https://colab.research.google.com/drive/1OlUJwTtPvj28h5tCVerf6ebEnAf8kPAh?usp=sharing) | \n| [Benchmarks](https://colab.research.google.com/drive/1nXvd25hDYO-j7QGOIIC0M7MDpovuPCaD?usp=sharing) | \n\n\n# Usage\n\n## ➡️ Usage For 0.3 and later version\n\n### Basic Text Embedding\n\n```python\nfrom embed_anything import EmbeddingModel, WhichModel, TextEmbedConfig\nimport embed_anything\n\n# Load a model from Hugging Face\nmodel = EmbeddingModel.from_pretrained_local(\n    WhichModel.Bert, \n    model_id=\"sentence-transformers/all-MiniLM-L12-v2\"\n)\n\n# Simple file embedding with default config\ndata = embed_anything.embed_file(\"test_files/test.pdf\", embedder=model)\n\n# Access results\nfor item in data:\n    print(f\"Text chunk: {item.text[:100]}...\")\n    print(f\"Embedding shape: {len(item.embedding)}\")\n```\n\n### Advanced Usage with Configuration\n\n```python\nfrom embed_anything import EmbeddingModel, WhichModel, TextEmbedConfig\nimport embed_anything\n\n# Load model\nmodel = EmbeddingModel.from_pretrained_hf(\n    model_id=\"jinaai/jina-embeddings-v2-small-en\"\n)\n\n# Configure embedding parameters\nconfig = TextEmbedConfig(\n    chunk_size=1000,              # Characters per chunk\n    batch_size=32,                # Process 32 chunks at once\n    buffer_size=64,               # Buffer size for streaming\n    splitting_strategy=\"sentence\" # Split by sentences\n)\n\n# Embed with custom configuration\ndata = embed_anything.embed_file(\n    \"test_files/document.pdf\", \n    embedder=model, \n    config=config\n)\n\n# Process embeddings\nfor item in data:\n    print(f\"Chunk: {item.text}\")\n    print(f\"Metadata: {item.metadata}\")\n```\n\n### Embedding Queries\n\n```python\nfrom embed_anything import EmbeddingModel, WhichModel\nimport embed_anything\nimport numpy as np\n\n# Load model\nmodel = EmbeddingModel.from_pretrained_hf(\n    model_id=\"sentence-transformers/all-MiniLM-L12-v2\"\n)\n\n# Embed a query\nqueries = [\"What is machine learning?\", \"How does neural networks work?\"]\nquery_embeddings = embed_anything.embed_query(queries, embedder=model)\n\n# Use embeddings for similarity search\nfor i, query_emb in enumerate(query_embeddings):\n    print(f\"Query: {queries[i]}\")\n    print(f\"Embedding shape: {len(query_emb.embedding)}\")\n```\n\n### Embedding Directories\n\n```python\nfrom embed_anything import EmbeddingModel, WhichModel, TextEmbedConfig\nimport embed_anything\n\n# Load model\nmodel = EmbeddingModel.from_pretrained_hf(\n    model_id=\"sentence-transformers/all-MiniLM-L12-v2\"\n)\n\n# Configure\nconfig = TextEmbedConfig(chunk_size=1000, batch_size=32)\n\n# Embed all files in a directory\ndata = embed_anything.embed_directory(\n    \"test_files/\", \n    embedder=model, \n    config=config\n)\n\nprint(f\"Total chunks: {len(data)}\")\n```\n\n\n\n### Using ONNX Models\n\nONNX models provide faster inference and lower memory usage. You can use pre-configured models via the `ONNXModel` enum or load custom ONNX models.\n\n#### Using Pre-configured ONNX Models (Recommended)\n\n```python\nfrom embed_anything import EmbeddingModel, WhichModel, ONNXModel, Dtype, TextEmbedConfig\nimport embed_anything\n\n# Use a pre-configured ONNX model (tested and optimized)\nmodel = EmbeddingModel.from_pretrained_onnx(\n    WhichModel.Bert,\n    model_id=ONNXModel.BGESmallENV15Q,  # Quantized BGE model\n    dtype=Dtype.Q4F16                    # Quantized 4-bit float16\n)\n\n# Embed files\nconfig = TextEmbedConfig(chunk_size=1000, batch_size=32)\ndata = embed_anything.embed_file(\"test_files/document.pdf\", embedder=model, config=config)\n```\n\n#### Using Custom ONNX Models\n\nFor custom or fine-tuned models, specify the Hugging Face model ID and path to the ONNX file:\n\n```python\nfrom embed_anything import EmbeddingModel, WhichModel, Dtype\n\n# Load a custom ONNX model from Hugging Face\nmodel = EmbeddingModel.from_pretrained_onnx(\n    WhichModel.Jina,\n    hf_model_id=\"jinaai/jina-embeddings-v2-small-en\",\n    path_in_repo=\"model.onnx\",  # Path to ONNX file in the repo\n    dtype=Dtype.F16              # Use half precision\n)\n\n# Use the model\ndata = embed_anything.embed_file(\"test_files/document.pdf\", embedder=model)\n```\n\n**Note**: Using pre-configured models (via `ONNXModel` enum) is recommended as these models are tested and optimized. For a complete list of supported ONNX models, see [ONNX Models Guide](/docs/guides/onnx_models.md).\n\n## ⁉️FAQ\n\n### Do I need to know rust to use or contribute to embedanything?\nThe answer is No. EmbedAnything provides you pyo3 bindings, so you can run any function in python without any issues. To contibute you should check out our guidelines and python folder example of adapters.\n\n### How is it different from fastembed?\n\nWe provide both backends, candle and onnx. On top of it we also give an end-to-end pipeline, that is you can ingest different data-types and index to any vector database, and inference any model. Fastembed is just an onnx-wrapper.\n\n### We've received quite a few questions about why we're using Candle.\n\nOne of the main reasons is that Candle doesn't require any specific ONNX format models, which means it can work seamlessly with any Hugging Face model. This flexibility has been a key factor for us. However, we also recognize that we’ve been compromising a bit on speed in favor of that flexibility.\n\n\n## 🚧 Contributing to EmbedAnything\n\nFirst of all, thank you for taking the time to contribute to this project. We truly appreciate your contributions, whether it's bug reports, feature suggestions, or pull requests. Your time and effort are highly valued in this project. 🚀\n\nThis document provides guidelines and best practices to help you to contribute effectively. These are meant to serve as guidelines, not strict rules. We encourage you to use your best judgment and feel comfortable proposing changes to this document through a pull request.\n\n\n\n\u003cli\u003e\u003ca href=\"##-RoadMap\"\u003eRoadmap\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"##-Quick-Start\"\u003eQuick Start\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"##-Contributing-Guidelines\"\u003eGuidelines\u003c/a\u003e\u003c/li\u003e\n\n\n# 🏎️ RoadMap \n\n## Accomplishments\n\nOne of the aims of EmbedAnything is to allow AI engineers to easily use state of the art embedding models on typical files and documents. A lot has already been accomplished here and these are the formats that we support right now and a few more have to be done. \u003cbr /\u003e\n\n\n### 🖼️ Modalities and Source\n\nWe’re excited to share that we've expanded our platform to support multiple modalities, including:\n\n- [x] Audio files\n\n- [x] Markdowns\n\n- [x] Websites\n\n- [x] Images\n\n- [ ] Videos\n\n- [ ] Graph\n\nThis gives you the flexibility to work with various data types all in one place! 🌐 \u003cbr /\u003e\n\n\n\n### ⚙️ Performance \n\n\nWe now support both candle and Onnx backend\u003cbr/\u003e\n➡️ Support for GGUF models \u003c/br \u003e\n\n\n### 🫐Embeddings:\n\nWe had multimodality from day one for our infrastructure. We have already included it for websites, images and audios but we want to expand it further to.\n\n➡️ Graph embedding -- build deepwalks embeddings depth first and word to vec \u003cbr /\u003e\n➡️ Video Embedding \u003cbr/\u003e\n➡️ Yolo Clip \u003cbr/\u003e\n\n\n### 🌊Expansion to other Vector Adapters\n\nWe currently support a wide range of vector databases for streaming embeddings, including:\n\n- Elastic: thanks to amazing and active Elastic team for the contribution \u003cbr/\u003e\n- Weaviate \u003cbr/\u003e\n- Pinecone \u003cbr/\u003e\n- Qdrant \u003cbr/\u003e\n- Milvus\u003cbr/\u003e\n- Chroma \u003cbr/\u003e\n\nHow to add an adpters: https://starlight-search.com/blog/2024/02/25/adapter-development-guide.md\n\n### 💥 Create WASM demos to integrate embedanything directly to the browser. \u003cbr/\u003e\n\n### 💜 Add support for ingestion from remote sources\n➡️ Support for S3 bucket \u003c/br \u003e\n➡️ Support for azure storage \u003c/br \u003e\n➡️ Support for google drive/dropbox\u003c/br \u003e\n\n\n\n\nBut we're not stopping there! We're actively working to expand this list.\n\nWant to Contribute?\nIf you’d like to add support for your favorite vector database, we’d love to have your help! Check out our contribution.md for guidelines, or feel free to reach out directly turingatverge@gmail.com . Let's build something amazing together! 💡\n\n## AWESOME Projects built on EmbedAnything.\n1. A Rust-based cursor like chat with your codebase tool: https://github.com/timpratim/cargo-chat\n2. A simple vector-based search engine, also supports ordinary text search : https://github.com/szuwgh/vectorbase2\n3. Semantic file tracker in CLI operated through daemon built with rust.: https://github.com/sam-salehi/sophist\n4. FogX-Store is a dataset store service that collects and serves large robotics datasets : https://github.com/J-HowHuang/FogX-Store\n5. A Dart Wrapper for EmbedAnything Crate: https://github.com/cotw-fabier/embedanythingindart\n6. Generate embeddings in Rust with tauri on MacOS : https://github.com/do-me/tauri-embedanything-ios\n7. RAG with EmbedAnything and Milvus: https://milvus.io/docs/v2.5.x/build_RAG_with_milvus_and_embedAnything.md\n\n\n\n\n## A big Thank you to all our StarGazers\n\n## Star History\n\n[![Star History Chart](https://api.star-history.com/svg?repos=StarlightSearch/EmbedAnything\u0026type=Date)](https://star-history.com/#StarlightSearch/EmbedAnything\u0026Date)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstarlightsearch%2Fembedanything","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fstarlightsearch%2Fembedanything","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstarlightsearch%2Fembedanything/lists"}