{"id":13455942,"url":"https://github.com/superagent-ai/super-rag","last_synced_at":"2025-04-04T20:11:07.575Z","repository":{"id":219331436,"uuid":"742949598","full_name":"superagent-ai/super-rag","owner":"superagent-ai","description":"Super performant RAG pipelines for AI apps. Summarization, Retrieve/Rerank and Code Interpreters in one simple API.","archived":false,"fork":false,"pushed_at":"2024-04-30T08:51:06.000Z","size":652,"stargazers_count":362,"open_issues_count":20,"forks_count":56,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-03-29T05:42:48.542Z","etag":null,"topics":["agents","ai","embeddings","inference","rag","vector-database"],"latest_commit_sha":null,"homepage":"https://docs.superagent.sh","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/superagent-ai.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-01-13T21:06:28.000Z","updated_at":"2025-03-26T16:50:13.000Z","dependencies_parsed_at":"2024-02-01T22:57:57.577Z","dependency_job_id":"fed72a39-a9df-4475-9e32-41a9d00bcf9e","html_url":"https://github.com/superagent-ai/super-rag","commit_stats":null,"previous_names":["superagent-ai/super-rag"],"tags_count":13,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/superagent-ai%2Fsuper-rag","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/superagent-ai%2Fsuper-rag/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/superagent-ai%2Fsuper-rag/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/superagent-ai%2Fsuper-rag/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/superagent-ai","download_url":"https://codeload.github.com/superagent-ai/super-rag/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247242678,"owners_count":20907134,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["agents","ai","embeddings","inference","rag","vector-database"],"created_at":"2024-07-31T08:01:13.788Z","updated_at":"2025-04-04T20:11:07.560Z","avatar_url":"https://github.com/superagent-ai.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n\t\u003cimg width=\"100px\" src=\"https://github.com/homanp/superagent/assets/2464556/eb51fa38-4a2a-4c41-b348-d3c1abc04234\" /\u003e\n\t\u003ch1\u003eSuper-Rag\u003c/h1\u003e\n\t\u003cp\u003e\n\t\t\u003cb\u003eSuper-performant RAG pipeline for AI apps.\u003c/b\u003e\n\t\u003c/p\u003e\n\t\u003cbr\u003e\n    \u003cp align=\"center\"\u003e\n        \u003ca href=\"#-key-features\"\u003eFeatures\u003c/a\u003e •\n        \u003ca href=\"#-installation\"\u003eInstallation\u003c/a\u003e •\n        \u003ca href=\"#-how-to-use\"\u003eHow to use\u003c/a\u003e •\n        \u003ca href=\"#-cloud-api\"\u003eCloud API\u003c/a\u003e\n    \u003c/p\u003e\n\u003c/div\u003e\n\n\n## ✅ Key features\n - Supports multiple document formats and vector databases.\n - Provides a production ready REST API.\n - Customizable splitting/chunking.\n - Includes options for encoding data using different encoding models both propriatory and open source.\n - Built in code interpreter mode for computational question \u0026 answer scenarios.\n - Allows session management through unique IDs for caching purposes.\n\n## ☁️ Cloud API \n\nEasiest way to get started is to use our [Cloud API](https://d3jvqvcd9u4534.cloudfront.net). This API is free to use (within reasonable limits).\n\n## 📦  Installation\n\n1. Clone the repository\n    ```bash\n    git clone https://github.com/superagent-ai/super-rag \n    cd super-rag \n    ```\n\n2. Setup virtual environment\n    ```bash\n    # Using virtualenv \n    virtualenv env \n    source env/bin/activate \n    \n    # Or using venv \n    python3 -m venv env \n    source env/bin/activate \n    ```\n\n3. Install requried packages\n    ```bash\n    poetry install\n    ```\n\n4. Rename `.env.example` to `.env` and set your environment variables\n\n5. Run server\n    ```bash\n    uvicorn main:app --reload\n    ```\n## 🤖 Interpreter mode\nSuper-Rag has built in support for running computational Q\u0026A using code interpreters powered by [E2B.dev](https://e2b.dev) custom runtimes. You can signup to receive an API key to leverage they sandboxes in a cloud environment or setup your own by following [these instructions](https://github.com/e2b-dev/infra). \n\n## 🚀 How to use \nSuper-Rag comes with a built in REST API powered by FastApi. \n\n### Ingest documents\n```json\n// POST: /api/v1/ingest\n\n// Payload\n{\n    \"files\": [\n        {\n            \"name\": \"My file\", // Optional\n            \"url\": \"https://path-to-my-file.pdf\",\n            \"metadata\": { // Optional\n                \"key\": \"value\"\n            }\n        }\n    ],\n    \"document_processor\": { // Optional\n        \"encoder\": {\n            \"dimensions\": 384,\n            \"model_name\": \"embed-multilingual-light-v3.0\",\n            \"provider\": \"cohere\"\n        },\n        \"unstructured\": {\n            \"hi_res_model_name\": \"detectron2_onnx\",\n            \"partition_strategy\": \"auto\",\n            \"process_tables\": false\n        },\n        \"splitter\": {\n            \"max_tokens\": 400,\n            \"min_tokens\": 30,\n            \"name\": \"semantic\",\n            \"prefix_summary\": true,\n            \"prefix_title\": true,\n            \"rolling_window_size\": 1\n        }\n    },\n    \"vector_database\": {\n        \"type\": \"qdrant\",\n        \"config\": {\n            \"api_key\": \"YOUR API KEY\",\n            \"host\": \"THE QDRANT HOST\"\n        }\n    },\n    \"index_name\": \"my_index\",\n    \"webhook_url\": \"https://my-webhook-url\"\n}\n```\n\n### Query documents\n```json\n// POST: /api/v1/query\n\n// Payload\n{\n    \"input\": \"What is ReAct\",\n    \"vector_database\": {\n            \"type\": \"qdrant\",\n            \"config\": {\n            \"api_key\": \"YOUR API KEY\",\n            \"host\": \"THE QDRANT HOST\"\n        }\n        },\n    \"index_name\": \"YOUR INDEX\",\n    \"interpreter_mode\": true,\n    \"encoder\": {\n        \"provider\": \"cohere\",\n        \"name\": \"embed-multilingual-light-v3.0\",\n        \"dimensions\": 384\n    },\n    \"filter\": { // Optional\n        // Each provider has its own filter schema,\n        // for Qdrant it looks like the following:\n        \"must\": [\n            {\n                \"key\": \"key\",\n                \"match\": {\n                    \"value\": \"value\"\n                }\n            }\n        ]\n    },\n    \"exclude_fields\": [\"metadata\"], // Exclude specific fields\n    \"interpreter_mode\": False, // Set to True if you wish to run computation Q\u0026A with a code interpreter\n    \"session_id\": \"my_session_id\" // keeps micro-vm sessions and enables caching \n}\n```\n\n### Delete document\n```json\n// POST: /api/v1/delete\n\n// Payload\n{\n    \"file_url\": \"A file url to delete\",\n    \"vector_database\": {\n        \"type\": \"qdrant\",\n        \"config\": {\n            \"api_key\": \"YOUR API KEY\",\n            \"host\": \"THE QDRANT HOST\"\n        }\n    },\n    \"index_name\": \"my_index\",\n}\n\n```\n\n## 🧠 Supportd encoders\n- OpenAi\n- Cohere\n- HuggingFace\n- FastEmbed\n- Mistral (coming soon)\n- Anthropic (coming soon)\n\n\n## 🗃 Supported vector databases\n- Pinecone\n- Qdrant\n- Weaviate\n- Astra\n- PGVector\n- Chroma (coming soon)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsuperagent-ai%2Fsuper-rag","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsuperagent-ai%2Fsuper-rag","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsuperagent-ai%2Fsuper-rag/lists"}