https://github.com/ksm26/prompt-compression-and-query-optimization
Enhance the performance and cost-efficiency of large-scale Retrieval Augmented Generation (RAG) applications. Learn to integrate vector search with traditional database operations and apply techniques like prefiltering, postfiltering, projection, and prompt compression.
https://github.com/ksm26/prompt-compression-and-query-optimization
cost-efficiency data-retrieval data-retrieval-and-display data-security database-operations developer-advocacy large-scale-applications mongodb performance-optimization postfiltering prefiltering projection prompt-compression query-optimization query-processing rag-applications reranking search-relevance vector-search vector-search-engine
Last synced: about 2 months ago
JSON representation
Enhance the performance and cost-efficiency of large-scale Retrieval Augmented Generation (RAG) applications. Learn to integrate vector search with traditional database operations and apply techniques like prefiltering, postfiltering, projection, and prompt compression.
- Host: GitHub
- URL: https://github.com/ksm26/prompt-compression-and-query-optimization
- Owner: ksm26
- Created: 2024-07-16T15:52:18.000Z (10 months ago)
- Default Branch: main
- Last Pushed: 2024-07-23T08:45:53.000Z (10 months ago)
- Last Synced: 2024-08-24T02:33:59.797Z (9 months ago)
- Topics: cost-efficiency, data-retrieval, data-retrieval-and-display, data-security, database-operations, developer-advocacy, large-scale-applications, mongodb, performance-optimization, postfiltering, prefiltering, projection, prompt-compression, query-optimization, query-processing, rag-applications, reranking, search-relevance, vector-search, vector-search-engine
- Language: Jupyter Notebook
- Homepage: https://www.deeplearning.ai/short-courses/prompt-compression-and-query-optimization/
- Size: 88.9 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# 📊 [Prompt Compression and Query Optimization](https://www.deeplearning.ai/short-courses/prompt-compression-and-query-optimization/)
🔍 Welcome to the "Prompt Compression and Query Optimization" course! Course will equip you with the skills to optimize the performance and cost-efficiency of large-scale Retrieval Augmented Generation (RAG) applications by integrating traditional database features with vector search capabilities.
## Course Summary
In this course, you'll learn to optimize large-scale RAG applications by integrating vector search capabilities with traditional database operations. Here’s what you can expect to learn and experience:
1. 📋 **Prefiltering and Postfiltering**: Filter results based on specific conditions. Prefiltering is done at the database index creation stage, while postfiltering is applied after the vector search is performed.
2. 📊 **Projection**: Select a subset of fields returned from a query to minimize the size of the output, enhancing performance and security.
3. 🔄 **Reranking**: Reorder search results based on other data fields to improve the relevance and quality of information retrieval.
4. ✂️ **Prompt Compression**: Reduce the length of prompts, which can be expensive to process in large-scale applications, optimizing both performance and cost.
## Key Points
- 🌐 **Vector Search and Database Operations**: Combine vector search capabilities with traditional database operations to build efficient and cost-effective RAG applications.
- 🚀 **Optimized Query Processing**: Use prefiltering, postfiltering, and projection techniques for faster query processing and optimized query output.
- 💡 **Prompt Compression**: Implement prompt compression techniques to reduce the length of prompts, making them more efficient to process in large-scale applications.
## About the Instructor
🌟 **Richmond Alake** is a Developer Advocate at MongoDB, bringing extensive expertise in database optimization and vector search capabilities to guide you through this course.
🔗 To enroll in the course or for further information, visit [deeplearning.ai](https://www.deeplearning.ai/short-courses/).