{"id":26051488,"url":"https://github.com/timerring/rag101","last_synced_at":"2026-04-10T23:09:27.891Z","repository":{"id":280666547,"uuid":"942758646","full_name":"timerring/rag101","owner":"timerring","description":"LangChain and RAG best practices","archived":false,"fork":false,"pushed_at":"2025-03-04T16:47:54.000Z","size":3936,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-04T17:36:13.768Z","etag":null,"topics":["101","chat","langchain","rag"],"latest_commit_sha":null,"homepage":"https://blog.timerring.com/posts/langchain-and-rag-best-practices/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/timerring.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-03-04T16:19:02.000Z","updated_at":"2025-03-04T16:51:38.000Z","dependencies_parsed_at":"2025-03-04T17:46:48.239Z","dependency_job_id":null,"html_url":"https://github.com/timerring/rag101","commit_stats":null,"previous_names":["timerring/rag101"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/timerring%2Frag101","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/timerring%2Frag101/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/timerring%2Frag101/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/timerring%2Frag101/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/timerring","download_url":"https://codeload.github.com/timerring/rag101/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":242501078,"owners_count":20139320,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["101","chat","langchain","rag"],"created_at":"2025-03-08T04:55:20.669Z","updated_at":"2026-04-10T23:09:27.856Z","avatar_url":"https://github.com/timerring.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n# LangChain and RAG best practices\n\n## Introduction\n\nThis is a quick start guide essay for LangChain and RAG which mainly refers to the [Langchain chat with your data](https://learn.deeplearning.ai/courses/langchain-chat-with-your-data/lesson/snupv/introduction?courseName=langchain-chat-with-your-data) course.\n\nYou can check the entire code in the [rag101 repository](https://github.com/timerring/rag101/).\n\n### LangChain\n\nLangChain is an Open-source developer framework for building LLM applications.\n\nIt components are as below:\n\n#### Prompt\n\n- Prompt Templates: used for generating model input.\n- Output Parsers: implementations for processing generated results.\n- Example Selectors: selecting appropriate input examples.\n\n#### Models\n\n- LLMs\n- Chat Models\n- Text Embedding Models\n\n#### Indexes\n\n- Document Loaders\n- Text Splitters\n- Vector Stores\n- Retrievers\n\n#### Chains\n\n- Can be used as a building block for other chains.\n- Provides over 20 types of application-specific chains.\n\n#### Agents\n\n- Supports 5 types of agents to help language models use external tools.\n- Agent Toolkits: provides over 10 implementations, agents execute tasks through specific tools.\n\n### RAG process\n\nThe whole RAG process lays on the Vector Store Loading and Retrieval-Augmented Generation.\n\n#### Vector Store Loading\n\nLoad the data from different sources, split and convert them into vector embeddings.\n\n\n#### Retrieval-Augmented Generation\n\n1. After the user's input **Query**, the system will retrieve the most relevant document fragments (Relevant Splits) from the vector store.\n2. The retrieved relevant fragments will be combined into a **Prompt**, which will be passed along with the context to the large language model (LLM).\n3. Finally, the language model will generate an answer based on the retrieved fragments and return it to the user.\n\n![](https://cdn.jsdelivr.net/gh/timerring/scratchpad2023/2024/2025-03-03-20-33-44.png)\n\n## Loaders\n\nYou can use loaders to deal with different kind and format of data.\n\nSome are public and some are proprietary. Some are structured and some are not.\n\n![](https://cdn.jsdelivr.net/gh/timerring/scratchpad2023/2024/2025-03-03-20-37-44.png)\n\nSome useful lib:\n\n- pdf: pypdf\n- youtube audio: yt_dlp pydub\n- web page: beautifulsoup4\n\nFor more loaders, you can check the [official docs](https://python.langchain.com/api_reference/community/document_loaders.html#module-langchain_community.document_loaders).\n\nYou can check the entire code [here](https://github.com/timerring/rag101/tree/main/loader).\n\n### PDF\n\nNow, we can practice:\n\nFirst, install the lib:\n\n```bash\npip install langchain-community \npip install pypdf\n```\nYou can check the demo in the \n\n```python\nfrom langchain.document_loaders import PyPDFLoader\n\n# In fact, the langchain calls the pypdf lib to load the pdf file\nloader = PyPDFLoader(\"ProbRandProc_Notes2004_JWBerkeley.pdf\")\npages = loader.load()\n\nprint(type(pages))\n# \u003cclass 'list'\u003e\nprint(len(pages))\n# Print the total num of pages\n\n# Using the first page as an example\npage = pages[0]\nprint(type(page))\n# \u003cclass 'langchain_core.documents.base.Document'\u003e\n\n# What is inside the page:\n# 1. page_content\n# 2. meta_data: the description of the page\n\nprint(page.page_content[0:500])\nprint(page.metadata)\n```\n\n### Web Base Loader\n\nAlso we install the lib first:\n\n```bash\npip install beautifulsoup4\n```\n\nThe WebBaseLoader is based on the beautifulsoup4 lib.\n\n```python\nfrom langchain_community.document_loaders import WebBaseLoader\n\nloader = WebBaseLoader(\"https://zh.d2l.ai/\")\npages = loader.load()\nprint(pages[0].page_content[:500])\n\n# You can also use json as the post processing\n# import json\n# convert_to_json = json.loads(pages[0].page_content)\n```\n\n## Splitters\n\nSplitting Documents into smaller chunks. Retaining the meaningful relationships.\n\n### Why split?\n\n- The limitation of GPU: the GPT model with more than 1B parameters. The forward propagation cannot process such a large parameters. So the split is necessary.\n- More efficient computation.\n- Some fixed size of sequence.\n- Better generalization.\n\n\u003e However, the split points may lose some information. So we split should consider the semantic.\n\n### Type of splitters\n\n- CharacterTextSplitter\n- MarkdownHeaderTextSplitter\n- TokenTextsplitter\n- SentenceTransformersTokenTextSplitter\n- **RecursiveCharacterTextSplitter**: Recursively tries to split by different characters to find one that works.\n- Language: for CPP, Python, Ruby, Markdown etc \n- NLTKTextSplitter: sentences using NLTK(Natural Language Tool Kit)\n- SpacyTextSplitter: sentences using Spacy\n\nFor more, check the [docs](https://python.langchain.com/api_reference/text_splitters/index.html#module-langchain_text_splitters).\n\n### Example CharacterTextSplitter and RecursiveCharacterTextSplitter\n\nYou can check the entire code [here](https://github.com/timerring/rag101/blob/main/splitter/text_splitter.py).\n\n```python\nfrom langchain.text_splitter import RecursiveCharacterTextSplitter, CharacterTextSplitter\n\nexample_text = \"\"\"When writing documents, writers will use document structure to group content. \\\nThis can convey to the reader, which idea's are related. For example, closely related ideas \\\nare in sentances. Similar ideas are in paragraphs. Paragraphs form a document. \\n\\n  \\\nParagraphs are often delimited with a carriage return or two carriage returns. \\\nCarriage returns are the \"backslash n\" you see embedded in this string. \\\nSentences have a period at the end, but also, have a space.\\\nand words are separated by space.\"\"\"\n\nc_splitter = CharacterTextSplitter(\n    chunk_size=450, # the size of the chunk\n    chunk_overlap=0, # the overlap of the chunk, which can be shared with the previous chunk\n    separator = ' '\n)\nr_splitter = RecursiveCharacterTextSplitter(\n    chunk_size=450,\n    chunk_overlap=0, \n    separators=[\"\\n\\n\", \"\\n\", \" \", \"\"] # priority of the separators\n)\n\nprint(c_splitter.split_text(example_text))\n# split at 450 characters\nprint(r_splitter.split_text(example_text))\n# split at first \\n\\n\n```\n\n## Vectorstores and Embeddings\n\nReview the RAG process:\n\n![](https://cdn.jsdelivr.net/gh/timerring/scratchpad2023/2024/2025-03-03-22-30-38.png)\n\nBenefits:\n\n1. Improve the accuracy of the query. When query the similar chunks, the accuracy will be higher.\n2. Improve the efficiency of the query. Minimize the computation when query the similar chunks.\n3. Improve the coverage of the query. The chunks can cover every point of the document.\n4. Facilitate the Embeddings.\n\n### Embeddings\n\n![](https://cdn.jsdelivr.net/gh/timerring/scratchpad2023/2024/2025-03-03-22-35-38.png)\n![](https://cdn.jsdelivr.net/gh/timerring/scratchpad2023/2024/2025-03-03-22-36-48.png)\n\nIf two sentences have similar meanings, then they will be closer in the high-dimensional semantic space.\n\n### Vector Stores\n\nStore every chunk in a vector store. When customer query, the query will be embedded and then find the most similar vectors which means the index of these chunks, and then return the chunks.\n\n![](https://cdn.jsdelivr.net/gh/timerring/scratchpad2023/2024/2025-03-03-22-37-35.png)\n![](https://cdn.jsdelivr.net/gh/timerring/scratchpad2023/2024/2025-03-03-22-41-22.png)\n\n### Practice\n\n#### Embeddings\n\nYou can check the entire code [here](https://github.com/timerring/rag101/blob/main/embeddings/zhipu.py).\n\nFirst, install the lib:\n\nThe `chromadb` is a lightweight vector database.\n\n```bash\npip install chromadb\n```\n\nWhat we need is a good embedding model, you can select what you like. Refer to the [docs](https://python.langchain.com/api_reference/community/embeddings.html#module-langchain_community.embeddings).\n\nHere I use the `ZhipuAIEmbeddings`. So you should install the lib:\n\n```bash\npip install zhipuai\n```\n\nHere is the test code:\n\n```python\nfrom langchain_community.embeddings import ZhipuAIEmbeddings\n\nembed = ZhipuAIEmbeddings(\n    model=\"embedding-3\",\n    api_key=\"Entry your own api key\"\n)\n\ninput_texts = [\"This is a test query1.\", \"This is a test query2.\"]\nprint(embed.embed_documents(input_texts))\n```\n\n#### Vector Stores\n\nYou can check the entire code [here](https://github.com/timerring/rag101/blob/main/vectorstores/chroma.py).\n\n```bash\npip install langchain-chroma\n```\n\nThen we can use the `Chroma` to store the embeddings.\n\n```python\nfrom langchain_chroma import Chroma\nfrom langchain_community.document_loaders import WebBaseLoader\nfrom langchain.text_splitter import RecursiveCharacterTextSplitter\nfrom langchain_community.embeddings import ZhipuAIEmbeddings\n\n# load the web page\nloader = WebBaseLoader(\"https://en.d2l.ai/\")\ndocs = loader.load()\n\n# split the text into chunks\ntext_splitter = RecursiveCharacterTextSplitter(\n    chunk_size = 1500,\n    chunk_overlap = 150\n)\nsplits = text_splitter.split_documents(docs)\n# print(len(splits))\n\n# set the embeddings models\nembeddings = ZhipuAIEmbeddings(\n    model=\"embedding-3\",\n    api_key=\"your own api key\"\n)\n\n# set the persist directory\npersist_directory = r'.'\n\n# create the vector database\nvectordb = Chroma.from_documents(\n    documents=splits,\n    embedding=embeddings,\n    persist_directory=persist_directory\n)\n# print(vectordb._collection.count())\n\n# query the vector database\nquestion = \"Recurrent\"\ndocs = vectordb.similarity_search(question, k=3)\n# print(len(docs))\nprint(docs[0].page_content)\n```\n\nThen you can find the `chorma.sqlite3` file in the specific directory.\n\n## Retrieval\n\nThis part is the core part of the RAG.\n\n![](https://cdn.jsdelivr.net/gh/timerring/scratchpad2023/2024/2025-03-04-20-58-14.png)\n\nLast part we have already used the `similarity_search` method. On top of that, we also have other methods.\n\n- Basic semantic similarity\n- Maximum Marginal Relevance(MMR)\n- Metadata\n- LLM Aided Retrieval\n\n\n### Similarity Search\n\nSimilarity Search calculates the similarity between the query vector and all document vectors in the database to find the most relevant document. \n\nThe similarity measurement methods include **cosine similarity** and **Euclidean distance**, which can effectively measure the closeness of two vectors in a high-dimensional space.\n\nHowever, relying solely on similarity search may result in insufficient diversity, as it only focuses on the match between the query and the content, ignoring the differences between different pieces of information. In some applications, especially when it is necessary to cover **multiple different aspects of information**, the extended method of Maximum Marginal Relevance (MMR) can better balance relevance and diversity.\n\n#### Practice\n\nThe practice part is on the pervious part.\n\n### Maximum Marginal Relevance (MMR)\n\nRetrieving only the most relevant documents may overlook the diversity of information. For example, if only the most similar response is selected, the **results may be very similar or even contain duplicate content**. The core idea of MMR is to balance relevance and diversity, that is, to select the information most relevant to the query while ensuring that the information is diverse in content. **By reducing the repetition of information between different pieces**, MMR can provide a more comprehensive and diverse set of results.\n\n![](https://cdn.jsdelivr.net/gh/timerring/scratchpad2023/2024/2025-03-04-21-14-54.png)\n\nThe process of MMR is as follows:\n\n![](https://cdn.jsdelivr.net/gh/timerring/scratchpad2023/2024/2025-03-04-21-21-02.png)\n\n1. Query the Vector Store: First convert the query into vectors using the embedding model.\n2. Choose the `fetch_k` most similar responses. Find the top `k` most similar vectors from the vector store.\n3. Within those responses choose the `k` most diverse. By calculating the similarity between each response, MMR will prefer results that are **more different from each other**, thus increasing the coverage of information. This process ensures that the returned results are not only \"most similar\", but also \"complementary\".\n\nThe key parameter is the `lambda` which is the weight of the relevance and diversity.\n\n- When lambda is close to 1, MMR will be more like the similarity search.\n- When lambda is close to 0, MMR will be more like the random search.\n\n#### Practice\n\nWe can adjust the code in `Vector stores` part to use the MMR method. The full code is in the [`retrieval/mmr.py` file](https://github.com/timerring/rag101/blob/main/retrieval/mmr.py).\n\n```python\n# query the vector database with MMR\nquestion = \"How the neural network works?\"\n# fetch the 8 most similar documents, and then choose the 2 most relevant documents\ndocs_mmr = vectordb.max_marginal_relevance_search(question, fetch_k=8, k=2)\nprint(docs_mmr[0].page_content[:100])\nprint(docs_mmr[1].page_content[:100])\n```\n\n### Metadata\n\nWhen our query is under some specific conditions, we can use the metadata to filter the results. \n\nFor example, the information such as page numbers, authors, timestamps, etc. These information can be used as filtering conditions during retrieval, thus improving the accuracy of the query.\n\n#### Practice\n\nYou can check the entire code [here](https://github.com/timerring/rag101/blob/main/retrieval/metadata.py).\n\nAdd new documents from another website, and then filter the results from the specific website.\n\n```python\nfrom langchain_chroma import Chroma\nfrom langchain_community.document_loaders import WebBaseLoader\nfrom langchain.text_splitter import RecursiveCharacterTextSplitter\nfrom langchain_community.embeddings import ZhipuAIEmbeddings\n\n# load the web page\nloader = WebBaseLoader(\"https://en.d2l.ai/\")\ndocs = loader.load()\n\n# split the text into chunks\ntext_splitter = RecursiveCharacterTextSplitter(\n    chunk_size = 1500,\n    chunk_overlap = 150\n)\nsplits = text_splitter.split_documents(docs)\n# print(len(splits))\n\n# set the embeddings models\nembeddings = ZhipuAIEmbeddings(\n    model=\"embedding-3\",\n    api_key=\"your_api_key\"\n)\n\n# set the persist directory\npersist_directory = r'.'\n\n# create the vector database\nvectordb = Chroma.from_documents(\n    documents=splits,\n    embedding=embeddings,\n    persist_directory=persist_directory\n)\n# print(vectordb._collection.count())\n\n# add new documents from another website\nnew_loader = WebBaseLoader(\"https://www.deeplearning.ai/\")\nnew_docs = new_loader.load()\n\n# split the text into chunks\nnew_splits = text_splitter.split_documents(new_docs)\n\n# add to the existing vector database\nvectordb.add_documents(new_splits)\n\n# Get all documents\nall_docs = vectordb.similarity_search(\"What is the difference between a neural network and a deep learning model?\", k=20)\n\n# Print the metadata of the documents\nfor i, doc in enumerate(all_docs):\n    print(f\"Document {i+1} metadata: {doc.metadata}\")\n# Document 1 metadata: {'language': 'en', 'source': 'https://en.d2l.ai/', 'title': 'Dive into Deep Learning — Dive into Deep Learning 1.0.3 documentation'}\n# Document 2 metadata: {'language': 'en', 'source': 'https://en.d2l.ai/', 'title': 'Dive into Deep Learning — Dive into Deep Learning 1.0.3 documentation'}\n# Document 3 metadata: {'language': 'en', 'source': 'https://en.d2l.ai/', 'title': 'Dive into Deep Learning — Dive into Deep Learning 1.0.3 documentation'}\n# Document 4 metadata: {'description': 'DeepLearning.AI | Andrew Ng | Join over 7 million people learning how to use and build AI through our online courses. Earn certifications, level up your skills, and stay ahead of the industry.', 'language': 'en', 'source': 'https://www.deeplearning.ai/', 'title': 'DeepLearning.AI: Start or Advance Your Career in AI'}\n\nquestion = \"how the neural network works?\"\n# filter the documents from the specific website\ndocs_meta = vectordb.similarity_search(question, k=1, filter={\"source\": \"https://www.deeplearning.ai/\"})\nprint(docs_meta[0].page_content[:100])\n```\n\n### LLM Aided Retrieval\n\nIt uses language models to automatically parse sentence semantics, extract filtering information.\n\n#### SelfQueryRetriever\n\nLangChain provides the SelfQueryRetriever module, which can analyze the semantics of the question sentence from the language model, and extract the search term and filter conditions.\n\n- The **search term** of the vector search\n- The **filter conditions** of the document metadata\n\n![](https://cdn.jsdelivr.net/gh/timerring/scratchpad2023/2024/2025-03-04-23-44-01.png)\n\nFor example, for the question \"Besides Wikipedia, which health websites are there?\", SelfQueryRetriever can infer that \"Wikipedia\" represents the filter condition, that is, to exclude the documents from Wikipedia.\n\n#### Practice\n\n```python\nfrom langchain.llms import OpenAI\nfrom langchain.retrievers.self_query.base import SelfQueryRetriever\nfrom langchain.chains.query_constructor.base import AttributeInfo\n\nllm = OpenAI(temperature=0)\n\nmetadata_field_info = [\n    AttributeInfo(\n        name=\"source\", #  source is to tell the LLM the data is from which document\n        description=\"The lecture the chunk is from, should be one of `docs/loaders.pdf`, `docs/text_splitters.pdf`, or `docs/vectorstores.pdf`\",\n        type=\"string\",\n    ),\n    AttributeInfo(\n        name=\"page\", # page is to tell the LLM the data is from which page\n        description=\"The page from the lecture\",\n        type=\"integer\",\n    ),\n]\n\ndocument_content_description = \"the lectures of retrieval augmentation generation\"\nretriever = SelfQueryRetriever.from_llm(\n    llm,\n    vectordb,\n    document_content_description,\n    metadata_field_info,\n    verbose=True\n)\n\nquestion = \"What is the main topic of second lecture?\"  \n```\n\n### Compression\n\nWhen using vector retrieval to get relevant documents, directly returning the entire document fragment may lead to resource waste, as the actual relevant part is only a small part of the document. To improve this, LangChain provides a \"compression\" retrieval mechanism. \n\nIts working principle is to first use standard vector retrieval to obtain candidate documents, and then **use a language model to compress these documents** based on the semantic meaning of the query sentence, only retaining the relevant part of the document. \n\nFor example, for the query \"the nutritional value of mushrooms\", the retrieval may return a long document about mushrooms. After compression, only the sentences related to \"nutritional value\" are extracted from the document.\n\n![](https://cdn.jsdelivr.net/gh/timerring/scratchpad2023/2024/2025-03-04-23-50-24.png)\n\n#### Practice\n\n```python\nfrom langchain.retrievers import ContextualCompressionRetriever\nfrom langchain.retrievers.document_compressors import LLMChainExtractor\n\ndef pretty_print_docs(docs):\n    print(f\"\\n{'-' * 100}\\n\".join([f\"Document {i+1}:\\n\\n\" + d.page_content for i, d in enumerate(docs)]))\n\nllm = OpenAI(temperature=0)\n# initialize the compressor\ncompressor = LLMChainExtractor.from_llm(llm)\n# initialize the compression retriever\ncompression_retriever = ContextualCompressionRetriever(\n    base_compressor=compressor, # llm chain extractor\n    base_retriever=vectordb.as_retriever() # vector database retriever\n)\n# compress the source documents\nquestion = \"What is the main topic of second lecture?\"\ncompressed_docs = compression_retriever.get_relevant_documents(question)\npretty_print_docs(compressed_docs)\n```\n\n## Question Answering\n\n![](https://cdn.jsdelivr.net/gh/timerring/scratchpad2023/2024/2025-03-04-22-03-35.png)\n\n1. Multiple relevant documents have been retrieved from the vector store\n2. Potentially compress the relevant splits to fit into the LLM context. The system will generate the necessary background information (System Prompt) and keep the user's question (Human Question), and then integrate all the information into a complete context input.\n3. Send the information along with our question to an LLM to select and format an answer\n\n### RetrievalQA Chain\n\n![](https://cdn.jsdelivr.net/gh/timerring/scratchpad2023/2024/2025-03-04-22-09-33.png)\n\nWe need to use Langchain to combine the prompts into the desired format and pass them to the large language model to generate the desired reply. This solution is better than the traditional method of inputting the question into the large language model because:\n\n- **Enhance the accuracy of the answer**: By combining the retrieval results with the generation ability of the large language model, the relevance and accuracy of the answer are greatly improved.\n- **Support real-time update of the knowledge base**: The retrieval process depends on the data in the vector store, which can be updated in real time according to needs, ensuring that the answer reflects the latest knowledge.\n- **Reduce the memory burden of the model**: By using the information in the knowledge base as the input context, the dependence on the model's internal parameters for storing knowledge is reduced.\n\nIn addition to the RetrievalQA Chain, there are other methods, such as `Map_reduce`, `Refine` and  `Map_rerank`.\n\n### Map_reduce\n\n`Map_reduce` method divides the documents into multiple chunks, and then passes each chunk to the language model (LLM) to generate an independent answer. After that, all the generated answers will be merged into the final answer, and the merging process (reduce) may include summarizing, voting, etc. \n\nThis method is suitable for **the large amount of documents parallel processing**, also with quick response.\n\n![](https://cdn.jsdelivr.net/gh/timerring/scratchpad2023/2024/2025-03-04-22-15-16.png)\n\n### Refine\n\nRefine method generates an initial answer from the first document chunk, and then processes each subsequent document one by one. Each block will supplement or correct the existing answer, and finally **obtain an optimized and improved answer** after all chunks are processed.\n\nThis method is suitable for the most quality answer.\n\n![](https://cdn.jsdelivr.net/gh/timerring/scratchpad2023/2024/2025-03-04-22-18-13.png)\n\n\n### Map_rerank\n\nMap_rerank divides the documents into multiple chunks, and then generates an independent answer for each chunk. The scoring is based on the relevance and quality of the answer. Finally, the answer with the highest score will be selected as the final output.\n\nThis method is suitable for the most match answer rather than combine with all the information.\n\n![](https://cdn.jsdelivr.net/gh/timerring/scratchpad2023/2024/2025-03-04-22-18-58.png)\n\n\n### Practice\n\nYou can check the entire code [here](https://github.com/timerring/rag101/tree/main/question_answering).\n\nFirst, install the lib:\n\n```bash\npip install pyjwt\n```\n\n\nYou can use the demo to check the model performance.\n\n```python\nfrom langchain_community.chat_models import ChatZhipuAI\nfrom langchain_core.messages import AIMessage, HumanMessage, SystemMessage\n\nchat = ChatZhipuAI(\n    model=\"glm-4-flash\",\n    temperature=0.5, # the temperature of the model\n    api_key=\"your_api_key\"\n)\n\nmessages = [\n    AIMessage(content=\"Hi.\"),  # AI generated message\n    SystemMessage(content=\"Your role is a poet.\"),  # the role of the model\n    HumanMessage(content=\"Write a short poem about AI in four lines.\"),  # the message from the user\n]\n\n# get the answer from the model\nresponse = chat.invoke(messages)\nprint(response.content)\n```\n\nThen we can use the `RetrievalQA chain` to get the answer from the model.\n\n```python\nfrom langchain_community.chat_models import ChatZhipuAI\nfrom langchain_core.messages import AIMessage, HumanMessage, SystemMessage\nfrom langchain_community.embeddings import ZhipuAIEmbeddings\nfrom langchain_chroma import Chroma\nfrom langchain_community.document_loaders import WebBaseLoader\nfrom langchain.text_splitter import RecursiveCharacterTextSplitter\nfrom langchain.chains import RetrievalQA\nfrom langchain.prompts import PromptTemplate # You can also import the PromptTemplate\n\nloader = WebBaseLoader(\"https://en.d2l.ai/\")\ndocs = loader.load()\n\ntext_splitter = RecursiveCharacterTextSplitter(\n    chunk_size = 1500,\n    chunk_overlap = 150\n)\nsplits = text_splitter.split_documents(docs)\n\npersist_directory = '.'\n\n# initialize the embeddings\nembeddings = ZhipuAIEmbeddings(\n    model=\"embedding-3\",\n    api_key=\"your_api_key\"\n)\n\n# initialize the vector database\nvectordb = Chroma.from_documents(\n    documents=splits,\n    embedding=embeddings,\n    persist_directory=persist_directory\n)\n\nchat = ChatZhipuAI(\n    model=\"glm-4-flash\",\n    temperature=0.5,\n    api_key =  \"your_api_key\"\n)\n\n# Now you can ask the question about the web to the model\nquestion = \"What is this book about?\"\n\n# You can also create a prompt template\ntemplate = \"\"\"\nPlease answer the question based on the following context.\nIf you don't know the answer, just say you don't know, don't try to make up an answer.\nAnswer in at most three sentences. Please answer as concisely as possible. Finally, always say \"Thank you for asking!\"\nContext: {context}\nQuestion: {question}\nHelpful answer:\n\"\"\"\nQA_CHAIN_PROMPT = PromptTemplate.from_template(template)\n\nqa_chain = RetrievalQA.from_chain_type(\n    chat,\n    retriever=vectordb.as_retriever(),\n    return_source_documents=True, # Return the source documents(optional)\n    chain_type_kwargs={\"prompt\": QA_CHAIN_PROMPT} # Add the prompt template to the chain\n)\nresult = qa_chain({\"query\": question})\nprint(result[\"result\"])\nprint(result[\"source_documents\"][0]) # If you set return_source_documents to True, you can get the source documents\n```\n\n## Conversational Retrieval Chain\n\nThe whole process of RAG is as follows:\n\n![](https://cdn.jsdelivr.net/gh/timerring/scratchpad2023/2024/2025-03-04-23-37-20.png)\n\nConversational Retrieval Chain is a technical architecture that combines dialogue history and intelligent retrieval capabilities.\n\n![](https://cdn.jsdelivr.net/gh/timerring/scratchpad2023/2024/2025-03-04-23-40-11.png)\n\n1. Chat History: The system will record the user's dialogue context as an important input for subsequent question processing.\n2. Question: The user's question is sent to the retrieval module.\n3. Retriever: The system retrieves the content related to the question from the vector database through the retriever.\n4. System \u0026 Human: The system integrates the user's question and the extracted relevant information into the Prompt, providing structured input to the language model.\n5. LLM: The language model generates the answer based on the context, and then returns the answer to the user.\n\n### Memory\n\n`ConversationBufferMemory` is a memory module in the LangChain framework, which is used to manage the dialogue history. Its main function is to store the dialogue content between users and AI in the form of a buffer, and then return these records when needed, so that the model can generate responses in a consistent context.\n\nThe demo of it is as follows:\n\n```python\nfrom langchain.memory import ConversationBufferMemory\nmemory = ConversationBufferMemory(\n    memory_key=\"chat_history\", # This key can be referenced in other modules (such as chains or tools).\n    return_messages=True # whether to return the messages in list, otherwise return the messages in block.\n)\n```\n\nBesides, we also need the corresponding RA module. Then we can test the memory.\n\n```python \nfrom langchain.chains import ConversationalRetrievalChain\nretriever=vectordb.as_retriever()\nqa = ConversationalRetrievalChain.from_llm(\n    chat,\n    retriever=retriever,\n    memory=memory\n)\nquestion = \"What is the main topic of this book?\"\nresult = qa.invoke({\"question\": question})\nprint(result['answer'])\n\nquestion = \"What is my last question?\"\nresult = qa.invoke({\"question\": question})\nprint(result['answer'])\n```\n\n### Practice\n\nYou can check the entire code [here](https://github.com/timerring/rag101/blob/main/conversational_retrieval_chain/conversational_retrieval_chain.py).\n\nThe best practice is as follows:\n\n```python\nfrom langchain_community.chat_models import ChatZhipuAI\nfrom langchain_core.messages import AIMessage, HumanMessage, SystemMessage\nfrom langchain_community.embeddings import ZhipuAIEmbeddings\nfrom langchain_chroma import Chroma\nfrom langchain_community.document_loaders import WebBaseLoader\nfrom langchain.text_splitter import RecursiveCharacterTextSplitter\nfrom langchain.memory import ConversationBufferMemory\nfrom langchain.chains import ConversationalRetrievalChain\n\nloader = WebBaseLoader(\"https://en.d2l.ai/\")\ndocs = loader.load()\n\ntext_splitter = RecursiveCharacterTextSplitter(\n    chunk_size = 1500,\n    chunk_overlap = 150\n)\nsplits = text_splitter.split_documents(docs)\n\npersist_directory = '.'\n\n# initialize the embeddings\nembeddings = ZhipuAIEmbeddings(\n    model=\"embedding-3\",\n    api_key=\"your_api_key\"\n)\n\n# initialize the vector database\nvectordb = Chroma.from_documents(\n    documents=splits,\n    embedding=embeddings,\n    persist_directory=persist_directory\n)\n\nchat = ChatZhipuAI(\n    model=\"glm-4-flash\",\n    temperature=0.5,\n    api_key =  \"your_api_key\"\n)\n\n# initialize the memory\nmemory = ConversationBufferMemory(\n    memory_key=\"chat_history\",\n    return_messages=True\n)\n\n# create the ConversationalRetrievalChain\nretriever = vectordb.as_retriever()\nqa = ConversationalRetrievalChain.from_llm(\n    chat,\n    retriever=retriever,\n    memory=memory\n)\n\n# First question\nquestion = \"What is the main topic of this book?\"\nresult = qa.invoke({\"question\": question})\nprint(result['answer'])\n\n# Second question\nquestion = \"Can you tell me more about it?\"\nresult = qa.invoke({\"question\": question})\nprint(result['answer'])\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftimerring%2Frag101","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftimerring%2Frag101","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftimerring%2Frag101/lists"}