{"id":15203959,"url":"https://github.com/kaylamarietorres/haystack_elasticsearch_ragbot","last_synced_at":"2026-03-06T21:30:54.525Z","repository":{"id":250503190,"uuid":"834642249","full_name":"kaylamarietorres/haystack_elasticsearch_ragbot","owner":"kaylamarietorres","description":"Internal Knowledge Base Search","archived":false,"fork":false,"pushed_at":"2024-08-01T02:27:58.000Z","size":6162,"stargazers_count":0,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-01-16T06:16:48.077Z","etag":null,"topics":["elasticsearch","haystack"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kaylamarietorres.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-07-27T22:43:33.000Z","updated_at":"2024-08-01T02:28:01.000Z","dependencies_parsed_at":"2024-09-24T06:01:00.901Z","dependency_job_id":"40cb4ed4-71b9-447f-b753-618189808067","html_url":"https://github.com/kaylamarietorres/haystack_elasticsearch_ragbot","commit_stats":{"total_commits":8,"total_committers":1,"mean_commits":8.0,"dds":0.0,"last_synced_commit":"dc7131fb14e97520f22831226f8a75f8d4545c5c"},"previous_names":["kaylamarietorres/haystack_elasticsearch_ragbot"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kaylamarietorres%2Fhaystack_elasticsearch_ragbot","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kaylamarietorres%2Fhaystack_elasticsearch_ragbot/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kaylamarietorres%2Fhaystack_elasticsearch_ragbot/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kaylamarietorres%2Fhaystack_elasticsearch_ragbot/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kaylamarietorres","download_url":"https://codeload.github.com/kaylamarietorres/haystack_elasticsearch_ragbot/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":242078477,"owners_count":20068556,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["elasticsearch","haystack"],"created_at":"2024-09-28T05:04:30.488Z","updated_at":"2025-03-05T18:27:40.633Z","avatar_url":"https://github.com/kaylamarietorres.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Knowledge Base Search\nRetrieval Augmented Generation (RAG) is an approach that combines both retrieval and generation techniques in AI language processing models\n\n![Architecture](img/architecture2.png)\n## Goal\nImplement a search system that allows users to quickly find keyword information within a knowledge base. This is helpful for large volumes of local text data to work with in order to find more centered, targeted keywords. \n\n![App Page](img/app_page.png)\n## Installation \nRun these commands to install and run this app\n```\ndocker pull docker.elastic.co/elasticsearch/elasticsearch:8.14.3\n```\n```\ndocker network create mynetwork\n```\n```\ndocker run -d --name elasticsearch --network mynetwork -p 9200:9200 -e \"discovery.type=single-node\" -e \"xpack.security.enabled=false\" -e \"xpack.security.transport.ssl.enabled=false\" -e \"ELASTIC_PASSWORD=testpassword\" docker.elastic.co/elasticsearch/elasticsearch:8.14.3\n\n```\n```\ncd Docker\n```\n```\ndocker build -t streamlit-app . --network host\n```\n```\ndocker run -d -p 8501:8501 --name streamlit-app --network mynetwork -e ELASTIC_PASSWORD=testpassword streamlit-app\n```\nSee app on localhost:8501\n\n# How to use\n1. Upload a PDF\n2. Click \"Proces PDF\"\n3. You should see \"PDF processed and indexed successfully!\" if done correctly\n4. Enter search query\n5. See results\n\n# How it works\n\n## [Haystack](https://haystack.deepset.ai/overview/intro) \nHaystack by Deepset is an open-source framework for building production-ready *LLM applications*, RAG *pipelines* and *state-of-the-art search systems* that work intelligently over large document collections. It lets you quickly try out the latest AI models while being flexible and easy to use.\n\nSome examples of what you can build include:\n\n- **Advanced RAG** on your own data source, powered by the latest retrieval and generation techniques.\n- **Chatbots and agents** powered by cutting-edge generative models like GPT-4, that can even call external functions and services.\n- **Generative multi-modal question answering** on a knowledge base containing mixed types of information: images, text, audio, and tables.\n- **Information extraction**from documents to populate your database or build a knowledge graph.\n\n\n## [Elasticsearch](https://www.elastic.co/guide/en/elasticsearch/reference/current/elasticsearch-intro.html) \nA distributed search and analytics engine designed for fast and efficient data retrieval and analysis. In this project we will be using it as a vector database to create, store, and search vector embeddings. \n\n## [Streamlit](https://www.geeksforgeeks.org/a-beginners-guide-to-streamlit/)\nWe have a lot of options in python for deploying our model. Some popular frameworks are Flask and Django. But the issue with using these frameworks is that we should have some knowledge of HTML, CSS, and JavaScript. Using streamlit you can deploy any machine learning model and any python project with ease and without worrying about the frontend. Streamlit is very user-friendly. \n\n## [Docker](https://docs.docker.com/guides/docker-overview/)\nDocker is a platform for developing, shipping, and running applications. It allows you to separate your applications from your infrastructure in order to deliver software quickly. In my opinion, it is one of the most important tools in software engineering. \n\n# Future Considerations\nKibana is used for data visualization and it works well with elasticsearch. It allows users to create dashbords and perform data exploration. I could see it being used in the future with this app or similar ones. \n\nIn the future this project could be built upon by using Retrieval Augmented Generation (RAG), which combines retrieval mechanisms with generative models like Large Language Models (LLMs). RAG works by first retrieving relevant documents and then generating a response based on the retrieved information using a generative model, which is helpful for contextual understanding and semantic relevance. Since the retrieval part is already provided here, you just need to add a generative model.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkaylamarietorres%2Fhaystack_elasticsearch_ragbot","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkaylamarietorres%2Fhaystack_elasticsearch_ragbot","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkaylamarietorres%2Fhaystack_elasticsearch_ragbot/lists"}