{"id":24175176,"url":"https://github.com/denser-org/denser-chat","last_synced_at":"2025-09-20T19:30:43.878Z","repository":{"id":262139342,"uuid":"883971270","full_name":"denser-org/denser-chat","owner":"denser-org","description":"Chat with PDF files with source highlights","archived":false,"fork":false,"pushed_at":"2024-12-06T06:43:11.000Z","size":32511,"stargazers_count":70,"open_issues_count":1,"forks_count":4,"subscribers_count":3,"default_branch":"main","last_synced_at":"2024-12-06T07:28:55.679Z","etag":null,"topics":["ai","chatbot","nlp","pdf","retrieval","retrieval-augmented-generation"],"latest_commit_sha":null,"homepage":"https://denser.ai","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/denser-org.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-11-05T22:45:59.000Z","updated_at":"2024-12-06T06:43:16.000Z","dependencies_parsed_at":"2024-11-10T19:51:51.715Z","dependency_job_id":null,"html_url":"https://github.com/denser-org/denser-chat","commit_stats":null,"previous_names":["denser-org/denser-chat"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/denser-org%2Fdenser-chat","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/denser-org%2Fdenser-chat/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/denser-org%2Fdenser-chat/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/denser-org%2Fdenser-chat/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/denser-org","download_url":"https://codeload.github.com/denser-org/denser-chat/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":233682464,"owners_count":18713553,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","chatbot","nlp","pdf","retrieval","retrieval-augmented-generation"],"created_at":"2025-01-13T02:01:33.550Z","updated_at":"2025-09-20T19:30:37.663Z","avatar_url":"https://github.com/denser-org.png","language":"Python","funding_links":[],"categories":["🎯 Marketing","Chatbots"],"sub_categories":["🟩 Development Tools 🛠️"],"readme":"## Denser Chat\n\n\u003cimg src=\"demo.gif\" width=\"800\" alt=\"Denser Chat\"\u003e\n\nDenser Chat is a chatbot that can answer questions from PDFs and webpages. This project is actively developed and maintained by [denser.ai](https://denser.ai). Feel free to contact support@denser.ai if you have feedback or questions.\n\nMain features:\n\n* Extract text and tables from PDFs and webpages.\n* Build a chatbot with [denser-retriever](https://github.com/denser-org/denser-retriever)\n* Support interactive Streamlit chatbot app with source highlights in PDFs\n* Elasticsearch is used for fast retrieval on indexed data, and the elastic keyword enable searching or filtering based on specific terms or criteria within that indexed data.\n\n## Installation\n\nFirst clone the repository.\n\n```bash\ngit clone https://github.com/denser-org/denser-chat.git\n```\n\nGo to the project directory and start a virtual environment. Make sure your python version is 3.11.\n\n```bash\ncd denser-chat\npython -m venv .venv\n# For Linux/Mac users\nsource .venv/bin/activate\n# For Windows users\n.\\.venv\\Scripts\\activate.bat\n```\n\nRun the following command to install the required packages.\n\n```bash\npip install -e .\n```\n\nOr use this [poetry](https://python-poetry.org/docs/) command\n\n```bash\npoetry install\n```\n\n## Quick Start\n\nBefore building an index, we need to run docker-compose to start Elasticsearch and Milvus services in the background,\nwhich are required for denser-retriever.\n\n```bash\ncd denser_chat\ndocker compose -f docker-compose.yml up -d\n```\n\nWe run the following command to build a chatbot index. The first argument is the sources file which specify files used to build chatbots. Files can be local PDF files, URL PDFs, or URLs. The second argument is the output directory, and the third argument is the index name.\n\n```bash\npython build.py sources.txt output test_index\n```\n\nThis command will build an index `test_index` via denser-retriever. Next we can start a streamlit app with the following\ncommand. As the app relies on ChatGPT or Claude API, we need to set their keys (one is sufficient) in the environment variables.\n\n```bash\nexport OPENAI_API_KEY=\"your-openai-key\"\nexport CLAUDE_API_KEY=\"your-claude-key\"\n```\n\nIn order to run the app, we need to start a local server to serve the PDFs. We can use the following command to start a server at root directory.\n\n```bash\npython -m http.server 8000 \n```\n\nThen we can start the streamlit app on a different terminal with the following command.\n\n```bash\ncd denser_chat\nstreamlit run demo.py -- --index_name test_index \n```\n\nThen we can start to ask questions such as \"What is in-batch negative sampling ?\" or \"what parts have stop pins?\". We can expect that the chatbot will return the answer with the source highlighted in the PDF.\n\n### License\n\nThis project is licensed under the MIT License.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdenser-org%2Fdenser-chat","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdenser-org%2Fdenser-chat","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdenser-org%2Fdenser-chat/lists"}