{"id":29486668,"url":"https://github.com/nazdridoy/docuchat","last_synced_at":"2026-05-19T09:06:49.065Z","repository":{"id":303730961,"uuid":"1016488560","full_name":"nazdridoy/docuchat","owner":"nazdridoy","description":"DocuChat is a document chat application that allows you to have conversations with your documents, powered by a serverless vector database for scalable, efficient retrieval. Upload your files and ask questions in natural language to get answers based on their content.","archived":false,"fork":false,"pushed_at":"2025-07-09T05:10:31.000Z","size":67,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-07-09T06:25:00.007Z","etag":null,"topics":["ai","docuchat","document-chatbot","libsql","llm","local-llm","oll","openai","pdf","vector-database"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"agpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nazdridoy.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-07-09T05:06:07.000Z","updated_at":"2025-07-09T05:12:41.000Z","dependencies_parsed_at":"2025-07-09T06:25:22.263Z","dependency_job_id":"02e9f25a-92a3-4215-9eba-bc3dbafe455d","html_url":"https://github.com/nazdridoy/docuchat","commit_stats":null,"previous_names":["nazdridoy/docuchat"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/nazdridoy/docuchat","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nazdridoy%2Fdocuchat","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nazdridoy%2Fdocuchat/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nazdridoy%2Fdocuchat/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nazdridoy%2Fdocuchat/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nazdridoy","download_url":"https://codeload.github.com/nazdridoy/docuchat/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nazdridoy%2Fdocuchat/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33209477,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-19T07:54:09.561Z","status":"ssl_error","status_checked_at":"2026-05-19T07:54:08.508Z","response_time":58,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","docuchat","document-chatbot","libsql","llm","local-llm","oll","openai","pdf","vector-database"],"created_at":"2025-07-15T08:01:26.719Z","updated_at":"2026-05-19T09:06:49.059Z","avatar_url":"https://github.com/nazdridoy.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# DocuChat\n\nDocuChat is a document chat application that allows you to have conversations with your documents, powered by a serverless vector database for scalable, efficient retrieval. Upload your files and ask questions in natural language to get answers based on their content.\n\n## Features\n\n-   Upload and process text documents (.txt, .pdf, etc.)\n-   Prevent duplicate file uploads using file hash comparison (SHA-256)\n-   Client-side file hash calculation before upload for fast duplicate detection\n-   Generate embeddings for document chunks\n-   Flexible database options: Use a local SQLite database for quick development or a serverless Turso database for scalable, production-ready vector storage with `better-sqlite3` and `sqlite-vss`.\n-   Query documents using natural language\n-   Get contextual responses based on your documents' content with citations.\n-   Real-time chat responses with Server-Sent Events (SSE).\n-   Deep search with HyDE (Hypothetical Document Embeddings) for improved retrieval.\n-   Maximal Marginal Relevance (MMR) for diverse search results.\n-   Contextual chat history integration for better conversations.\n-   Markdown rendering for chat responses and source citations.\n\n## Technologies\n\n-   Node.js\n-   Express\n-   `better-sqlite3` with `sqlite-vss` (for local and Turso-compatible serverless vector storage)\n-   OpenAI / Ollama compatible APIs (for chat and embeddings)\n-   `pdf-parse` for PDF document processing.\n-   Modular OpenAI client.\n-   Server-Sent Events (SSE).\n\n## Prerequisites\n\n-   Node.js\n-   npm\n-   Git\n-   Optionally, [Ollama](https://ollama.ai/) for local embeddings.\n\n## Installation\n\n1.  Clone this repository:\n    ```bash\n    git clone https://github.com/your-username/docuchat.git\n    cd docuchat\n    ```\n\n2.  Install dependencies:\n    ```bash\n    npm install\n    ```\n\n3.  Create a `.env` file in the root directory and populate it with the necessary environment variables as described in the **Configuration** section.\n\n## Configuration\n\nThe application is configured through environment variables.\n\n### Environment Variables\n\n```\n# Server Configuration\nPORT=3000\n\n# Database Configuration\n# For local SQLite database:\nDATABASE_URL=\"file:./data.db\"\nUSE_LOCAL_DB=\"true\"\n\n# For remote Turso database (experimental with better-sqlite3):\n# DATABASE_URL=\"libsql://your-database.turso.io\"\n# DATABASE_TOKEN=\"your-database-token\"\n# USE_LOCAL_DB=\"false\"\n\n# OpenAI API Configuration (for chat)\nOPENAI_BASE_URL=\"https://api.openai.com/v1\"\nOPENAI_MODEL=\"gpt-3.5-turbo\"\nOPENAI_API_KEY=\"your-openai-api-key\"\n\n# RAG API Configuration (for embeddings)\n# For OpenAI:\nRAG_BASE_URL=\"https://api.openai.com/v1\"\nRAG_MODEL=\"text-embedding-3-small\"\nRAG_API_KEY=\"your-openai-api-key\"\n\n# Embedding Rate Limit Handling\nEMBEDDING_MAX_RETRIES=5 # Maximum number of retries for rate-limited embedding requests\nEMBEDDING_RETRY_DELAY=10000 # Delay in milliseconds (10 seconds) between retries\n\n# For Ollama (recommended for local development):\n# RAG_BASE_URL=\"http://127.0.0.1:11434/v1\"\n# RAG_MODEL=\"mxbai-embed-large:latest\"\n# RAG_API_KEY=\"ollama\" # Any value works here\n# EMBEDDING_DIMENSIONS=1024 # Required for local models to set correct chunking defaults and vector dimensions in DB\n\n# CRITICAL: If your embedding API server (OpenAI or Ollama) is offline, you MUST set this value\n# or the server will not start. Common values:\n# - OpenAI text-embedding-3-small: 1536\n# - OpenAI text-embedding-3-large: 3072\n# - Ollama mxbai-embed-large: 1024\n# - Ollama nomic-embed-text: 768\n# EMBEDDING_DIMENSIONS=1536\n\n# Document Processing Configuration\nCHUNK_SIZE=1000\nCHUNK_OVERLAP=200\n# These values are dynamically determined based on the embedding model's dimensions\n# if not explicitly set. Overriding them may be useful for fine-tuning performance.\n\n# RAG Configuration\nSIMILARITY_THRESHOLD=0.5 # Minimum score for a chunk to be included in context. (Default: 0.5)\nDEEP_SEARCH_INITIAL_THRESHOLD=0.30\nCONTEXT_MAX_LENGTH=4096 # Max characters for context sent to the LLM. (Default: 4096)\n\n\n# Document Upload Configuration\nMAX_FILE_SIZE=10485760 # 10MB\nUPLOAD_DIRECTORY=\"./uploads\"\n```\n\n## CLI Tool(debug): View Database Entries\n\nA Python script `view_embedding.py` is provided to inspect entries in the SQLite database, including document chunks and their associated embeddings. This is particularly useful for debugging and understanding the stored data.\n\nTo use it:\n1.  Ensure you have Python and `numpy` installed (`pip install numpy`).\n2.  Run the script, specifying the database path and whether to view chunks or embeddings.\n\nExample:\n```bash\npython view_embedding.py --database ./data.db --rowid 1\npython view_embedding.py --database ./data.db --rowid 1 --embedding\n\n```\n\n## License\n\nThis project is licensed under the AGPL-3.0 License. See the `LICENSE` file for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnazdridoy%2Fdocuchat","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnazdridoy%2Fdocuchat","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnazdridoy%2Fdocuchat/lists"}