{"id":24801367,"url":"https://github.com/syedt1/django_chatbot_with_vector_db","last_synced_at":"2026-01-06T07:02:31.460Z","repository":{"id":274764956,"uuid":"923971726","full_name":"SyedT1/django_chatbot_with_vector_db","owner":"SyedT1","description":"RAG Application","archived":false,"fork":false,"pushed_at":"2025-03-05T16:41:28.000Z","size":2670,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-25T03:55:21.750Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"HTML","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/SyedT1.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-01-29T07:04:01.000Z","updated_at":"2025-03-05T16:41:31.000Z","dependencies_parsed_at":"2025-01-29T09:26:00.603Z","dependency_job_id":"62567d42-ebd8-46d9-8c71-fdef5b68c555","html_url":"https://github.com/SyedT1/django_chatbot_with_vector_db","commit_stats":null,"previous_names":["syedt1/django_chatbot_with_vector_db"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/SyedT1/django_chatbot_with_vector_db","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SyedT1%2Fdjango_chatbot_with_vector_db","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SyedT1%2Fdjango_chatbot_with_vector_db/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SyedT1%2Fdjango_chatbot_with_vector_db/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SyedT1%2Fdjango_chatbot_with_vector_db/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/SyedT1","download_url":"https://codeload.github.com/SyedT1/django_chatbot_with_vector_db/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SyedT1%2Fdjango_chatbot_with_vector_db/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28223114,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2026-01-06T02:00:07.049Z","response_time":56,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-01-30T04:28:23.442Z","updated_at":"2026-01-06T07:02:31.428Z","avatar_url":"https://github.com/SyedT1.png","language":"HTML","funding_links":[],"categories":[],"sub_categories":[],"readme":"## RAG Application\n\nThis project is a Django-based chatbot that uses a vector database for storing and querying text embeddings. It leverages the Ollama API for generating embeddings and the Stanza library for text processing.\n\n## Ollama Setup Instructions\n\n1. Download Ollama from [here](https://ollama.com/).\n2. Install Ollama by following the instructions provided on the download page.\n3. To pull the Llama 3.2 model, use the following command:\n    ```sh\n    ollama pull llama3.2\n    ```\n4. For other models, replace `llama3.2` with the desired model name in the command above.\n\n\n## Setup Instructions\n\n1. Clone the repository:\n    ```sh\n    git clone https://github.com/SyedT1/django_chatbot_with_vector_db.git\n    ```\n2. Navigate to the project directory:\n    ```sh\n    cd django_chatbot_with_vector_db\n    ```\n3. Create a virtual environment:\n    ```sh\n    python -m venv myenv\n    ```\n4. Activate the virtual environment:\n    - On Windows:\n        ```sh\n        myenv\\Scripts\\activate\n        ```\n    - On macOS and Linux:\n        ```sh\n        source myenv/bin/activate\n        ```\n5. Install the required dependencies:\n    ```sh\n    pip install -r requirements.txt\n    ```\n6. Open MySQL WorkBench and select File (top left), and then click Open SQL Script from this project titled __create_db4ai.sql__ . Execute it\n\n7. Complete Migrations:\n    ```sh\n    python manage.py migrate\n    ```\n8. Run the development server:\n    ```sh\n    python manage.py runserver\n    ```\n9. Open Powershell as Administrator and run:\n    ```sh\n    ollama serve\n    ```\n## Setup db in mysql (if you didnt already in point number 6)\n- Create a database db4ai\n\n## Installing Chromadb you might encounter the following error\n- error: Microsoft Visual C++ 14.0 or greater is required. Get it with \"Microsoft C++ Build Tools\": https://visualstudio.microsoft.com/visual-cpp-build-tools/ \n- [Solution is here](https://stackoverflow.com/questions/73969269/error-could-not-build-wheels-for-hnswlib-which-is-required-to-install-pyprojec)\n    \n\n## Usage\n1. Access the chatbot interface by navigating to `http://127.0.0.1:8000` in your web browser.\n2. To upload text files and insert embeddings into the vector database (Chroma DB), navigate to the endpoint `http://127.0.0.1:8000/upload`.\n3. The chatbot will respond based on the text embeddings stored in the vector database.\n\n\n## Supported File Type  \n(.txt)\n# [Latest] Version-3 (Easy to store, but tough to retrieve) ( EaToR )\n## Improvements\n1. Include data and time\n2. Responses should include source of the knowledge i.e text file names\n\n\n## Interface [Before finetuning] (http://127.0.0.1:8000/list_file)\n### List of uploaded files where the user can delete and update the knowledge base\n![Pic 1](interface/interface_3.png)\n\n### Initially it looks like this (http://127.0.0.1:8000/upload)\n![Pic 2](interface/interface_4.png)\n\n### Embedding Process in the background where query is embedded and the fetched embedded vector with nearest distance \n![Pic 3](interface/interface_5.png)\n\n### The vector db is empty hence nothing significant was achieved. (http://127.0.0.1:8000/)\n![Pic 4](interface/interface_6.png)\n\n### Embedding Generation of the text files\n![Pic 5](interface/interface_7.png)\n\n\n### Uploading the same file results in rejection\n![Pic 6](interface/interface_8.png)\n\n\n### Aftermath of training a text file about Linux (10 Lines)\n![Pic 7](interface/interface_9.png)\n\n\n## Limitations\n1. Authentication incomplete for admin\n2. Slower retrieval process\n3. Text Deletion leading to embedding deletion task incomplete\n4. Many More\n\n\n#  Version-2 (Easy to store, but tough to retrieve) ( EaToR )\n\n## Improvements\n1. User gets more accurate responses\n2. UI is more developed\n3. Persistant storage enabled by chrome_db, hence session-ends won't lose embeddings.\n\n\n\n## Limitations\n1. Late Responses\n2. Responses should include source of the knowledge i.e text file names\n3. Include data and time\n\n## Interface\n![Pic 1](interface/interface_1.png)\n![Pic 2](interface/interface_2.png)\n\n\n\n\n# Version-1 (Easy to store, but tough to retrieve) ( EaToR )\n\n\n## Todo\n- Exception and Error Handling (All modules)\n- Memory Optimization\n- Minimization of Inference time\n- Create diagrammatic representations of the processes.\n- Include diagrams in every version to show the state of development.\n- Implement fetching queries from the vector database (under development).\n- Complete the role of admins in deleting and uploading text files.\n    - Ensure embeddings are deleted from the vector database upon file deletion.\n\n## Diagrammatic Processes\n### Process --upload\n![file](diagrams/upload.png)\n- Incomplete\n\n### Process --query\n- TODO\n\n### Process --login\n- TODO\n\n## Benchmarks\n- TODO\n\n## Things to work on\n### Processing text files of various sorts:\n    1. File can't be empty -\u003e render alert on empty files and wont process\n    2. Same file name can't exist -\u003e render alert\n    3. Same file content can't exist\n    4. Same file name and Same file content can't exist\n    5. File can't process bangla content. Presence of bangla text will generate alert\n### Embeddings:\n    1. Query must be able to capture as much as information possible\n    2. Benchmarking on various models\n    3. Embeddings similarity must be compared with various tools","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsyedt1%2Fdjango_chatbot_with_vector_db","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsyedt1%2Fdjango_chatbot_with_vector_db","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsyedt1%2Fdjango_chatbot_with_vector_db/lists"}