{"id":15175541,"url":"https://github.com/aktersnurra/rag","last_synced_at":"2026-02-28T18:01:22.225Z","repository":{"id":231416324,"uuid":"781662281","full_name":"aktersnurra/rag","owner":"aktersnurra","description":"Retrieval Augmented Generation","archived":false,"fork":false,"pushed_at":"2024-04-13T11:55:16.000Z","size":703,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2024-04-14T00:44:25.751Z","etag":null,"topics":["cohere","ollama","qdrant-vector-database","retrieval-augmented-generation"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/aktersnurra.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2024-04-03T20:03:14.000Z","updated_at":"2024-04-15T07:53:32.207Z","dependencies_parsed_at":"2024-04-04T00:59:36.801Z","dependency_job_id":"be7b6f73-9747-44c7-96bf-c25de037df65","html_url":"https://github.com/aktersnurra/rag","commit_stats":null,"previous_names":["aktersnurra/rag"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/aktersnurra/rag","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aktersnurra%2Frag","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aktersnurra%2Frag/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aktersnurra%2Frag/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aktersnurra%2Frag/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/aktersnurra","download_url":"https://codeload.github.com/aktersnurra/rag/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aktersnurra%2Frag/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29946463,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-28T17:57:52.716Z","status":"ssl_error","status_checked_at":"2026-02-28T17:57:31.974Z","response_time":90,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cohere","ollama","qdrant-vector-database","retrieval-augmented-generation"],"created_at":"2024-09-27T12:39:25.590Z","updated_at":"2026-02-28T18:01:22.183Z","avatar_url":"https://github.com/aktersnurra.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Retrieval Augmented Generation\n\nRAG with ollama (and optionally cohere) and qdrant. This is basically a glorified\n(bloated) `grep`.\n\n## Usage\n\n### Setup\n\n#### 1. Environment Variables\n\nCreate a .env file or set the following parameters:\n\n```.env\nCHUNK_SIZE=4096\nCHUNK_OVERLAP=256\n\nENCODER_MODEL=nomic-embed-text\nEMBEDDING_DIM=768\nRETRIEVER_TOP_K=15\nRETRIEVER_SCORE_THRESHOLD=0.5\n\nRERANK_MODEL=mixedbread-ai/mxbai-rerank-large-v1\nRERANK_TOP_K=5\n\nGENERATOR_MODEL=llama3\n\nDOCUMENT_DB_NAME=rag\nDOCUMENT_DB_USER=aktersnurra\n\nQDRANT_URL=http://localhost:6333\nQDRANT_COLLECTION_NAME=knowledge-base\n\nCOHERE_API_KEY = \u003cCOHERE_API_KEY\u003e # OPTIONAL\nCOHERE_RERANK_MODEL = \"rerank-english-v3.0\"\n```\n\n#### 2. Install Python Dependencies\n\n```\npoetry install\n```\n\n#### 3. Ollama\n\nMake sure ollama is running:\n\n```sh\nollama serve\n```\n\nDownload the encoder and generator models with ollama:\n\n```sh\nollama pull $GENERATOR_MODEL\nollama pull $ENCODER_MODEL\n```\n\n#### 4. Qdrant\n\nQdrant is used to store the embeddings of the chunks from the documents.\n\nDownload and run qdrant.\n\n#### 5. Postgres\n\nPostgres is used to save hashes of the document to prevent documents from\nbeing added to the vector db more than ones.\n\nDownload and run qdrant.\n\n#### 6. Cohere\n\nGet an API from their website, but is optional.\n\n### Running\n\nActivate the poetry shell:\n\n```sh\npoetry shell\n```\n\nUse the cli:\n\n```sh\npython rag/cli.py\n```\n\nor the ui using a browser:\n\n```sh\nstreamlit run rag/ui.py\n```\n\n### Notes\n\nYes, it is inefficient/dumb to use ollama when you can just load the models with python\nin the same process.\n\n### TODO\n\n- [x] Rerank history if it is relevant.\n\n- [x] message ollama/cohere\n\n- [x] create db script\n\n- [x] write a general model for cli/ui\n\n- [ ] ~~use huggingface instead of ollama~~\n  - Huggingface is too slow...\n\n- [ ] Refactor messages\n\n- [ ] Rewrite in functional style\n\n- [ ] Try out nemotron-mini\n\n- [ ] Try out llama3-chatqa\n\n### Inspiration\n\n\nI took some inspiration from these tutorials:\n\n[rag-openai-qdrant](https://colab.research.google.com/github/qdrant/examples/blob/master/rag-openai-qdrant/rag-openai-qdrant.ipynb)\n\n[building-rag-application-using-langchain-openai-faiss](https://medium.com/@solidokishore/building-rag-application-using-langchain-openai-faiss-3b2af23d98ba)\n[knowledge_gpt](https://github.com/mmz-001/knowledge_gpt)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faktersnurra%2Frag","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Faktersnurra%2Frag","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faktersnurra%2Frag/lists"}