{"id":45157424,"url":"https://github.com/couchbase-examples/evaluate-rag","last_synced_at":"2026-02-20T05:02:42.858Z","repository":{"id":292804072,"uuid":"862798361","full_name":"couchbase-examples/evaluate-rag","owner":"couchbase-examples","description":"Evaluation of RAG systems using Ragas","archived":false,"fork":false,"pushed_at":"2025-05-12T08:01:25.000Z","size":16,"stargazers_count":1,"open_issues_count":1,"forks_count":0,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-05-12T08:56:33.786Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/couchbase-examples.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-09-25T07:56:04.000Z","updated_at":"2025-04-12T15:44:09.000Z","dependencies_parsed_at":"2025-05-12T09:08:07.621Z","dependency_job_id":null,"html_url":"https://github.com/couchbase-examples/evaluate-rag","commit_stats":null,"previous_names":["couchbase-examples/evaluate-rag"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/couchbase-examples/evaluate-rag","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/couchbase-examples%2Fevaluate-rag","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/couchbase-examples%2Fevaluate-rag/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/couchbase-examples%2Fevaluate-rag/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/couchbase-examples%2Fevaluate-rag/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/couchbase-examples","download_url":"https://codeload.github.com/couchbase-examples/evaluate-rag/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/couchbase-examples%2Fevaluate-rag/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29641929,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-20T03:21:14.183Z","status":"ssl_error","status_checked_at":"2026-02-20T03:18:24.455Z","response_time":59,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-02-20T05:02:38.777Z","updated_at":"2026-02-20T05:02:42.847Z","avatar_url":"https://github.com/couchbase-examples.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Evaluate RAG using Ragas\n\nThe code examples to evaluate RAG responses using [Ragas](https://ragas.io/) using the LLM (OpenAI) as the judge approach. The RAG pipelines is based off the RAG application in this [repo](https://github.com/couchbase-examples/qa-bot-demo).\n\n\u003e Note that you need Couchbase Server 7.6 or higher for Vector Search.\n\nA detailed tutorial for this demo can be found on the [Developer Portal](https://developer.couchbase.com/tutorial-evaluate-rag-responses-using-ragas)\n\n## How to Run\n\n- Install the dependencies\n\n  `pip install -r requirements.txt`\n\n- Set the environment variables\n  Copy the .env.example file and rename it to .env and replace the placeholders with the actual values for your environment.\n\n  ```\n  OPENAI_API_KEY=\u003copen-ai-api-key\u003e\n  DB_CONN_STR=\u003cconnection string for couchbase cluster\u003e\n  DB_USERNAME=\u003cyour database user\u003e\n  DB_PASSWORD=\u003cyour database password\u003e\n  DB_BUCKET=\u003cyour database bucket\u003e\n  DB_SCOPE=\u003cyour database scope\u003e\n  DB_COLLECTION=\u003cyour database collection\u003e\n  INDEX_NAME=\u003cvector search index name\u003e\n  EMBEDDING_MODEL=text-embedding-3-small\n  LANGCHAIN_ENDPOINT=https://api.smith.langchain.com\n  LANGCHAIN_API_KEY=\u003cyour langsmith api key\u003e\n  ```\n\n- Run the evaluation script\n  `python rag_evaluation.py`\n\nNote: For the RAG pipeline to work, you would need to ingest some documents to answer the questions related to Couchbase. You could do that based on the ingestion script in this [repo](https://github.com/couchbase-examples/qa-bot-demo).\n\n## Adding more Test Data\n\nThe test data can be updated by adding more question \u0026 answer pairs to the list in the `create_dataset.py` file.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcouchbase-examples%2Fevaluate-rag","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcouchbase-examples%2Fevaluate-rag","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcouchbase-examples%2Fevaluate-rag/lists"}