{"id":14524342,"url":"https://github.com/comet-ml/opik","last_synced_at":"2026-05-14T12:02:11.164Z","repository":{"id":173701998,"uuid":"638951438","full_name":"comet-ml/opik","owner":"comet-ml","description":"Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.","archived":false,"fork":false,"pushed_at":"2026-01-24T01:01:14.000Z","size":447413,"stargazers_count":17481,"open_issues_count":119,"forks_count":1317,"subscribers_count":115,"default_branch":"main","last_synced_at":"2026-01-24T04:57:56.052Z","etag":null,"topics":["evaluation","hacktoberfest","hacktoberfest2025","langchain","llama-index","llm","llm-evaluation","llm-observability","llmops","open-source","openai","playground","prompt-engineering"],"latest_commit_sha":null,"homepage":"https://www.comet.com/docs/opik/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/comet-ml.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":"AGENTS.md","dco":null,"cla":"CLA.md"}},"created_at":"2023-05-10T12:57:13.000Z","updated_at":"2026-01-24T03:25:37.000Z","dependencies_parsed_at":"2023-11-28T14:29:34.716Z","dependency_job_id":"815f0607-e1b0-48cf-a174-76e31b7dc2b0","html_url":"https://github.com/comet-ml/opik","commit_stats":null,"previous_names":["comet-ml/comet-llm","comet-ml/opik"],"tags_count":4824,"template":false,"template_full_name":null,"purl":"pkg:github/comet-ml/opik","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/comet-ml%2Fopik","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/comet-ml%2Fopik/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/comet-ml%2Fopik/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/comet-ml%2Fopik/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/comet-ml","download_url":"https://codeload.github.com/comet-ml/opik/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/comet-ml%2Fopik/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28847053,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-28T15:15:36.453Z","status":"ssl_error","status_checked_at":"2026-01-28T15:15:13.020Z","response_time":57,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["evaluation","hacktoberfest","hacktoberfest2025","langchain","llama-index","llm","llm-evaluation","llm-observability","llmops","open-source","openai","playground","prompt-engineering"],"created_at":"2024-09-04T12:01:32.648Z","updated_at":"2026-01-28T17:01:49.300Z","avatar_url":"https://github.com/comet-ml.png","language":"Python","funding_links":[],"categories":["LLMOps","Frameworks \u0026 Libraries","MCP Servers","A01_文本生成_文本对话","Python","🤖 AI \u0026 Machine Learning","Tools \u0026 Code","Playgrounds and Alternative UIs","Evaluation and Monitoring","📊 LLM Ops \u0026 Observability","LLM Deployment","Web apps","TypeScript","Testing and Monitoring (Observability)","Coding","语言资源库","Recently Updated","🌟🌟 Repo of the Month!","Tools","\u003ca id=\"tools\"\u003e\u003c/a\u003e🛠️ Tools","Tracing","Cloud Services","LLM Applications","Repos","Large Language Models (LLMs)","The Data Science Toolbox","Technical Resources","Tools \u0026 Platforms","人工智能","Training","Visual Analysis and Debugging","Librerías para usar NLP en español","Observability and Debugging","Platforms","🚀 MLOps","3. Prompt Optimization","📊 Metrics \u0026 Evaluation","Table of Contents","Agent Observability and Testing","Monitoring \u0026 Observability","LLM and Agent Observability","Monitoring","10. LLM \u0026 AI Observability","LLM-as-Judge Evaluation","8. MLOps / LLMOps \u0026 Production","LLM Testing / Monitoring","Supporting Infrastructure","Catalog","Chatbots \u0026 Virtual Companions","Cost Tracking, Observability, and Budgets","Observability \u0026 Tracing","Tools and Platforms","*Ops for AI"],"sub_categories":["LLM Observability \u0026 Tracing","Tracing \u0026 Observability Tools","Software Development","大语言对话模型及数据","Observability","Embedding Models","Hosted and self-hosted","Developer tools","python","[Dec 23, 2024](/content/2024/12/23/README.md)","Testing, Evaluation and Observability","Model Lifecycle","LLM Evaluations and Benchmarks","Contribute to our Repository","General-Purpose Machine Learning","Miscellaneous Tools","Machine Learning Environment Management Tools","Open Source Frameworks","Benchmark","Herramientas de observabilidad","Open Source Platforms","Tools","Rust","Response Evaluation Metrics","Model Tools","Benchmark Reality Check (real-world tool use)","Platforms","AI Services","Observability \u0026 Reliability Operations","Open source","LLM-as-Judge Evaluation","Scanners","LLMOps"],"readme":"\u003ch1 align=\"center\" style=\"border-bottom: none\"\u003e\n    \u003cdiv\u003e\n        \u003ca href=\"https://www.comet.com/site/products/opik/?from=llm\u0026utm_source=opik\u0026utm_medium=github\u0026utm_content=header_img\u0026utm_campaign=opik\"\u003e\u003cpicture\u003e\n            \u003csource media=\"(prefers-color-scheme: dark)\" srcset=\"/apps/opik-documentation/documentation/static/img/logo-dark-mode.svg\"\u003e\n            \u003csource media=\"(prefers-color-scheme: light)\" srcset=\"/apps/opik-documentation/documentation/static/img/opik-logo.svg\"\u003e\n            \u003cimg alt=\"Comet Opik logo\" src=\"/apps/opik-documentation/documentation/static/img/opik-logo.svg\" width=\"200\" /\u003e\n        \u003c/picture\u003e\u003c/a\u003e\n        \u003cbr\u003e\n        Opik\n    \u003c/div\u003e\n    Open source LLM evaluation framework\u003cbr\u003e\n\u003c/h1\u003e\n\n\u003cp align=\"center\"\u003e\nFrom RAG chatbots to code assistants to complex agentic pipelines and beyond, build LLM systems that run better, faster, and cheaper with tracing, evaluations, and dashboards.\n\u003c/p\u003e\n\n\u003cdiv align=\"center\"\u003e\n\n[![Python SDK](https://img.shields.io/pypi/v/opik)](https://pypi.org/project/opik/)\n[![License](https://img.shields.io/github/license/comet-ml/opik)](https://github.com/comet-ml/opik/blob/main/LICENSE)\n[![Build](https://github.com/comet-ml/opik/actions/workflows/build_apps.yml/badge.svg)](https://github.com/comet-ml/opik/actions/workflows/build_apps.yml)\n\u003ca target=\"_blank\" href=\"https://colab.research.google.com/github/comet-ml/opik/blob/master/apps/opik-documentation/documentation/docs/cookbook/opik_quickstart.ipynb\"\u003e\n\n  \u003c!-- \u003cimg src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open Quickstart In Colab\"/\u003e --\u003e\n\u003c/a\u003e\n\n\u003c/div\u003e\n\n\u003cp align=\"center\"\u003e\n    \u003ca href=\"https://www.comet.com/site/products/opik/?from=llm\u0026utm_source=opik\u0026utm_medium=github\u0026utm_content=website_button\u0026utm_campaign=opik\"\u003e\u003cb\u003eWebsite\u003c/b\u003e\u003c/a\u003e •\n    \u003ca href=\"https://chat.comet.com\"\u003e\u003cb\u003eSlack community\u003c/b\u003e\u003c/a\u003e •\n    \u003ca href=\"https://x.com/Cometml\"\u003e\u003cb\u003eTwitter\u003c/b\u003e\u003c/a\u003e •\n    \u003ca href=\"https://www.comet.com/docs/opik/?from=llm\u0026utm_source=opik\u0026utm_medium=github\u0026utm_content=docs_button\u0026utm_campaign=opik\"\u003e\u003cb\u003eDocumentation\u003c/b\u003e\u003c/a\u003e\n\u003c/p\u003e\n\n![Opik thumbnail](readme-thumbnail.png)\n\n## 🚀 What is Opik?\n\nOpik is an open-source platform for evaluating, testing and monitoring LLM applications. Built by [Comet](https://www.comet.com?from=llm\u0026utm_source=opik\u0026utm_medium=github\u0026utm_content=what_is_opik_link\u0026utm_campaign=opik).\n\n\u003cbr\u003e\n\nYou can use Opik for:\n* **Development:**\n\n  * **Tracing:** Track all LLM calls and traces during development and production ([Quickstart](https://www.comet.com/docs/opik/quickstart/?from=llm\u0026utm_source=opik\u0026utm_medium=github\u0026utm_content=quickstart_link\u0026utm_campaign=opik), [Integrations](https://www.comet.com/docs/opik/tracing/integrations/overview/?from=llm\u0026utm_source=opik\u0026utm_medium=github\u0026utm_content=integrations_link\u0026utm_campaign=opik)\n\n  * **Annotations:** Annotate your LLM calls by logging feedback scores using the [Python SDK](https://www.comet.com/docs/opik/tracing/annotate_traces/#annotating-traces-and-spans-using-the-sdk?from=llm\u0026utm_source=opik\u0026utm_medium=github\u0026utm_content=sdk_link\u0026utm_campaign=opik) or the [UI](https://www.comet.com/docs/opik/tracing/annotate_traces/#annotating-traces-through-the-ui?from=llm\u0026utm_source=opik\u0026utm_medium=github\u0026utm_content=ui_link\u0026utm_campaign=opik).\n\n  * **Playground:**: Try out different prompts and models in the [prompt playground](https://www.comet.com/docs/opik/evaluation/playground/?from=llm\u0026utm_source=opik\u0026utm_medium=github\u0026utm_content=playground_link\u0026utm_campaign=opik)\n\n* **Evaluation**: Automate the evaluation process of your LLM application:\n\n    * **Datasets and Experiments**: Store test cases and run experiments ([Datasets](https://www.comet.com/docs/opik/evaluation/manage_datasets/?from=llm\u0026utm_source=opik\u0026utm_medium=github\u0026utm_content=datasets_link\u0026utm_campaign=opik), [Evaluate your LLM Application](https://www.comet.com/docs/opik/evaluation/evaluate_your_llm/?from=llm\u0026utm_source=opik\u0026utm_medium=github\u0026utm_content=eval_link\u0026utm_campaign=opik))\n\n    * **LLM as a judge metrics**: Use Opik's LLM as a judge metric for complex issues like [hallucination detection](https://www.comet.com/docs/opik/evaluation/metrics/hallucination/?from=llm\u0026utm_source=opik\u0026utm_medium=github\u0026utm_content=hallucination_link\u0026utm_campaign=opik), [moderation](https://www.comet.com/docs/opik/evaluation/metrics/moderation/?from=llm\u0026utm_source=opik\u0026utm_medium=github\u0026utm_content=moderation_link\u0026utm_campaign=opik) and RAG evaluation ([Answer Relevance](https://www.comet.com/docs/opik/evaluation/metrics/answer_relevance/?from=llm\u0026utm_source=opik\u0026utm_medium=github\u0026utm_content=alex_link\u0026utm_campaign=opik), [Context Precision](https://www.comet.com/docs/opik/evaluation/metrics/context_precision/?from=llm\u0026utm_source=opik\u0026utm_medium=github\u0026utm_content=context_link\u0026utm_campaign=opik)\n\n    * **CI/CD integration**: Run evaluations as part of your CI/CD pipeline using our [PyTest integration](https://www.comet.com/docs/opik/testing/pytest_integration/?from=llm\u0026utm_source=opik\u0026utm_medium=github\u0026utm_content=pytest_link\u0026utm_campaign=opik)\n\n* **Production Monitoring**:\n    \n    * **Log all your production traces**: Opik has been designed to support high volumes of traces, making it easy to monitor your production applications. Even small deployments can ingest more than 40 million traces per day!\n    \n    * **Monitoring dashboards**: Review your feedback scores, trace count and tokens over time in the [Opik Dashboard](https://www.comet.com/docs/opik/self-host/opik_dashboard/?from=llm\u0026utm_source=opik\u0026utm_medium=github\u0026utm_content=dashboard_link\u0026utm_campaign=opik).\n\n    * **Online evaluation metrics**: Easily score all your production traces using LLM as a Judge metrics and identify any issues with your production LLM application thanks to [Opik's online evaluation metrics](https://www.comet.com/docs/opik/production/rules/?from=llm\u0026utm_source=opik\u0026utm_medium=github\u0026utm_content=dashboard_link\u0026utm_campaign=opik)\n\n\u003e [!TIP]  \n\u003e If you are looking for features that Opik doesn't have today, please raise a new [Feature request](https://github.com/comet-ml/opik/issues/new/choose) 🚀\n\n\u003cbr\u003e\n\n## 🛠️ Installation\nOpik is available as a fully open source local installation or using Comet.com as a hosted solution.\nThe easiest way to get started with Opik is by creating a free Comet account at [comet.com](https://www.comet.com/signup?from=llm\u0026utm_source=opik\u0026utm_medium=github\u0026utm_content=install\u0026utm_campaign=opik).\n\nIf you'd like to self-host Opik, you can do so by cloning the repository and starting the platform using Docker Compose:\n\n```bash\n# Clone the Opik repository\ngit clone https://github.com/comet-ml/opik.git\n\n# Navigate to the opik/deployment/docker-compose directory\ncd opik/deployment/docker-compose\n\n# Start the Opik platform\ndocker compose up --detach\n\n# You can now visit http://localhost:5173 on your browser!\n```\n\nFor more information about the different deployment options, please see our deployment guides:\n\n| Installation methods | Docs link |\n| ------------------- | --------- |\n| Local instance | [![Local Deployment](https://img.shields.io/badge/Local%20Deployments-%232496ED?style=flat\u0026logo=docker\u0026logoColor=white)](https://www.comet.com/docs/opik/self-host/local_deployment?from=llm\u0026utm_source=opik\u0026utm_medium=github\u0026utm_content=self_host_link\u0026utm_campaign=opik)\n| Kubernetes | [![Kubernetes](https://img.shields.io/badge/Kubernetes-%23326ce5.svg?\u0026logo=kubernetes\u0026logoColor=white)](https://www.comet.com/docs/opik/self-host/kubernetes/#kubernetes-installation?from=llm\u0026utm_source=opik\u0026utm_medium=github\u0026utm_content=kubernetes_link\u0026utm_campaign=opik)\n\n\n## 🏁 Get Started\n\nTo get started, you will need to first install the Python SDK:\n\n```bash\npip install opik\n```\n\nOnce the SDK is installed, you can configure it by running the `opik configure` command:\n\n```bash\nopik configure\n```\n\nThis will allow you to configure Opik locally by setting the correct local server address or if you're using the Cloud platform by setting the API Key\n\n\u003e [!TIP]  \n\u003e You can also call the `opik.configure(use_local=True)` method from your Python code to configure the SDK to run on the local installation.\n\nYou are now ready to start logging traces using the [Python SDK](https://www.comet.com/docs/opik/python-sdk-reference/?from=llm\u0026utm_source=opik\u0026utm_medium=github\u0026utm_content=sdk_link2\u0026utm_campaign=opik).\n\n### 📝 Logging Traces\n\nThe easiest way to get started is to use one of our integrations. Opik supports:\n\n| Integration | Description                                                                  | Documentation                                                                                                                                                      | Try in Colab                                                                                                                                                                                                                      |\n|-------------|------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| OpenAI      | Log traces for all OpenAI LLM calls                                          | [Documentation](https://www.comet.com/docs/opik/tracing/integrations/openai/?utm_source=opik\u0026utm_medium=github\u0026utm_content=openai_link\u0026utm_campaign=opik)          | [![Open Quickstart In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/comet-ml/opik/blob/master/apps/opik-documentation/documentation/docs/cookbook/openai.ipynb)      |\n| LiteLLM     | Call any LLM model using the OpenAI format                                   | [Documentation](/tracing/integrations/litellm.md)                                                                                                                  | [![Open Quickstart In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/comet-ml/opik/blob/master/apps/opik-documentation/documentation/docs/cookbook/litellm.ipynb)     |\n| LangChain   | Log traces for all LangChain LLM calls                                       | [Documentation](https://www.comet.com/docs/opik/tracing/integrations/langchain/?utm_source=opik\u0026utm_medium=github\u0026utm_content=langchain_link\u0026utm_campaign=opik)    | [![Open Quickstart In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/comet-ml/opik/blob/master/apps/opik-documentation/documentation/docs/cookbook/langchain.ipynb)   |\n| Haystack    | Log traces for all Haystack calls                                            | [Documentation](https://www.comet.com/docs/opik/tracing/integrations/haystack/?utm_source=opik\u0026utm_medium=github\u0026utm_content=haystack_link\u0026utm_campaign=opik)      | [![Open Quickstart In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/comet-ml/opik/blob/master/apps/opik-documentation/documentation/docs/cookbook/haystack.ipynb)    |\n| Anthropic   | Log traces for all Anthropic LLM calls                                       | [Documentation](https://www.comet.com/docs/opik/tracing/integrations/anthropic?utm_source=opik\u0026utm_medium=github\u0026utm_content=anthropic_link\u0026utm_campaign=opik)     | [![Open Quickstart In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/comet-ml/opik/blob/master/apps/opik-documentation/documentation/docs/cookbook/anthropic.ipynb)   |\n| Bedrock     | Log traces for all Bedrock LLM calls                                         | [Documentation](https://www.comet.com/docs/opik/tracing/integrations/bedrock?utm_source=opik\u0026utm_medium=github\u0026utm_content=bedrock_link\u0026utm_campaign=opik)         | [![Open Quickstart In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/comet-ml/opik/blob/master/apps/opik-documentation/documentation/docs/cookbook/bedrock.ipynb)     |\n| CrewAI      | Log traces for all CrewAI calls                                              | [Documentation](https://www.comet.com/docs/opik/tracing/integrations/crewai?utm_source=opik\u0026utm_medium=github\u0026utm_content=crewai_link\u0026utm_campaign=opik)           | [![Open Quickstart In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/comet-ml/opik/blob/master/apps/opik-documentation/documentation/docs/cookbook/crewai.ipynb)      |\n| DeepSeek    | Log traces for all DeepSeek LLM calls                                        | [Documentation](https://www.comet.com/docs/opik/tracing/integrations/deepseek?utm_source=opik\u0026utm_medium=github\u0026utm_content=deepseek_link\u0026utm_campaign=opik)       | |\n| DSPy        | Log traces for all DSPy runs                                                 | [Documentation](https://www.comet.com/docs/opik/tracing/integrations/dspy?utm_source=opik\u0026utm_medium=github\u0026utm_content=dspy_link\u0026utm_campaign=opik)               | [![Open Quickstart In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/comet-ml/opik/blob/master/apps/opik-documentation/documentation/docs/cookbook/dspy.ipynb)        |\n| Gemini      | Log traces for all Gemini LLM calls                                          | [Documentation](https://www.comet.com/docs/opik/tracing/integrations/gemini?utm_source=opik\u0026utm_medium=github\u0026utm_content=gemini_link\u0026utm_campaign=opik)           | [![Open Quickstart In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/comet-ml/opik/blob/master/apps/opik-documentation/documentation/docs/cookbook/gemini.ipynb)      |\n| Groq        | Log traces for all Groq LLM calls                                            | [Documentation](https://www.comet.com/docs/opik/tracing/integrations/groq?utm_source=opik\u0026utm_medium=github\u0026utm_content=groq_link\u0026utm_campaign=opik)               | [![Open Quickstart In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/comet-ml/opik/blob/master/apps/opik-documentation/documentation/docs/cookbook/groq.ipynb)        |\n| Guardrails  | Log traces for all Guardrails validations                                    | [Documentation](https://www.comet.com/docs/opik/tracing/integrations/guardrails/?utm_source=opik\u0026utm_medium=github\u0026utm_content=guardrails_link\u0026utm_campaign=opik)    | [![Open Quickstart In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/comet-ml/opik/blob/master/apps/opik-documentation/documentation/docs/cookbook/guardrails-ai.ipynb)   |\n| Instructor  | Log traces for all LLM calls made with Instructor                            | [Documentation](https://www.comet.com/docs/opik/tracing/integrations/instructor/?utm_source=opik\u0026utm_medium=github\u0026utm_content=instructor_link\u0026utm_campaign=opik)    | [![Open Quickstart In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/comet-ml/opik/blob/master/apps/opik-documentation/documentation/docs/cookbook/instructor.ipynb)   |\n| LangGraph   | Log traces for all LangGraph executions                                      | [Documentation](https://www.comet.com/docs/opik/tracing/integrations/langgraph/?utm_source=opik\u0026utm_medium=github\u0026utm_content=langchain_link\u0026utm_campaign=opik)    | [![Open Quickstart In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/comet-ml/opik/blob/master/apps/opik-documentation/documentation/docs/cookbook/langgraph.ipynb)   |\n| LlamaIndex  | Log traces for all LlamaIndex LLM calls                                      | [Documentation](https://www.comet.com/docs/opik/tracing/integrations/llama_index?utm_source=opik\u0026utm_medium=github\u0026utm_content=llama_index_link\u0026utm_campaign=opik) | [![Open Quickstart In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/comet-ml/opik/blob/master/apps/opik-documentation/documentation/docs/cookbook/llama-index.ipynb) |\n| Ollama      | Log traces for all Ollama LLM calls                                          | [Documentation](https://www.comet.com/docs/opik/tracing/integrations/ollama?utm_source=opik\u0026utm_medium=github\u0026utm_content=ollama_link\u0026utm_campaign=opik)           | [![Open Quickstart In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/comet-ml/opik/blob/master/apps/opik-documentation/documentation/docs/cookbook/ollama.ipynb)      |\n| Predibase   | Fine-tune and serve open-source Large Language Models                        | [Documentation](https://www.comet.com/docs/opik/tracing/integrations/predibase?utm_source=opik\u0026utm_medium=github\u0026utm_content=predibase_link\u0026utm_campaign=opik)     | [![Open Quickstart In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/comet-ml/opik/blob/master/apps/opik-documentation/documentation/docs/cookbook/predibase.ipynb)   |\n| Ragas       | Evaluation framework for your Retrieval Augmented Generation (RAG) pipelines | [Documentation](https://www.comet.com/docs/opik/tracing/integrations/ragas?utm_source=opik\u0026utm_medium=github\u0026utm_content=ragas_link\u0026utm_campaign=opik)             | [![Open Quickstart In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/comet-ml/opik/blob/master/apps/opik-documentation/documentation/docs/cookbook/ragas.ipynb)       |\n| watsonx     | Log traces for all watsonx LLM calls                                         | [Documentation](https://www.comet.com/docs/opik/tracing/integrations/watsonx?utm_source=opik\u0026utm_medium=github\u0026utm_content=watsonx_link\u0026utm_campaign=opik)         | [![Open Quickstart In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/comet-ml/opik/blob/master/apps/opik-documentation/documentation/docs/cookbook/watsonx.ipynb)     |\n\n\u003e [!TIP]  \n\u003e If the framework you are using is not listed above, feel free to [open an issue](https://github.com/comet-ml/opik/issues) or submit a PR with the integration.\n\nIf you are not using any of the frameworks above, you can also use the `track` function decorator to [log traces](https://www.comet.com/docs/opik/tracing/log_traces/?from=llm\u0026utm_source=opik\u0026utm_medium=github\u0026utm_content=traces_link\u0026utm_campaign=opik):\n\n```python\nimport opik\n\nopik.configure(use_local=True) # Run locally\n\n@opik.track\ndef my_llm_function(user_question: str) -\u003e str:\n    # Your LLM code here\n\n    return \"Hello\"\n```\n\n\u003e [!TIP]  \n\u003e The track decorator can be used in conjunction with any of our integrations and can also be used to track nested function calls.\n\n### 🧑‍⚖️ LLM as a Judge metrics\n\nThe Python Opik SDK includes a number of LLM as a judge metrics to help you evaluate your LLM application. Learn more about it in the [metrics documentation](https://www.comet.com/docs/opik/evaluation/metrics/overview/?from=llm\u0026utm_source=opik\u0026utm_medium=github\u0026utm_content=metrics_2_link\u0026utm_campaign=opik).\n\nTo use them, simply import the relevant metric and use the `score` function:\n\n```python\nfrom opik.evaluation.metrics import Hallucination\n\nmetric = Hallucination()\nscore = metric.score(\n    input=\"What is the capital of France?\",\n    output=\"Paris\",\n    context=[\"France is a country in Europe.\"]\n)\nprint(score)\n```\n\nOpik also includes a number of pre-built heuristic metrics as well as the ability to create your own. Learn more about it in the [metrics documentation](https://www.comet.com/docs/opik/evaluation/metrics/overview?from=llm\u0026utm_source=opik\u0026utm_medium=github\u0026utm_content=metrics_3_link\u0026utm_campaign=opik).\n\n### 🔍 Evaluating your LLM Application\n\nOpik allows you to evaluate your LLM application during development through [Datasets](https://www.comet.com/docs/opik/evaluation/manage_datasets/?from=llm\u0026utm_source=opik\u0026utm_medium=github\u0026utm_content=datasets_2_link\u0026utm_campaign=opik) and [Experiments](https://www.comet.com/docs/opik/evaluation/evaluate_your_llm/?from=llm\u0026utm_source=opik\u0026utm_medium=github\u0026utm_content=experiments_link\u0026utm_campaign=opik).\n\nYou can also run evaluations as part of your CI/CD pipeline using our [PyTest integration](https://www.comet.com/docs/opik/testing/pytest_integration/?from=llm\u0026utm_source=opik\u0026utm_medium=github\u0026utm_content=pytest_2_link\u0026utm_campaign=opik).\n\n## ⭐ Star Us on GitHub\n\nIf you find Opik useful, please consider giving us a star! Your support helps us grow our community and continue improving the product.\n\n\u003cimg src=\"https://github.com/user-attachments/assets/ffc208bb-3dc0-40d8-9a20-8513b5e4a59d\" alt=\"Opik GitHub Star History\" width=\"600\"/\u003e\n\n\n\n## 🤝 Contributing\n\nThere are many ways to contribute to Opik:\n\n* Submit [bug reports](https://github.com/comet-ml/opik/issues) and [feature requests](https://github.com/comet-ml/opik/issues)\n* Review the documentation and submit [Pull Requests](https://github.com/comet-ml/opik/pulls) to improve it\n* Speaking or writing about Opik and [letting us know](https://chat.comet.com)\n* Upvoting [popular feature requests](https://github.com/comet-ml/opik/issues?q=is%3Aissue+is%3Aopen+label%3A%22enhancement%22) to show your support\n\nTo learn more about how to contribute to Opik, please see our [contributing guidelines](CONTRIBUTING.md).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcomet-ml%2Fopik","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcomet-ml%2Fopik","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcomet-ml%2Fopik/lists"}