Ecosyste.ms: Awesome

https://github.com/agentops-ai/agentops

Python SDK for AI agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most LLMs and agent frameworks like CrewAI, Langchain, and Autogen

agent agentops ai anthropic autogen cost-estimation crewai evals evaluation-metrics groq langchain llm mistral ollama openai

Last synced: 17 Dec 2024

https://github.com/AgentOps-AI/agentops

Python SDK for AI agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most LLMs and agent frameworks like CrewAI, Langchain, and Autogen

agent agentops ai anthropic autogen cost-estimation crewai evals evaluation-metrics groq langchain llm mistral ollama openai

Last synced: 30 Oct 2024

https://github.com/lmnr-ai/lmnr

Laminar - open-source all-in-one platform for engineering AI products. Traces, Evals, Datasets, Labels. YC S24.

agents ai ai-observability aiops analytics developer-tools evals evaluation llm-evaluation llm-observability llm-workflow llmops monitoring observability open-source pipeline-builder rag rust-lang self-hosted

Last synced: 20 Dec 2024

https://github.com/superlinear-ai/raglite

🥤 RAGLite is a Python toolkit for Retrieval-Augmented Generation (RAG) with PostgreSQL or SQLite

chainlit colbert evals hybrid-search late-chunking late-interaction llm markdown pdf pgvector postgres postgresql query-adapter rag reranker reranking retrieval-augmented-generation sqlite tsvector vector-search

Last synced: 17 Dec 2024

https://github.com/mattpocock/evalite

Test your LLM-powered apps with TypeScript. No API key required.

ai evals typescript

Last synced: 21 Dec 2024

https://github.com/aianytime/rag-evaluator

A library for evaluating Retrieval-Augmented Generation (RAG) systems (The traditional ways).

eval evals rag

Last synced: 18 Dec 2024

https://github.com/dustalov/evalica

Evalica, your favourite evaluation toolkit

arena bradley-terry elo evalica evals evaluation hacktoberfest leaderboard library llm pagerank pairwise-comparison pyo3 python ranking rating rust serbia statistics winrate

Last synced: 07 Nov 2024

https://github.com/nirantk/rag-to-riches

evals rag search

Last synced: 14 Oct 2024

https://github.com/openlayer-ai/templates

Our curated collection of templates. Use these patterns to set up your AI projects for evaluation with Openlayer.

ai evals examples

Last synced: 10 Nov 2024

https://github.com/gokayfem/dspy-ollama-colab

dspy with ollama and llamacpp on google colab

agents colab-notebook dspy evals evaluation llamacpp llm ollama vlm

Last synced: 02 Dec 2024