{"id":16949799,"url":"https://github.com/yuyz0112/relia","last_synced_at":"2025-04-30T22:24:34.299Z","repository":{"id":243272688,"uuid":"810236897","full_name":"Yuyz0112/relia","owner":"Yuyz0112","description":"Find the Best LLM for Your Needs through E2E Testing","archived":false,"fork":false,"pushed_at":"2024-06-12T01:39:01.000Z","size":165,"stargazers_count":80,"open_issues_count":0,"forks_count":2,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-04-20T23:56:07.008Z","etag":null,"topics":["ai","e2e","llm"],"latest_commit_sha":null,"homepage":"https://relia.dev/","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Yuyz0112.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"docs/contributing.en-US.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-06-04T10:03:45.000Z","updated_at":"2025-04-06T04:32:06.000Z","dependencies_parsed_at":"2024-10-27T12:37:25.299Z","dependency_job_id":null,"html_url":"https://github.com/Yuyz0112/relia","commit_stats":null,"previous_names":["yuyz0112/relia"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Yuyz0112%2Frelia","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Yuyz0112%2Frelia/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Yuyz0112%2Frelia/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Yuyz0112%2Frelia/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Yuyz0112","download_url":"https://codeload.github.com/Yuyz0112/relia/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251790554,"owners_count":21644236,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","e2e","llm"],"created_at":"2024-10-13T21:56:06.149Z","updated_at":"2025-04-30T22:24:34.279Z","avatar_url":"https://github.com/Yuyz0112.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Relia\n\n[Try Relia](https://relia.dev/)\n\n[中文文档](./README.zh-CN.md)\n\nRelia is an E2E testing framework for LLMs, designed to help you build AI benchmarks tailored to your specific use cases.\n\nIt identifies the most suitable LLM model for your needs and ensures that model upgrades do not cause performance regressions through continuous testing.\n\nBuilt specifically for function calling (or \"tool use\") scenarios, which are at the core of agent-based AI applications.\n\n## Documents\n\n[Usage Guide](./docs/guide.en-US.md)\n\n[Self-Deployment](./docs/deployment.en-US.md)\n\n[How to Contribute](./docs/contributing.en-US.md)\n\n## Use Cases\n\n### Selecting the Most Suitable LLM\n\nWhen selecting models, identify the best LLM for your specific use case, ensuring high performance and cost efficiency.\n\n### Optimizing Prompts\n\nWhen developing an application, compare the results of multiple sets of prompts on the same model to understand the impact of different prompts and complete optimization.\n\n### Continuous Testing to Prevent Regressions\n\nAfter the application is released, continuously test different versions of the same model to avoid regressions during upgrades.\n\n## Roadmap\n\n- [ ] Enable customization of provider titles and suite titles in test reports for better organization and clarity.\n- [ ] Improve the efficiency and reliability of executing large-scale test plans.\n- [ ] Expand support to include more LLM providers.\n- [x] Develop a form UI for editing test plans, making it easier and more intuitive to create and manage tests.\n- [ ] Implement persistent storage for test plans and reports.\n- [ ] Allow custom scoring for different suites to better evaluate and compare the performance of test cases.\n\nFeel free to follow our project on [GitHub](https://github.com/Yuyz0112/relia), [X](https://x.com/Aryu0112), and [Bilibili](https://space.bilibili.com/489667127).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyuyz0112%2Frelia","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fyuyz0112%2Frelia","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyuyz0112%2Frelia/lists"}