{"id":18897024,"url":"https://github.com/scale3-labs/evaluate-llm","last_synced_at":"2026-01-07T05:37:42.125Z","repository":{"id":248192324,"uuid":"828022696","full_name":"Scale3-Labs/evaluate-llm","owner":"Scale3-Labs","description":"Example React component for scoring LLM responses with Langtrace.ai","archived":false,"fork":false,"pushed_at":"2024-09-18T02:40:49.000Z","size":22991,"stargazers_count":2,"open_issues_count":2,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-04-15T02:40:21.851Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Scale3-Labs.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-07-12T23:01:22.000Z","updated_at":"2024-07-24T16:31:57.000Z","dependencies_parsed_at":"2024-11-08T08:37:44.850Z","dependency_job_id":null,"html_url":"https://github.com/Scale3-Labs/evaluate-llm","commit_stats":null,"previous_names":["scale3-labs/evaluate-llm"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/Scale3-Labs/evaluate-llm","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Scale3-Labs%2Fevaluate-llm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Scale3-Labs%2Fevaluate-llm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Scale3-Labs%2Fevaluate-llm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Scale3-Labs%2Fevaluate-llm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Scale3-Labs","download_url":"https://codeload.github.com/Scale3-Labs/evaluate-llm/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Scale3-Labs%2Fevaluate-llm/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28232982,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2026-01-07T02:00:05.975Z","response_time":58,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-08T08:36:02.879Z","updated_at":"2026-01-07T05:37:42.113Z","avatar_url":"https://github.com/Scale3-Labs.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# evaluate-llm\n\n![NextJS Project Example](assets/react_eval.gif)\n\n![LLM React Evaluation Component](assets/llm_eval_component.png)\n\nExample NextJS project with React component for scoring LLM responses with Langtrace.ai.\n\n## How it Works\n\nThe Feedback component allows users to provide feedback on LLM responses by simply clicking the thumbs up or thumbs down icon. When a user interacts with the component, it sends feedback to Langtrace.ai, which helps track and evaluate the performance of LLM applications. The feedback includes user-specific details like userId, userScore, traceId, and spanId, ensuring precise tracking and analysis.\n\nThe backend function handles the feedback submission, sending the data to Langtrace.ai's API using the Langtrace TypeScript SDK. This seamless integration ensures that user feedback is recorded and can be analyzed for improving the LLM's responses.\n\n## How to Get Started\n\n### Step 1: Sign Up at Langtrace.ai\n\nVisit Langtrace.ai and sign up for an account. Once registered, create a new project and generate an API key for your project.\n\n### Step 2: Add your API Keys\n\nIn the `/app/api/upload/route.ts` file add your Langtrace API key and your OpenAI API key.\n\n### Step 3: Run the NextJS Application\n\nRun `npm i` to install all dependencies.\n\nRun `npm run dev` to run the NextJS project. You can view the application at `http://localhost:3000`\n\n### Step 4: Test the upload and feedback capabilites.\n\nUpload a photo of your favorite meal by selecting a file or dropping in a jpeg. Then select `Upload Meal`.\n\nAfter a few seconds, you will see the nutrition information for the uploaded meal. You can view the traces, metrics, and evaluations by navigating to langtrace.ai and selecting the project whose API key you are using.\n\nFinally, click on either the Thumbs Up or Thumbs Down icon within the web application to evaluate the LLM response. You can view the user feedback by navigating to the `/annotations` page within Langtrace.ai and viewing the `User Score` column.\n\n### Note\n\nIf you are looking to use any features of this project they are located in the following:\n\n- React component for sending feedback: `/components/shared/evals.tsx`\n- API route for calling OpenAI API and sending trace to Langtrace Cloud: `/app/api/upload/route.ts`\n- API route for sending user feedback: `/app/api/feedback/route.ts`\n\nEvaluation in action:\n![evaluated response](assets/evaluated_response.png)\n\nEvaluation seen in Langtrace.ai:\n![langtrace evalulation](assets/langtrace_evaluation.png)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fscale3-labs%2Fevaluate-llm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fscale3-labs%2Fevaluate-llm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fscale3-labs%2Fevaluate-llm/lists"}