{"id":21354355,"url":"https://github.com/samestrin/comparing-llm-api-performance","last_synced_at":"2025-03-16T05:21:46.651Z","repository":{"id":246079191,"uuid":"819171041","full_name":"samestrin/comparing-llm-api-performance","owner":"samestrin","description":"A simple Node.js script that compares LLM API performance using llm-interface.","archived":false,"fork":false,"pushed_at":"2024-06-26T17:57:14.000Z","size":3218,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-01-22T17:46:52.082Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/samestrin.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-06-24T01:12:51.000Z","updated_at":"2024-10-03T11:25:55.000Z","dependencies_parsed_at":"2024-06-25T19:12:14.498Z","dependency_job_id":"8665c37a-c2fd-47c7-b4a3-a81596838c8d","html_url":"https://github.com/samestrin/comparing-llm-api-performance","commit_stats":null,"previous_names":["samestrin/comparing-llm-api-performance"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/samestrin%2Fcomparing-llm-api-performance","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/samestrin%2Fcomparing-llm-api-performance/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/samestrin%2Fcomparing-llm-api-performance/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/samestrin%2Fcomparing-llm-api-performance/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/samestrin","download_url":"https://codeload.github.com/samestrin/comparing-llm-api-performance/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243828095,"owners_count":20354433,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-22T04:13:12.477Z","updated_at":"2025-03-16T05:21:46.610Z","avatar_url":"https://github.com/samestrin.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Comparing LLM API Performance\n\nThis repository contains a test script that sends the simple prompt to each API provider ten times, using a small and large model (via the `llm-interface` model aliases). The script measures the average response time, standard deviation of the response time, latency, and success rate. After each request, the script sleeps to prevent rate limit exceeded errors. (The default sleep is 1 second but is configurable.)\n\nThis is the example code for the **[Comparing 13 LLM Providers API Performance with Node.js: Latency and Response Times Across Models](https://dev.to/samestrin/comparing-13-llm-providers-api-performance-with-nodejs-latency-and-response-times-across-models-2ka4)** article.\n\n## Prerequisites\n\n- Node.js\n- npm (Node Package Manager)\n- [API keys](https://github.com/samestrin/llm-interface/blob/main/docs/APIKEYS.md) for the LLM providers you want to test\n\n## Installation\n\n1. Clone the repository:\n\n```sh\ngit clone https://github.com/samestrin/comparing-llm-api-performance.git\ncd comparing-llm-api-performance\n```\n\n2. Install the required npm packages:\n\n```sh\nnpm install llm-interface ping cli-progress dotenv\n```\n\n3. Create a `.env` file in the root directory and add your API keys:g\n\n```sh\nOPENAI_API_KEY=your_openai_api_key\nAI21_API_KEY=your_ai21_api_key\nANTHROPIC_API_KEY=your_anthropic_api_key\nCOHERE_API_KEY=your_cohere_api_key\nGEMINI_API_KEY=your_gemini_api_key\nGOOSE_API_KEY=your_goose_api_key\nGROQ_API_KEY=your_groq_api_key\nHUGGINGFACE_API_KEY=your_huggingface_api_key\nMISTRAL_API_KEY=your_mistral_api_key\nPERPLEXITY_API_KEY=your_perplexity_api_key\nREKA_API_KEY=your_reka_api_key\n```\n\n## Usage\n\n1. Run the performance test script:\n\n```sh\nnpm run test-llm-performance\n```\n\nor\n\n```sh\nnode testLLMPerformance.js\n```\n\n2. The script will append the average response time in milliseconds for each provider in CSV format to `results.csv`.\n\n3. You should now have a `results.csv` file in your current directory. Since CSV is a text-based format, you can open this file using any basic text editor. However, this will display the data in raw format without any table structure. For a more user-friendly view, you can use a freely available online spreadsheet like Google Sheets or Microsoft Excel Online.\n\n4. You should also have `sampleSmall.md` and `sampleLarge.md`. Markdown is also a text-based format, so you can open either file using any basic text editor. If prefer a markdown editor instead, StackEdit is a freely available online markdown editor that is easy to use.\n\n## LLM API Performance: Response Times (ms)\n\nTo see the results of my testing you can read [Comparing 13 LLM Providers API Performance with Node.js: Latency and Response Times Across Models](https://dev.to/samestrin/comparing-13-llm-providers-api-performance-with-nodejs-latency-and-response-times-across-models-2ka4).\n\n## License\n\nThis project is licensed under the MIT License - see the [LICENSE](/LICENSE) file for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsamestrin%2Fcomparing-llm-api-performance","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsamestrin%2Fcomparing-llm-api-performance","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsamestrin%2Fcomparing-llm-api-performance/lists"}