https://github.com/nexmoe/lm-speed
Help developers optimize AI application performance through comprehensive speed testing and analysis
https://github.com/nexmoe/lm-speed
llm openai speed test
Last synced: 2 months ago
JSON representation
Help developers optimize AI application performance through comprehensive speed testing and analysis
- Host: GitHub
- URL: https://github.com/nexmoe/lm-speed
- Owner: nexmoe
- License: mit
- Created: 2025-02-14T12:23:22.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2025-03-31T15:29:55.000Z (about 1 year ago)
- Last Synced: 2025-06-10T19:17:08.316Z (about 1 year ago)
- Topics: llm, openai, speed, test
- Language: TypeScript
- Homepage: https://lmspeed.net
- Size: 1.64 MB
- Stars: 57
- Watchers: 1
- Forks: 3
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
[δΈζ](README_zh.md) | English
# LM Speed - Simple LLM Speed Analysis Tool
Portal:
Provides precise and reliable OpenAI API performance testing solutions for AI application developers. Through multi-dimensional real-time data analysis, it helps users quickly identify performance bottlenecks and optimize model calling strategies. It also offers an intuitive ranking feature, allowing users to easily compare and select the most suitable models and service providers.
## Features
- π Real-time Performance Monitoring: Multi-dimensional real-time data display
- π Comprehensive Performance Evaluation: Core metrics including first token latency and response time
- π Data Visualization: Rich chart displays for intuitive performance trend understanding
- π Automated Testing: Five-round continuous stress testing for reliable data
- π One-click Reports: Automatic professional test report generation with export capabilities
## Quick Start
### Deploy with Docker Compose
```yaml
version: '3.8'
services:
app:
image: nexmoe/lmspeed:latest
ports:
- "8650:3000"
environment:
- DATABASE_URL=postgresql://postgres:postgres@db:5432/nexmoe
- NODE_ENV=production
depends_on:
- db
restart: always
db:
image: postgres:16
restart: always
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: nexmoe
volumes:
- postgres_data:/var/lib/postgresql/data
volumes:
postgres_data:
```
### Manual Deployment
1. Clone the repository
```bash
git clone https://github.com/yourusername/lm-speed.git
cd lm-speed
```
2. Install dependencies
```bash
npm install
```
3. Configure environment variables
```bash
cp .env.example .env
# Edit .env file with necessary configuration
```
4. Start the service
```bash
npm run dev
```
## URL Parameters Usage
LM Speed supports quick test initiation through URL parameters without manual form filling. You can use it in the following way:
```
https://lmspeed.net/?baseUrl=YOUR_BASE_URL&apiKey=YOUR_API_KEY&modelId=YOUR_MODEL_ID
```
Parameter Description:
- `baseUrl`: The base URL of the API service, e.g., `https://api.deepseek.com/v1`
- `apiKey`: Your API key
- `modelId`: The model ID to test
Example:
```
http://lmspeed.net/zh-CN?baseUrl=https://api.suanli.cn/v1&apiKey=sk-W0rpStc95T7JVYVwDYc29IyirjtpPPby6SozFMQr17m8KWeo&modelId=free:QwQ-32B
```
Notes:
1. All parameters are required; missing any parameter will prevent the test from starting automatically
2. For security reasons, it's recommended not to pass API keys directly in URLs, but rather use the form for manual input
3. If the URL contains special characters, make sure to URL encode them
## Three Core Pain Points Solved
### 1. Response Quality Opacity
Is DeepSeek's official API unusable? Is Silicon Flow's API too slow? When choosing LLM API services, developers often face challenges in evaluating service quality. Different service providers' APIs have significant performance variations, and there's a lack of objective evaluation standards. LM Speed provides standardized performance testing solutions, allowing you to accurately assess each API's actual performance before development.
### 2. Performance Fluctuation Monitoring
Unsure about the LLM API speed? Uncertain about supplier reliability? Traditional performance testing tools often only provide simple response time data, failing to comprehensively reflect API's actual performance. LM Speed employs five-round continuous stress testing + dynamic streaming monitoring mechanism, using tiktoken for precise token calculation, combined with response time analysis, automatically generating three-dimensional evaluation maps of maximum/minimum/average performance to help you fully understand API's performance characteristics.
### 3. Test Results Accumulation
Performance test data is often scattered and difficult to systematically accumulate and analyze. LM Speed provides one-click test report generation, automatically integrating performance metrics, test environment, and other key information, supporting report export and team sharing. It also offers historical data storage and trend analysis, helping teams establish a complete performance evaluation system.
## Key Value Created for Users
Data-driven decision support. Through comprehensive performance data analysis, helping you make wiser API selection decisions:
- **Real-time Performance Insights**: Real-time monitoring of TPoS (Tokens per Second), giving you complete visibility into API performance. Supports multi-dimensional real-time data display for intuitive performance trend understanding.
- **Comprehensive Evaluation System**: Covers core metrics including first token latency, response time, etc., providing the most comprehensive performance profile.
- **Visual Decision Support**: One-click professional test report generation, supporting multi-user real-time collaboration, saving an average of 80% decision time. Provides rich data visualization charts to assist team decision-making.
## Tech Stack
- **Frontend**:
- Next.js 15
- React 19
- TypeScript
- TailwindCSS
- Radix UI Components
- SWR for data fetching
- next-intl for internationalization
- next-themes for dark mode
- **Backend**:
- Next.js API Routes
- Drizzle ORM
- PostgreSQL
- OpenAI SDK
- tiktoken for token counting
- **Development**:
- ESLint
- TypeScript
- Drizzle Kit for database management
- TailwindCSS for styling
- **Deployment**:
- Docker
- Docker Compose
- PostgreSQL
## Contributing
Issues and Pull Requests are welcome! Before submitting a PR, please ensure:
1. Code follows project coding standards
2. Necessary tests are added
3. Related documentation is updated
## License
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details