https://github.com/vltpkg/benchmarks-semver
Benchmarking against `node-semver`
https://github.com/vltpkg/benchmarks-semver
Last synced: 4 months ago
JSON representation
Benchmarking against `node-semver`
- Host: GitHub
- URL: https://github.com/vltpkg/benchmarks-semver
- Owner: vltpkg
- Created: 2025-08-20T18:06:01.000Z (5 months ago)
- Default Branch: main
- Last Pushed: 2025-08-20T19:01:12.000Z (5 months ago)
- Last Synced: 2025-08-28T22:09:12.209Z (5 months ago)
- Language: JavaScript
- Size: 621 KB
- Stars: 1
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Semver Benchmarks
A comprehensive benchmark suite comparing semantic version libraries in Node.js, specifically:
- **`node-semver`** - The standard semver parser used by npm
- **`@vltpkg/semver`** - High-performance semver library by the vlt team
## Features
This benchmark suite tests the most common semver operations:
- **Parsing** - Converting version strings to version objects
- **Comparison** - Comparing two versions (compare, gt, lt, eq)
- **Satisfies** - Checking if a version satisfies a range
## Requirements
- Node.js 22+ (required by @vltpkg/semver)
- [vlt](https://vlt.sh) package manager
- [hyperfine](https://github.com/sharkdp/hyperfine) (optional, for detailed benchmarking)
## Installation
```bash
# Install dependencies using vlt
vlt install
# Or use the package script
vlr install-deps
```
## Usage
### Run Tests
First, verify that both libraries produce consistent results:
```bash
vlr test
```
### Run Basic Benchmarks
```bash
# Run the comprehensive benchmark suite
vlr benchmark
# Run individual benchmark categories
vlr benchmark:parsing
vlr benchmark:comparison
vlr benchmark:satisfies
```
### Run Benchmarks with Chart Generation
The new chart generation feature automatically creates performance tables and charts:
```bash
# Run benchmarks and generate charts/tables in README
vlr benchmark:generate
# Just collect benchmark data (saves to benchmark-results.json)
vlr benchmark:collect
# Generate charts from existing results
vlr benchmark:charts
```
### Run Hyperfine Benchmarks
For more detailed performance analysis with statistical data:
```bash
# Quick hyperfine benchmark
vlr benchmark:quick
# Full hyperfine benchmark suite (exports JSON and Markdown results)
vlr benchmark:hyperfine
# Run all benchmarks (tests + charts + hyperfine)
vlr benchmark:all
```
## Results
The hyperfine benchmarks will generate result files:
- `results-parsing.json/md` - Parsing operation results
- `results-comparison.json/md` - Comparison operation results
- `results-satisfies.json/md` - Satisfies operation results
- `results-combined.json/md` - Full benchmark suite results
## Expected Performance
Based on the vlt team's documentation, `@vltpkg/semver` should show:
- **40-50% faster** at parsing versions
- **15-20% faster** at parsing ranges
- **60-70% faster** at testing versions against ranges
## 🏆 Summary
**@vltpkg/semver** wins overall, taking 2/3 categories.
### Performance Highlights:
- **Parsing**: node-semver is 60% faster
- **Comparison**: @vltpkg/semver is 84% faster
- **Satisfies**: @vltpkg/semver is 10% faster
## 📊 Performance Results
| Operation | node-semver | @vltpkg/semver | Winner | Improvement |
|-----------|-------------|----------------|--------|-------------|
| Parsing | 131,968 ops/sec | 82,468 ops/sec | 🏆 node-semver | 60% |
| Comparison | 122,921 ops/sec | 225,656 ops/sec | 🏆 @vltpkg/semver | 84% |
| Satisfies | 17,860 ops/sec | 19,685 ops/sec | 🏆 @vltpkg/semver | 10% |
## 📈 Performance Comparison Chart
```
parsing │
node-semver │████████████████████████████████████████ 131,968
@vltpkg │█████████████████████████ 82,468
│
comparison │
node-semver │██████████████████████ 122,921
@vltpkg │████████████████████████████████████████ 225,656
│
satisfies │
node-semver │████████████████████████████████████ 17,860
@vltpkg │████████████████████████████████████████ 19,685
│
```
## 🔬 Detailed Results
### Parsing Benchmark
| Metric | node-semver | @vltpkg/semver |
|--------|-------------|----------------|
| Operations/sec | 131,968 | 82,468 |
| Mean time (ms) | 0.008 | 0.012 |
| Margin of error | ±0ms | ±0ms |
| Relative margin | ±0.97% | ±0.41% |
| Sample runs | 85 | 96 |
### Comparison Benchmark
| Metric | node-semver | @vltpkg/semver |
|--------|-------------|----------------|
| Operations/sec | 122,921 | 225,656 |
| Mean time (ms) | 0.008 | 0.004 |
| Margin of error | ±0ms | ±0ms |
| Relative margin | ±0.11% | ±0.13% |
| Sample runs | 98 | 97 |
### Satisfies Benchmark
| Metric | node-semver | @vltpkg/semver |
|--------|-------------|----------------|
| Operations/sec | 17,860 | 19,685 |
| Mean time (ms) | 0.056 | 0.051 |
| Margin of error | ±0ms | ±0.001ms |
| Relative margin | ±0.23% | ±1.37% |
| Sample runs | 101 | 95 |
## 🖥️ System Information
- **Node.js**: v24.1.0
- **Platform**: darwin (arm64)
- **node-semver**: v7.7.2
- **@vltpkg/semver**: v0.0.0-22
- **Test Date**: 2025-08-20, 2:10:09 p.m.
## Test Data
The benchmarks use a comprehensive set of test cases including:
**Versions:**
- Standard versions (1.0.0, 2.1.3)
- Prerelease versions (1.0.0-alpha.1, 1.2.3-beta.4)
- Build metadata (1.2.3+build.5)
- Complex prerelease strings
- Edge cases
**Ranges:**
- Caret ranges (^1.0.0)
- Tilde ranges (~1.2.3)
- Comparison ranges (>=1.2.7 <1.3.0)
- Hyphen ranges (1.2.3 - 2.3.4)
- Complex logical operators
## Project Structure
```
semver-benchmarks/
├── benchmarks/
│ ├── index.js # Main benchmark suite
│ ├── parsing.js # Parsing benchmarks
│ ├── comparison.js # Comparison benchmarks
│ └── satisfies.js # Satisfies benchmarks
├── test/
│ └── test.js # Compatibility tests
├── run-hyperfine.sh # Hyperfine benchmark script
├── package.json # Project configuration
└── README.md # This file
```
## Contributing
Feel free to add more test cases, additional libraries, or improve the benchmark methodology.