https://github.com/suites-dev/benchmark
https://github.com/suites-dev/benchmark
Last synced: 8 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/suites-dev/benchmark
- Owner: suites-dev
- License: mit
- Created: 2023-12-01T14:52:57.000Z (over 2 years ago)
- Default Branch: master
- Last Pushed: 2024-06-08T18:30:12.000Z (about 2 years ago)
- Last Synced: 2024-12-27T00:14:27.730Z (over 1 year ago)
- Language: JavaScript
- Homepage:
- Size: 357 KB
- Stars: 2
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE.md
Awesome Lists containing this project
README
# TypeScript Compiler Benchmark Analysis
## Overview
This document provides a detailed analysis of TypeScript compiler performance across various Node.js versions.
Benchmarks were conducted on 1000 test suites in Automock and NestJS environments.
**Special thanks to [@soryy708](https://github.com/soryy708) for his contribution!**
### Testing Environment
- **Machine**: GitHub Native Workflow (Ubuntu Latest)
- **Jest Version**: 29.7.0
- **TypeScript Version**: 5.3.2
- **Relevant Dependencies**:
- @automock/adapters.nestjs: ^2.0.0
- @automock/jest: ^2.0.0
- @golevelup/ts-jest: ^0.4.0
- @nestjs/common: ^10.2.10
- @nestjs/core: ^10.2.10
- @nestjs/testing: ^10.2.10
- @swc/jest: ^0.2.29
### Benchmark Results
#### Node v20
| Compiler | automock (ms) | NestJS (ms) |
|------------------|---------------|-------------|
| **tsc** | 174569 | 245705 |
| **tsc-isolated** | 99139 | 134079 |
| **swc** | 90483 | 129110 |
#### Node v18
| Compiler | automock (ms) | NestJS (ms) |
|------------------|---------------|-------------|
| **tsc** | 321784 | 431411 |
| **tsc-isolated** | 230833 | 303219 |
| **swc** | 226546 | 303635 |
#### Node v16
| Compiler | automock (ms) | NestJS (ms) |
|------------------|---------------|-------------|
| **tsc** | 394804 | 548740 |
| **tsc-isolated** | 282173 | 326732 |
| **swc** | 224413 | 349720 |
### Performance Comparison (Compared to Fastest - SWC)
#### Node v20
| Environment | tsc (Slower by %) | tsc-isolated (Slower by %) |
|--------------|-------------------|----------------------------|
| **automock** | +93.1% | +9.6% |
| **NestJS** | +90.4% | +3.8% |
#### Node v18
| Environment | tsc (Slower by %) | tsc-isolated (Slower by %) |
|--------------|-------------------|----------------------------|
| **automock** | +42.0% | +1.9% |
| **NestJS** | +42.1% | -0.1% |
#### Node v16
| Environment | tsc (Slower by %) | tsc-isolated (Slower by %) |
|--------------|-------------------|----------------------------|
| **automock** | +75.9% | +25.7% |
| **NestJS** | +56.9% | -8.0% |
### Environment Speed Comparison (Automock vs. Nestjs)
#### Node v20
| Compiler | Speed Increase (%) |
|------------------|--------------------|
| **tsc** | +28.9% |
| **tsc-isolated** | +26.0% |
| **swc** | +23.2% |
#### Node v18
| Compiler | Speed Increase (%) |
|------------------|--------------------|
| **tsc** | +25.4% |
| **tsc-isolated** | +23.9% |
| **swc** | +25.4% |
#### Node v16
| Compiler | Speed Increase (%) |
|------------------|--------------------|
| **tsc** | +28.0% |
| **tsc-isolated** | +13.6% |
| **swc** | +35.8% |
### Key Insights
- **Performance Across Node Versions**:
- All compilers show decreased performance with older Node.js versions.
- The impact is more pronounced in the NestJS environment.
- **Comparing Compilers**:
- `swc` consistently outperforms `tsc` and `tsc-isolated` in all Node versions.
- **tsc vs. tsc-isolated**:
- `tsc-isolated` demonstrates better performance than `tsc` across all Node versions.
- **Automock vs. Nestjs**:
- All compilers are faster in Automock than in NestJS, indicating higher complexity or resource demands
in NestJS.
### Speed Difference Between Automock and NestJS
- **Node v20 Analysis**:
- All compilers exhibited significantly faster performance in the Automock environment compared to NestJS.
- `tsc` showed a 28.9% speed increase in Automock, while `tsc-isolated` and `swc` displayed increases of 26.0% and
23.2%, respectively.
- This suggests that Automock is substantially less resource-intensive than NestJS.
- **Node v18 Insights**:
- The trend of faster performance in Automock persists in Node v18.
- `tsc` was 25.4% faster in Automock, closely followed by `swc` with a similar 25.4% increase, and `tsc-isolated`
showed a 23.9% increase.
- These results further underline the efficiency and lower complexity of Automock compared to NestJS.
- **Node v16 Observations**:
- The performance gap between Automock and NestJS remains notable in Node v16.
- `tsc` registered a 28.0% speed increase in Automock, while `tsc-isolated` had a 13.6% increase. Notably, `swc`
showed the most significant difference with a 35.8% increase.
- This indicates that Automock maintains its efficiency advantage over NestJS even in older Node.js environments.
In summary, across all tested Node.js versions, Automock consistently outperforms NestJS in terms of speed for all
compilers. The difference is marked and suggests that Automock is a more efficient environment, possibly due to lower
complexity or better resource optimization.
### Visualizations
