https://github.com/cipher982/py-perf-compare
Benchmarking Python performance across different compilation and interpretation methods (CPython, Cython, PyPy). Includes CPU, memory, and mixed workload tests.
https://github.com/cipher982/py-perf-compare
Last synced: 3 months ago
JSON representation
Benchmarking Python performance across different compilation and interpretation methods (CPython, Cython, PyPy). Includes CPU, memory, and mixed workload tests.
- Host: GitHub
- URL: https://github.com/cipher982/py-perf-compare
- Owner: cipher982
- License: apache-2.0
- Created: 2024-12-10T18:12:36.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2024-12-31T19:47:19.000Z (over 1 year ago)
- Last Synced: 2024-12-31T20:17:00.143Z (over 1 year ago)
- Language: Python
- Size: 218 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Python Performance Comparison
## Project Goals
A benchmarking suite that compares performance across different Python implementations (CPython, Cython, PyPy) for various computational tasks. The project uses Docker to ensure consistent testing environments and reproducible results.
## Overview
This project provides performance comparisons across three types of computational tasks:
- CPU-bound operations (prime number calculations)
- Memory-bound operations (matrix multiplication)
- Mixed operations (Fibonacci sequence with memoization)
## Project Structure
```
.
├── benchmarks/
│ ├── performance_runner.py # Performance measurement script
│ └── results_processor.py # Results processing and visualization
├── src/
│ ├── pure/ # Pure Python implementations
│ │ ├── cpu_test_python.py # Pure Python CPU-bound test
│ │ ├── cpu_test_cython.pyx # Cython CPU-bound test
│ │ ├── memory_test_python.py # Pure Python memory-bound test
│ │ ├── memory_test_cython.pyx # Cython memory-bound test
│ │ ├── mixed_test_python.py # Pure Python mixed test
│ │ └── mixed_test_cython.pyx # Cython mixed test
│ └── numpy/ # NumPy-optimized implementations
│ ├── cpu_test_numpy.py # NumPy CPU-bound test
│ ├── cpu_test_cython.pyx # Cython+NumPy CPU-bound test
│ ├── memory_test_python.py # NumPy memory-bound test
│ └── memory_test_cython.pyx # Cython+NumPy memory-bound test
├── docker/
│ ├── Dockerfile.base # Base Docker configuration
│ ├── Dockerfile.cpython # CPython environment
│ └── Dockerfile.pypy # PyPy environment
├── results/ # Benchmark results
│ ├── cpython/ # CPython test results
│ ├── pypy/ # PyPy test results
│ ├── logs/ # Benchmark run logs
│ └── performance_comparison.png # Visualization of results
├── docker-compose.yml # Docker services configuration
├── docker-entrypoint.sh # Container startup script
├── pyproject.toml # Project configuration and dependency management
├── setup.py # Additional build and setup configurations
├── uv.lock # Dependency lock file
├── .pre-commit-config.yaml # Pre-commit hooks configuration
├── .env # Default benchmark parameters
└── requirements-pypy.txt # PyPy-specific dependencies
```
## Project Dependencies and Configuration
- **Dependency Management**:
- `pyproject.toml`: Primary configuration for project dependencies
- `uv.lock`: Ensures consistent dependency versions across environments
- `setup.py`: Additional build and packaging configurations
- **Code Quality**:
- Pre-commit hooks configured in `.pre-commit-config.yaml`
- Ensures code style and quality standards
## Benchmark Implementations
### Test Variations
Each computational task is implemented in multiple variations:
- Pure Python implementations
- NumPy-optimized implementations
- Cython-optimized implementations
- PyPy-compatible implementations
### Benchmark Types
#### CPU-bound Test
Measures pure computational performance through prime number calculations.
- Implementation: Sieve of Eratosthenes algorithm
- Metrics: Raw computation speed
- Variations:
- Pure Python implementation
- NumPy-accelerated implementation
- Cython-optimized implementation
- PyPy-compatible implementation
#### Memory-bound Test
Evaluates memory handling through matrix operations.
- Implementation: Matrix multiplication
- Metrics: Memory usage and operation speed
- Variations:
- Pure Python implementation
- NumPy-accelerated implementation
- Cython-optimized implementation
- PyPy-compatible implementation
#### Mixed Test
Tests both CPU and memory performance using Fibonacci sequence.
- Implementation: Iterative Fibonacci calculation
- Metrics: Combined CPU and memory performance
- Variations:
- Pure Python implementation
- Cython-optimized implementation
- PyPy-compatible implementation
## Prerequisites
- Docker
- Docker Compose
- Python 3.11+
## Running Benchmarks
### Quick Start with Docker (Recommended)
```bash
# Run all implementations
IMPLEMENTATION=all RUNS=3 PRIME_UPPER_BOUND=1000000 MATRIX_DIMENSION=200 FIBONACCI_LENGTH=500 docker compose up
# Or run specific implementation
IMPLEMENTATION=cpython RUNS=3 PRIME_UPPER_BOUND=1000000 MATRIX_DIMENSION=200 FIBONACCI_LENGTH=500 docker compose up
```
### Configuration
Default values for all parameters are provided in `.env`. You can override them using environment variables:
- `IMPLEMENTATION`: Target implementation (all, cpython, cython, pypy)
- `RUNS`: Number of test iterations
- `PRIME_UPPER_BOUND`: Upper limit for prime number calculations
- `MATRIX_DIMENSION`: Size of matrices (NxN) for multiplication
- `FIBONACCI_LENGTH`: Number of Fibonacci numbers to calculate
Results will be saved to:
- `./results/cpython/` - CPython and Cython results
- `./results/pypy/` - PyPy results
- `./results/logs/` - Detailed benchmark logs
## Results Interpretation
- Detailed CSV results available for each test type and implementation
- Performance comparison visualizations generated in `results/`
- Comprehensive logging provides in-depth insights into benchmark performance
## License
See the `LICENSE` file for details about project licensing.
## Contributing
Contributions are welcome! Please refer to project guidelines and code of conduct.