Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/actuarialopensource/benchmarks
Some performance tests for actuarial applications
https://github.com/actuarialopensource/benchmarks
Last synced: 9 days ago
JSON representation
Some performance tests for actuarial applications
- Host: GitHub
- URL: https://github.com/actuarialopensource/benchmarks
- Owner: actuarialopensource
- License: mit
- Created: 2022-09-14T19:28:56.000Z (about 2 years ago)
- Default Branch: main
- Last Pushed: 2024-06-01T19:28:27.000Z (6 months ago)
- Last Synced: 2024-08-14T07:09:18.843Z (4 months ago)
- Language: Python
- Size: 3.13 MB
- Stars: 13
- Watchers: 1
- Forks: 4
- Open Issues: 5
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- jimsghstars - actuarialopensource/benchmarks - Some performance tests for actuarial applications (Python)
README
# Benchmarking
We provide benchmarks to encourage collaboration and competition in making actuarial software run faster than ever before.
## Containerized benchmarks
We currently only have 1 benchmark, we are working to expand the benchmarks. Open a discussion in this repository if you have any thoughts to share.
Time measurement is currently the best of 3 runs.
| benchmark | classification | container |A100-SXM4-40GB | H100-SXM5-80GB |
|---------------|-|-|----------------|----------------|
| BasicTerm_ME 100 Million | recursive PyTorch | [link](https://hub.docker.com/repository/docker/actuarial/basicterm_me_python/general) | 15.8284s | 7.205s |
| BasicTerm_ME 100 Million | compiled iterative JAX | [link](https://hub.docker.com/repository/docker/actuarial/basicterm_me_python/general) | 3.448s | 1.551s |
### Notes
* BasicTerm_ME 100 Million
* You can find lifelib's modelpoint file with 10,000 modelpoints as `model_point_table.xlsx`. We use these modelpoints, but repeat them 10,000 times for 100,000,000 modelpoints.
## GitHub-hosted runners
These benchmarks run in GitHub-hosted runners in GitHub actions. Used for benchmarks that are not computationally intensive.
Benchmarks in this repository:
* `basic_term_benchmark`: Replicate the cashflows of the [LifeLib BasicTerm model](https://github.com/lifelib-dev/lifelib/tree/main/lifelib/libraries/basiclife/BasicTerm_M)
* Python [LifeLib BasicTerm_M](https://github.com/lifelib-dev/lifelib/tree/main/lifelib/libraries/basiclife/BasicTerm_M)
* Julia [using LifeSimulator](https://github.com/JuliaActuary/LifeSimulator.jl)
* Python using recursive formulas with [PyTorch](https://github.com/actuarialopensource/benchmarks/blob/main/Python/basicterm_recursive_pytorch.py) and [NumPy](https://github.com/actuarialopensource/benchmarks/blob/main/Python/basicterm_recursive_numpy.py)
* Python using matrix operations (no recursion) on [PyTorch arrays](https://github.com/actuarialopensource/benchmarks/blob/main/Python/basicterm_array_pytorch.py) and [NumPy arrays](https://github.com/actuarialopensource/benchmarks/blob/main/Python/basicterm_array_numpy.py)
* `exposures`: Create date partitions for experience studies
* Julia [ExperienceAnalysis](https://github.com/JuliaActuary/ExperienceAnalysis.jl)
* R [actxps](https://github.com/mattheaphy/actxps)
* `mortality`: Read SOA mortality tables and use them in a simple calculation
* Julia [MortalityTables](https://github.com/JuliaActuary/MortalityTables.jl)
* Python [Pymort](https://github.com/actuarialopensource/pymort)
The below results are generated by the benchmarking scripts in the folders for each language. These scripts are run automatically by GitHub Actions and populate the results below.
```yaml
basic_term_benchmark:
- Julia array basic_term:
minimum time: TrialEstimate(30.279 ms)
result: 1.4489630534602132e7
Julia recursive basic_term:
minimum time: TrialEstimate(84.812 ms)
result: 1.4489630534602132e7
- Python array numpy basic_term_m:
minimum time: 83.45314299992879 milliseconds
result: 14489630.534603368
Python array pytorch basic_term_m:
minimum time: 51.54931000004126 milliseconds
result: 14489630.534603368
Python lifelib basic_term_m:
minimum time: 618.0503439999256 milliseconds
result: 14489630.534601536
Python recursive numpy basic_term_m:
minimum time: 63.08064000006652 milliseconds
result: 14489630.534603368
Python recursive pytorch basic_term_m:
minimum time: 75.6699999999455 milliseconds
result: 14489630.53460337
basic_term_me_benchmark:
- Python heavylight numpy basic_term_me:
minimum time: 354.6492100000478 milliseconds
result: 215146132.0684811
Python lifelib basic_term_me:
minimum time: 1191.792300999964 milliseconds
result: 215146132.06848112
Python recursive numpy basic_term_me:
minimum time: 309.30894900006933 milliseconds
result: 215146132.0684814
exposures:
- Julia ExperienceAnalysis.jl:
minimum time: TrialEstimate(29.659 ms)
num_rows: 141281
- R actxps:
min: 486.248656 ms
num_rows: 141281
mortality:
- Julia MortalityTables.jl:
minimum time: TrialEstimate(229.507 μs)
result: 1904.4865526636793
- Python PyMort:
minimum time: 9.425531999909253 milliseconds
result: 1904.4865526636793
savings_benchmark:
- Julia Benchmarks savings:
minimum time: TrialEstimate(119.226 ms)
result: 3.507113709040273e12
- Python lifelib cashvalue_me_ex4:
minimum time: 596.2715760000492 milliseconds
result: 3507113709040.141
Python recursive numpy cashvalue_me_ex4:
minimum time: 543.3022800000344 milliseconds
result: 3507113709040.124
```