https://github.com/actuarialopensource/benchmarks

Some performance tests for actuarial applications
https://github.com/actuarialopensource/benchmarks

Last synced: 3 months ago
JSON representation

Some performance tests for actuarial applications

• Host: GitHub
• URL: https://github.com/actuarialopensource/benchmarks
• Owner: actuarialopensource
• License: mit
• Created: 2022-09-14T19:28:56.000Z (about 2 years ago)
• Default Branch: main
• Last Pushed: 2024-06-01T19:28:27.000Z (5 months ago)
• Last Synced: 2024-06-02T21:20:15.000Z (5 months ago)
• Language: Python
• Size: 3.13 MB
• Stars: 12
• Watchers: 1
• Forks: 4
• Open Issues: 5
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

• jimsghstars - actuarialopensource/benchmarks - Some performance tests for actuarial applications (Python)

README

        
# Benchmarking

We provide benchmarks to encourage collaboration and competition in making actuarial software run faster than ever before.

## Containerized benchmarks

We currently only have 1 benchmark, we are working to expand the benchmarks. Open a discussion in this repository if you have any thoughts to share.

Time measurement is currently the best of 3 runs.

| benchmark     | classification | container |A100-SXM4-40GB | H100-SXM5-80GB |

|---------------|-|-|----------------|----------------|

| BasicTerm_ME 100 Million | recursive PyTorch | [link](https://hub.docker.com/repository/docker/actuarial/basicterm_me_python/general) | 15.8284s        | 7.205s         |

| BasicTerm_ME 100 Million | compiled iterative JAX | [link](https://hub.docker.com/repository/docker/actuarial/basicterm_me_python/general) | 3.448s        | 1.551s         |

### Notes

* BasicTerm_ME 100 Million

  * You can find lifelib's modelpoint file with 10,000 modelpoints as `model_point_table.xlsx`. We use these modelpoints, but repeat them 10,000 times for 100,000,000 modelpoints.

## GitHub-hosted runners

These benchmarks run in GitHub-hosted runners in GitHub actions. Used for benchmarks that are not computationally intensive.

Benchmarks in this repository:

* `basic_term_benchmark`: Replicate the cashflows of the [LifeLib BasicTerm model](https://github.com/lifelib-dev/lifelib/tree/main/lifelib/libraries/basiclife/BasicTerm_M)

    * Python [LifeLib BasicTerm_M](https://github.com/lifelib-dev/lifelib/tree/main/lifelib/libraries/basiclife/BasicTerm_M)

    * Julia [using LifeSimulator](https://github.com/JuliaActuary/LifeSimulator.jl)

    * Python using recursive formulas with [PyTorch](https://github.com/actuarialopensource/benchmarks/blob/main/Python/basicterm_recursive_pytorch.py) and [NumPy](https://github.com/actuarialopensource/benchmarks/blob/main/Python/basicterm_recursive_numpy.py)

    * Python using matrix operations (no recursion) on [PyTorch arrays](https://github.com/actuarialopensource/benchmarks/blob/main/Python/basicterm_array_pytorch.py) and [NumPy arrays](https://github.com/actuarialopensource/benchmarks/blob/main/Python/basicterm_array_numpy.py)

* `exposures`: Create date partitions for experience studies

    * Julia [ExperienceAnalysis](https://github.com/JuliaActuary/ExperienceAnalysis.jl)

    * R [actxps](https://github.com/mattheaphy/actxps)

* `mortality`: Read SOA mortality tables and use them in a simple calculation

    * Julia [MortalityTables](https://github.com/JuliaActuary/MortalityTables.jl)

    * Python [Pymort](https://github.com/actuarialopensource/pymort)

The below results are generated by the benchmarking scripts in the folders for each language. These scripts are run automatically by GitHub Actions and populate the results below. 

```yaml 

basic_term_benchmark:

- Julia array basic_term:

    minimum time: TrialEstimate(30.279 ms)

    result: 1.4489630534602132e7

  Julia recursive basic_term:

    minimum time: TrialEstimate(84.812 ms)

    result: 1.4489630534602132e7

- Python array numpy basic_term_m:

    minimum time: 83.45314299992879 milliseconds

    result: 14489630.534603368

  Python array pytorch basic_term_m:

    minimum time: 51.54931000004126 milliseconds

    result: 14489630.534603368

  Python lifelib basic_term_m:

    minimum time: 618.0503439999256 milliseconds

    result: 14489630.534601536

  Python recursive numpy basic_term_m:

    minimum time: 63.08064000006652 milliseconds

    result: 14489630.534603368

  Python recursive pytorch basic_term_m:

    minimum time: 75.6699999999455 milliseconds

    result: 14489630.53460337

basic_term_me_benchmark:

- Python heavylight numpy basic_term_me:

    minimum time: 354.6492100000478 milliseconds

    result: 215146132.0684811

  Python lifelib basic_term_me:

    minimum time: 1191.792300999964 milliseconds

    result: 215146132.06848112

  Python recursive numpy basic_term_me:

    minimum time: 309.30894900006933 milliseconds

    result: 215146132.0684814

exposures:

- Julia ExperienceAnalysis.jl:

    minimum time: TrialEstimate(29.659 ms)

    num_rows: 141281

- R actxps:

    min: 486.248656 ms

    num_rows: 141281

mortality:

- Julia MortalityTables.jl:

    minimum time: TrialEstimate(229.507 μs)

    result: 1904.4865526636793

- Python PyMort:

    minimum time: 9.425531999909253 milliseconds

    result: 1904.4865526636793

savings_benchmark:

- Julia Benchmarks savings:

    minimum time: TrialEstimate(119.226 ms)

    result: 3.507113709040273e12

- Python lifelib cashvalue_me_ex4:

    minimum time: 596.2715760000492 milliseconds

    result: 3507113709040.141

  Python recursive numpy cashvalue_me_ex4:

    minimum time: 543.3022800000344 milliseconds

    result: 3507113709040.124

```