Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

https://github.com/vprusso/toqito-bench

Benchmarking suite for the toqito software package.
https://github.com/vprusso/toqito-bench

Last synced: 28 days ago
JSON representation

Benchmarking suite for the toqito software package.

Awesome Lists containing this project

README 

          
# toqito-bench

Benchmarking suite for the toqito software package.



## Benchmark coverage

The following table lists the functions we have benchmarked across toqito, QuTiPy, and Ket.jl:



| **toqito**                 | **QuTiPy**                  | **Ket.jl**                              | **QETLAB**|

| -------------------------- | --------------------------- | --------------------------------------- |------------

| `basis`                    | `ket`                       | `ket`                                   |

| `is_positive_semidefinite` |                             |                                         |

| `swap`                     |                             |                                         |

| `swap_operator`            | `SWAP`                      |                                         |

| `to_density_matrix`        |                             |                                         |

| `permute_systems`          | `syspermute`                | `permute_systems`                       |

| `kraus_to_choi`            | `choi_representation`       | `choi`                                  |

| `pauli`                    | `generate_nQubit_Pauli`     | `pauli`                                 |

| `permutation_operator`     | `permute_tensor_factors`    | `permutation_matrix`                    |

| `apply_channel`            | `apply_channel`             |                                         |

| `partial_transpose`        | `partial_transpose`         |                                         |

| `random_povm`              |                             | `random_povm`                           |

| `partial_trace`            | `partial_trace`             | `partial_trace`                         | `PartialTrace`|

| `random_density_matrix`    | `random_density_matrix`     |                                         |

| `random_unitary`           | `RandomUnitary`             | `random_unitary`                        |

| `random_psd_operator`      | `random_PSD_operator`       |                                         |

| `trace_distance`           | `norm_trace_dist`           |                                         |

| `trace_norm`               | `trace_norm`                | `trace_norm`                            |

| `log_negativity`           | `log_negativity`            |                                         |

| `von_neumann_entropy`      | `entropy`                   | `entropy`                               |

| `natural_representation`   | `natural_representation`    |                                         |

| `amplitude_damping`        | `amplitude_damping_channel` | `channel_amplitude_damping_generalized` |

| `bitflip`                  | `bit_flip_channel`          | `channel_bit_flip`                      |

| `dephasing`                | `dephasing_channel`         |                                         |



## Setup Environments General



### Python Environment

```bash

# Checks if the Python virtual environment exists at env/python-env.

make check-env-python



# Creates a Python 3.10 virtual environment in env/python-env, upgrades pip.

make setup-python



# Ensures the Python environment exists, creating it if missing.

make ensure-python



# Installs/updates pytest-benchmark in the virtual environment and runs setup/test.py.

make test-python



# Removes the Python virtual environment.

make clean-python



# Cleans and reinstalls the Python environment from scratch.

make reinstall-python

```

### Julia Environment



```bash

# Performs a fresh Julia installation, removing previous installations and setting up a new environment with BenchmarkTools.

make install-julia-fresh



# Checks for Julia installation; installs or updates as needed. Ensures BenchmarkTools is available in the project environment.

make setup-julia



# Runs Julia tests via setup/test.jl in the configured environment.

make test-julia



# Removes all Julia installations and environments.

make clean-julia

```



## Toqito Environment Setup & Benchmarking



### Setup Environment



```bash

# Checks if a Poetry environment for toqito exists at env/toqito-env.

make check-env-toqito



# Initializes a poetry environment for Toqito in env/toqito-env.

make setup-toqito



# Ensures the toqito poetry environment exists, creating it if missing.

make ensure-toqito



# Runs a setup test script (setup/test_toqito.py) inside the toqito environment.

make test-toqito-setup 



# Displays installed packages and environment info for toqito.

make toqito-info



#removes the Toqito Poetry environment.

make clean-toqito



# Reinstalls the toqito poetry environment from scratch.

make reinstall-toqito

```

### Benchmark



1. Runs all the tests in `scripts/benchmark_toqito.py` with `--benchmark-warmup=on` and `--benchmark-verbose`. JSON report is stored in `results/toqito/full`

```bash

make benchmark-full-toqito

```

2. Runs a simple benchmark with optional filtering and saving. Results are saved in `results/toqito//` with only required columns obtained from `--benchmark-columns=mean,median,stddev,ops`

As an example,

```bash

# runs benchmarks only for partial_trace function with varying only dim and displays the results but does not save them.

make benchmark-simple-toqito FILTER="TestPartialTraceBenchmarks" FUNCTION="test_bench__partial_trace__vary__dim" SAVE=false

```



3. Generates histogram visualizations from benchmark results or runs `make benchmark-simple-toqito` if none exist and then generate them. Histogram SVG files saved to `results/toqito///images/`. As an example,



```bash

# If benchmarks for partial_trace function with varying only dim exist in isolation then constructs histogram through it else runs simple benchmarks for toqito with specified arguments and then construct it.

make benchmark-histogram-toqito FILTER="TestPartialTraceBenchmarks" FUNCTION="test_bench__partial_trace__vary__dim"

```



4. Runs benchmarks with memory profiling using `cProfile` (currently in development)

```bash

make test-toqito-benchmark-memory

```

## QuTIpy Environment Setup & Benchmarking



### Setup Environment



```bash

# Checks if a Poetry environment for qutipy exists at env/qutipy-env.

make check-env-qutipy



# Initializes a poetry environment for qutipy in env/qutipy-env.

make setup-qutipy



# Ensures the qutipy poetry environment exists, creating it if missing.

make ensure-qutipy



# Runs a setup test script (setup/test_qutipy.py) inside the qutipy environment.

make test-qutipy-setup 



#removes the qutipy Poetry environment.

make clean-qutipy



# Reinstalls the qutipy poetry environment from scratch.

make reinstall-qutipy

```



### Benchmark



1. Runs all the tests in `scripts/benchmark_qutipy.py` with `--benchmark-warmup=on` and `--benchmark-verbose`. JSON report is stored in `results/qutipy/full`

```bash

make benchmark-full-qutipy

```

2. Runs a simple benchmark with optional filtering and saving. Results are saved in `results/qutipy//` with only required columns obtained from `--benchmark-columns=mean,median,stddev,ops`

As an example,

```bash

# runs benchmarks only for partial_trace function with varying only dim and displays the results but does not save them.

make benchmark-simple-qutipy FILTER="TestPartialTraceBenchmarks" FUNCTION="test_bench__partial_trace__vary__dim" SAVE=false

```



3. Generates histogram visualizations from benchmark results or runs `make benchmark-simple-qutipy` if none exist and then generate them. Histogram SVG files saved to `results/qutipy///images/`. As an example,



```bash

# If benchmarks for partial_trace function with varying only dim exist in isolation then constructs histogram through it else runs simple benchmarks for qutipy with specified arguments and then construct it.

make benchmark-histogram-qutipy FILTER="TestPartialTraceBenchmarks" FUNCTION="test_bench__partial_trace__vary__dim"

```



## Ketjl Environment Setup & Benchmarking



### Setup Environment



```bash

# Checks if an environment for ketjl exists at env/ketjl-env.

make check-env-ketjl



# Initializes an environment for ketjl in env/ketjl-env.

make setup-ketjl



# Ensures the ketjl environment exists, creating it if missing.

make ensure-ketjl



# Runs a setup test script (setup/test_ketjl.jl) inside the ketjl environment.

make test-ketjl-setup 



#removes the ketjl environment.

make clean-ketjl



# Reinstalls the ketjl environment from scratch.

make reinstall-ketjl

```



### Benchmark



1. Runs all the tests in `scripts/benchmark_ketjl.jl`. JSON report is stored in `results/ketjl/full`.

```bash

make benchmark-full-ketjl

```

2. Runs a simple benchmark with optional filtering and saving. Results are saved in `results/ketjl//` with all the required data.

As an example,

```bash

# runs benchmarks only for partial_trace function with varying only dim and saves the results.

make benchmark-simple-ketjl FILTER="TestPartialTraceBenchmarks" FUNCTION="test_bench__partial_trace__vary__dim" SAVE=true

```



3. **TODO:** Generates histogram visualizations from benchmark results or runs `make benchmark-simple-ketjl` if none exist and then generate them. Histogram SVG files saved to `results/ketjl///images/`. As an example,



```bash

# If benchmarks for partial_trace function with varying only dim exist in isolation then constructs histogram through it else runs simple benchmarks for ketjl with specified arguments and then construct it.

make benchmark-histogram-ketjlFILTER="TestPartialTraceBenchmarks" FUNCTION="test_bench__partial_trace__vary__dim"

```