{"id":16469465,"url":"https://github.com/btschwertfeger/biasadjustcxx-performance-test","last_synced_at":"2026-02-13T09:13:28.861Z","repository":{"id":132514035,"uuid":"605167183","full_name":"btschwertfeger/BiasAdjustCXX-Performance-Test","owner":"btschwertfeger","description":"Performmance/execution time test of the bias correction tool BiasAdjustCXX v1.8 in comparison to python-cmethods v0.6.1 and xclim v0.40.0.","archived":false,"fork":false,"pushed_at":"2023-02-28T15:59:08.000Z","size":17174,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-09-10T01:05:07.444Z","etag":null,"topics":["bias-correction","biasadjustcxx","climate-analysis","climate-science","delta-method","linear-scaling","performance-testing","python-cmethods","quantile-delta-mapping","quantile-mapping","variance-scaling","xclim"],"latest_commit_sha":null,"homepage":"","language":"HTML","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/btschwertfeger.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2023-02-22T15:34:02.000Z","updated_at":"2023-02-26T09:38:10.000Z","dependencies_parsed_at":"2023-06-08T07:30:29.407Z","dependency_job_id":null,"html_url":"https://github.com/btschwertfeger/BiasAdjustCXX-Performance-Test","commit_stats":{"total_commits":14,"total_committers":2,"mean_commits":7.0,"dds":0.0714285714285714,"last_synced_commit":"9d3ae017563c1f5f66bf46dbdfdd2b30dc70aea3"},"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/btschwertfeger/BiasAdjustCXX-Performance-Test","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/btschwertfeger%2FBiasAdjustCXX-Performance-Test","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/btschwertfeger%2FBiasAdjustCXX-Performance-Test/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/btschwertfeger%2FBiasAdjustCXX-Performance-Test/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/btschwertfeger%2FBiasAdjustCXX-Performance-Test/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/btschwertfeger","download_url":"https://codeload.github.com/btschwertfeger/BiasAdjustCXX-Performance-Test/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/btschwertfeger%2FBiasAdjustCXX-Performance-Test/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":276622503,"owners_count":25675125,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-23T02:00:09.130Z","response_time":73,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bias-correction","biasadjustcxx","climate-analysis","climate-science","delta-method","linear-scaling","performance-testing","python-cmethods","quantile-delta-mapping","quantile-mapping","variance-scaling","xclim"],"created_at":"2024-10-11T12:07:41.090Z","updated_at":"2025-09-23T17:59:49.793Z","avatar_url":"https://github.com/btschwertfeger.png","language":"HTML","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n\u003ch1 style=\"center\"\u003ePerformance test of the \u003ca href=\"https://zenodo.org/record/7652734\" target=\"_blank\"\u003eBiasAdjustCXX v1.8\u003c/a\u003e command-line tool\u003c/h1\u003e\n\u003ch3 style=\"centter\"\u003e ... and comparison to \u003ca href=\"https://zenodo.org/record/7535677\" target=\"blank\"\u003exclim v0.40.0\u003c/a\u003e and \u003ca href=\"https://zenodo.org/record/7652756\" target=\"_blank\"\u003epython-cmethods v0.6.1\u003c/a\u003e\u003c/h3\u003e \n\n___\n[![DOI](https://zenodo.org/badge/605167183.svg)](https://zenodo.org/badge/latestdoi/605167183)\n\n\n📍 Check out the results here: https://btschwertfeger.github.io/BiasAdjustCXX-Performance-Test/\n\n📍 Check out the Notebook: https://github.com/btschwertfeger/BiasAdjustCXX-Performance-Test/blob/master/index.ipynb\n\n\n\u003ca name=\"introduction\"\u003e\u003c/a\u003e\n# Introduction\n\n\nThere are many different tools for bias correction in climate data. One of these tools is `BiasAdjustCXX`, whose performance in terms of speed (execution time; i.e. the whole time between the program start until the end) will be examined in more detail. This notebook also serves as a guide for reproducing the results of these tests. Within this notebook scripts are called, which install the tool, generate test data and afterwards all procedures provided in `BiasAdjustCXX` (v1.8) are executed to adjust the generated data sets. \n\nTo be able to compare this with something, the Python modules \u003ca href=\"https://zenodo.org/record/7535677\" target=\"blank\"\u003exclim\u003c/a\u003e (v0.40.0) and \u003ca href=\"https://zenodo.org/record/7652756\" target=\"_blank\"\u003epython-cmethods \u003c/a\u003e(v0.6.1) are also used to measure their speed when applying the quantile mapping (QM) for `python-cmethods` and the quantile delta mapping (QDM) for both. Since all tools bring a variety of methods with them, but these differ both in terms of implementation and approach, only QM and QDM are tested here for `python-cmethods`. For the `xclim` module, only QDM is tested. \n\n____\n\u003ca name=\"tools\"\u003e\u003c/a\u003e\n## Tools\n\nComparison of the speed performance of the bias correction tools: \n- [BiasAdjustCXX v1.8](https://zenodo.org/record/7652734) (C++ command-line tool)\n- [python-cmethods v0.6.1](https://zenodo.org/record/7652756) (Python3 module)\n- [xclim v0.40.0](https://zenodo.org/record/7535677) (Python3 module)\n\n____\n\n\u003ca name=\"test-env\"\u003e\u003c/a\u003e\n## Test environment and approach\nThis tests was run on MacBook Pro 13-inch 2017 (tests run in February of 2023)\n\nwith:\n\n* Processor: 2.3 GHz Dual-Core Intel Core i5\n* L2-Cache (per core): 256 KB\n* L3-Cache: 4 MB\n* Hyper-Threading Technology: enabled\n* RAM: 8 GB 2133 MHz LPDDR3\n* OS: MacOS Ventura 13.2.1\n* Disk: 512 GB APPLE SSD AP0512J\n\nAll tests have been executed on the same machine. The scripts `scripts/test_performance_*.sh` were used for this purpose. The adjustments are executed using a minimal setup for all tools. The input data sets were randomly generated, this can be reproduced using the script `scripts/generate_test_data.py`. All data sets have exactly 10950 float values per grid cell, so this is comparable to 30 years of daily temperature values. A seed was set so that re-execution would generate the same data. \n\nAll methods provided in the BiasAdjustCXX tool are tested. For `python-cmthods`, the corrections are only performed for the QM and QDM. For `xclim` only the QDM is used, because this is the only method with a comparable implementation in `BiasAdjustCXX` and `python-cmethods`. All methods are given the same parameters, that is, they are given the same input data sets and all distribution-based methods are given the value of 250 for the number of quantiles to be considered.\n\nData sets are generated, which contain a grid of 10x10, 20x20, 30x30, ..., 100x100 with 10950 values per grid cell. Since especially with the Python modules the time to load, prepare and correct a data set varies strongly despite seemingly equal starting-/preconditions, the data sets are corrected multiple times per tool. This provides a better overview of the outliers and also makes it possible to map an average value.\n\nIt has been found that the python modules have difficulty processing large data sets, for this reason only data sets with a resolution up to 60x60 are tested for these python modules. \n\n`BiasAdjustCXX` also offers the computation using multiple threads. This is als tested here.\n\n#### Notes:\n* In this tests, the data sets with a grid of 100x100 are about 836 MB in size ($T_{obs,h}$, $T_{sim,h}$ and $T_{sim,p}$ together thus 2.508 GB). A dataset with a resolution of 500x500 would therefore be 25 times larger ($100 \\cdot 100 = 10000 \\rightarrow 500 \\cdot 500 = 250000$), i.e. 62.5 GB in total. Since I don't have this capacity at the time of writing, I don't currently have access to more powerful machines, and I'm making this supplement within 3 weeks besides of my regular work, the largest grid examined here is 100x100.\n* The results of all tests can be foudn in the `performance_results` dirctory.\n____\n\u003ca name=\"requirements\"\u003e\u003c/a\u003e\n## Requirements\n\nTo run the python modules, python==3.10.18 is required, since this is the latest version, that `xclim` supports. It is recommended to create a virtual environment and install the requirements like this:\n\n```bash\npython3 -m venv test_performance_venv\nsource test_performance_venv/bin/activate\npython3 -m pip install -r requirements.txt\n``` \n\nFor the compilation of [BiasAdjustCXX v1.8](https://zenodo.org/record/7652734) the following libraries and tools are needed:\n* NetCDF-4 C library ([How to install NetCDF-4 C](https://docs.geoserver.org/stable/en/user/extensions/netcdf-out/nc4.html))\n* CMake v3.10+ ([How to install CMake](https://cmake.org/install/))\n\nThe compilation can be done using:\n\nCompilation:\n\n```bash\nmkdir build \u0026\u0026 cd build\ncmake .. \u0026\u0026 cmake --build .\n```\n(... or see the README.md of [BiasAdjustCXX v1.8](https://zenodo.org/record/7652734))\n\n\n____\n\u003ca name=\"disclaimer\"\u003e\u003c/a\u003e\n## Disclaimer\n\n* The scripts contained in this project/repository were created by Benjamin T. Schwertfeger in February 2023 and serve as a supplement to a manuscript introducing the command-line tool \u003ca href=\"https://zenodo.org/record/7652734\" target=\"_blank\"\u003eBiasAdjustCXX v1.8\u003c/a\u003e. This manuscript was submitted to the journal [SoftwareX](https://www.sciencedirect.com/journal/softwarex) in January 2023. \n\n* Errors and inaccuracies can happen. The tests performed here have been executed with the best of conscience. However, it is requested that you do not execute any of these scripts without understanding their contents. The author assumes no liability for errors, data loss, exploits or other damages caused by this code. \n\n____\n\n\u003ca name=\"toc\"\u003e\u003c/a\u003e\n## Table of contents \n(https://btschwertfeger.github.io/BiasAdjustCXX-Performance-Test/)\n\n1. Download and compilation BiasAdjustCXX v1.8\u003cbr\u003e\n2. Generate test data sets\u003cbr\u003e\n3. Run the performance tests for BiasAdjustCXX v1.8, xclim v0.40.0 and python-cmethods v0.6.1\u003cbr\u003e\n4. Evaluation of the results\u003cbr\u003e\n    4.1 Results of BiasAdjustCXX v1.8 (DM, LS, VS, QM, and QDM)\u003cbr\u003e\n    4.2 Results of xclim v0.40.0 (QDM)\u003cbr\u003e\n    4.3 Results of python-cmethods v0.6.1 (QM and QDM)\u003cbr\u003e\n    4.4 Comparison of the execution time of QDM\u003cbr\u003e\n5. Conclusion\u003cbr\u003e\n___\n\u003ca name=\"abbrev\"\u003e\u003c/a\u003e\n## Abbreviations\n\n|Phrase|Definitiion|\n|-----|------|\n|DM|Delta Method|\n|LS|Linear Scaling|\n|VS|Variance Scaling|\n|QM|Quantile Mapping|\n|QDM|Quantile Delta Mapping|\n\n____\n\n\u003ca name=\"references\"\u003e\u003c/a\u003e\n\n## References\n\nFor further information regarding the mathematical basis of the correction procedures of `BiasAdjustCXX` I refer to the following articles:\n\n* Linear Scaling and Variance Scaling based on: Teutschbein, Claudia and Seibert, Jan (2012) Bias correction of regional climate model simulations for hydrological climate-change impact studies: Review and evaluation of different methods (https://doi.org/10.1016/j.jhydrol.2012.05.052)\n* Delta Method based on: Beyer, R. and Krapp, M. and Manica, A. (2020): An empirical evaluation of bias correction methods for palaeoclimate simulations (https://doi.org/10.5194/cp-16-1493-2020)\n* Quantile Mapping based on: Alex J. Cannon and Stephen R. Sobie and Trevor Q. Murdock Bias Correction of GCM Precipitation by Quantile Mapping: How Well Do Methods Preserve Changes in Quantiles and Extremes? (https://doi.org/10.1175/JCLI-D-14-00754.1)\n* Quantile Delta Mapping based on: Tong, Y., Gao, X., Han, Z. et al. Bias correction of temperature and precipitation over China for RCM simulations using the QM and QDM methods. Clim Dyn 57, 1425–1443 (2021). (https://doi.org/10.1007/s00382-020-05447-4)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbtschwertfeger%2Fbiasadjustcxx-performance-test","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbtschwertfeger%2Fbiasadjustcxx-performance-test","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbtschwertfeger%2Fbiasadjustcxx-performance-test/lists"}