{"id":37625479,"url":"https://github.com/verificarlo/fuzzy","last_synced_at":"2026-01-16T10:47:33.236Z","repository":{"id":44572042,"uuid":"218554957","full_name":"verificarlo/fuzzy","owner":"verificarlo","description":"A fuzzy ecosystem for evaluating the stability of your computational tools.","archived":false,"fork":false,"pushed_at":"2025-06-12T20:34:10.000Z","size":15543,"stargazers_count":16,"open_issues_count":2,"forks_count":14,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-06-12T21:47:19.790Z","etag":null,"topics":["brainweb","floating-point","fuzzy","monte-carlo","numerical-analysis","stability"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/verificarlo.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG","contributing":".github/CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2019-10-30T15:07:03.000Z","updated_at":"2025-06-12T20:33:37.000Z","dependencies_parsed_at":"2024-05-21T17:08:26.262Z","dependency_job_id":null,"html_url":"https://github.com/verificarlo/fuzzy","commit_stats":null,"previous_names":[],"tags_count":8,"template":false,"template_full_name":null,"purl":"pkg:github/verificarlo/fuzzy","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/verificarlo%2Ffuzzy","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/verificarlo%2Ffuzzy/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/verificarlo%2Ffuzzy/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/verificarlo%2Ffuzzy/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/verificarlo","download_url":"https://codeload.github.com/verificarlo/fuzzy/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/verificarlo%2Ffuzzy/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28478059,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-16T06:30:42.265Z","status":"ssl_error","status_checked_at":"2026-01-16T06:30:16.248Z","response_time":107,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["brainweb","floating-point","fuzzy","monte-carlo","numerical-analysis","stability"],"created_at":"2026-01-16T10:47:32.468Z","updated_at":"2026-01-16T10:47:33.213Z","avatar_url":"https://github.com/verificarlo.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Fuzzy v2.1.0\n\n[![DOI](https://zenodo.org/badge/218554957.svg)](https://zenodo.org/badge/latestdoi/218554957)\n[![Build Fuzzy Environments](https://github.com/verificarlo/fuzzy/actions/workflows/build-fuzzy.yml/badge.svg?branch=master)](https://github.com/verificarlo/fuzzy/actions/workflows/build-fuzzy.yml)\n[![Docker Pulls](https://img.shields.io/docker/pulls/verificarlo/fuzzy)](https://hub.docker.com/r/verificarlo/fuzzy)\n[![License](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](https://releases.llvm.org/13.0.0/LICENSE.TXT)\n\n*A fuzzy ecosystem for evaluating the effect of numerical error on computational tools.*\n\n\u003cdiv align=\"center\"\u003e\n  \u003ca href=\"./img/fuzzy.png\"\u003e\n    \u003cimg src=\"https://github.com/verificarlo/fuzzy/raw/master/img/fuzzy.png\" alt=\"Fuzzy Marimo\"\u003e\n  \u003c/a\u003e\n\u003c/div\u003e\n\n## Table of Contents\n- [Fuzzy v2.1.0](#fuzzy-v210)\n  - [Table of Contents](#table-of-contents)\n  - [Motivation](#motivation)\n  - [Usage](#usage)\n      - [Building \\\u0026 booting the environment](#building--booting-the-environment)\n      - [Adding your software](#adding-your-software)\n      - [Fuzzy-libmath](#fuzzy-libmath)\n      - [Using Fuzzy in Multi-stage builds](#using-fuzzy-in-multi-stage-builds)\n      - [Running Fuzzy workflows](#running-fuzzy-workflows)\n      - [Quick overview of Monte Carlo Arithmetic](#quick-overview-of-monte-carlo-arithmetic)\n      - [Common failures](#common-failures)\n  - [Contributing](#contributing)\n  - [Presentation](#presentation)\n  - [References](#references)\n  - [License](#license)\n\n## Motivation\n\nComputational analysis has become an essential component of science. The software tools\nused bear the weight of the countless models, discoveries, and interventions based\nupon them. However, computers were never intended to be perfect, and one doesn't\nneed to look very far to find examples of where conceptually simple operations\nmay fail due to floating point arithmetic (e.g. ...\n\n```\n$ python3 -c \"print(sum([0.001] * 1000))\"\n1.0000000000000007\n```\n\n). While small issues like the one above are unlikely to cause catestrophic failures on\ntheir own, they can cascade over the course of an execution and it isn't uncommon to\nfind a [plane that requires a power cycle every two months](https://www.theregister.com/2020/04/02/boeing_787_power_cycle_51_days_stale_data)\nor an [unstable series that consistently converges to the wrong solution](https://nbviewer.jupyter.org/github/verificarlo/fuzzy/blob/master/tutorial/evaluating_unstable_sequence.ipynb).\n\n*Fuzzy* allows you to study the stability or *trustworthiness* of tools and pipelines.\nYou start by *instrumenting* libraries in your pipeline which do the bulk of numerical\nheavy lifting, then run your tool multiple times, and finally analyze the variability\nin your results. This project aims to provide an environment in which the stability of\nprograms can be evaluated, reducing the overhead on both tool developers and consumers.\n\n\n## Usage\n\n#### Building \u0026 booting the environment\nYou can get started with *Fuzzy* quite simply, just launch a\n[Docker](https://www.docker.com/) container as follows:\n\n```bash\ndocker run -ti verificarlo/fuzzy\n```\n\nIf you're on a shared system and would prefer to use [Singularity](https://sylabs.io/),\nthat's no problem, just convert the container using the appropriate method for\nyour system ([e.g.](https://docs.computecanada.ca/wiki/Singularity#Creating_images)).\n\nIf you would like to build the environment locally on your system, look at the\nDockerfiles in `docker/base/` to see how installation was performed. At the end of the\nbuild chain, you'll find instrumented versions of `libmath`, `lapack`, `python3`,\n`numpy`, and several other recompiled libraries.\n\nAn example for how to verify your installation for Python could be the following:\n```bash\n$ python3 -c \"print([sum([.001]*1000) for _ in range(3)])\"\n[1.0, 0.9999999999999997, 1.0000000000000007, 1.0000000000000002]\n```\n\n#### Adding your software\nIf your code is written in, say, Python, then you can simply mount your code to\nthis environment, run it using the contained interpreter, and experince the *:fuzz:*.\nIf your code loads shared libraries, such as `libmath` or `lapack`, make sure that\nthey are using the instrumented versions of these libraries made available in\n`/usr/local/lib/`. You can check whether this is the case using `ldd \u003c/path/to/yourbinary\u003e`.\n\nThe next step is to make sure the environment is configured to introduce perturbations\nthe way you expect. You can start with a configuration which performs perturbations\nakin to randomizing machine error with the following:\n\n```bash\necho \"libinterflop_mca.so -m mca --precision-binary32=24 --precision-binary64=53\" \u003e $VFC_BACKENDS_FROM_FILE\n```\n\nFor more usage instructions on how to control how perturbations are introduced\nand where they occur within operations, please refer to the\n[Verificarlo](https://github.com/verificarlo/verificarlo) repository.\n\nA simple sanity check to verify that your code is using the perturbed libraries\nis to dramatically reduce the precision in your configuration by changing the\n`precision-binary32` and `precision-binary64` values to something small (e.g. 1),\nboot up a debugging session in your environment (e.g. a Python shell, GDB, etc.),\nload a math library, and run a basic math function a few times (e.g. `sin(1)`) —\nyou should get different results if your instrumentation is setup properly.\n***Important***: Don't forget to set the precision back to your desired level\nprior to performing your experiments! *Fuzzy* should print a log message to the\nterminal when you run your commands, including the configuration, so you can verify\nthat the parameters were properly specified.\n\n#### Fuzzy-libmath\n\nSince version v0.8.0, fuzzy-libmath comes with three modes (standard, quad, and\n mpfr). Internally, the fuzzy-libmath computes the actual result of the math\nfunction using the standard library of the OS and then perturbs the output by\nadding MCA noise. Since the accuracy of the `libm` used can vary from one\nversion to another (see this\n[article](https://hal.inria.fr/hal-03141101v2/document)), one\ncan use higher precision to ensure accurate intermediate computations.\nA new mode called `fast` was introduced in version v0.9.1 that is equivalent \nto a MCA noise with a fixed virtual precision (t=24 for binary32, t=53 for binary64)\nthat cannot be changed. \nThese modes specify the library used:\n- `fast`: uses the standard libm provided by the OS with a virtual precision fixed to the last bit (fastest)\n- `standard`: uses the standard libm provided by the OS\n- `quad`: uses the libquadmath.so library\n- `mpfr`: uses the MPFR library with 113 bits of precision, equivalent to\n  the binary128 precision (slowest)\n\nNote that a higher precision implies a larger slowdown. The `standard` mode is\nthe fastest and can be used if one does not require high accuracy. To switch\nfrom one version to another, please use the `set-fuzzy-libmath` tool already\ninstalled in the docker image as follow:\n\n```bash\nusage: set-fuzzy-libmath [-h] --version {no-fuzzy,standard,quad,mpfr}\n\noptional arguments:\n  -h, --help            show this help message and exit\n  --version {no-fuzzy,standard,quad,mpfr}\n\nFuzzy libmath version to use:\n        - no-fuzzy: Disable fuzzy libmath\n        - fast:     Use the standard libmath available on the system.\n                    Not possible to modify the virtual precision!\n                    Fastest instrumentation but possibly not accurate\n        - standard: Use the standard libmath available on the system.\n                    Fast instrumentation but possibly not accurate\n        - quad:     Use the libquadmath for intermediate results.\n                    Slower than the standard version but more accurate.\n        - mpfr:     Use the mpfr library for intermediate results.\n                    Slowest version but gives the correct rounding.\n```\n\n\n\u003e [!TIP] \n\u003e The script [build_fuzzy_libmath_dockerfile.sh](docker/resources/build_fuzzy_libmath_dockerfile.sh) helps you turn your Docker image into a fuzzy-libmath one.\n\n```bash\nusage: ./build_fuzzy_libmath_dockerfile.sh \u003cDOCKER_IMAGE\u003e \u003cTAG\u003e [FUZZY_IMAGE]\n          \u003cDOCKER_IMAGE\u003e: Name of the base Docker image to build\n          \u003cTAG\u003e:          Tag of the new image to build\n          [FUZZY_IMAGE]:  Name of the fuzzy image to copy from (optional)\n                          Requires a fuzzy version \u003e= 0.9.1\n```\n\nTo test your instrumentation:\n\n**Python**\n```python\n\u003e\u003e\u003e import math\n\u003e\u003e\u003e from collections import Counter\n\u003e\u003e\u003e Counter( math.cos(42) for i in range(1000) )\nCounter({-0.39998531498835127: 506, -0.3999853149883513: 249, -0.3999853149883512: 245})\n```\n\n**Octave**\n```matlab\n\u003e\u003e\u003e x = repmath(42, 1000);\n\u003e\u003e\u003e y = cos(x);\n\u003e\u003e\u003e printf(\"%.17f\\n\", unique(y))\n-0.39998531498835133\n-0.39998531498835127\n-0.39998531498835121\n```\n\n\n#### Using Fuzzy in Multi-stage builds\nFuzzy provides a set of recompiled shared objects and tools that facilitate adding\nMonte Carlo Arithmetic to tools. If you've got a Docker container which relies on\nsome of these libraries, you can easily add *Fuzzy* with a [Multi-stage Docker build](https://docs.docker.com/develop/develop-images/multistage-build/).\n\nFor example:\n\n```bash\nFROM verificarlo/fuzzy:latest as fuzzy\n\n# Your target image\nFROM user/image:version\n\n# Copy libmath fuzzy environment from fuzzy image, for example\nRUN mkdir -p /opt/mca-libmath/{fast,standard,quad,mpfr}\nCOPY --from=${2} /opt/mca-libmath/set-fuzzy-libmath.py /usr/local/bin/set-fuzzy-libmath\nCOPY --from=${2} /opt/mca-libmath/fast/libmath.so /opt/mca-libmath/fast/libmath.so\nCOPY --from=${2} /opt/mca-libmath/standard/libmath.so /opt/mca-libmath/standard/libmath.so\nCOPY --from=${2} /opt/mca-libmath/quad/libmath.so /opt/mca-libmath/quad/libmath.so\nCOPY --from=${2} /opt/mca-libmath/mpfr/libmath.so /opt/mca-libmath/mpfr/libmath.so\nCOPY --from=${2} /usr/local/lib/libinterflop* /usr/local/lib/\n\n# If you will also want to recompile more libraries with verificarlo, add these lines\nCOPY --from=fuzzy /usr/local/bin/verificarlo* /usr/local/bin/\nCOPY --from=fuzzy /usr/local/include/* /usr/local/include/\n\n# Preloading the instrumented shared library\nARG FUZZY_LIBMATH_VERSION=standard\nRUN set-fuzzy-libmath --version=${FUZZY_LIBMATH_VERSION}\n\nENV VFC_BACKENDS 'libinterflop_mca.so --precision-binary32=24 --precision-binary64=53 --mode=mca'\n```\n\n#### Running Fuzzy workflows\nIn the context of *Fuzzy* experiments, it is important to remember that by default\neach execution will be evaluated with a unique random state, meaning that when you\nrun it again you may get slightly (or very) different results. If your goal is to\ncharacterize the variability in your results, or obtain a robust estimate of the\n\"true\" mean answer, you will need to then run your tool multiple times and compare\neach execution.\n\nIt's also important to remember that the execution time will be increased when using\n*Fuzzy*, as compared to running tools in a determinitic environment. Depending on\nthe tool, the instrumentation, and the MCA mode, this additional overhead may range\nfrom negligible to the order of a ***30x*** slowdown.\n\nIncluded in the references below are some references which can be referred to when\ndeciding how many perturbations to run, how to consider groups of results, and\ndemonstrating the differences in overhead that may exist between different\ninstrumentations.\n\n#### Quick overview of Monte Carlo Arithmetic\nA detailed explanation of Monte Carlo Arithmetic can be found in the references below\nor the [Verificarlo repository](https://github.com/verificarlo/verificarlo). In short,\nhere is some terminology to get you started.\n\nIn this form of stochastic arithmetic you have three modes for perturbing your\nfloating-point operations `x = a op b` where `op` is in `{+,-,*,/}`:\n\n1. Random Rounding (RR): `x = inexact(a op b)`\n2. Precision Bounding (PB): `x = inexact(a) op inexact(b)`\n3. Full MCA (MCA): `x = inexact(inexact(a) op inexact(b))`\n\nIn this implementation, the `inexact` operation is the addition of a `0`-centered\nuniform random variable at the target bit of precision.\n\n#### Common failures\nAs you become familiar with *Fuzzy*, you may run into some of the following common\nsources of error in your (or other) software:\n\n- Fuzzy not running on your system, and giving `Illegal instruction` errors (likely exit-code `132`)? This is probably because our environments were built in a different architecture than yours! We tried to turn off as many optimizations as we could, increasing the portability of these images, but sometimes libraries don't listen to instructions very well... Try rebuilding the images on your local machine. to fix the problem. To our knowledge, `Scipy` is the first package in the install chain which has some dependencies that may ignore the optimization-disabling compiler flags, so you could try starting your rebuild from the `Numpy` image.\n\n- When using MCA or PB modes of perturbation, operations which rely on integer\nvalues which happen to be stored in floating point containers may crash. For instance,\nimagine you're creating an array and store the desired length as `3.0` instead of `3`;\nintroduced perturbations may shift your length away from the exact-value of `3.0`, and\nthen your tool could (justifiably) crash when trying to allocate a `2.99..` element array.\nYou can fix these types of bugs by simply casting your length variable to an integer.\n\n- In cases where piecewise approximations are used to solve complex functions (e.g.\n[the sin function in libc](https://github.com/lattera/glibc/blob/master/sysdeps/ieee754/dbl-64/s_sin.c).),\nit is possible that perturbations will trigger distinct branching and result in\npossibly large discontinuity between results. *Fuzzy* appraches instrumenting these\nlibraries via wrappers which ultimately perturb the function inputs and outputs,\nrather than the internal arithmetic operations which may take place. An example\nof this can be found for [`libmath`](/docker/resources/libmath/).\n\n\n## Contributing\n\nFor instructions on how to contribute, please refer to the [Contribution Guide](.github/CONTRIBUTING.md).\n\n## Presentation\nIf you prefer a visual illustration of everything we've written above, feel free to check-out [the slides\nwe recently presented at Scipy2021](https://figshare.com/articles/presentation/Fuzzy_Environments/14991825)!\n\n## References\nThe *Fuzzy* ecosystem has emerged from — and been used in — several scientific\npublications. Below is a list of papers which present the techniques used, the\ntools which have been developed accordingly, and demonstrate how decision-making\nand applications can be built atop them:\n\n\u003e Parker, Douglas Stott, Brad Pierce, and Paul R. Eggert. \"Monte Carlo arithmetic: how to gamble with floating point and win.\" Computing in Science \u0026 Engineering 2.4 (2000): 58-68.\n\u003e\n\u003e Frechtling, Michael, and Philip HW Leong. \"Mcalib: Measuring sensitivity to rounding error with monte carlo programming.\" ACM Transactions on Programming Languages and Systems (TOPLAS) 37.2 (2015): 1-25.\n\u003e\n\u003e C. Denis, P. De Oliveira Castro and E. Petit,  \"Verificarlo: Checking Floating Point Accuracy through Monte Carlo Arithmetic,\" in 2016 IEEE 23nd Symposium on Computer Arithmetic (ARITH), Silicon Valley, CA, USA, 2016 pp. 55-62.\n\u003e\n\u003e Chatelain, Yohan, et al. \"VeriTracer: Context-enriched tracer for floating-point arithmetic analysis.\" 2018 IEEE 25th Symposium on Computer Arithmetic (ARITH). IEEE, 2018.\n\u003e\n\u003e Kiar, Gregory, et al. \"Comparing perturbation models for evaluating stability of neuroimaging pipelines.\" The International Journal of High Performance Computing Applications 34.5 (2020): 491-501.\n\u003e\n\u003e Sohier, Devan, et al. \"Confidence Intervals for Stochastic Arithmetic.\" ACM Transactions on Mathematical Software (TOMS) 47.2 (2021): 1-33.\n\u003e\n\u003e Kiar, Gregory, et al. \"Data Augmentation Through Monte Carlo Arithmetic Leads to More Generalizable Classification in Connectomics.\" bioRxiv (2020).\n\u003e\n\u003e Chatelain, Yohan, et al. \"PyTracer: Automatically profiling numerical instabilities in Python.\" IEEE Transactions on Computers (2022).\n\u003e\n\u003e Des Ligneris, Morgane, et al. \"Reproducibility of tumor segmentation outcomes with a deep learning model.\" 2023 IEEE 20th International Symposium on Biomedical Imaging (ISBI). IEEE, 2023.\n\u003e \n\u003e Pepe, Inés Gonzalez, et al. \"Numerical Uncertainty of Convolutional Neural Networks Inference for Structural Brain MRI Analysis.\" International Workshop on Uncertainty for Safe Utilization of Machine Learning in Medical Imaging. Cham: Springer Nature Switzerland, 2023.\n\u003e\n\u003e Chatelain, Yohan, et al. \"A numerical variability approach to results stability tests and its application to neuroimaging.\" arXiv preprint arXiv:2307.01373 (2023).\n\u003e \n\u003e Vila, Gaël, et al. \"The Impact of Hardware Variability on Applications Packaged with Docker and Guix: a Case Study in Neuroimaging.\" (2024).\n\u003e\n\u003e Gonzalez Pepe, Inés, et al. \"Numerical stability of DeepGOPlus inference.\" Plos one 19.1 (2024): e0296725.\n\n## License\nThe *Fuzzy* copyright belongs to all contributors of this repository, and it is\n[licensed](./LICENSE) for public use under the same terms as the\n[LLVM project](https://foundation.llvm.org/relicensing/LICENSE.txt), which is a\nmodified version of the Apache 2.0 license.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fverificarlo%2Ffuzzy","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fverificarlo%2Ffuzzy","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fverificarlo%2Ffuzzy/lists"}