{"id":21094632,"url":"https://github.com/ise-uiuc/nablafuzz","last_synced_at":"2025-08-17T10:40:17.100Z","repository":{"id":71821576,"uuid":"599623430","full_name":"ise-uiuc/NablaFuzz","owner":"ise-uiuc","description":"Fuzzing Automatic Differentiation in Deep-Learning Libraries (ICSE'23)","archived":false,"fork":false,"pushed_at":"2024-03-02T14:34:23.000Z","size":11210,"stargazers_count":22,"open_issues_count":0,"forks_count":4,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-05-16T14:45:07.678Z","etag":null,"topics":["autodiff","autograd","deep-learning","fuzzing","jax","oneflow","pytorch","tensorflow"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ise-uiuc.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-02-09T14:35:43.000Z","updated_at":"2024-05-29T19:25:33.000Z","dependencies_parsed_at":"2025-05-16T14:52:56.674Z","dependency_job_id":null,"html_url":"https://github.com/ise-uiuc/NablaFuzz","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/ise-uiuc/NablaFuzz","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ise-uiuc%2FNablaFuzz","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ise-uiuc%2FNablaFuzz/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ise-uiuc%2FNablaFuzz/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ise-uiuc%2FNablaFuzz/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ise-uiuc","download_url":"https://codeload.github.com/ise-uiuc/NablaFuzz/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ise-uiuc%2FNablaFuzz/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":270837424,"owners_count":24654379,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-17T02:00:09.016Z","response_time":129,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["autodiff","autograd","deep-learning","fuzzing","jax","oneflow","pytorch","tensorflow"],"created_at":"2024-11-19T22:18:12.024Z","updated_at":"2025-08-17T10:40:17.057Z","avatar_url":"https://github.com/ise-uiuc.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ∇Fuzz\n\nThis is the artifact of the research paper, \"Fuzzing Automatic Differentiation in Deep-Learning Libraries\" (in submission). The authors make their best attempt to anonymize the artifacts. Please note that it may still be possible that the authors' identities are revealed by deeply studying this repository.\n\n## About\n\n∇Fuzz is a fully automated API-level fuzzer targeting AD in DL libraries, which utilizes differential testing on different execution scenarios to test both first-order and high-order gradients, and also includes automated filtering strategies to remove false positives caused by numerical instability.\n\nThis is the ∇Fuzz's implementation on PyTorch, TensorFlow, JAX, and OneFlow.\n\n## Detected Bugs\n\nUntil submission, ∇Fuzz has detected 173 bugs in total for PyTorch, TensorFlow, JAX and OneFlow, with 144 already confirmed by developers.\n\nWe provide a list of confirmed bug reports on [PyTorch](https://github.com/NablaFuzz/NablaFuzz/blob/main/PyTorch-confirmed-bugs.csv), [TensorFlow](https://github.com/NablaFuzz/NablaFuzz/blob/main/TensorFlow-confirmed-bugs.csv), [JAX](https://github.com/NablaFuzz/NablaFuzz/blob/main/Jax-confirmed-bugs.csv) and [OneFlow](https://github.com/NablaFuzz/NablaFuzz/blob/main/OneFlow-confirmed-bugs.csv).\n\u003e Please kindly note that this might reveal our identity, please think twice before clicking into this.\n\n## Getting Started\n\n### Step 1: Requirements\n\n1. Our testing framework leverages [MongoDB](https://www.mongodb.com/) so you need to [install and run MongoDB](https://docs.mongodb.com/manual/installation/) first.\n\n\t- After installing MongoDB and before loading the database, run the command `ulimit -n 640000` to adjust the limit that the system resources a process may use. You can see this [document](https://docs.mongodb.com/manual/reference/ulimit/) for more details.\n\n2. Python version \u003e= 3.9.0 (It must support f-string.)\n\n\t- highly recommend to use Python 3.9\n\n3. Check our dependent python libraries in `requirements.txt` and install with:\n\n\t```\n\tpip install -r requirements.txt\n\t```\n\t\n\t- For OneFlow, please use the following command to install:\n\t\n\t```shell\n\tpython3 -m pip install --find-links https://release.oneflow.info oneflow==0.7.0\n\t```\n\n4. JAX install\n\nFor JAX, if you want to reproduce our results on version `0.3.14`, you could use the following commands to install jaxlib on CPU via the wheel archive:\n\n```\n# Install jaxlib on CPU via the wheel archive\npip install jax[cpu]==0.3.14 -f https://storage.googleapis.com/jax-releases/jax_releases.html\n\n# Install the jaxlib 0.3.25 CPU wheel directly\npip install jaxlib==0.3.14 -f https://storage.googleapis.com/jax-releases/jax_releases.html\n```\n\nFor more information about JAX installation, please refer to the [official website](https://jax.readthedocs.io/en/latest/installation.html#installing-jax).\n\n### Step 2: Set up Database\n\nRun the following commands in current directory (`NablaFuzz`) to load the database.\n\n```shell\nmongorestore NablaFuzz-PyTorch-Jax/dump\nmongorestore NablaFuzz-Oneflow/dump\nmongorestore NablaFuzz-TensorFlow/dump\n```\n\n\n### Step 3: Run\n\nBefore running, you can set the number of mutants for each API (which is 1000 by default):\n\n```shell\nNUM_MUTANT=100\n```\n\nAlso, you can set the number of APIs you want to test (which is -1 by default, meaning all APIs will be tested)\n\n```shell\nNUM_API=100\n```\n\n#### PyTorch \u0026 JAX\n\nFirst go into the `NablaFuzz-PyTorch-Jax/src` directory,\n\n```shell\ncd NablaFuzz-PyTorch-Jax/src\n```\n\nRun the following command to start ∇Fuzz to test PyTorch\n\n```shell\npython torch_test.py --num $NUM_MUTANT --max_api $NUM_API\n```\n\nThe output will be put in `NablaFuzz-PyTorch-Jax/output-ad/torch/union` directory by default. \n\nTo filter out the inconsistent gradients caused by numerical instability, you can run the folloing commands:\n\n```shell\npython torch-diff-filter.py --num $NUM_MUTANT --max_api $NUM_API\n```\n\nRun the following command to start ∇Fuzz to test JAX\n\n```shell\npython jax_test.py\n```\n\nThe output will be put in `NablaFuzz-PyTorch-Jax/output-ad/jax/union` directory by default. \n\nTo filter out the inconsistent gradients caused by numerical instability, you can run the folloing commands:\n\n```shell\npython jax-diff-filter.py\n```\n\n#### OneFlow\n\nFirst go into the `NablaFuzz-Oneflow` directory,\n\n```shell\ncd NablaFuzz-Oneflow\n```\n\nRun the following commands to start ∇Fuzz to test Oneflow\n\n```shell\npython oneflow_test.py --num $NUM_MUTANT --max_api $NUM_API\n```\n\nThe error code snippets will be put in `NablaFuzz-Oneflow/errors`. \n\n\n#### TensorFlow\n\nFirst go into the `NablaFuzz-TensorFlow/src` directory,\n\n```shell\ncd NablaFuzz-TensorFlow/src\n```\n\nRun the following commands to start ∇Fuzz to test TensorFlow\n\n```shell\npython tf_adtest.py --num $NUM_MUTANT --max_api $NUM_API\npython tf_filter.py \n```\n\nThe outputs will be put in `NablaFuzz-TensorFlow/expr_outputs` by default. \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fise-uiuc%2Fnablafuzz","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fise-uiuc%2Fnablafuzz","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fise-uiuc%2Fnablafuzz/lists"}