{"id":21094624,"url":"https://github.com/ise-uiuc/freefuzz","last_synced_at":"2025-05-16T14:33:48.219Z","repository":{"id":43956547,"uuid":"448804659","full_name":"ise-uiuc/FreeFuzz","owner":"ise-uiuc","description":"Free Lunch for Testing: Fuzzing Deep-Learning Libraries from Open Source (ICSE'22)","archived":false,"fork":false,"pushed_at":"2022-11-02T15:43:15.000Z","size":8810,"stargazers_count":71,"open_issues_count":5,"forks_count":15,"subscribers_count":7,"default_branch":"main","last_synced_at":"2024-07-19T09:19:52.263Z","etag":null,"topics":["fuzzing","mining","pytorch","tensorflow"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ise-uiuc.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-01-17T08:07:35.000Z","updated_at":"2024-06-04T14:43:19.000Z","dependencies_parsed_at":"2023-01-20T21:18:52.823Z","dependency_job_id":null,"html_url":"https://github.com/ise-uiuc/FreeFuzz","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ise-uiuc%2FFreeFuzz","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ise-uiuc%2FFreeFuzz/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ise-uiuc%2FFreeFuzz/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ise-uiuc%2FFreeFuzz/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ise-uiuc","download_url":"https://codeload.github.com/ise-uiuc/FreeFuzz/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":225433449,"owners_count":17473599,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["fuzzing","mining","pytorch","tensorflow"],"created_at":"2024-11-19T22:18:11.262Z","updated_at":"2024-11-19T22:18:11.834Z","avatar_url":"https://github.com/ise-uiuc.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# FreeFuzz\n\nThis is the artifact of the research paper, \"Free Lunch for Testing: Fuzzing Deep-Learning Libraries from Open Source\", at ICSE 2022.\n\n## About\n\nFreeFuzz is the first approach to fuzzing DL libraries via mining from open source. It collects code/models from three different sources: 1) code snippets from the library documentation, 2) library developer tests, and 3) DL models in the wild. Then, FreeFuzz automatically runs all the collected code/models with instrumentation to collect the dynamic information for each covered API. Lastly, FreeFuzz will leverage the traced dynamic information to perform fuzz testing for each covered API.\n\nThis is the FreeFuzz's implementation for testing PyTorch and TensorFlow.\n\n## Getting Started\n\n### 1. Requirements\n\n1. Our testing framework leverages [MongoDB](https://www.mongodb.com/) so you should [install and run MongoDB](https://docs.mongodb.com/manual/installation/) first.\n\t- Run the command `ulimit -n 64000` to adjust the limit that the system resources a process may use. You can see this [document](https://docs.mongodb.com/manual/reference/ulimit/) for more details.\n2. You should check our dependent python libraries in `requirements.txt` and run `pip install -r requirements.txt` to install them\n3. Python version \u003e= 3.8.0 (It must support f-string.)\n\n### 2. Setting Up with Dataset\n\n#### Using Our Dataset\n\nRun the following command to load the database.\n\n```shell\nmongorestore dump/\n```\n\n#### Collecting Data by Yourself\n\n1. Go to `src/instrumentation/{torch, tensorflow}` to see how to intrument the dynamic information and add them into the database\n2. After adding invocation data, you should run the following command to preprocess the data for PyTorch\n\n```shell\ncd src \u0026\u0026 python preprocess/process_data.py torch\n```\n\nor for TensorFlow\n```shell\ncd src \u0026\u0026 python preprocess/process_data.py tf\n```\n\n### 3. Configuration\n\nThere are some hyper-parameters in FreeFuzz and they could be easily configured as follows.\n\nIn `src/config/demo.conf`:\n\n1. MongoDB database configuration.\n\n```conf\n[mongodb]\n# your-mongodb-server\nhost = 127.0.0.1\n# mongodb port\nport = 27017 \n# name of pytorch database\ntorch_database = freefuzz-torch\n# name of tensorflow database\ntf_database = freefuzz-tf\n```\n\n2. Output directory configuration.\n\n```conf\n[output]\n# output directory for pytorch\ntorch_output = torch-output\n# output directory for tensorflow\ntf_output = tf-output\n```\n\n3. Oracle configuration.\n\n```conf\n[oracle]\n# enable crash oracle\nenable_crash = true\n# enable cuda oracle\nenable_cuda = true\n# enable precision oracle\nenable_precision = true\n# float difference bound: if |a-b| \u003e bound, a is different than b\nfloat_difference_bound = 1e-5\n# max time bound: if time(low_precision) \u003e bound * time(high_precision),\n# it will be considered as a potential bug\nmax_time_bound = 10\n# only consider the call with time(call) \u003e time_thresold\ntime_thresold = 1e-3\n```\n\n4. Mutation stratgy configuration.\n\n```conf\n[mutation]\nenable_value_mutation = true\nenable_type_mutation = true\nenable_db_mutation = true\n# the number of times each api is executed\neach_api_run_times = 1000\n```\n\n### 4. Start\n\nAfter finishing above steps, run the following command to start FreeFuzz to test PyTorch\n\n```shell\ncd src \u0026\u0026 python FreeFuzz.py --conf demo_torch.conf\n```\n\nOr run this command to test TensorFlow\n\n```shell\ncd src \u0026\u0026 python FreeFuzz.py --conf demo_tf.conf\n```\n\nTo run the full experiment, run the following command\n```shell\ncd src \u0026\u0026 python FreeFuzz.py --conf expr.conf\n```\nIf you want to use another configuration file, you can put it in `src/config`.\n\nNote that you should specify the configuration file you want to use.\n\n## Notes\n\n1. Some APIs will be skipped since they may crash the program. You can set what you want to skip in the file `src/config/skip_torch.txt` or `src/config/skip_tf`.\n2. For the details of three mutation strategies, please refer to our paper.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fise-uiuc%2Ffreefuzz","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fise-uiuc%2Ffreefuzz","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fise-uiuc%2Ffreefuzz/lists"}