{"id":25237205,"url":"https://github.com/devdacian/solidity-fuzzing-comparison","last_synced_at":"2025-10-26T12:30:38.727Z","repository":{"id":209827262,"uuid":"725040975","full_name":"devdacian/solidity-fuzzing-comparison","owner":"devdacian","description":"A comparison of solidity fuzzing tools Foundry, Echidna \u0026 Medusa","archived":false,"fork":false,"pushed_at":"2025-02-01T00:57:51.000Z","size":237,"stargazers_count":145,"open_issues_count":0,"forks_count":26,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-02-01T01:31:00.681Z","etag":null,"topics":["echidna","fuzz-testing","fuzz-testing-foundry","fuzzing","smart-contracts","solidity"],"latest_commit_sha":null,"homepage":"","language":"Solidity","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/devdacian.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-11-29T10:11:50.000Z","updated_at":"2025-02-01T00:57:56.000Z","dependencies_parsed_at":"2023-12-04T11:24:14.677Z","dependency_job_id":"585e7e57-7e51-4467-ba24-cd4641037434","html_url":"https://github.com/devdacian/solidity-fuzzing-comparison","commit_stats":null,"previous_names":["devdacian/solidity-fuzzing-comparison"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/devdacian%2Fsolidity-fuzzing-comparison","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/devdacian%2Fsolidity-fuzzing-comparison/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/devdacian%2Fsolidity-fuzzing-comparison/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/devdacian%2Fsolidity-fuzzing-comparison/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/devdacian","download_url":"https://codeload.github.com/devdacian/solidity-fuzzing-comparison/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":238320490,"owners_count":19452563,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["echidna","fuzz-testing","fuzz-testing-foundry","fuzzing","smart-contracts","solidity"],"created_at":"2025-02-11T15:33:38.611Z","updated_at":"2025-10-26T12:30:38.272Z","avatar_url":"https://github.com/devdacian.png","language":"Solidity","funding_links":[],"categories":["Practical Code Samples"],"sub_categories":["Utils"],"readme":"# Solidity Fuzzing Challenge: Foundry vs Echidna vs Medusa (plus Halmos \u0026 Certora) #\n\nA comparison of solidity fuzzing tools [Foundry](https://book.getfoundry.sh/), [Echidna](https://secure-contracts.com/program-analysis/echidna/index.html) \u0026 [Medusa](https://github.com/crytic/medusa) also considering Formal Verification tools such as [Halmos](https://github.com/a16z/halmos) and [Certora](https://docs.certora.com/en/latest/docs/user-guide/tutorials.html). This challenge set is not intended to be an academically rigorous benchmark but rather to present the experiences of an auditor \"in the trenches\"; the primary goal is finding the best performance \"out of the box\" with as little guidance \u0026 tweaking as possible.\n\nMany of the challenges are simplified versions of audit findings from my private audits at [Cyfrin](https://www.cyfrin.io). These findings could have been found by the protocol developers themselves prior to an external audit if the protocol had written the correct [fuzz testing invariants](https://dacian.me/find-highs-before-external-auditors-using-invariant-fuzz-testing). Hence a secondary goal of this repo is to show developers how to write better fuzz testing invariants to improve their protocol security prior to engaging external auditors.\n\n## Setup ##\n\nEnsure you are using recent versions of [Foundry](https://github.com/foundry-rs/foundry), [Echidna](https://github.com/crytic/echidna) and [Medusa](https://github.com/crytic/medusa).\n\nConfigure [solc-select](https://github.com/crytic/solc-select) for Echidna \u0026 Medusa:\n\n`solc-select install 0.8.23`\\\n`solc-select use 0.8.23`\n\nTo compile this project:\n\n`forge build`\n\nEvery exercise has a `basic` configuration and/or `advanced` fuzz configuration for Foundry, Echidna \u0026 Medusa. The `basic` configuration does not guide the fuzzer at all; it simply sets up the scenario and allows the fuzzer to do whatever it wants. The `advanced` configuration guides the fuzzer to the functions it should call and helps to eliminate invalid inputs which result in useless fuzz runs.\n\n## Results ##\n\n### Challenge #1 Naive Receiver: (Winner TIED ALL) ###\n\nIn `basic` configuration Foundry, Echidna \u0026 Medusa are able to break the simpler invariant but not the more valuable and difficult one. In `advanced` configuration all 3 fuzzers can break both invariants. All 3 fuzzers reduce the exploit chain to a very concise \u0026 optimized transaction set and present this to the user in an easy to understand output. As a result they are tied and there is no clear winner.\n\n### Challenge #2 Unstoppable: (Winner TIED ALL) ###\n\nAll Fuzzers in `basic` configuration can break both invariants; Foundry appears to be the slightly faster.\n\n### Challenge #3 Proposal: (Winner TIED ALL) ###\n\nFoundry, Echidna \u0026 Medusa in `basic` mode are able to easily break the invariant, resulting in a tie.\n\n### Challenge #4 Voting NFT: (Winner TIED ALL) ###\n\nIn `basic` configuration Foundry, Echidna \u0026 Medusa are all able to break the easier invariant but not the more difficult one. All Fuzzers are able to provide the user with a minimal transaction set to generate the exploit. Hence they are tied, there is no clear winner.\n\n### Challenge #5 Token Sale: (Winner MEDUSA) ###\n\nIn `basic` configuration Foundry \u0026 Echidna can only break the easier and more valuable invariant which leads to a Critical exploit but not the harder though less valuable invariant which leads to a High/Medium. However Medusa is able to almost immediately break both invariants in unguided `basic` mode, making Medusa the clear winner.\n\n### Challenge #6 Rarely False: (Winner TIED HALMOS \u0026 CERTORA) ###\n\nBoth Echidna \u0026 Foundry are unable to break the assertion in this stateless fuzzing challenge. Medusa [used](https://twitter.com/DevDacian/status/1732199452344221913) to be able to break it almost instantly but has [regressed](https://github.com/crytic/medusa/issues/305) in performance after recent changes and is now unable to break it. Halmos and Certora can break it so they are the winners.\n\n### Challenge #7 Byte Battle: (Winner TIED ALL)\n\nAll tools are able to quickly break this challenge.\n\n### Challenge #8 Omni Protocol: (Winner MEDUSA)\n\nAll 3 Fuzzers configured in `advanced` guided mode attempted to break 16 invariants on Beta Finance [Omni Protocol](https://github.com/beta-finance/Omni-Protocol). Medusa is typically able to break 2 invariants within 5 minutes (often much sooner on subsequent runs) though on the first run can take a bit longer. Echidna can sometimes break 1 invariant within 5 minutes and Foundry appears to never be able to break any invariants within 5 minutes. Hence Medusa is the clear winner. The fuzzers written for this challenge were [contributed](https://github.com/beta-finance/Omni-Protocol/pull/2) to Beta Finance.\n\n### Challenge #9 -\u003e #14\n\nSome additional solvers have been added based upon real-world findings from my private audits.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdevdacian%2Fsolidity-fuzzing-comparison","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdevdacian%2Fsolidity-fuzzing-comparison","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdevdacian%2Fsolidity-fuzzing-comparison/lists"}