{"id":13440118,"url":"https://github.com/xoreaxeaxeax/sandsifter","last_synced_at":"2025-10-07T15:31:36.220Z","repository":{"id":40238903,"uuid":"98572277","full_name":"xoreaxeaxeax/sandsifter","owner":"xoreaxeaxeax","description":"The x86 processor fuzzer","archived":false,"fork":true,"pushed_at":"2024-02-20T17:57:28.000Z","size":5160,"stargazers_count":4898,"open_issues_count":63,"forks_count":350,"subscribers_count":167,"default_branch":"master","last_synced_at":"2024-09-26T00:41:18.675Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":"Battelle/sandsifter","license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/xoreaxeaxeax.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-07-27T19:18:40.000Z","updated_at":"2024-09-25T02:23:55.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/xoreaxeaxeax/sandsifter","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xoreaxeaxeax%2Fsandsifter","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xoreaxeaxeax%2Fsandsifter/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xoreaxeaxeax%2Fsandsifter/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xoreaxeaxeax%2Fsandsifter/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/xoreaxeaxeax","download_url":"https://codeload.github.com/xoreaxeaxeax/sandsifter/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":219877431,"owners_count":16554892,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-31T03:01:19.939Z","updated_at":"2025-10-07T15:31:30.588Z","avatar_url":"https://github.com/xoreaxeaxeax.png","language":"Python","funding_links":[],"categories":["Python","Tools","\u003ca name=\"security\"\u003e\u003c/a\u003eSecurity and encryption"],"sub_categories":["Open Source"],"readme":"## s a n d s i f t e r \n: the x86 processor fuzzer\n\n### Overview\n\nThe sandsifter audits x86 processors for hidden instructions and hardware bugs,\nby systematically generating machine code to search through a processor's\ninstruction set, and monitoring execution for anomalies. Sandsifter has\nuncovered secret processor instructions from every major vendor; ubiquitous\nsoftware bugs in disassemblers, assemblers, and emulators; flaws in enterprise\nhypervisors; and both benign and security-critical hardware bugs in x86 chips.\n\nWith the multitude of x86 processors in existence, the goal of the tool is to\nenable users to check their own systems for hidden instructions and bugs.\n\nTo run a basic audit against your processor:\n\n```\nsudo ./sifter.py --unk --dis --len --sync --tick -- -P1 -t\n```\n\n![demo_sandsifter](references/sandsifter.gif)\n\nThe computer is systematically scanned for anomalous instructions.  In the upper\nhalf, you can view the instructions that the sandsifter is currently testing on\nthe processor.  In the bottom half, the sandsifter reports anomalies it finds.\n\nThe search will take from a few hours to a few days, depending on the speed of\nand complexity of your processor.  When it is complete, summarize the results:\n\n```\n./summarize.py data/log\n```\n\n![demo_summarizer](references/summarizer.png)\n\nTypically, several million undocumented instructions on your processor will be\nfound, but these generally fall into a small number of different groups.  After\nbinning the anomalies, the summarize tool attempts to assign each instruction to\nan issue category:\n\n* Software bug (for example, a bug in your hypervisor or disassembler),\n* Hardware bug (a bug in your CPU), or\n* Undocumented instruction (an instruction that exists in the processor, but is\n  not acknowledged by the manufacturer)\n\nPress 'Q' to quit and obtain a text based summary of the system scan:\n\nThe results of a scan can sometimes be difficult for the tools to automatically\nclassify, and may require manual analysis. For help analyzing your results, feel\nfree to send the ./data/log file to xoreaxeaxeax@gmail.com.  No personal\ninformation, other than the processor make, model, and revision (from\n/proc/cpuinfo) are included in this log.\n\n\n### Results\n\nScanning with the sandsifter has uncovered undocumented processor features\nacross dozens of opcode categories, flaws in enterprise hypervisors, bugs in\nnearly every major disassembly and emulation tool, and critical hardware bugs\nopening security vulnerabilities in the processor itself.\n\nDetails of the results can be found in the project \n[whitepaper](./references/domas_breaking_the_x86_isa_wp.pdf).\n\n(TODO: detailed results enumeration here)\n\n\n### Building\n\nSandsifter requires first installing the Capstone disassembler:\nhttp://www.capstone-engine.org/.  Capstone can typically be installed with:\n\n```\nsudo apt-get install libcapstone3 libcapstone-dev\nsudo pip install capstone\n```\n\nSandsifter can be built with:\n\n```\nmake\n```\n\nand is then run with \n\n```\nsudo ./sifter.py --unk --dis --len --sync --tick -- -P1 -t\n```\n\n### Flags\n\nFlags are passed to the sifter with --flag, and to the injector with -- -f.\n\nExample:\n\n```\nsudo ./sifter.py --unk --dis --len --sync --tick -- -P1 -t\n```\n\nSifter flags:\n\n```\n--len\n\tsearch for length differences in all instructions (instructions that\n\texecuted differently than the disassembler expected, or did not\n\texist when the disassembler expected them to\n\n--dis\n\tsearch for length differences in valid instructions (instructions that\n\texecuted differently than the disassembler expected)\n\n--unk\n\tsearch for unknown instructions (instructions that the disassembler doesn't\n\tknow about but successfully execute)\n\n--ill\n\tthe inverse of --unk, search for invalid disassemblies (instructions that do\n\tnot successfully execute but that the disassembler acknowledges)\n\n--tick\n\tperiodically write the current instruction to disk\n\n--save\n\tsave search progress on exit\n\n--resume\n\tresume search from last saved state\n\n--sync\n\twrite search results to disk as they are found\n\n--low-mem\n\tdo not store results in memory\n```\n\nInjector flags:\n\n```\n-b\n\tmode: brute force\n\n-r\n\tmode: randomized fuzzing\n\n-t\n\tmode: tunneled fuzzing\n\n-d\n\tmode: externally directed fuzzing\n\n-R\n\traw output mode\n\n-T\n\ttext output mode\n\n-x\n\twrite periodic progress to stderr\n\n-0\n\tallow null dereference (requires sudo)\n\n-D\n\tallow duplicate prefixes\n\n-N\n\tno nx bit support\n\n-s seed\n\tin random search, seed value\n\n-B brute_depth\n\tin brute search, maximum search depth\n\n-P max_prefix\n\tmaximum number of prefixes to search\n\n-i instruction\n\tinstruction at which to start search (inclusive)\n\n-e instruction\n\tinstruction at which to end search (exclusive)\n\n-c core\n\tcore on which to perform search\n\n-X blacklist\n\tblacklist the specified instruction\n\n-j jobs\n\tnumber of simultaneous jobs to run\n\n-l range_bytes\n\tnumber of base instruction bytes in each sub range\n```\n\n\n### Keys\n\nm: Mode - change the search mode (brute force, random, or tunnel) for the sifter\n\nq: Quit - exit the sifter\n\np: Pause - pause or unpause the search\n\n\n### Algorithms\n\nThe scanning supports four different search algorithms, which can be set at the\ncommand line, or cycled via hotkeys.\n\n* Random searching generates random instructions to test; it generally produces\n  results quickly, but is unable to find complex hidden instructions and bugs.\n* Brute force searching tries instructions incrementally, up to a user-specified\n  length; in almost all situations, it performs worse than random searching.\n* Driven or mutation driven searching is designed to create new, increasingly\n  complex instructions through genetic algorithms; while promising, this\n  approach was never fully realized, and is left as a stub for future research.\n* Tunneling is the approach described in the presentation and white paper, and\n  in almost all cases provides the best trade-off between thoroughness and\n  speed.\n\n\n### Tips\n\n* sudo\n\n\tFor best results, the tool should be run as the root user.  This is necessary so\n\tthat the process can map into memory a page at address 0, which requires root\n\tpermissions.  This page prevents many instructions from seg-faulting on memory\n\taccesses, which allows a more accurate fault analysis.\n\n* Prefixes\n\n\tThe primary limitation for the depth of an instruction search is the number\n\tof prefix bytes to explore, with each additional prefix byte increasing the\n\tsearch space by around a factor of 10.  Limit prefix bytes with the -P flag.\n\n* Colors\n\n\tThe interface for the sifter is designed for a 256 color terminal.  While\n\tthe details vary greatly depending on your terminal, this can roughly be\n\taccomplished with:\n\n\t```\n\texport TERM='xterm-256color'\n\t```\n\n* GUI\n\n\tThe interface assumes the terminal is of at least a certain size; if the\n\tinterface is not rendering properly, try increasing the terminal size; this\n\tcan often be accomplished by decreasing the terminal font size.\n\n\tIn some cases, it may be desirable or necessary to run the tool without the\n\tgraphical front end.  This can be done by running the injector directly:\n\n\t```\n\tsudo ./injector -P1 -t -0\n\t```\n\n\tTo filter the results of a direct injector invocation, grep can be used.\n\tFor example,\n\n\t```\n\tsudo ./injector -P1 -r -0 | grep '\\.r' | grep -v sigill\n\t```\n\n\tsearches for instructions for which the processor and disassembler disagreed\n\ton the instruction length (grep '\\.r'), but the instruction successfully\n\texecuted (grep -v sigill).\n\n* Targeted fuzzing\n\n\tIn many cases, it is valuable to direct the fuzzer to a specific target.\n\tFor example, if you suspect that an emulator has flaws around repeated 'lock'\n\tprefixes (0xf0), you could direct the fuzzer to search this region of the\n\tinstruction space with the -i and -e flags:\n\n\t```\n\tsudo ./sifter.py --unk --dis --len --sync --tick -- -t -i f0f0 -e f0f1 -D -P15\n\t```\n\n* Legacy systems\n\n\tFor scanning much older systems (i586 class processors, low memory systems),\n\tpass the --low-mem flag to the sifter and the -N flag to the injector:\n\n\t```\n\tsudo ./sifter.py --unk --dis --len --sync --tick --low-mem -- -P1 -t -N\n\t```\n\n\tIf you observe your scans completing too quickly (for example, a scan\n\tcompletes in seconds), it is typically because these flags are required for\n\tthe processor you are scanning.\n\n* 32 vs. 64 bit\n\n\tBy default, sandsifter is built to target the bitness of the host operating\n\tsystem.  However, some instructions have different behaviors when run in a\n\t32 bit process compared to when run in a 64 bit process.  To explore these\n\tscenarios, it is sometimes valuable to run a 32 bit sandsifter on a 64 bit\n\tsystem.\n\n\tTo build a 32 bit sandsifter on a 64 bit system, Capstone must be installed\n\tas 32 bit; the instructions for this can be found at http://www.capstone-engine.org/.\n\n\tThen sandsifter must be built for a 32 bit architecture:\n\n\t```\n\tmake CFLAGS=-m32\n\t```\n\n\tWith this, the 32 bit instruction space can be explored on a 64 bit system.\n\n\n### References\n\n* A discussion of the techniques and results can be found in the Black Hat\n  [presentation](https://www.youtube.com/watch?v=KrksBdWcZgQ).\n* Technical details are described in the\n  [whitepaper](./references/domas_breaking_the_x86_isa_wp.pdf).\n* Slides from the Black Hat presentation are\n  [here](./references/domas_breaking_the_x86_isa.pdf).\n\n\n### Author\n\nsandsifter is a research effort from Christopher Domas\n([@xoreaxeaxeax](https://twitter.com/xoreaxeaxeax)).\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxoreaxeaxeax%2Fsandsifter","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fxoreaxeaxeax%2Fsandsifter","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxoreaxeaxeax%2Fsandsifter/lists"}