{"id":13797371,"url":"https://github.com/jaybosamiya/security-notes","last_synced_at":"2026-02-05T05:34:49.096Z","repository":{"id":78706466,"uuid":"86937218","full_name":"jaybosamiya/security-notes","owner":"jaybosamiya","description":":notebook: Some security related notes","archived":false,"fork":false,"pushed_at":"2017-08-13T01:54:57.000Z","size":22,"stargazers_count":459,"open_issues_count":0,"forks_count":41,"subscribers_count":34,"default_branch":"master","last_synced_at":"2025-07-16T17:21:03.410Z","etag":null,"topics":["binary-analysis","hacking","notes","pwning","reverse-engineering","security"],"latest_commit_sha":null,"homepage":"https://www.jaybosamiya.com/blog/security-notes/","language":null,"has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jaybosamiya.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2017-04-01T19:25:39.000Z","updated_at":"2025-05-31T22:43:30.000Z","dependencies_parsed_at":null,"dependency_job_id":"836d831e-7ffd-4a1c-b5b8-9ff6f23eb78d","html_url":"https://github.com/jaybosamiya/security-notes","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/jaybosamiya/security-notes","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jaybosamiya%2Fsecurity-notes","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jaybosamiya%2Fsecurity-notes/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jaybosamiya%2Fsecurity-notes/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jaybosamiya%2Fsecurity-notes/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jaybosamiya","download_url":"https://codeload.github.com/jaybosamiya/security-notes/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jaybosamiya%2Fsecurity-notes/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29113722,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-05T05:31:32.482Z","status":"ssl_error","status_checked_at":"2026-02-05T05:31:29.075Z","response_time":65,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["binary-analysis","hacking","notes","pwning","reverse-engineering","security"],"created_at":"2024-08-03T23:01:29.101Z","updated_at":"2026-02-05T05:34:49.076Z","avatar_url":"https://github.com/jaybosamiya.png","language":null,"readme":"# Some security related notes\n\nI have started to write down notes on the security related videos I\nwatch (as a way of quick recall).\n\nThese might be more useful to beginners.\n\nThe order of notes here is _not_ in order of difficulty, but in\nreverse chronological order of how I write them (i.e., latest first).\n\n## License\n\n[![CC BY-NC-SA 4.0](https://i.creativecommons.org/l/by-nc-sa/4.0/88x31.png)](http://creativecommons.org/licenses/by-nc-sa/4.0/)\n\nThis work is licensed under a [Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License](http://creativecommons.org/licenses/by-nc-sa/4.0/).\n\n## The Notes Themselves\n\n### Misc RE tips\n\nWritten on Aug 12 2017\n\n\u003e Influenced by Gynvael's CONFidence CTF 2017\n\u003e Livestreams [here](https://www.youtube.com/watch?v=kZtHy9GqQ8o)\n\u003e and [here](https://www.youtube.com/watch?v=W7s5CWaw6I4); and by his\n\u003e Google CTF Quals 2017\n\u003e Livestream [here](https://www.youtube.com/watch?v=KvyBn4Btv8E)\n\nSometimes, a challenge might implement a complicated task by\nimplementing a VM. It is not always necessary to completely reverse\nengineer the VM and work on solving the challenge. Sometimes, you can\nRE a little bit, and once you know what is going on, you can hook into\nthe VM, and get access to stuff that you need. Additionally, timing\nbased side-channel attacks become easier in VMs (mainly due to more\nnumber of _\"real\"_ instructions executed.\n\nCryptographically interesting functions in binaries can be recognized\nand quickly RE'd simply by looking for the constants and searching for\nthem online. For standard crypto functions, these constants are\nsufficient to quickly guess at a function. Simpler crypto functions\ncan be recognized even more easily. If you see a lot of XORs and stuff\nlike that happening, and no easily identifiable constants, it is\nprobably hand-rolled crypto (and also possibly broken).\n\nSometimes, when using IDA with HexRays, the disassembly view might be\nbetter than the decompilation view. This is especially true if you\nnotice that there seems to be a lot of complication going on in the\ndecompilation view, but you notice repetitive patterns in the\ndisassembly view. (You can quickly switch b/w the two using the space\nbar). For example, if there is a (fixed size) big-integer library\nimplemented, then the decompilation view is terrible, but the\ndisassembly view is easy to understand stuff (and easily recognizable\ndue to the repetitive \"with-carry\" instructions such as\n`adc`). Additionally, when analyzing like this, using the \"Group\nNodes\" feature in IDA's graph view is extremely useful to quickly\nreduce the complexity of your graph, as you understand what each node\ndoes.\n\nFor weird architectures, having a good emulator is extremely\nuseful. Especially, an emulator that can give you a dump of the memory\ncan be used to quickly figure out what is going on, and recognize\ninteresting portions, once you have the memory out of the\nemulator. Additionally, using an emulator implemented in a comfortable\nlanguage (such as Python), means that you could run things exactly how\nyou like. For example, if there is some interesting part of the code\nyou might wish to run multiple times (for example, to brute force or\nsomething), then using the emulator, you can quickly code up something\nthat does only that part of the code, rather than having to run the\ncomplete program.\n\nBeing lazy is good, when REing. Do NOT waste time reverse engineering\neverything, but spend enough time doing recon (even in an RE\nchallenge!), so as to be able to reduce the time spent on actually\ndoing the more difficult task of REing. What recon, in such a\nsituation means, is to just take quick looks at different functions,\nwithout spending too much time on analyzing each function\nthoroughly. You just quickly gauge what the function might be about\n(for example \"looks like a crypto thing\", or \"looks like a memory\nmanagement thing\", etc.)\n\nFor unknown hardware or architecture, spend enough time looking it up\non Google, you might get lucky with a bunch of useful tools or\ndocuments that might help you build tools quicker. Often times, you'll\nfind toy emulator etc implementations that might be useful as a quick\npoint to start off from. Alternatively, you might get some interesting\ninfo (such as how bitmaps are stored, or how strings are stored, or\nsomething) with which you can write a quick \"fix\" script, and then use\nnormal tools to see if interesting stuff is there.\n\nGimp (the image manipulation tool), has a very cool open/load\nfunctionality to see raw pixel data. You can use this to quickly look\nfor assets or repetitive structures in raw binary data. Do spend time\nmessing around with the settings to see if more info can be gleaned\nfrom it.\n\n### Analysis for RE and Pwning tasks in CTFs\n\nWritten on Jul 2 2017\n\n\u003e Influenced by a discussion with [@p4n74](https://github.com/p4n74/)\n\u003e and [@h3rcul35](https://github.com/aazimcr) on\n\u003e the [InfoSecIITR](https://github.com/InfoSecIITR/) #bin chat. We\n\u003e were discussing on how sometimes beginners struggle to start with a\n\u003e larger challenge binary, especially when it is stripped.\n\nTo either solve the RE challenge, or to be able to pwn it, one must\nfirst analyze the given binary, in order to be able to effectively\nexploit it. Since the binary might possibly be stripped etc (found\nusing `file`) one must know where to begin analysis, to get a foothold\nto build up from.\n\nThere's a few styles of analysis, when looking for vulnerabilities in\nbinaries (and from what I have gathered, different CTF teams have\ndifferent preferences):\n\n1. Static Analysis\n\n 1.1. Transpiling complete code to C\n\n  This kind of analysis is sort of rare, but is quite useful for\n  smaller binaries. The idea is to go in an reverse engineer the\n  entirety of the code. Each and every function is opened in IDA\n  (using the decompiler view), and renaming (shortcut: n) and retyping\n  (shortcut: y) are used to quickly make the decompiled code much more\n  readable. Then, all the code is copied/exported into a separate .c\n  file, which can be compiled to get an equivalent (but not same)\n  binary to the original. Then, source code level analysis can be\n  done, to find vulns etc. Once the point of vulnerability is found,\n  then the exploit is built on the original binary, by following along\n  in the nicely decompiled source in IDA, side by side with the\n  disassembly view (use Tab to quickly switch between the two; and use\n  Space to switch quickly between Graph and Text view for\n  disassembly).\n\n 1.2. Minimal analysis of decompilation\n\n  This is done quite often, since most of the binary is relatively\n  useless (from the attacker's perspective). You only need to analyze\n  the functions that are suspicious or might lead you to the vuln. To\n  do this, there are some approaches to start off:\n\n  1.2.1. Start from main\n\n   Now usually, for a stripped binary, even main is not labelled (IDA\n   6.9 onwards does mark it for you though), but over time, you learn\n   to recognize how to reach the main from the entry point (where IDA\n   opens at by default). You jump to that and start analyzing from\n   there.\n\n  1.2.2. Find relevant strings\n\n   Sometimes, you know some specific strings that might be outputted\n   etc, that you know might be useful (for example \"Congratulations,\n   your flag is %s\" for an RE challenge). You can jump to Strings View\n   (shortcut: Shift+F12), find the string, and work backwards using\n   XRefs (shortcut: x). The XRefs let you find the path of functions\n   to that string, by using XRefs on all functions in that chain,\n   until you reach main (or some point that you know).\n\n  1.2.3. From some random function\n\n   Sometimes, not specific string might be useful, and you don't want\n   to start from main. So instead, you quickly flip through the whole\n   functions list, looking for functions that look suspicious (such as\n   having lots of constants, or lots of xors, etc) or call important\n   functions (XRefs of malloc, free, etc), and you start off from\n   there, and go both forwards (following functions it calls) and\n   backwards (XRefs of the function)\n\n 1.3. Pure disassembly analysis\n\n  Sometimes, you cannot use the decompilation view (because of weird\n  architecture, or anti-decompilation techniques, or hand written\n  assembly, or decompilation looking too unnecessarily complex). In\n  that case, it is perfectly valid to look purely at the disassembly\n  view. It is extremely useful (for new architectures) to turn on Auto\n  Comments, which shows a comment explaining each\n  instruction. Additionally, the node colorization and group nodes\n  functionalities are immensely helpful. Even if you don't use any of\n  these, regularly marking comments in the disassembly helps a lot. If\n  I am personally doing this, I prefer writing down Python-like\n  comments, so that I can quickly then transpile in manually into\n  Python (especially useful for RE challenges, where you might have to\n  use Z3 etc).\n\n 1.4. Using platforms like BAP, etc.\n\n  This kind of analysis is (semi-)automated, and is usually more\n  useful for much larger software, and is rarely directly used in\n  CTFs.\n\n2. Fuzzing\n\n Fuzzing can be an effective technique to quickly get to the vuln,\n without having to actually understand it initially. By using a\n fuzzer, one can get a lot of low-hanging-fruit style of vulns, which\n then need to be analyzed and triaged to get to the actual vuln. See\n my notes\n on\n [basics of fuzzing](https://github.com/jaybosamiya/security-notes#basics-of-fuzzing) and\n [genetic fuzzing](https://github.com/jaybosamiya/security-notes#genetic-fuzzing) for\n more info.\n\n3. Dynamic Analysis\n\n Dynamic Analysis can be used after finding a vuln using static\n analysis, to help build exploits quickly. Alternatively, it can be\n used to find the vuln itself. Usually, one starts up the executable\n inside a debugger, and tries to go along code paths that trigger the\n bug. By placing breakpoints at the right locations, and analyzing the\n state of the registers/heap/stack/etc, one can get a good idea of\n what is going on.  One can also use debuggers to quickly identify\n interesting functions. This can be done, for example, by setting\n temporary breakpoints on all functions initially; then proceeding to\n do 2 walks - one through all uninteresting code paths; and one\n through only a single interesting path. The first walk trips all the\n uninteresting functions and disables those breakpoints, thereby\n leaving the interesting ones showing up as breakpoints during the\n second walk.\n\nMy personal style for analysis, is to start with static analysis,\nusually from main (or for non-console based applications, from\nstrings), and work towards quickly finding a function that looks\nodd. I then spend time and branch out forwards and backwards from\nhere, regularly writing down comments, and continuously renaming and\nretyping variables to improve the decompilation. Like others, I do use\nnames like Apple,Banana,Carrot,etc for seemingly useful, but as of yet\nunknown functions/variables/etc, to make it easier to analyze (keeping\ntrack of func_123456 style of names is too difficult for me). I also\nregularly use the Structures view in IDA to define structures (and\nenums) to make the decompilation even nicer. Once I find the vuln, I\nusually move to writing a script with pwntools (and use that to call a\n`gdb.attach()`). This way, I can get a lot of control over what is\ngoing on. Inside gdb, I usually use plain gdb, though I have added a\ncommand `peda` that loads peda instantly if needed.\n\nMy style is definitely evolving though, as I have gotten more\ncomfortable with my tools, and also with custom tools I have written\nto speed things up. I would be happy to hear of other analysis styles,\nas well as possible changes to my style that might help me get\nfaster. For any comments/criticisms/praise you have, as always, I can\nbe reached on Twitter [@jay\\_f0xtr0t](http://twitter.com/@jay_f0xtr0t).\n\n### Return Oriented Programming\n\nWritten on Jun 4 2017\n\n\u003e Influenced by [this](https://www.youtube.com/watch?v=iwRSFlZoSCM)\n\u003e awesome live stream by Gynvael Coldwind, where he discusses the\n\u003e basics of ROP, and gives a few tips and tricks\n\nReturn Oriented Programming (ROP) is one of the classic exploitation\ntechniques, that is used to bypass the NX (non executable memory)\nprotection. Microsoft has incorporated NX as DEP (data execution\nprevention). Even Linux etc, have it effective, which means that with\nthis protection, you could no longer place shellcode onto heap/stack\nand have it execute just by jumping to it. So now, to be able to\nexecute code, you jump into pre-existing code (main binary, or its\nlibraries -- libc, ldd etc on Linux; kernel32, ntdll etc on\nWindows). ROP comes into existence by re-using fragments of this code\nthat is already there, and figuring out a way to combine those\nfragments into doing what you want to do (which is of course, HACK THE\nPLANET!!!).\n\nOriginally, ROP started with ret2libc, and then became more advanced\nover time by using many more small pieces of code. Some might say that\nROP is now \"dead\", due to additional protections to mitigate it, but\nit still can be exploited in a lot of scenarios (and definitely\nnecessary for many CTFs).\n\nThe most important part of ROP, is the gadgets. Gadgets are \"usable\npieces of code for ROP\". That usually means pieces of code that end\nwith a `ret` (but other kinds of gadgets might also be useful; such as\nthose ending with `pop eax; jmp eax` etc). We chain these gadgets\ntogether to form the exploit, which is known as the _ROP chain_.\n\nOne of the most important assumptions of ROP is that you have control\nover the stack (i.e., the stack pointer points to a buffer that you\ncontrol). If this is not true, then you will need to apply other\ntricks (such as stack pivoting) to gain this control before building a\nROP chain.\n\nHow do you extract gadgets? Use downloadable tools (such\nas [ropgadget](http://shell-storm.org/project/ROPgadget/)) or online\ntool (such as [ropshell](http://ropshell.com/)) or write your own\ntools (might be more useful for more difficult challenges sometimes,\nsince you can tweak it to the specific challenge if need\nbe). Basically, we just need the addresses that we can jump to for\nthese gadgets. This is where there might be a problem with ASLR etc\n(in which case, you get a leak of the address, before moving on to\nactually doing ROP).\n\nSo now, how do we use these gadgets to make a ropchain? We first look\nfor \"basic gadgets\". These are gadgets that can do _simple_ tasks for\nus (such as `pop ecx; ret`, which can be used to load a value into ecx\nby placing the gadget, followed by the value to be loaded, followed by\nrest of chain, which is returned to after the value is loaded). The\nmost useful basic gadgets, are usually \"set a register\", \"store\nregister value at address pointed to by register\", etc.\n\nWe can build up from these primitive functions to gain higher level\nfunctionality (similar to my post\ntitled [exploitation abstraction](#exploitation-abstraction)). For\nexample, using the set-register, and store-value-at-address gadgets,\nwe can come up with a \"poke\" function, that lets us set any specific\naddress with a specific value. Using this, we can build a\n\"poke-string\" function that lets us store any particular string at any\nparticular location in memory. Now that we have poke-string, we are\nbasically almost done, since we can create any structures that we want\nin memory, and can also call any functions we want with the parameters\nwe want (since we can set-register, and can place values on stack).\n\nOne of the most important reasons to build from these lower order\nprimitives to larger functions that do more complex things, is to\nreduce the chances of making mistakes (which is common in ROP\notherwise).\n\nThere are more complex ideas, techniques, and tips for ROP, but that\nis possibly a topic for a separate note, for a different time :)\n\nPS: Gyn has a blogpost\non [Return-Oriented Exploitation](http://gynvael.coldwind.pl/?id=149)\nthat might be worth a read.\n\n### Genetic Fuzzing\n\nWritten on May 27 2017; extended on May 29 2017\n\n\u003e Influenced by [this](https://www.youtube.com/watch?v=JhsHGms_7JQ)\n\u003e amazing live stream by Gynvael Coldwind, where he talks about the\n\u003e basic theory behind genetic fuzzing, and starts to build a basic\n\u003e genetic fuzzer.  He then proceeds to complete the implementation\n\u003e in [this](https://www.youtube.com/watch?v=HN_tI601jNU) live stream.\n\n\"Advanced\" fuzzing (compared to a blind fuzzer, described in\nmy [\"Basics of Fuzzing\"](#basics-of-fuzzing) note). It also\nmodifies/mutates bytes etc, but it does it a little bit smarter than\nthe blind \"dumb\" fuzzer.\n\nWhy do we need a genetic fuzzer?\n\nSome programs might be \"nasty\" towards dumb fuzzers, since it is\npossible that a vulnerability might require a whole bunch of\nconditions to be satisfied to be reached. In a dumb fuzzer, we have\nvery low probability of this happening since it doesn't have any idea\nif it is making any progress or not. As a specific example, if we have\nthe code `if a: if b: if c: if d: crash!` (let's call it the CRASHER\ncode), then in this case we need 4 conditions to be satisfied to crash\nthe program. However, a dumb fuzzer might be unable to get past the\n`a` condition, just because there is very low chance that all 4\nmutations `a`, `b`, `c`, `d`, happen at same time. In fact, even if it\nprogresses by doing just `a`, the next mutation might go back to `!a`\njust because it doesn't know anything about the program.\n\nWait, when does this kind of \"bad case\" program show up?\n\nIt is quite common in file format parsers, to take one example. To\nreach some specific code paths, one might need to go past multiple\nchecks \"this value must be this, and that value must be that, and some\nother value must be something of something else\" and so\non. Additionally, almost no real world software is \"uncomplicated\",\nand most software has many many many possible code paths, some of\nwhich can be accessed only after many things in the state get set up\ncorrectly. Thereby, many of these programs' code paths are basically\ninaccessible to dumb fuzzers. Additionally, sometimes, some paths\nmight be completely inaccessible (rather than just crazily improbable)\ndue to not enough mutations done whatsoever. If any of these paths\nhave bugs, a dumb fuzzer would never be able to find them.\n\nSo how do we do better than dumb fuzzers?\n\nConsider the Control Flow Graph (CFG) of the above mentioned CRASHER\ncode. If by chance a dumb fuzzer suddenly got `a` correct, then too it\nwould not recognize that it reached a new node, but it would continue\nignoring this, discarding the sample. On the other hand, what AFL (and\nother genetic or \"smart\" fuzzers) do, is they recognize this as a new\npiece of information (\"a newly reached path\") and store this sample as\na new initial point into the corpus. What this means is that now the\nfuzzer can start from the `a` block and move further. Of course,\nsometimes, it might go back to the `!a` from the `a` sample, but most\nof the time, it will not, and instead might be able to reach `b`\nblock. This again is a new node reached, so adds a new sample into the\ncorpus. This continues, allowing more and more possible paths to be\nchecked, and finally reaches the `crash!`.\n\nWhy does this work?\n\nBy adding mutated samples into the corpus, that explore the graph more\n(i.e. reach parts not explored before), we can reach previously\nunreachable areas, and can thus fuzz such areas. Since we can fuzz\nsuch areas, we might be able to uncover bugs in those regions.\n\nWhy is it called genetic fuzzing?\n\nThis kind of \"smart\" fuzzing is kind of like genetic\nalgorithms. Mutation and crossover of specimens causes new\nspecimens. We keep specimens which are better suited to the conditions\nwhich are tested. In this case, the condition is \"how many nodes in\nthe graph did it reach?\". The ones that traverse more can be\nkept. This is not exactly like genetic algos, but is a variation\n(since we keep all specimens that traverse unexplored territory, and\nwe don't do crossover) but is sufficiently similar to get the same\nname. Basically, choice from pre-existing population, followed by\nmutation, followed by fitness testing (whether it saw new areas), and\nrepeat.\n\nWait, so we just keep track of unreached nodes?\n\nNope, not really. AFL keeps track of edge traversals in the graph,\nrather than nodes. Additionally, it doesn't just say \"edge travelled\nor not\", it keeps track of how many times an edge was traversed. If an\nedge is traversed 0, 1, 2, 4, 8, 16, ... times, it is considered as a\n\"new path\" and leads to addition into the corpus. This is done because\nlooking at edges rather than nodes is a better way to distinguish\nbetween application states, and using an exponentially increasing\ncount of the edge traversals gives more info (an edge traversed once\nis quite different from traversed twice, but traversed 10 is not too\ndifferent from 11 times).\n\nSo, what and all do you need in a genetic fuzzer?\n\nWe need 2 things, the first part is called the tracer (or tracing\ninstrumentation). It basically tells you which instructions were\nexecuted in the application. AFL does this in a simple way by jumping\nin between the compilation stages. After the generation of the\nassembly, but before assembling the program, it looks for basic blocks\n(by looking for endings, by checking for jump/branch type of\ninstructions), and adds code to each block that marks the block/edge\nas executed (probably into some shadow memory or something). If we\ndon't have source code, we can use other techniques for tracing (such\nas pin, debugger, etc). Turns out, even ASAN can give coverage\ninformation (see docs for this).\n\nFor the second part, we then use the coverage information given by the\ntracer to keep track of new paths as they appear, and add those\ngenerated samples into the corpus for random selection in the future.\n\nThere are multiple mechanisms to make the tracer. They can be software\nbased, or hardware based. For hardware based, there are, for example,\nsome Intel CPU features exist where given a buffer in memory, it\nrecords information of all basic blocks traversed into that buffer. It\nis a kernel feature, so the kernel has to support it and provide it as\nan API (which Linux does). For software based, we can do it by adding\nin code, or using a debugger (using temporary breakpoints, or through\nsingle stepping), or use address sanitizer's tracing abilities, or use\nhooks, or emulators, or a whole bunch of other ways.\n\nAnother way to differentiate the mechanisms is by either black-box\ntracing (where you can only use the unmodified binary), or software\nwhite-box tracing (where you have access to the source code, and\nmodify the code itself to add in tracing code).\n\nAFL uses software instrumentation during compilation as the method for\ntracing (or through QEMU emulation). Honggfuzz supports both software\nand hardware based tracing methods. Other smart fuzzers might be\ndifferent. The one that Gyn builds uses the tracing/coverage provided\nby address sanitizer (ASAN).\n\nSome fuzzers use \"speedhacks\" (i.e. increase fuzzing speed) such as by\nmaking a forkserver or other such ideas. Might be worth looking into\nthese at some point :)\n\n### Basics of Fuzzing\n\nWritten on 20th April 2017\n\n\u003e Influenced by [this](https://www.youtube.com/watch?v=BrDujogxYSk)\n\u003e awesome live stream by Gynvael Coldwind, where he talks about what\n\u003e fuzzing is about, and also builds a basic fuzzer from scratch!\n\nWhat is a fuzzer, in the first place? And why do we use it?\n\nConsider that we have a library/program that takes input data. The\ninput may be structured in some way (say a PDF, or PNG, or XML, etc;\nbut it doesn't need to be any \"standard\" format). From a security\nperspective, it is interesting if there is a security boundary between\nthe input and the process / library / program, and we can pass some\n\"special input\" which causes unintended behaviour beyond that\nboundary. A fuzzer is one such way to do this. It does this by\n\"mutating\" things in the input (thereby _possibly_ corrupting it), in\norder to lead to either a normal execution (including safely handled\nerrors) or a crash. This can happen due to edge case logic not being\nhandled well.\n\nCrashing is the easiest way for error conditions. There might be\nothers as well. For example, using ASAN (address sanitizer) etc might\nlead to detecting more things as well, which might be security\nissues. For example, a single byte overflow of a buffer might not\ncause a crash on its own, but by using ASAN, we might be able to catch\neven this with a fuzzer.\n\nAnother possible use for a fuzzer is that inputs generated by fuzzing\none program can also possibly be used in another library/program and\nsee if there are differences. For example, some high-precision math\nlibrary errors were noticed like this. This doesn't usually lead to\nsecurity issues though, so we won't concentrate on this much.\n\nHow does a fuzzer work?\n\nA fuzzer is basically a mutate-execute-repeat loop that explores the\nstate space of the application to try to \"randomly\" find states of a\ncrash / security vuln. It does _not_ find an exploit, just a vuln. The\nmain part of the fuzzer is the mutator itself. More on this later.\n\nOutputs from a fuzzer?\n\nIn the fuzzer, a debugger is (sometimes) attached to the application\nto get some kind of a report from the crash, to be able to analyze it\nlater as security vuln vs a benign (but possibly important) crash.\n\nHow to determine what areas of programs are best to fuzz first?\n\nWhen fuzzing, we want to usually concentrate on a single piece or\nsmall set of piece of the program. This is usually done mainly to\nreduce the amount of execution to be done. Usually, we concentrate on\nthe parsing and processing only. Again, the security boundary matters\na _lot_ in deciding which parts matter to us.\n\nTypes of fuzzers?\n\nInput samples given to the fuzzer are called the _corpus_. In\noldschool fuzzers (aka \"blind\"/\"dumb\" fuzzzers) there was a necessity\nfor a large corpus. Newer ones (aka \"genetic\" fuzzers, for example\nAFL) do not necessarily need such a large corpus, since they explore\nthe state on their own.\n\nHow are fuzzers useful?\n\nFuzzers are mainly useful for \"low hanging fruit\". It won't find\ncomplicated logic bugs, but it can find easy to find bugs (which are\nactually sometimes easy to miss out during manual analysis).  While I\nmight say _input_ throughout this note, and usually refer to an _input\nfile_, it need not be just that. Fuzzers can handle inputs that might\nbe stdin or input file or network socket or many others. Without too\nmuch loss of generality though, we can think of it as just a file for\nnow.\n\nHow to write a (basic) fuzzer?\n\nAgain, it just needs to be a mutate-run-repeat loop. We need to be\nable to call the target often (`subprocess.Popen`). We also need to be\nable to pass input into the program (eg: files) and detect crashes\n(`SIGSEGV` etc cause exceptions which can be caught). Now, we just\nhave to write a mutator for the input file, and keep calling the\ntarget on the mutated files.\n\nMutators? What?!?\n\nThere can be multiple possible mutators. Easy (i.e. simple to\nimplement) ones might be to mutate bits, mutate bytes, or mutate to\n\"magic\" values. To increase chance of crash, instead of changing only\n1 bit or something, we can change multiple (maybe some parameterized\npercentage of them?). We can also (instead of random mutations),\nchange bytes/words/dwords/etc to some \"magic\" values. The magic values\nmight be `0`, `0xff`, `0xffff`, `0xffffffff`, `0x80000000` (32-bit\n`INT_MIN`), `0x7fffffff` (32-bit `INT_MAX`) etc. Basically, pick ones\nthat are common to causing security issues (because they might trigger\nsome edge cases). We can write smarter mutators if we know more info\nabout the program (for example, for string based integers, we might\nwrite something that changes an integer string to `\"65536\"` or `-1`\netc). Chunk based mutators might move pieces around (basically,\nreorganizing input). Additive/appending mutators also work (for\nexample causing larger input into buffer). Truncators also might work\n(for example, sometimes EOF might not be handled well). Basically, try\na whole bunch of creative ways of mangling things. The more experience\nwith respect to the program (and exploitation in general), the more\nuseful mutators might be possible.\n\nBut what is this \"genetic\" fuzzing?\n\nThat is probably a discussion for a later time. However, a couple of\nlinks to some modern (open source) fuzzers\nare [AFL](http://lcamtuf.coredump.cx/afl/)\nand [honggfuzz](https://github.com/google/honggfuzz).\n\n### Exploitation Abstraction\n\nWritten on 7th April 2017\n\n\u003e Influenced from a nice challenge\n\u003e in [PicoCTF 2017](http://2017.picoctf.com/) (name of challenge\n\u003e withheld, since the contest is still under way)\n\nWARNING: This note might seem simple/obvious to some readers, but it\nnecessitates saying, since the layering wasn't crystal clear to me\nuntil very recently.\n\nOf course, when programming, all of us use abstractions, whether they\nbe classes and objects, or functions, or meta-functions, or\npolymorphism, or monads, or functors, or all that jazz. However, can\nwe really have such a thing during exploitation? Obviously, we can\nexploit mistakes that are made in implementing the aforementioned\nabstractions, but here, I am talking about something different.\n\nAcross multiple CTFs, whenever I've written an exploit previously, it\nhas been an ad-hoc exploit script that drops a shell. I use the\namazing pwntools as a framework (for connecting to the service, and\nconverting things, and DynELF, etc), but that's about it. Each exploit\ntended to be an ad-hoc way to work towards the goal of arbitrary code\nexecution. However, this current challenge, as well as thinking about\nmy previous note\non\n[\"Advanced\" Format String Exploitation](#advanced-format-string-exploitation),\nmade me realize that I could layer my exploits in a consistent way,\nand move through different abstraction layers to finally reach the\nrequisite goal.\n\nAs an example, let us consider the vulnerability to be a logic error,\nwhich lets us do a read/write of 4 bytes, somewhere in a small range\n_after_ a buffer. We want to abuse this all the way to gaining code\nexecution, and finally the flag.\n\nIn this scenario, I would consider this abstraction to be a\n`short-distance-write-anything` primitive. With this itself, obviously\nwe cannot do much. Nevertheless, I make a small Python function\n`vuln(offset, val)`. However, since just after the buffer, there may\nbe some data/meta-data that might be useful, we can abuse this to\nbuild both `read-anywhere` and `write-anything-anywhere`\nprimitives. This means, I write short Python functions that call the\npreviously defined `vuln()` function. These `get_mem(addr)` and\n`set_mem(addr, val)` functions are made simply (in this current\nexample) simply by using the `vuln()` function to overwrite a pointer,\nwhich can then be dereferenced elsewhere in the binary.\n\nNow, after we have these `get_mem()` and `set_mem()` abstractions, I\nbuild an anti-ASLR abstraction, by basically leaking 2 addresses from\nthe GOT through `get_mem()` and comparing against\na [libc database](https://github.com/niklasb/libc-database) (thanks\n@niklasb for making the database). The offsets from these give me a\n`libc_base` reliably, which allows me to replace any function in\nthe GOT with another from libc.\n\nThis has essentially given me control over EIP (the moment I can\n\"trigger\" one of those functions _exactly_ when I want to). Now, all\nthat remains is for me to call the trigger with the right parameters.\nSo I set up the parameters as a separate abstraction, and then call\n`trigger()` and I have shell access on the system.\n\nTL;DR: One can build small exploitation primitives (which do not have\ntoo much power), and by combining them and building a hierarchy of\nstronger primitives, we can gain complete execution.\n\n### \"Advanced\" Format String Exploitation\n\nWritten on 6th April 2017\n\n\u003e Influenced by [this](https://www.youtube.com/watch?v=xAdjDEwENCQ)\n\u003e awesome live stream by Gynvael Coldwind, where he talks about format\n\u003e string exploitation\n\nSimple format string exploits:\n\nYou can use the `%p` to see what's on the stack. If the format string\nitself is on the stack, then one can place an address (say _foo_) onto\nthe stack, and then seek to it using the position specifier `n$` (for\nexample, `AAAA %7$p` might return `AAAA 0x41414141`, if 7 is the\nposition on the stack). We can then use this to build a **read-where**\nprimitive, using the `%s` format specifier instead (for example, `AAAA\n%7$s` would return the value at the address 0x41414141, continuing the\nprevious example). We can also use the `%n` format specifier to make\nit into a **write-what-where** primitive. Usually instead, we use\n`%hhn` (a glibc extension, iirc), which lets us write one byte at a\ntime.\n\nWe use the above primitives to initially beat ASLR (if any) and then\noverwrite an entry in the GOT (say `exit()` or `fflush()` or ...) to\nthen raise it to an **arbitrary-eip-control** primitive, which\nbasically gives us **arbitrary-code-execution**.\n\nPossible difficulties (that make it \"advanced\" exploitation):\n\nIf we have **partial ASLR**, then we can still use format strings and\nbeat it, but this becomes much harder if we only have one-shot exploit\n(i.e., our exploit needs to run instantaneously, and the addresses are\nrandomized on each run, say). The way we would beat this is to use\naddresses that are already in the memory, and overwrite them partially\n(since ASLR affects only higher order bits). This way, we can gain\nreliability during execution.\n\nIf we have a **read only .GOT** section, then the \"standard\" attack of\noverwriting the GOT will not work. In this case, we look for\nalternative areas that can be overwritten (preferably function\npointers). Some such areas are: `__malloc_hook` (see `man` page for\nthe same), `stdin`'s vtable pointer to `write` or `flush`, etc. In\nsuch a scenario, having access to the libc sources is extremely\nuseful. As for overwriting the `__malloc_hook`, it works even if the\napplication doesn't call `malloc`, since it is calling `printf` (or\nsimilar), and internally, if we pass a width specifier greater than\n64k (say `%70000c`), then it will call malloc, and thus whatever\naddress was specified at the global variable `__malloc_hook`.\n\nIf we have our format string **buffer not on the stack**, then we can\nstill gain a **write-what-where** primitive, though it is a little\nmore complex. First off, we need to stop using the position specifiers\n`n$`, since if this is used, then `printf` internally copies the stack\n(which we will be modifying as we go along). Now, we find two pointers\nthat point _ahead_ into the stack itself, and use those to overwrite\nthe lower order bytes of two further _ahead_ pointing pointers on the\nstack, so that they now point to `x+0` and `x+2` where `x` is some\nlocation further _ahead_ on the stack. Using these two overwrites, we\nare able to completely control the 4 bytes at `x`, and this becomes\nour **where** in the primitive. Now we just have to ignore more\npositions on the format string until we come to this point, and we\nhave a **write-what-where** primitive.\n\n### Race Conditions \u0026 Exploiting Them\n\nWritten on 1st April 2017\n\n\u003e Influenced by [this](https://www.youtube.com/watch?v=kqdod-ATGVI)\n\u003e amazing live stream by Gynvael Coldwind, where he explains about race\n\u003e conditions\n\nIf a memory region (or file or any other resource) is accessed _twice_\nwith the assumption that it would remain same, but due to switching of\nthreads, we are able to change the value, we have a race condition.\n\nMost common kind is a TOCTTOU (Time-of-check to Time-of-use), where a\nvariable (or file or any other resource) is first checked for some\nvalue, and if a certain condition for it passes, then it is used. In\nthis case, we can attack it by continuously \"spamming\" this check in\none thread, and in another thread, continuously \"flipping\" it so that\ndue to randomness, we might be able to get a flip in the middle of the\n\"window-of-opportunity\" which is the (short) timeframe between the\ncheck and the use.\n\nUsually the window-of-opportunity might be very small. We can use\nmultiple tricks in order to increase this window of opportunity by a\nfactor of 3x or even up to ~100x. We do this by controlling how the\nvalue is being cached, or paged. If a value (let's say a `long int`)\nis not aligned to a cache line, then 2 cache lines might need to be\naccessed and this causes a delay for the same instruction to\nexecute. Alternatively, breaking alignment on a page, (i.e., placing\nit across a page boundary) can cause a much larger time to\naccess. This might give us higher chance of the race condition being\ntriggered.\n\nSmarter ways exist to improve this race condition situation (such as\nclearing TLB etc, but these might not even be necessary sometimes).\n\nRace conditions can be used, in (possibly) their extreme case, to get\nring0 code execution (which is \"higher than root\", since it is kernel\nmode execution).\n\nIt is possible to find race conditions \"automatically\" by building\ntools/plugins on top of architecture emulators. For further details,\nhttp://vexillium.org/pub/005.html\n\n### Types of \"basic\" heap exploits\n\nWritten on 31st Mar 2017\n\n\u003e Influenced by [this](https://www.youtube.com/watch?v=OwQk9Ti4mg4jjj)\n\u003e amazing live stream by Gynvael Coldwind, where he is experimenting\n\u003e on the heap\n\nUse-after-free:\n\nLet us say we have a bunch of pointers to a place in heap, and it is\nfreed without making sure that all of those pointers are updated. This\nwould leave a few dangling pointers into free'd space. This is\nexploitable by usually making another allocation of different type\ninto the same region, such that you control different areas, and then\nyou can abuse this to gain (possibly) arbitrary code execution.\n\nDouble-free:\n\nFree up a memory region, and the free it again. If you can do this,\nyou can take control by controlling the internal structures used by\nmalloc. This _can_ get complicated, compared to use-after-free, so\npreferably use that one if possible.\n\nClassic buffer overflow on the heap (heap-overflow):\n\nIf you can write beyond the allocated memory, then you can start to\nwrite into the malloc's internal structures of the next malloc'd\nblock, and by controlling what internal values get overwritten, you\ncan usually gain a read-what-where primitive, that can usually be\nabused to gain higher levels of access (usually arbitrary code\nexecution, via the `GOT PLT`, or `__fini_array__` or similar).\n","funding_links":[],"categories":["Others","Others (1002)","\u003ca id=\"9f9fed5b730bc5bfceaaf77da3aa719e\"\u003e\u003c/a\u003e笔记\u0026\u0026文章\u0026\u0026教程","Programming/Comp Sci/SE Things"],"sub_categories":["Challenge 4"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjaybosamiya%2Fsecurity-notes","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjaybosamiya%2Fsecurity-notes","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjaybosamiya%2Fsecurity-notes/lists"}