{"id":15115027,"url":"https://github.com/otsmr/blackbox-fuzzing","last_synced_at":"2025-05-06T10:31:53.463Z","repository":{"id":205781375,"uuid":"715064260","full_name":"otsmr/blackbox-fuzzing","owner":"otsmr","description":"Fuzzing IoT Devices Using the Router TL-WR902AC as Example","archived":false,"fork":false,"pushed_at":"2024-03-13T11:16:52.000Z","size":2872,"stargazers_count":111,"open_issues_count":0,"forks_count":14,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-05-06T04:47:18.864Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://tsmr.eu/blackbox-fuzzing.html","language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/otsmr.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-11-06T12:09:22.000Z","updated_at":"2025-05-02T15:06:24.000Z","dependencies_parsed_at":null,"dependency_job_id":"4f7b90c3-44c6-4a94-99af-9dfb50573ac2","html_url":"https://github.com/otsmr/blackbox-fuzzing","commit_stats":null,"previous_names":["otsmr/blackbox-fuzzing"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/otsmr%2Fblackbox-fuzzing","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/otsmr%2Fblackbox-fuzzing/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/otsmr%2Fblackbox-fuzzing/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/otsmr%2Fblackbox-fuzzing/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/otsmr","download_url":"https://codeload.github.com/otsmr/blackbox-fuzzing/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252666076,"owners_count":21785192,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-09-26T01:43:39.583Z","updated_at":"2025-05-06T10:31:52.157Z","avatar_url":"https://github.com/otsmr.png","language":"C","funding_links":[],"categories":["C"],"sub_categories":[],"readme":"# Blackbox-Fuzzing of IoT Devices Using the Router TL-WR902AC as Example\n\nThis is the HTML version of my term paper which can be downloaded as PDF\n[here](Paper.pdf).\n\n## Introduction\n\nFuzzing has become \"one of the most effective ways\" of finding bugs in software. With this or\nsimilar claims, many current fuzzing-related papers start\n[\\[google-scholar\\]](https://scholar.google.com/scholar?start=40\u0026q=Fuzzing+%2B%22most+effective%22\u0026hl=de\u0026as_sdt=0,5).\nThe main goal of our last term paper about the topic \"Internet of Vulnerable Things\" was to find a\nmemory-related bug and then write an exploit for this vulnerability. We were able to find a\nvulnerability by reversing the firmware, but no memory related bugs were found. Finding a buffer\noverflow by reversing a binary by hand is not only time-consuming, but also requires a lot of\nexperience. Fuzzing at the same time aims to be the \"most effective way\" to find such memory related\nvulnerabilities.  Google, for example, introduced OSS-Fuzz, which continuously fuzzes open source\nsoftware and has already found over 10,000 vulnerabilities across 1,000 projects\n[\\[oss-fuzz\\]](https://github.com/google/oss-fuzz).\n\nThe goal of this term paper is again to find a memory-related vulnerability, but this time by using\nfuzzing. The goal vulnerability should be exploitable over the network without knowledge of the\nadmin credentials. This paper describes the way to achieve this goal. For this, the paper is\nseparated into two parts. The first part focuses on how to find a potent target, which tools can be\nused, and what a good fuzzing target should consist of. The second part then describes how to\ndevelop and debug a harness that is able to fuzz a specific function in a binary. Then the developed\nharness is used by AFL++ to fuzz the target function. In the following, a short background is given\nand what the current state of the art is when it comes to IoT device fuzzing.\n\nAll files created in context of this term paper are also published in full on GitHub and can be\naccessed using the following URL:\n[otsmr/blackbox-fuzzing](https://github.com/otsmr/blackbox-fuzzing).\n\n### State of the Art\n\nFuzzing IoT devices is not as easy as fuzzing an open source project.  Often the source code is\nproprietary, which makes gray-box fuzzing, which instruments the source code for the best fuzzing\nperformance, impossible\n[\\[afl-persistent\\]](https://github.com/AFLplusplus/AFLplusplus/blob/stable/instrumentation/README.persistent_mode.md).\nAlso, the CPU architecture is often not natively supported by fuzzers which requires an emulator\nlike QEMU [\\[qemu\\]](https://www.qemu.org/) which also slows down the fuzzing speed\n[\\[afl-persistent\\]](https://github.com/AFLplusplus/AFLplusplus/blob/stable/instrumentation/README.persistent_mode.md).\nAnother issue is the hardware peripherals, which complicates the development of a general approach.\nThe paper \"Embedded Fuzzing: A Review of Challenges, Tools, and Solutions\"\n[\\[embedded-fuzzing\\]](https://cybersecurity.springeropen.com/articles/10.1186/s42400-022-00123-y#Sec12)\ngives an overview of different fuzzing strategies, like hardware-based embedded fuzzing. Most of\nthese strategies need the source code of the target program, like when porting the fuzzers source\ncode, like AFL, to ARM-based IoT devices to run the fuzzer on the IoT hardware. Running the fuzzer\non the device's hardware also has performance problems because they often have low-level CPUs, which\nare slower than normal desktop CPUs. Another approach presented in this paper is emulation-based\nembedded fuzzing. Where either a single targeted program is executed in an emulator to perform\ncoverage-guided fuzzing or the full system.\n\nThe above-mentioned approaches all target a binary directly by using an emulator or by instrumenting\nthe source code. These approaches require a fuzzing setup that must often be specifically crafted\nfor a single IoT device and are hard to generalize. For that, researchers created a program\n`IoTFuzzer` which aims to be an automated fuzzing framework aiming to \"finding memory corruption\nvulnerabilities without access to their firmware images\n[\\[iotfuzzer\\]](http://staff.ie.cuhk.edu.hk/~khzhang/my-papers/2018-ndss-iot.pdf).\" `IoTFuzzers`\nbased on the observation that most IoT devices have a mobile app to control them, and such apps\ncontain information about the protocol used to communicate with the device. The program then\nidentifies and reuses program-specific logic to mutate the test cases to effectively test IoT\ntargets [\\[iotfuzzer\\]](http://staff.ie.cuhk.edu.hk/~khzhang/my-papers/2018-ndss-iot.pdf).\n\n### Background\n\n#### Harness\n\nA harness describes a sequence of API calls processing the fuzzer provided inputs. On the contrary\nto a normal application, which often does not need a harness, a library that implements reusable\nfunctions must be called with the correct parameters and also in the right sequence, so the state\nbetween multiple shared function calls can be called. Randomly fuzzing the library without building\nthe state machine is unlikely to be successful and will, in contrast, create a lot of false-positive\ncrashes when the library dependencies are not enforced.  This can happen when, for example, a buffer\nsize check is skipped by the fuzzer resulting in a spurious buffer overflow.\n\nIn this paper, normal applications will be fuzzed, but because of the hardware dependencies of the\nuse of sockets and multi-threading, we need to create a harness for them as well. The harness is\nloaded in the context of the binary and can call internal functions of the targeted program, as\nshown in [Code 10](#c10).\n\n#### Corpus\n\nThe term \"corpus\" describes valid input samples or test cases and serves as a foundational reference\nfor generating new input data during the fuzzing process. In [Code 10](#c10) this would be, for\nexample, an HTTP request. Fuzzers then leverage this corpus to create mutated or diversified test\ncases, aiding in the detection of software vulnerabilities through the exploration of various input\nscenarios.\n\n## Finding a potent target\n\nThe most time-consuming part of black box fuzzing is finding a potential\nvulnerable function in the firmware. The first step is to find\ninteresting binaries that, for example, are accessible over the network,\nuse insecure functions or do not have security features like stack\ncanary enabled, which is buffer overflow protection. Our last paper\n([\\[iovt\\]](https://raw.githubusercontent.com/otsmr/internet-of-vulnerable-things/main/Internet_of_Vulnerable_Things.pdf)) already described how to extract the firmware\nfrom the targeted router and how to find a potentially dangerous binary.\nFor this, the tool EMBA [\\[emba\\]](https://github.com/e-m-b-a/emba) was used. EMBA ranks all\nbinaries found in the firmware by the count of unsecure functions like\n`strcpy`, network access, and security protection like stack canary or\nthe NX-Bit which become interesting when exploiting a buffer overflow,\nwhich can be found in [Code 1](#c1).\n\n\u003cdiv id=\"c1\"\u003e\u003c/div\u003e\n\n```txt\n[+] STRCPY - top 10 results:\n 235   : libcmm.so       : common linux file: no  |  No RELRO  |  No Canary  |  NX disabled  |  No Symbols  |  No Networking |\n 77    : wscd            : common linux file: no  |  No RELRO  |  No Canary  |  NX disabled  |  No Symbols  |  Networking    |\n [snip]\n 28    : httpd           : common linux file: yes |  RELRO     |  No Canary  |  NX enabled   |  No Symbols  |  Networking    |\n 27    : cli             : common linux file: no  |  No RELRO  |  No Canary  |  NX disabled  |  No Symbols  |  No Networking |\n```\n\u003cp style=\"text-align: center\"\u003eCode 1: EMBAs result of unsecure uses of the function strcpy.\u003c/p\u003e\n\nBecause the goal of this paper is to find a memory vulnerability that can be exploited over the\nnetwork without the knowledge of the admin credentials, the vulnerable function must be callable\nover the network and should interact directly with the provided user input. But having network\ninteraction does not mean the binary is also directly accessible over the network. To find out which\nbinaries are listening, we can use the UART root shell, which was already established in\n[\\[iovt\\]](https://raw.githubusercontent.com/otsmr/internet-of-vulnerable-things/main/Internet_of_Vulnerable_Things.pdf).\n\n\u003cdiv id=\"c2\"\u003e\u003c/div\u003e  \n\n```txt\n ~ # netstat -tulpn\nActive Internet connections (only servers)\nProto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name\ntcp        0      0 127.0.0.1:20002         0.0.0.0:*               LISTEN      1045/tmpd\ntcp        0      0 0.0.0.0:1900            0.0.0.0:*               LISTEN      1034/upnpd\ntcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      1027/httpd\ntcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      1224/dropbear\nudp        0      0 0.0.0.0:20002           0.0.0.0:*                           1048/tdpd\n[...]\n```\n\u003cp style=\"text-align: center\"\u003eCode 2: Using the UART root shell to execute netstat\u003c/p\u003e\n\n### Reversing the binary\n\nThe first binary that looks promising is `wscd`. The binary has the most unsafe `strcpy` calls\n(except for the libcmm.so library) and network interaction, which in the case of `wscd` means it\nconnects to a `UPnP` device and does not listen on a specific port. It has, as shown later, an easy\nfunction to fuzz, which is why this binary was selected as an example in this paper to explain the\ngeneral procedure. Before reversing, we can use the UART root shell to find out if the binary is\nrunning and how it was started.\n\n\u003cdiv id=\"c3\"\u003e\u003c/div\u003e\n\n```txt\n$ ps\n PID USER       VSZ STAT COMMAND\n 962 admin     1096 S    wscd -i ra0 -m 1 -w /var/tmp/wsc_upnp/\n1018 admin     1080 S    wscd_5G -i rai0 -m 1 -w /var/tmp/wsc_upnp_5G/\n```\n\u003cp style=\"text-align: center\"\u003eCode 3: Using the command ps to display all running programs.\u003c/p\u003e\n\nWith `ps` we not only see that the binary is running but also what the arguments are, which are\nimportant to verify if a potential function is called at all. The meaning of these arguments can be\ngained from the CLI help, which is displayed when calling the binary without any arguments.\n\n\u003cdiv id=\"c4\"\u003e\u003c/div\u003e\n\n```txt\n$ chroot root /qemu-mipsel-static /usr/bin/wscd\nUsage: wscd [-i infName] [-a ipaddress] [-p port] [-f descDoc] [-w webRootDir] -m UPnPOpMode -D [-d debugLevel] -h\n -i:  Interface name this daemon will run wsc protocol(if not set, will use the default interface name - ra0)\n       e.g.: ra0\n -w: Filesystem path where descDoc and web files related to the device are stored\n       e.g.: /etc/xml/\n -m: UPnP system operation mode\n       1: Enable UPnP Device service(Support Enrolle or Proxy functions)\n       2: Enable UPnP Control Point service(Support Registratr function)\n       3: Enable both UPnP device service and Control Point services.\n [...]\n```\n\u003cp style=\"text-align: center\"\u003eCode 4: Options of the binary wscd.\u003c/p\u003e\n\nAs shown in [Code 4](#c4) `wscd` is started with \"Enabled UPnP Device service\" which looks\npromising. After verifying that the binary is actually running on the router, the binary can then be\nanalyzed using [Ghidra](https://ghidra-sre.org/) to search for suspect functions. For fuzzing,\nparsing functions are especially interesting because they are usually complex, and often the input\nthat is parsed has length fields for the containing data, like the TCP packet contains the length of\nthe payload.\n\n\u003cdiv id=\"f1\"\u003e\u003c/div\u003e\n\u003cfigure\u003e\n  \u003cp\u003e \u003cimg src=\"blackbox-fuzzing/ghidra_search.png\" style=\"width:90.0%\" /\u003e\u003c/p\u003e\n  \u003cfigcaption\u003e\n    \u003cp\u003eFigure 1: Using Ghidra to search for parsing functions.\u003c/p\u003e\n  \u003c/figcaption\u003e\n\u003c/figure\u003e\n\nAnother benefit of a parsing functions is that they often do not interact with other parts of the\ncode or have user interaction over the network. So the parsing function can be called directly with\nthe input without modifying the binary or overwriting other functions, so the function can be\nfuzzed.\n\nBefore starting to fuzz the function, it should be checked if the function is triggered at all,\nbecause the function is only interesting when it is called with a user-controlled input. For this,\nGhidra can be used to search for references to the target function. In the case of the\n`parser_parse` function there are multiple ways. Because we know how the program is started, the\ncalls can be reduced to a single function call tree, shown in [Code 5](#c5).\n\n\u003cdiv id=\"c5\"\u003e\u003c/div\u003e\n\n```c\nmain()\n if ((WscUPnPOpMode \u0026 1) != 0) // Argument -m 1\n  WscUPnPDevStart()\n   UpnpDownloadXmlDoc() -\u003e my_http_Download() -\u003e http_Download()\n     if (http_MakeMessage())\n      http_RequestAndResponse()\n       http_RecvMessage()\n```\n\u003cp style=\"text-align: center\"\u003eCode 5: Call tree of the function parser_parse\u003c/p\u003e\n\nAfter a target function is found, we can now create a fuzzing setup to fuzz the function, which is\ndescribed in the next part. But first other potent functions are presented.\n\n### Other potential vulnerable functions\n\nFor this paper, multiple potential binaries were manually analyzed for suspect functions. The\nfollowing is a short summary of other possible targets which were found.\n\nThe binary **httpd** is the backend for the admin web interface. The binary is accessible over the\nnetwork on port 80. One interesting function in `httpd` is the `httpd_parser_main` function. While\nskimming the parser implementation using Ghidra several different suspect code parts could be\nidentified. One of the suspect parts is the parsing of the `Content-Type`. In the following, a basic\nHTTP request can be found.\n\n```txt\nPOST / HTTP/1.1\\r\\n\nContent-Type: multipart/form-data; boundary=X;\\r\\n\nHost: example.com\\r\\n\n\\r\\n\n\\r\\n\nDATA\\r\\n\n```\n\nBelow is a snippet from the `httpd_parser_main` function which parses the `Content-Type` from the\nuser provided http request.\n\n\n\u003cdiv id=\"c6\"\u003e\u003c/div\u003e\n\n```c\n// user_input_ptr points to\n//  \"Content-Type: multipart/form-data; boundary=X;\\r\\nHost: example.com\\r\\n...\"\ncursor = strstr(user_input_ptr,\"multipart/form-data\");\n\nif (user_input_ptr == cursor) {\n cursor = strstr(user_input_ptr,\"boundary=\");\n user_input_ptr = cursor + 9;\n\n // user_input_ptr points now to \"X;\\r\\nHost: example.com\\r\\n...\"\n\n if (cursor != (char *)0x0) {\n\n  do {\n    while (cursor = user_input_ptr, *cursor == \" \") {\n      user_input_ptr = cursor + 1;\n    }\n    user_input_ptr = cursor + 1;\n  } while (*cursor == \"\\t\");\n\n  // cursor points now to \"X;\\r\\nHost: example.com\\r\\n...\"\n\n  // strchr returns a pointer to the first occurrence of \";\" in the user request.\n  // If \";\" is not found, the function returns a null pointer.\n  user_input_ptr = strchr(cursor, \";\");\n  if (user_input_ptr != (char *)0x0) {\n    // The character \";\" is replaced by an null byte to terminate the string\n    *user_input_ptr = \"\\0\";\n    // cursor points now to \"X\\0\\r\\nHost: example.com\\r\\n...\"\n  }\n\n  // DAT_00444050 global array from 0x00444050 to 0x0044414f (255 Bytes)\n  strcpy(\u0026DAT_00444050, cursor);\n  // DAT_00444050 contains now \"X\"\n }\n}\n```\n\u003cp style=\"text-align: center\"\u003eCode 6: Call tree of the function parser_parse\u003c/p\u003e\n\nThe vulnerability in this code is the function call `strcpy` and the assumption that the\n`Content-Type` ends with a semicolon. Because `strcpy` copies the buffer until the next null byte,\nand as shown in [Code 6](#c6) the null byte is only added when a semicolon is found. By removing the\nsemicolon, the next null byte is at the end of the input buffer, e.g., the end of the HTTP request.\nSo the global variable DAT_00444050 can be overflowed, which then overwrites data beyond the address\n0x0044414f. The challenging part is not only to find an interesting global variable beyond this\naddress that could be overwritten, but also that no null bytes can be used because of `strcpy`. But\nwhen there is one such mistake, there are probably more to find.\n\nThe binary **tdpd** is used by the mobile app and is accessible over UDP on the local network.\n`tdpd` has almost the same functions as the `tmpd` which are mostly just never called. The main\nfunction only listens for messages over the UDP port and always responds with basic information\nabout the router, like the name or model. There is barely any interaction with the user-provided\ninput, which is therefore not interesting to fuzz.\n\nAnother interesting pair of binaries are **upnpd** and **ushare**. Both binaries are handling `UPnP`\nmessages which therefore need to parse XML.  Because a copyright string can be found in the binary,\nit can be assumed that these programs were not developed by TP-Link.\n\n```sh\n$ strings usr/bin/ushare | grep \"(C)\"\nBenjamin Zores (C) 2005-2007, for GeeXboX Team.\n```\n\nBoth binaries are loading the shared libraries `libupnp.so` and `libixml.so` which have the same\nfunctions as the open source project `pupnp` [\\[pupnp\\]](https://github.com/pupnp/pupnp/). Because\nthe focus of this paper is black box fuzzing, these binaries are ignored. But gray box fuzzing this\nlibrary could have potential because in 2021 a memory leak was found in `libixml.so`\n[\\[pupnp-mem-leak\\]](https://github.com/pupnp/pupnp/issues/249).\n\nThe binary **tmpd** is the backend of the mobile app. The interesting part is that the router and\nthe mobile app are communicating over a custom binary protocol. In the following, a message from the\nclient to the server is shown.\n\n\u003cdiv id=\"c7\"\u003e\u003c/div\u003e\n\n```txt\n00000000  01 00 05 00 00 08 00 00  00 00 00 17 50 7b 6e fe  |............P{n.|\n00000010  01 01 02 00 00 00 00 00                           |........        |\n```\n\u003cp class=\"text-align: center\"\u003eCode 7: Message from the mobile app to the router.\u003c/p\u003e\n\nTo understand the binary protocol, the binary `tmpd` was reversed using Ghidra. With this\ninformation, the message in [Code 7](#c7) can be broken down into the following:\n\n\n```txt\n01 00 05 00 : Version\n00 08 00 00 : Size (8 Bytes)\n00 00 00 17 : Datatype\n50 7b 6e fe : Checksum (CRC32)\n01 01       : Options\n02 00       : Function id\n00 00 00 00 : Function parameters\n```\n\u003cp class=\"text-align: center\"\u003eCode 8: Custom binary protocol broken down.\u003c/p\u003e\n\nThis looks promising because such binary protocols must be parsed. But the most suspect part of the\nbinary protocol is not the length field, but the use of the function ID and function parameters.\n\n\n\u003cfigure id=\"f2\"\u003e\n  \u003cp\u003e\u003cimg src=\"blackbox-fuzzing/ghidra_tmpd.png\" style=\"width:90.0%\" /\u003e\u003c/p\u003e\n  \u003cfigcaption\u003e\n    \u003cp style=\"text-align: center\"\u003eFigure 2: Reversed function from tmpd which parses the function id and their parameters.\u003c/p\u003e\n  \u003c/figcaption\u003e\n\u003c/figure\u003e\n\n[Figure 2](#f2) shows a part of the decompiled parser function of the custom protocol. In line 16,\nthe function ID is extracted, and the corresponding function is then called in line 29. The suspect\nbehavior is that the function is called with parameters extracted without any check from the\nuser-controlled input buffer. We could now try to find a function in the jumping table shown in\n[Figure 3](#f3) where this could be dangerous, like when the parameter is used to index a buffer\nor interpreted as a string. Instead of manually reversing and searching the over 100 functions,\nwhich would be time-consuming, we can use a fuzzer which would do this automatically.\n\n\u003cfigure id=\"f3\"\u003e\n\u003cp\u003e\u003cimg src=\"blackbox-fuzzing/tmpd_function_list.png\"\nstyle=\"width:90.0%\" /\u003e\u003c/p\u003e\n\u003cfigcaption\u003e\u003cp style=\"text-align: center\"\u003eFigure 3: Reversed function from tmpd which parses the function ID\nand its parameters.\u003c/p\u003e\u003c/figcaption\u003e\n\u003c/figure\u003e\n\nUnfortunately, the `tmpd` binary is only locally reachable over the network, as shown in [Code\n2](#c2). To connect to this binary, the app first connects to the router via SSH in the mode\n`direct-tcpip` which just forwards the packets to the local process. And the SSH connection is\nprotected by the admin credentials. But as described in\n[\\[iovt\\]](https://raw.githubusercontent.com/otsmr/internet-of-vulnerable-things/main/Internet_of_Vulnerable_Things.pdf)\nthe SSH connection can easily be compromised because the server host key is never checked by the\napp. By dropping every packet routed to the internet, the admin can be tricked into logging in to\nthe router while a man-in-the-middle attack is performed to steal the credentials.\n\n## Fuzzing with AFL++ and QEMU\n\nIn this section, a harness is developed targeting one of the previously found function. After the\nharness is developed, the state-of-the-art fuzzer AFL++\n[\\[aflpp\\]](https://github.com/AFLplusplus/AFLplusplus) is used to fuzz the target function. Because\nthe binaries are compiled for the `mipsel` architecture, the emulator QEMU is used to execute the\nbinary. The basic fuzzing setup used in this paper is mostly inspired by the blog entry \"Firmware\nFuzzing 101\" by Adam Van Prooyen [\\[b101\\]](https://www.mayhem.security/blog/firmware-fuzzing-101).\n\n### Fuzzing environment \n\nTo easily create a reproducible fuzzing environment, Docker is the best choice. We created a\nDockerfile that installs every necessary tool, like a cross-compiler for `mipsel` CPU architecture\nor `gdb-multiarch` which can be used to debug the harness.\n\nFurthermore, AFLplusplus is downloaded and compiled together with QEMU which is built in a version\nwith minor tweaks to allow non-instrumented binaries to be run under afl-fuzz.\n\n```docker\nFROM debian:latest\n\nRUN apt update \u0026\u0026 apt install -y \\\n      curl \\\n      vim \\\n      gcc-mipsel-linux-gnu \\\n      openssh-server \\\n      qemu-user-static \\\n      gdb-multiarch\n# Qemu statics are installed at /usr/bin/qemu-mipsel-static\n\n# Compiling AFL++\nRUN apt install -y git make build-essential clang ninja-build pkg-config libglib2.0-dev libpixman-1-dev\nRUN git clone https://github.com/AFLplusplus/AFLplusplus /AFLplusplus\nWORKDIR /AFLplusplus\nRUN make all\nWORKDIR /AFLplusplus/qemu_mode\nRUN CPU_TARGET=mipsel ./build_qemu_support.sh\n\nRUN echo \"#!/bin/bash\\n\\nsleep infinity\" \u003e\u003e /entry.sh\nRUN chmod +x /entry.sh\n\nWORKDIR /share\nENTRYPOINT [ \"/entry.sh\" ]\n```\n\nDockerfile which installs necessary tools.\n\nThe image can then be built using `docker build`.\n\n```sh\ndocker build -t fuzz .\n```\n\nWhen the image is built, it can be easily used with `docker run` which\nthen starts the container.\n\n```sh\ndocker run -d --rm -v $PWD/:/share --name fuzz fuzz\n```\n\nUsing the option `-d` will start the container in the background. With `docker exec` multiple shells\ncan be started inside the container, which is helpful to start the executable in one session using\nQEMU and in the other session `gdb-multiarch`.\n\n```sh\ndocker exec -it fuzz /bin/bash\n```\n\n### Overwrite the main function\n\nIn the previous section, a potent fuzz target was identified. The problem is that when executing the\nbinary, we will never reach the function call because the `parser_parse` function is only called if\na TCP packet is received over a socket. This would be not only bad for performance, but also hard to\nset up. This is why the entry of the fuzzer should be at an different location than the normal main\nfunction.  For this, the environment variable `LD_PRELOAD` which enables injecting a harness that\nhas access to internal functions, can be used. As the man page of `ld.so`, which is responsible for\nlinking the shared libraries needed by an executable at runtime, describes, `LD_PRELOAD` can be used\n\"to selectively override functions in other shared objects\n[\\[man-pages\\]](https://www.man7.org/linux/man-pages/man8/ld.so.8.html).\"\n\nThe function `__uClibc_main` is best suited for this purpose. To overwrite this function, a C file\nmust be created that contains a function with the same name.\n\n```c\nvoid __uClibc_main(void *main, int argc, char** argv) {\n    // Harness code, e.g. call the function parser_append\n    printf(\"My custom __uClibc_main was called!\");\n}\n```\n\nThe C file can then be cross-compiled to a shared object in the mipsel architecture using\n`mipsel-linux-gnu-gcc`. The option `-fPIC` enables \"Position Independent Code\" which means that the\nmachine code does not depend on being located at a specific address by using relative addressing\ninstead of absolute.\n\n```txt\n$ mipsel-linux-gnu-gcc parser_parse_hook.c -o parser_parse_hook.o -shared -fPIC\n```\n\nThe newly created shared library can then be loaded by adding the environment variable `LD_PRELOAD`\nto the QEMU command.\n\n```txt\n$ chroot root /qemu-mipsel-static -E LD_PRELOAD=/parser_parse_hook.o /usr/bin/wscd\nMy custom __uClibc_main was called!\n```\n\nWith the command `chroot` the current and root directories can be changed for the command provided.\nThis is helpful because the executable `wscd` opens other files, like shared libraries from the\nfirmware. We can see this behavior by adding the argument `-strace` to QEMU.\n\n```txt\nchroot root /qemu-mipsel-static -E LD_PRELOAD=/parser_parse_hook.o -strace /usr/bin/wscd /corpus/notify.txt\n38180 mmap(NULL,4096,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANONYMOUS|0x4000000,-1,0) = 0x7f7e7000\n38180 stat(\"/etc/ld.so.cache\",0x7ffffa48) = -1 errno=2 (No such file or directory)\n38180 open(\"/parser_parse_hook.o\",O_RDONLY) = 3\n38180 fstat(3,0x7ffff920) = 0\n38180 close(3) = 0\n38180 munmap(0x7f7e6000,4096) = 0\n38180 open(\"/lib/libpthread.so.0\",O_RDONLY) = 3\n38180 open(\"/lib/libc.so.0\",O_RDONLY) = 3\n[...]\n```\n\nAs we can see, the executable opens multiple libraries in the `/lib/` folder on the firmware and not\non the host.\n\n### Developing and debug the harness\n\nAfter the setup is created, we can now start developing a harness. As described in the background\nsection, the harness is the driver between the fuzzer and the target function. The harness loads the\nfuzz input, which is stored by AFL++ in a file. With the file path as parameters, the harness then\ncalls the fuzzing target; in this case, this would be `parser_append`. The functions can be called\nby using the address.\n\n\u003cdiv id=\"c10\"\u003e\u003c/div\u003e\n\n```c\nvoid __uClibc_main(void *main, int argc, char** argv)\n{\n  // Verify that a filename is provided\n  if (argc != 2) exit(1);\n\n  // Create function pointer to the fuzz target\n  int (*parser_request_init)(void *, int) = (void *) 0x00412564;\n  int (*parser_append)(void *, void *, int) = (void *) 0x00412e98;\n\n  // Open the fuzz input file\n  int fd = open(argv[1], O_RDONLY);\n  char fuzz_buf[2048 + 1];\n  int fuzz_buf_len = read(fd, fuzz_buf, sizeof(fuzz_buf) - 1);\n  if (fuzz_buf_len \u003c 0) exit(1);\n  fuzz_buf[fuzz_buf_len] = 0;\n\n  // Call the target functions\n  uint8_t parsed_data[220]; \n  parser_request_init(parsed_data, 8);\n  int status = parser_append(parsed_data, fuzz_buf, fuzz_buf_len);\n  printf(\"Response is %d\\n\", status);\n  exit(0);\n}\n```\n\u003cp style=\"text-align: center\"\u003eCode 10: Harness code with the fuzz target `parser_append` in the binary wscd.\u003c/p\u003e\n\nAs shown in [Code 10](#c10) the function `parser_parse` is not called directly but by using the\nfunction `parser_append`. Before this function is called, the initialization function\n`parser_request_init` must be called, which initializes the output struct of the `parser_parse`\nfunction.\n\nWhile in the case of the `parser_parse` the harness is pretty easy to set up, other targets require\nmore sophisticated harnesses like the `httpd_parser_main` function. For example, before calling the\ntarget, the function `http_init_main` must be called, which ends in a SIGSEGV.  To find out where\nthis segmentation fault is caused, it is useful to debug the code with a debugger like `gdb`. To do\nthis, QEMU can be started with the option `-g` which spawns a `gdb-server` at the provided port.\n\n```sh\nchroot root /qemu-mipsel-static -strace -g 1234 -E LD_PRELOAD=\"/httpd_parser_main.o\" /usr/bin/httpd\ncorpus/httpd/simple.txt\n```\n\nBecause the binary is in the `mipsel` architecture `gdb-multiarch` must be used. After gdb is\nstarted, the following init script can be loaded with gdb using `sources \u003cpath to script\u003e`.\n\n```sh\nset solib-absolute-prefix /share/root/\nfile /share/root/usr/bin/httpd\ntarget remote :1234\n# break bevor fuzz target is called\n# break __uClibc_main\nbreak http_parser_main\ndisplay/4i $pc\n```\n\nBecause of the chroot the script first changed the absolute prefix path so that when the binary\nloads a shared object, gdb will find the file.  Then the targeted file is set, because QEMUs\ngdb-server does not support file transfer, so gdb tries to load the files from the disk instead.\nAfter gdb is configured, the script then connects to the gdb-server with `target remote` and creates\na breakpoint at the start of the target function. With display, the output is just improved, so when\nstepping through, the next four lines of assembly will be shown. Using `si` we can step one\ninstruction, which is useful when the harness has a segmentation fault using the default corpus,\nwhich should always work.  As shown in [Code 11](#c11) the binary has a segmentation fault in the\nfunction `fprintf`.\n\n\u003cdiv id=\"c11\"\u003e\u003c/div\u003e\n\n```sh\n(gdb) si\n0x004059b0 in http_parser_makeHeader ()\n1: x/4i \\$pc\n=\u003e 0x4059b0 \u003chttp_parser_makeHeader+120\u003e:       jalr    t9\n   0x4059b4 \u003chttp_parser_makeHeader+124\u003e:       addiu   a1,a1,16248\n   0x4059b8 \u003chttp_parser_makeHeader+128\u003e:       li      v0,200\n   0x4059bc \u003chttp_parser_makeHeader+132\u003e:       lw      gp,16(sp)\n(gdb) ni\n0x7f56a8ac in fprintf () from /share/root/lib/libc.so.0\n1: x/4i \\$pc\n=\u003e 0x7f56a8ac \\\u003cfprintf+44\u003e:     bal     0x7f56db80 \u003cvfprintf\u003e\n   0x7f56a8b0 \\\u003cfprintf+48\u003e:     nop\n   0x7f56a8b4 \\\u003cfprintf+52\u003e:     lw      ra,36(sp)\n   0x7f56a8b8 \\\u003cfprintf+56\u003e:     jr      ra\n   0x7f56a8bc \\\u003cfprintf+60\u003e:     addiu   sp,sp,40\n(gdb) n\nSingle stepping until exit from function fprintf,\nwhich has no line number information.\n\nProgram received signal SIGSEGV, Segmentation fault.\n```\n\u003cp style=\"text-align: center\"\u003eCode 11: Segmentation fault in printf.\u003c/p\u003e\n\nTo investigate the error, Ghidra can be used to find out with which parameters the function is\ncalled.\n\n```c\nfprintf(\n *(FILE **)(iVar1 + 0x101c),\n \"HTTP/1.1 %d %s\\r\\n\",\n *(undefined4 *)(\u0026DAT_0042ee68 + (uint)(byte)(\u0026DAT_00414570)[statuscode \u0026 0x3f] * 8),\n (\u0026PTR_DAT_0042ee6c)[(uint)(byte)(\u0026DAT_00414570)[statuscode \u0026 0x3f] * 2]\n);\n```\n\nThe SIGSEGV is probably caused by the fact that the first parameter is not a file descriptor but a\nnull pointer. Where `iVar1`` is just a reference to the input of the `httpd_parser_main` function.\nThis means that the fuzzing input must have a file descriptor at position 0x101c.  So the input must\nbe adjusted to the following struct.\n\n```c\ntypedef struct  {\n  int _a;     // 4 Bytes\n  int _b;     // 4 Bytes\n  int socket; // 4 Bytes\n  int ip;     // 4 Bytes\n  int mac;    // 4 Bytes\n  unsigned char body[0x1008]; 0x101c - 4*5 = 0x1008 Bytes\n  FILE * fd_out; // expected to be a valid file descriptor\n} HttpMainT;\n```\n\nBecause `fd_out` must just be a valid file descriptor pointer, it can easily be set to `stdout`.\nExecuting the `httpd_parser_main` again will now produce a valid HTTP output.\n\n```c\n$ chroot root /qemu-mipsel-static -E LD_PRELOAD=/httpd_parser_main.o \\\n    /usr/bin/httpd /httpd_corpus.txt\n\nbind: No such file or directory\n[ dm_shmInit ] 086:  shmget to exitst shared memory failed. Could not create shared memory.\nrdp_getObj is called with: 4274932gdpr_getSystemGDPREntry Error\ngdpr_getNewSystemGDPREntry OK\n#Msg: getsockname error\nHTTP/1.1 200 OK\nContent-Type: text/html; charset=utf-8\nContent-Length: 24257\nSet-Cookie: JSESSIONID=deleted; Expires=Thu, 01 Jan 1970 00:00:01 GMT; Path=/; HttpOnly\nConnection: close\n\n\u003c!DOCTYPE html\u003e\n[...]\n```\n\nThe harness works now and can be used to fuzz the function using AFL++ which will be explained in\nthe next section.\n\n### Generate corpus data\n\nAs mentioned in the background, a seed corpus describes valid input samples, which serves as a\nfoundational reference for generating new input data during the fuzzing process.\n\nThese inputs are typically chosen to represent different aspects of the target programs. The seed\ncorpus is used by a fuzzer to generate mutated or evolved test cases that are then run against the\ntarget software to uncover bugs, crashes, or other problems. This corpus plays an important role in\ndirecting the fuzzer to relevant areas of the program and increasing the probability of detecting\nvulnerabilities or unexpected behaviors. By providing a diverse and representative set of initial\ninputs, the seed corpus helps the fuzzer explore different paths in the target faster and thereby\nincreases coverage.\n\nWhen it comes to functions parsing network data, these inputs can be created by using Wireshark to\nrecord different packets.\n\nFor the function, `httpd_parse_main` four different corpora were created. Each targeting different\npaths in the binary. One example is the login request, which contains the username and password. For\nthis corpus, the harness had to be modified because TP-Link uses (weak) cryptography to \"protect\"\nthe password. For this, the password is encrypted in the browser using AES and then decrypted on the\nbackend.  Whereby the password is generated in the browser and then encrypted using RSA. Then the\nencrypted data is signed. Because a fuzzer can not create a signature or encrypt data, some\nfunctions were overwritten and now just decodes the data from base64. For this, the data were first\nextracted in plaintext from the browser using the debugger shown in [Figure 4](#f4).\n\n\u003cdiv id=\"f4\"\u003e\u003c/div\u003e\n\n\u003cfigure\u003e\n  \u003cp\u003e\u003cimg src=\"blackbox-fuzzing/firefox.png\" style=\"width:95.0%\" /\u003e\u003c/p\u003e\n\u003cfigcaption\u003e\u003cp style=\"text-align: center\"\u003eFigure 4: Extracting the data bevor encryption.\u003c/p\u003e\u003c/figcaption\u003e\n\u003c/figure\u003e\n\nIn the target, the function `rsa_tmp_decrypt_bypart` was then overwritten to replace the logic from\ndecrypting the data to just decoding from base64.\n\n```c\n// Replacing the logic with b64_decode\nint rsa_tmp_decrypt_bypart(uint8_t *input, int input_len, uint8_t *output) { // other params just key data\n  int (*b64_decode)(uint8_t *, int, uint8_t *, int) = (void *) 0x0040bf00;\n  b64_decode(output, 0x1000, input, input_len);\n  int * seqnumber = (int *) 0x00444db0;\n  *seqnumber = 0x3ac28e29-input_len+12;\n  return 0; // says it was okay\n}\n```\n\u003cp style=\"text-align: center\"\u003eCode 12: Function rsa_tmp_decrypt_bypart now just decodes base64\ninstead of decrypt the data.\u003c/p\u003e\n\nWhile executing the corpus, the target function always returns an HTML document with the error \"408\nRequest Timeout\". Using Ghidra and GDB the problem could be identified. The error always happens\nafter the function call to `http_stream_fgets`. The problematic line was the check for the line\nbreak character `\\n`.\n\n```c\nif (((cVar1 == '\\n') \u0026\u0026 (param_3 \u003c pcVar4)) \u0026\u0026 (pcVar4[-1] == '\\r')) {\n```\n\nThis condition enforces that after every line break, a carriage return must follow. After adding the\ncarriage return, all the created corpora worked.\n\n### Fuzz the target\n\nIn the last section, we developed multiple harnesses and executed them using QEMU. In this section,\nQEMU is replaced by AFL++ which gets the generated corpora as seed input to fuzz the target\nfunction. In the section \"Fuzzing environment\" a docker image was created that already pulls AFL++\nfrom GitHub and then uses an AFL++-provided script to build a patched version of QEMU. So AFL++ can\nnow be started with the following command that gets different parameters, like `-Q` which tells\nAFL++ to use the patched version of QEMU.\n\n```sh\nQEMU_LD_PREFIX=/share/root AFL_PRELOAD=/share/root/httpd_parser_main.o \\\n  /AFLplusplus/afl-fuzz -Q \\\n  -i /share/root/corpus/httpd/ -o /share/afl-out/httpd/ \\\n  -- /share/root/usr/bin/httpd @@\n```\n\u003cp style=\"text-align: center\"\u003eCode 13: Fuzzing the binary \u003ccode\u003ehttpd\u003c/code\u003e using the harness\nand \u003ccode\u003eafl-fuzz\u003c/code\u003e.\u003c/p\u003e\n\nUnlike before, the command `chroot` is no longer necessary and is replaced by the variable\n`QEMU_LD_PREFIX`. Which tells QEMU where to search for shared objects. Also, the `LD_PRELOAD`\nvariable is replaced by the AFL-specific version `AFL_PRELOAD`. The last argument in the command is\nthe two `@` characters. They will be replaced by AFL++ with a file path that holds the fuzzing\ninput. When started, AFL++ shows the progress using the terminal UI shown in [Figure 5](#f5).\n\n\u003cdiv id=\"f5\"\u003e\u003c/div\u003e\n\u003cfigure\u003e\n\u003cp\u003e\u003cimg src=\"blackbox-fuzzing/afl++.png\" style=\"width:95.0%\" /\u003e\u003c/p\u003e\n\u003cfigcaption\u003e\u003cp style=\"text-align: center\"\u003eFigure 5: The status screen from AFL++.\u003c/p\u003e\u003c/figcaption\u003e\n\u003c/figure\u003e\n\nThe `AFL++` status screen provides essential insights into the current fuzzing process. The docs of\n`AFL++` have a nice overview of the terms used in the status screen\n[\\[afl-screen\\]](https://aflplus.plus/docs/status_screen/).  When debugging the corpus with the\nfollowing environment variables, the UI can be disabled and with `AFL_DEBUG` a detailed logging\nenabled, which shows the current fuzzer input and the `stdout` from the target program.\n\n```sh\nexport AFL_DEBUG=1 \u0026\u0026 export AFL_NO_UI=1\nunset AFL_DEBUG \u0026\u0026 unset AFL_NO_UI\n```\n\nAs shown in [Figure 5](#f5) fuzzing a binary can take quite some time. According to the docs it\n\"should be expected to run for days or weeks\" and \"some jobs will be allowed to run for months.\" To\nimprove the time needed, the exec speed should be above 100 execs/sec. When, for example, the target\n`httpd_main_parser` was fuzzed, the exec speed was at the beginning by around 30/sec. To improve the\nspeed, the target binary was searched for suspect functions, which are probably the cause of the\nslowdown. One of the suspect functions was `rsa_gdpr_generate_key` because generating an RSA key is\nknown to be slow. After overwriting the function, the speed improved to 600 executions per second.\n\nOne indicator that helps indicate when to stop fuzzing is the cycle counter. AFL++ will highlight\nthe number in green when \"the fuzzer has not been seeing any action for a longer while,\" which helps\nto make the call to stop the fuzzer.\n\nBut the most interesting number is probably \"total crashes\". This shows when the program crashes\nbecause of the current fuzzing input and is probably a memory-related bug. To verify that this is a\nreal bug `gdb` can be used again to find the position of the bug.\n\n## Conclusion\n\nFuzzing may be the most effective way to find security vulnerabilities.  In this term paper, three\ndifferent functions were fuzzed, but none were found. While the black box fuzzing setup itself is\nnot that complex and time-consuming, finding a potent target and developing a working harness are.\nMost of the time, the harness has to be debugged, and then the underlining logic in the binary must\nbe reversed, which again consumes a long time.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fotsmr%2Fblackbox-fuzzing","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fotsmr%2Fblackbox-fuzzing","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fotsmr%2Fblackbox-fuzzing/lists"}