Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

https://github.com/nathanreb/ocaml-afl-examples

Small examples of how to use AFL to fuzz OCaml programs
https://github.com/nathanreb/ocaml-afl-examples

Last synced: about 1 year ago
JSON representation

Small examples of how to use AFL to fuzz OCaml programs

Awesome Lists containing this project

README 

          
# OCaml AFL fuzzing examples



Small examples of how to use AFL to fuzz OCaml programs



These examples are here to help you quickly get set up with AFL and to illustrate an upcoming

article on the [Tarides blog](https://tarides.com/blog.html).



## Setup



To be able to correctly run the examples in this repo and toy around with fuzzing you will need to

install `afl` and have a `+afl` opam switch so that the binaries are properly instrumented for

fuzzing.



You can setup the afl switch by running:

```

$ opam switch create fuzzing-switch 4.07.1+afl

```



You can either install AFL from your distribution, e.g. on Debian:

```

$ apt update && apt install afl

```



Or by using the convenience opam package:

```

$ opam install --switch=fuzzing-switch afl

```



Some of the examples have extra dependencies such as `crowbar` and `bun`. You can install all of

them by running:



```

$ opam install --switch=fuzzing-switch crowbar bun

```



## Simple parser



The `simple-parser` folder contains the most basic example and shows how you can use afl-fuzz to

fuzz a simple parsing function written in OCaml.



The `lib` subfolder contains a library with a single `parse_int` function that parses an int from a

string, with a little twist.



The `fuzz` subfolder contains the code to be compiled to the fuzzing binary `fuzz_me.exe` which

must be passed to afl and an `inputs/` folder with a couple starting test cases.



You can try fuzzing it by yourself:

```

$ dune build simple-parser/fuzz/fuzz_me.exe

$ afl-fuzz -i simple-parser/fuzz/inputs -o _build/default/simple-parser/fuzz/findings _build/default/simple-parser/fuzz/fuzz_me.exe @@

```



Or simply run:

```

$ dune build @simple-parser/fuzz --no-buffer

```



which will do pretty much exactly the above.



AFL should find the crash fairly quickly. It will show up in the top right corner of `afl-fuzz`'s

output, under `uniq crashes`, see the picture below.



![afl-output-screenshot-emphasized-crashes](img/afl-output-screenshot-emphasized-crashes.png)



You can inspect the input that triggered the crash by running:

```

$ cat _build/default/simple-parser/fuzz/findings/crashes/id*

abc

```



and reproduce it by running:

```

$ cd _build/default/simple-parser/fuzz

$ ./fuzz_me.exe findings/crashes/id*

Fatal error: exception Failure("secret crash")

```



## Awesome list



The `awesome-list` folder contains an example of how AFL can be used conjointly with the `crowbar`

library to both find crashes and do some property based testing.



The `lib` subfolder contains a library with a single `sort` function that sorts lists of integers.

Again it mostly works fine except in two specific cases where it will either crash or sort the list

in reverse order.



The `fuzz` subfolder contains the code for the fuzzing binary. It is slightly different from the

previous example as here we use `crowbar` to build the correct binary instead of doing it by hand.

Furthermore, we don't check for crashes only anymore but also want to know if the function under

test invariant, i.e. the resulting list is sorted in increasing order, stands. The resulting fuzzing

binary has roughly two modes of execution: an AFL one and a QuickCheck one.



In QuickCheck mode it uses OCaml's randonmess source to try a fixed number of inputs. To try that

mode you can run:

```

$ dune exec awesome-list/fuzz/fuzz_me.exe

```



Alternatively you can also run the QuickCheck mode until a test failure is discovered with the `-i`

option of the binary like this:

```

$ dune exec -- awesome-list/fuzz/fuzz_me.exe -i

```



In AFL mode, it just use the input supplied by AFL as a source of randomness to supply values of the

right form to your test functions. You can run it just like with a regular fuzzing binary:

```

$ dune build awesome-list/fuzz/fuzz_me.exe

$ afl-fuzz -i awesome-list/fuzz/inputs -o _build/default/awesome-list/fuzz/findings ./_build/default/awesome-list/fuzz/fuzz_me.exe @@

```



Or use the convenience dune alias:

```

$ dune build @awesome-list/fuzz --no-buffer

```



Both modes should find the bugs in a split second. In QuickCheck mode it'll pretty print the input

value that triggered the failure. In AFL mode you can proceed as in the above example, i.e. kill the

`afl-fuzz` process once it found the two unique crashes. From there, inspecting the input files

won't tell you much as it's just used to seed `crowbar` PRNG but you can run the fuzz binary on

those and the input values will be pretty printed the same way they are in QuickCheck mode:

```

$ cd _build/default/awesome-list/fuzz

$ ./fuzz_me.exe findings/crashes/

Awesome_list.sort: ....

Awesome_list.sort: FAIL



When given the input:



    [4; 5; 6]



the test failed:



    check false



Fatal error: exception Crowbar.TestFailure

```



## Bun and fuzz testing in CI



This repo also contain Drone CI scripts that run fuzz testing for both our examples.

`afl-fuzz` is not very CI friendly by essence so we use

[`bun`](https://github.com/yomimono/ocaml-bun). `bun` is a CLI wrapper for `afl-fuzz`, written in

OCaml. It takes care of a few things for you, such as running several fuzzing processes in parallel

while correctly setting `afl-fuzz` to do so but most of all it wraps each of those processes so that

the execution fail whenever one of them finds a crash. Finally, it will also display the input that

lead to the test failure so that you can try to reproduce and debug it locally.



The CI script itself is in the `.drone.yml` file and it looks like that:

```yml

kind: pipeline

name: amd



platform:

  os: linux

  arch: amd64



steps:

- name: build

  image: ocaml/opam2:4.07

  commands:

  - sudo apt-get update && sudo apt-get -y install afl

  - sudo chown -R opam .

  - git -C /home/opam/opam-repository pull origin && opam update

  - opam switch 4.07+afl

  - opam depext crowbar bun

  - opam install -y crowbar bun

  - opam exec -- dune build @bun-fuzz --no-buffer

```



As you can see, there's nothing special here. We install the dependencies and build the `bun-fuzz`

alias.



There's one for `simple-parser` and one  for `awesome-list`. The aliases are identical and defined

as:

```

(alias

 (name bun-fuzz)

 (locks %{project_root}/bun)

 (deps

  (:exe fuzz_me.exe)

  (source_tree input))

 (action

  (run bun --input inputs --output findings -- ./%{exe})))

```



The default `bun` invocation is very similar to a regular `afl-fuzz` invocation.

You'll note the use of `(locks %{project_root}/bun)` to prevent concurrent execution of fuzz tests

by dune. This is required because by default `bun` will use all available cores and `afl-fuzz` won't

use a core that's already running an `afl-fuzz` process.



You may try it locally:

```

$ dune build @awesome-list/bun-fuzz

09:05.39:Fuzzers launched: [1 (pid=7357); 2 (pid=7358); 3 (pid=7359);

                            4 (pid=7360); 5 (pid=7361); 6 (pid=7362);

                            7 (pid=7363); 8 (pid=7364)].

09:05.39:Fuzzer 1 (pid=7357) finished

Crashes found! Take a look; copy/paste to save for reproduction:

echo J3JhaWl0IA== | base64 -d > crash_0.$(date -u +%s)

09:05.39:[ERROR]All fuzzers finished, but some crashes were found!

```



or you can take a look at the [latest

build](https://cloud.drone.io/NathanReb/ocaml-afl-examples/16/1/2).



`bun` comes with a bunch of configuration CLI options and I invite you to take a look at its

documentation by running `bun --help` to find out how to make the most out of it in your particular

use case.



One useful bun feature is its `no-kill` mode. There's a `bun-fuzz-no-kill` alias available for

`awesome-list` fuzz tests, it's defined as:

```

(alias

 (name bun-fuzz-no-kill)

 (locks %{project_root}/bun)

 (deps

  (:exe fuzz_me.exe)

  (source_tree input))

 (action

  (run timeout --preserve-status 1m bun --no-kill --input inputs --output

    findings -- ./%{exe})))

```



The `--no-kill` option tells `bun` to let the fuzzing processes run even after they found their

first crash. That can be convenient if you're not very confident about an implementation you're

fuzzing and you are expecting to find several bugs in it.

What makes this mode so nice is that it integrates with `timeout` fairly well. When receiving

`SIGTERM`, `bun` will shutdown all the `afl-fuzz` processes and will pretty print the crash

triggering inputs to the standard output in the same way it does in regular mode which makes this

a viable option for running fuzz tests in CI as well.