{"id":19662173,"url":"https://github.com/efeslab/asplos22-hardware-debugging-artifact","last_synced_at":"2026-01-27T18:17:35.428Z","repository":{"id":127885482,"uuid":"433156575","full_name":"efeslab/asplos22-hardware-debugging-artifact","owner":"efeslab","description":null,"archived":false,"fork":false,"pushed_at":"2024-06-12T20:48:16.000Z","size":19,"stargazers_count":12,"open_issues_count":0,"forks_count":3,"subscribers_count":9,"default_branch":"master","last_synced_at":"2025-06-04T02:58:26.221Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/efeslab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-11-29T18:37:37.000Z","updated_at":"2025-04-07T20:52:41.000Z","dependencies_parsed_at":null,"dependency_job_id":"a7a1be46-1390-43cb-971c-f8a68a3b0a4e","html_url":"https://github.com/efeslab/asplos22-hardware-debugging-artifact","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/efeslab/asplos22-hardware-debugging-artifact","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/efeslab%2Fasplos22-hardware-debugging-artifact","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/efeslab%2Fasplos22-hardware-debugging-artifact/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/efeslab%2Fasplos22-hardware-debugging-artifact/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/efeslab%2Fasplos22-hardware-debugging-artifact/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/efeslab","download_url":"https://codeload.github.com/efeslab/asplos22-hardware-debugging-artifact/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/efeslab%2Fasplos22-hardware-debugging-artifact/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28817797,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-27T18:01:38.485Z","status":"ssl_error","status_checked_at":"2026-01-27T18:01:27.499Z","response_time":168,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-11T16:09:50.912Z","updated_at":"2026-01-27T18:17:35.410Z","avatar_url":"https://github.com/efeslab.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# Debugging in the Brave New World of Reconfigurable Hardware  (Artifact)\n\nThis artifact includes 20 hardware bugs, each of them can be reproduced with Verilator in a push-button manner. It also includes the five tools we designed to help bug localization (i.e., SignalCat, FSM Monitor, Statistics Monitor, Dependency Monitor, and LossCheck), as well as examples of using these tools and the instructions of reproducing the figures in the paper.\n\nThe full list of 68 bugs we studied can be found [here](https://docs.google.com/spreadsheets/d/1GonADjkm878iRs2noQFXW5AidY4KiiJfTPJyIq-3Z4I/edit?usp=sharing).\n\nIf you have an interesting bug that you can reproduce, feel free to submit a pull request and we will add it to this repo. If you notice a bug that's not reproducible but still want to share to others, you may request edit access to the spreadsheet and add it there.\n\n## Table of Contents\n\n- [0. Downloading the Repository](#0-downloading-the-repository)\n\n- [1. Reproducible Bugs](#1-reproducible-bugs)\n  - [1.1 Installation](#11-installation)\n  - [1.2 Reproducing Bugs with Verilator](#12-table-of-bugs)\n- [2. Debugging Tools](#2-debugging-tools)\n  - [2.1 Installation](#21-installation)\n  - [2.2 SignalCat and the Monitors](#22-signalcat-and-monitors)\n    - [2.2.1 Debugging Logs with SignalCat and the Monitors](#221-debugging-logs-with-signalcat-and-the-monitors)\n    - [2.2.2 Reproducing the Resource Overhead](#222-reproducing-the-resource-overhead)\n  - [2.3 LossCheck](#23-losscheck)\n    - [2.3.1 Data Loss Localization for the 6 Data Loss Bugs](#231-data-loss-localization-for-the-6-data-loss-bugs)\n    - [2.3.2 Reproducing the Resource Overhead](#232-reproducing-the-resource-overhead)\n- [3. Licenses and Terms](#3-licenses-and-terms)\n\n## 0. Downloading the Repository\n\nUse the following command to download the artifact repository:\n\n```bash\ngit clone --recursive https://github.com/efeslab/asplos22-hardware-debugging-artifact\n```\n\nAfter this command, you are expected to see the following directory hierarchy:\n\n```bash\nasplos22-hardware-debugging-artifact\n├── hardware-bugbase\n│   ├── c1-dead-lock-sdspi\n│   ├── c2-producer-consumer-mismatch-optimus\n│   ├── c3-signal-asynchrony-sdspi\n│   ├── c4-signal-asynchrony-axi-stream-fifo\n│   ├── common\n│   ├── d10-failure-to-update-sha512\n│   ├── d11-failure-to-update-frame-fifo\n│   ├── d12-failure-to-update-frame-fifo\n│   ├── d13-failure-to-update-frame-len\n│   ├── d1-buffer-overflow-rsd\n│   ├── d2-buffer-overflow-grayscale\n│   ├── d3-buffer-overflow-optimus\n│   ├── d4-buffer-overflow-frame-buffer\n│   ├── d5-bit-truncation-sha512\n│   ├── d6-bit-truncation-fft\n│   ├── d7-misindexing-fadd\n│   ├── d8-misindexing-axis-switch\n│   ├── d9-endianness-mismatch-sdspi\n│   ├── manual_debug_log\n│   ├── n1-frame-len-failure-to-update\n│   ├── n3-frame-fifo-fail-to-update\n│   ├── n8-axis-adapter-incomplete-implementation\n│   ├── n9-frame-fifo-failure-to-update\n│   ├── s1-protocol-violation-axi-lite\n│   ├── s2-protocol-violation-axi-stream\n│   ├── s3-incomplete-implementation-axis-adapter\n│   └── scripts\n└── veripass\n    ├── dbgtools\n    ├── model\n    ├── passes\n    ├── Pyverilog\n    ├── recording\n    ├── utils\n    └── verilator\n```\n\n## 1. Reproducible Bugs\n\nThe `hardware-bugbase` directory contains all the reproducible bugs. Each bug is located in a directory, together with a simplified code snippet that helps understanding.\n\n### 1.1 Installation\n\nYou will need to install compile `Verilator` to reproduce these bugs. `Verilator` is located under the `veripass` directory.\n\nBefore compilation, you will need to install a few dependencies:\n\n```bash\nsudo apt-get install perl python3 make autoconf g++ flex bison ccache\nsudo apt-get install libgoogle-perftools-dev numactl perl-doc\nsudo apt-get install libfl2 libfl-dev  # Ubuntu only (ignore if gives error)\nsudo apt-get install zlibc zlib1g zlib1g-dev  # Ubuntu only (ignore if gives error)\n```\n\nThen compile `Verilator`:\n\n```bash\ncd asplos22-hardware-debugging-artifact/veripass/verilator\nautoconf\n./configure\nmake -j8\n```\n\nCompilation is enough. You do not need to install it. Scripts in the bug database will find the location of `Verilator` themselves.\n\n### 1.2 Reproducing Bugs with Verilator\n\nBugs are listed in the table below. You may `cd` into the directory of each bug to reproduce it and read its documentations.\n\nTo reproduce a specific bug:\n\n```bash\ncd asplos22-hardware-debugging-artifact/hardware-bugbase/\u003cbug-dir\u003e\nmake -j8 # compile the verilog code for simulation\nmake sim  # run the simulation\nmake wave # open the generated waveform with GTKWave\n```\n\nYou are expected to see an error message after `make sim`. `make wave` requires you using GUI, or the `DISPLAY` environment variable being set correctly. After simulation, you can also find a `.fst` file or a `.vcd` file under the directory. These are the waveforms generated by `Verilator`. You can copy the file to another computer and open it with `GTKWave` or other waveform-viewing software.\n\n| Bug ID | Bug Name                                                     |\n| ------ | ------------------------------------------------------------ |\n| D1     | [Buffer Overflow - RSD](https://github.com/efeslab/hardware-bugbase/tree/bugs/d1-buffer-overflow-rsd) |\n| D2     | [Buffer Overflow - Grayscale](https://github.com/efeslab/hardware-bugbase/tree/bugs/d2-buffer-overflow-grayscale) |\n| D3     | [Buffer Overflow - Optimus](https://github.com/efeslab/hardware-bugbase/tree/bugs/d3-buffer-overflow-optimus) |\n| D4     | [Buffer Overflow - Frame FIFO](https://github.com/efeslab/hardware-bugbase/tree/bugs/d4-buffer-overflow-frame-buffer) |\n| D5     | [Bit Truncation - SHA512](https://github.com/efeslab/hardware-bugbase/tree/bugs/d5-bit-truncation-sha512) |\n| D6     | [Bit Truncation - FFT](https://github.com/efeslab/hardware-bugbase/tree/bugs/d6-bit-truncation-fft) |\n| D7     | [Misindexing - FADD](https://github.com/efeslab/hardware-bugbase/tree/bugs/d7-misindexing-fadd) |\n| D8     | [Misindexing - AXI-Stream Switch](https://github.com/efeslab/hardware-bugbase/tree/bugs/d8-misindexing-axis-switch) |\n| D9     | [Endianness Mismatch - SDSPI](https://github.com/efeslab/hardware-bugbase/tree/bugs/d9-endianness-mismatch-sdspi) |\n| D10    | [Failure-to-Update - SHA512](https://github.com/efeslab/hardware-bugbase/tree/bugs/d10-failure-to-update-sha512) |\n| D11    | [Failure-to-Update - Frame FIFO](https://github.com/efeslab/hardware-bugbase/tree/bugs/d11-failure-to-update-frame-fifo) |\n| D12    | [Failure-to-Update - Frame FIFO](https://github.com/efeslab/hardware-bugbase/tree/bugs/d12-failure-to-update-frame-fifo) |\n| D13    | [Failure-to-Update - Frame Length Measurer](https://github.com/efeslab/hardware-bugbase/tree/bugs/d13-failure-to-update-frame-len) |\n| C1     | [Dead Lock - SDSPI](https://github.com/efeslab/hardware-bugbase/tree/bugs/c1-dead-lock-sdspi) |\n| C2     | [Producer-Consumer Mismatch - Optimus](https://github.com/efeslab/hardware-bugbase/tree/bugs/c2-producer-consumer-mismatch-optimus) |\n| C3     | [Signal Asynchrony - SDSPI](https://github.com/efeslab/hardware-bugbase/tree/bugs/c3-signal-asynchrony-sdspi) |\n| C4     | [Signal Asynchrony - AXI-Stream FIFO](https://github.com/efeslab/hardware-bugbase/tree/bugs/c4-signal-asynchrony-axi-stream-fifo) |\n| S1     | [Protocol Violation - AXI-Lite](https://github.com/efeslab/hardware-bugbase/tree/bugs/s1-protocol-violation-axi-lite) |\n| S1     | [Protocol Violation - AXI-Stream](https://github.com/efeslab/hardware-bugbase/tree/bugs/s2-protocol-violation-axi-stream) |\n| S3     | [Incomplete Implementation - AXI-Stream Adapter](https://github.com/efeslab/hardware-bugbase/tree/bugs/s3-incomplete-implementation-axis-adapter) |\n\n## 2. Debugging Tools\n\nOur debugging tools locate in the `veripass` directory. In the `hardware-bugbase` directory, we provide `make` scripts to invoke these debugging tools.\n\n**Warning: A full evaluation of this part takes days, because FPGA synthesis is slow (e.g., up to several hours per-run). We encourage you to evaluate the non-synthesis part (e.g., [2.3.1](#231-data-loss-localization-for-the-6-data-loss-bugs)) first.**\n\n### 2.1 Installation\n\nTo run the debugging tools, you will need to compile `Verilator` and `Pyverilog` if you have not done so already:\n\n```bash\ncd asplos22-hardware-debugging-artifact/veripass\nmake -j8\n```\n\nAnd install the following python packages:\n\n```bash\npip3 install jinja2 sympy ply gephistreamer\n```\n\nAnd add the following lines to your `.bashrc` or `.zshrc` to help the scripts find `Vivado`, `Quartus`, and `VCS`. `Vivado` must be the `Design Suite` edition, `Quartus` must be the `Pro` edition with version `17.0`, and `VCS` must be the `MX` edition.\n\n```bash\n# Quartus Pro\nexport QUARTUS_HOME=\u003cyour-quartus-home\u003e/17.0/quartus\nexport PATH=$QUARTUS_HOME/bin:$PATH\nexport LM_LICENSE_FILE=\u003cyour-quartus-license\u003e\n\n# Vivado\nexport XILINX_VIVADO=\u003cyour-vivado-home\u003e/Vivado/2020.2\nexport PATH=$XILINX_VIVADO/bin:$PATH\nexport XILINXD_LICENSE_FILE=\u003cyour-vivado-license\u003e\n\n# VCS MX\nexport VCS_HOME=\u003cyour-vcs-home\u003e\nexport PATH=$VCS_HOME/bin:$PATH\nexport SNPSLMD_LICENSE_FILE=\u003cyour-vcs-license\u003e\n```\n\nIn order to synthesize projects for Intel HARP, you will need to download a supported version of Intel FPGA Basic Building Blocks, a set of platform files for HARP, and have the following additional lines in `.bashrc` or `.zshrc`. You can ask your Intel contact for `BBS_6.4.0`. You may want to read [this](https://www.intel.com/content/dam/www/programmable/us/en/pdfs/literature/manual/mnl-ias-ccip.pdf) to understand the interface of the HARP platform. It is theoretically possible to compile these HARP projects for the PAC platform (which is more widely available); however, we did not evaluate it.\n\n```bash\nexport OPAE_PLATFORM_ROOT=\u003cyour-opae-platform-root-location\u003e/BBS_6.4.0\nexport PATH=$OPAE_PLATFORM_ROOT/bin:$PATH\n```\n\nThe original framework for HARP simulation requires Python 2 as the default `python` command. As a result, you may need to set up a `virtualenv` with the following command:\n\n```bash\nvirtualenv --python=/usr/bin/python2 \u003cpath-to-virtualenv\u003e\n```\n\n### 2.2 SignalCat and the Monitors\n\n#### 2.2.1 Debugging Logs with SignalCat and the Monitors\n\nIn Section 6.2 of the paper, we demonstrated that a developer can use SignalCat and the Monitors to localize all the 20 bugs in this artifact. We provide the mental debugging logs of a developer localizing these bugs in [this sheet](https://github.com/efeslab/hardware-bugbase/blob/f3f391be341e2f36462dd94bba98b8bffa81c34b/manual_debug_log/manual_debugging_agreements.xlsx). For each bug, the sheet includes the tools the developer would use at each step. The configurations for invoking these tools are located in a `.cfg` file under each bug's directory; you can invoke the tools using the following commands under each bug's directory:\n\n```bash\nmake withtask.v\n```\n\nAfter running this command, a file called `withtask.v` will be generated. This file contains the flattened verilog code with the debugging instrumentations described in the configuration.\n\n#### 2.2.2 Reproducing the Resource Overhead\n\nTo synthesize the instrumented circuit, you may run the following command:\n\n```bash\nsource \u003cpath-to-virtualenv\u003e/bin/activate # switch to a python virtualenv where python2 is the default\nmake sweep_depth\n```\n\nThis command will generate a number of files (e.g., instrumented circuit with different buffer size, the TCL scripts to invoke synthesis, etc) and invoke the synthesis script for the circuit and run syntheses with different recording buffer size. This command would froze for a long time, because each synthesis takes hours.\n\nAfter the command finishes, you can run the following command to report resource utilization.\n\n```bash\nmake report_depth_sweep\n```\n\nFor `D4`, `D6`, `D7`, `D8`, `D9`, `D11`, `D12`, `D13`, `C1`, `C3`, `C4`, `S1`, `S2`, and `S3`, you will see something like the following.\n\n```\nlog2(Depth),10,11,12,13\nTotal LUTs,2225,2208,2191,2287\nFFs,2870,2881,2892,2905\nRAMB36,4,7,15,30\nRAMB18,0,1,0,0\n\nbuild_notask: Total LUTs,FFs,RAMB36,RAMB18\n        858;516;0;0\n```\n\nThe upper block shows the resource utilization of instrumented circuit, and the bottom block shows the resource utilization of the uninstrumented circuit. In the paper, we use the word `Logic` for `LUT`, `Register` for `FF`, and calculate the total number of bits from `RAM36` (36Kbit per instance) and `RAM18` (18Kbit per instance). In the above example, the register overhead of an instrumented circuit with a `1024`-depth buffer is `2870-517=2354`.\n\nFor `D1`, `D2`, `D3`, `D5`, `D10`, and `C2`, you will see something like the following. We use the `Logic` for `ALM`, `Register` for `FF`, and use the number of `BRAM Blocks` to calculate BRAM size (each block contains 20Kbits).\n\n```\nlog2(Depth),10,11,12,13\nALM,101170,101173,101185,101191\nBRAM#B,326,343,376,477\nBRAMbit,3989920,4332960,5019040,6391200\nFFs,111356,111371,111397,111447\n\nbuild_notask: ALM BRAM#B BRAMbit FFs\n100245;309;3646880;108734\n```\n\n### 2.3 LossCheck\n\nBug `D1`, `D2`, `D3`, `D4`, `C2`, and `C4` are the six data loss bugs that can be localized by LossCheck.\n\n#### 2.3.1 Data Loss Localization for the 6 Data Loss Bugs\n\nYou can use the following command to invoke LossCheck under the directories of these four bugs.\n\n```bash\nmake -f Makefile.lc\n```\n\nFor `D1`, `D2`, `D3`, and `C4`:\nThis will generate two `.v` files (e.g., a `\u003cbenchmark\u003e.losscheck.0.v` and a `\u003cbenchmark\u003e.losscheck.1.v`). `\u003cbenchmark\u003e.losscheck.0.v` is the first instrumentation, which does not filter false positives (as discussed in Section 4.5.3). Our scripts run the original testbench of the circuit on the first instrumentation, and generate a list of signals that should be filtered out (i.e., storing in `filter.txt`). Then, our scripts invoke LossCheck again, generating the second instrumentation (i.e., `\u003cbenchmark\u003e.losscheck.1.v`), with the signals in `filter.txt` filtered out.\n\nFor `D4` and `C4`:\nThis will generate a `test.v` file, which is the flattened design with LossCheck's instrumentation. These two bugs do not need false positive filtering. As a result, no `filter.txt` file will be generated.\n\n\nTo verify that the second instrumentation actually detects the data loss, run the following command:\n\n```bash\nmake -f Makefile.lc sim\n```\n\nYou are expected to see some error message with regard to data loss. For `D2`, `D3`, `D4`, `C2`, and `C4`, there should be no false positives. For `D1`, you are expected to see one register that's misidentified. (You will see several rows misreporting the same register.)\n\n#### 2.3.2 Reproducing the Resource Overhead\n\nYou can use the following command to synthesize the circuit with and without LossCheck instrumentation. Please note each synthesis can take hours.\n\n```bash\nmake -f Makefile.lc synth\n```\n\nAnd use the following command to report resource utilization.\n\n```bash\nmake -f Makefile.lc report_util\n```\n\nFor `D1`, `D2`, `D3`, and `C2`, you will get something like this:\n\n```\nbuild_withlosscheck: ALM BRAM#B BRAMbit FFs\n115428;775;11146672;139645\nbuild_notask: ALM BRAM#B BRAMbit FFs\n109694;413;5238192;130390\n```\n\nThese four bugs are on the Intel HARP platform. This platform contains a vendor-provided shell and an user-implemented accelerator. Because the shell is a fixed region and is not usable by the accelerator, the resource overhead in Figure 3 is normalized to the total available resource of the accelerator region (i.e., without the shell). You may use the following data as the available resource of the accelerator-usable region.\n\n| ALM    | FFs    |\n| ------ | ------ |\n| 327029 | 1600141 |\n\nIn the above example, the uninstremented accelerator uses 9523 ALMs, and the instrumented accelerator uses 15257 ALMs. As a result, the ALM (logic) overhead is `(15257-9523)/327029=1.7%`.\n\nSpecifically, as we mentioned in our paper, the frequency of D3 and C2 (i.e., the Optimus bugs) will be reduced from 400MHz to 200MHz after LossCheck's instrumentation. As a result, we need to add an asynchronous fifo which helps clock domain crossing. When generating the verilog files for compilation, the makefile will add the fifo.\n\nFor `D4` and `C4`, you will get something like this:\n\n```\nbuild_withlosscheck: Total LUTs,FFs,RAMB36,RAMB18\n        1435;2415;16;1\nbuild_notask: Total LUTs,FFs,RAMB36,RAMB18\n        45;83;0;0\n```\nThese two bugs are on the Xilinx platform. There's no shell in the platform so the accelerator can use all resource on the FPGA. You may use the following data as the available resource.\n\n| LUT   | FFs   |\n| ----- | ----- |\n| 203800 | 407600 |\n\n## 3. Licenses and Terms\n\nThis artifact includes modified versions of Pyverilog (`veripass/Pyverilog`) and Verilator (`veripass/verilator`), which are released under their original licenses. Bugs in the `hardware-bugbase` directory are collected (and organized) from different sources, and are also released under the original licenses of the original implementation.\n\nOur debugging tools under the `veripass` directory are released under the `GPLv3` license, whatever it means. Please also note that these tools are academic prototypes and may not be stable, reliable, or always correct; use it at your own risk.\n\nBy downloading/cloning/forking the `veripass` repository, you have known and agreed to all terms included in `GPLv3`, and that the developers/authors of these tools will not be responsible for any of your losses and/or damages, including but not limited to the tools not working as expected and your loved ones being unhappy of you working/hacking at 3am.\n\nIf you find our work interesting, please cite our paper.\n\n```\n@inproceedings{ma2022debugging,\n  title={Debugging in the Brave New World of Reconfigurable Hardware},\n  author={Ma, Jiacheng and Zuo, Gefei and Loughlin, Kevin and Zhang, Haoyang and Quinn, Andrew and Kasikci, Baris},\n  booktitle={Proceedings of the Twenty-Seventh International Conference on Architectural Support for Programming Languages and Operating Systems},\n  year={2022}\n}\n```\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fefeslab%2Fasplos22-hardware-debugging-artifact","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fefeslab%2Fasplos22-hardware-debugging-artifact","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fefeslab%2Fasplos22-hardware-debugging-artifact/lists"}