{"id":13648886,"url":"https://github.com/chiselverify/chiselverify","last_synced_at":"2026-01-11T17:01:28.626Z","repository":{"id":37911446,"uuid":"267338934","full_name":"chiselverify/chiselverify","owner":"chiselverify","description":"A dynamic verification library for Chisel.","archived":false,"fork":false,"pushed_at":"2024-11-09T16:18:23.000Z","size":5364,"stargazers_count":151,"open_issues_count":8,"forks_count":23,"subscribers_count":12,"default_branch":"master","last_synced_at":"2025-06-06T00:28:58.668Z","etag":null,"topics":["bus-functional-model","chisel","chisel-test","constrained-random-verification","coverage","functional-coverage","scala","testing","timed-assertions","verification"],"latest_commit_sha":null,"homepage":"","language":"Scala","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-2-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/chiselverify.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2020-05-27T14:18:26.000Z","updated_at":"2025-05-19T17:36:00.000Z","dependencies_parsed_at":"2024-01-14T10:59:44.358Z","dependency_job_id":"47c0db55-99be-4927-9fe0-a1cef7155ac6","html_url":"https://github.com/chiselverify/chiselverify","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"purl":"pkg:github/chiselverify/chiselverify","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chiselverify%2Fchiselverify","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chiselverify%2Fchiselverify/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chiselverify%2Fchiselverify/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chiselverify%2Fchiselverify/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/chiselverify","download_url":"https://codeload.github.com/chiselverify/chiselverify/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chiselverify%2Fchiselverify/sbom","scorecard":{"id":278204,"data":{"date":"2025-08-11","repo":{"name":"github.com/chiselverify/chiselverify","commit":"1a48e0e96bddbb983825a894714cfcc00e32e14b"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":3.5,"checks":[{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Dangerous-Workflow","score":10,"reason":"no dangerous workflow patterns detected","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Code-Review","score":1,"reason":"Found 5/26 approved changesets -- score normalized to 1","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Token-Permissions","score":0,"reason":"detected GitHub workflow tokens with excessive permissions","details":["Warn: no topLevel permission defined: .github/workflows/ci.yml:1","Info: no jobLevel write permissions found"],"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Pinned-Dependencies","score":0,"reason":"dependency not pinned by hash detected -- score normalized to 0","details":["Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:10: update your workflow using https://app.stepsecurity.io/secureworkflow/chiselverify/chiselverify/ci.yml/master?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/ci.yml:13: update your workflow using https://app.stepsecurity.io/secureworkflow/chiselverify/chiselverify/ci.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:18: update your workflow using https://app.stepsecurity.io/secureworkflow/chiselverify/chiselverify/ci.yml/master?enable=pin","Info:   0 out of   2 GitHub-owned GitHubAction dependencies pinned","Info:   0 out of   1 third-party GitHubAction dependencies pinned"],"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Vulnerabilities","score":10,"reason":"0 existing vulnerabilities detected","details":null,"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: BSD 2-Clause \"Simplified\" License: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'master'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 9 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}}]},"last_synced_at":"2025-08-17T15:03:24.547Z","repository_id":37911446,"created_at":"2025-08-17T15:03:24.547Z","updated_at":"2025-08-17T15:03:24.547Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28314259,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-11T14:58:17.114Z","status":"ssl_error","status_checked_at":"2026-01-11T14:55:53.580Z","response_time":60,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bus-functional-model","chisel","chisel-test","constrained-random-verification","coverage","functional-coverage","scala","testing","timed-assertions","verification"],"created_at":"2024-08-02T01:04:37.521Z","updated_at":"2026-01-11T17:01:28.467Z","avatar_url":"https://github.com/chiselverify.png","language":"Scala","funding_links":[],"categories":["Scala","Hardware Verification"],"sub_categories":["Tools"],"readme":"# ChiselVerify: A Hardware Verification Library for Chisel\n\nIn this repository, we proprose ChiselVerify, which is the beginning of a verification library within Scala for digital hardware described in [Chisel](https://github.com/chipsalliance/chisel3), but also supporting legacy components in VHDL, Verilog, or SystemVerilog. The library runs off of [ChiselTest](https://github.com/ucb-bar/chisel-testers2) for all of the DUT interfacing. An early technical report describing the initial version of the library in detail is available online: [Open-Source Verification with Chisel and Scala](https://arxiv.org/abs/2102.13460). For the most up-to-date writing on topic, please refer to [Verification of Chisel Hardware Designs with ChiselVerify](https://www.sciencedirect.com/science/article/pii/S0141933122002666).  \n\nWhen you use this library in a research project, please cite it as:\n```\n@article{dobis2023verification,\n  title = {Verification of Chisel Hardware Designs with ChiselVerify},\n  author = {Dobis, Amelia and Laeufer, Kevin and Damsgaard, Hans Jakob and Petersen, Tjark and Rasmussen, Kasper Juul Hesse and Tolotto, Enrico and Andersen, Simon Thye and Lin, Richard and Schoeberl, Martin},\n  journal = {Microprocessors and Microsystems},\n  volume = {96},\n  pages={104737},\n  year={2023},\n  publisher={Elsevier}\n  issn = {0141-9331},\n  doi = {https://doi.org/10.1016/j.micpro.2022.104737},\n  url = {https://www.sciencedirect.com/science/article/pii/S0141933122002666}\n}\n```\n\nChiselVerify is published on Maven. To use it, add following line to your\n```build.sbt```:\n\n```\nlibraryDependencies += \"io.github.chiselverify\" % \"chiselverify\" % \"0.4.0\"\n```\n\nRun tests with\n```\nmake\n```\n\nThis README contains a brief overview of the library and its functionalities. For a more in-depth tutorial, please check-out the [ChiselVerify Wiki](https://github.com/chiselverify/chiselverify/wiki). Other general documentation, such as technical reports, journal, and conference papers related to the chiselverify project can be found in the [documentation repository](https://github.com/chiselverify/documentation).\n  \n**********************************\n\n# Verification Library for Chisel\nThe library can be divided into 4 main parts:\n1. __Functional Coverage__: Enabling Functional Coverage features like Cover Points, Cross Coverage, Timed Coverage and Conditional Coverage.\n2. __Constrained Random Verification__: Allowing for constraints and random variables to be defined and used directly in Scala.\n3. __Bus Functional Models__: Enabling Transactional modeling for standardized Buses like _AXI4_.\n4. __Approximate Design Verification__: Provides comparative port samplers, as well as several error metrics that simplify the verification of approximate deisgns.    \n  \n\u003e __THE API PRESENTED IN THIS README MIGHT BE OUT-DATED, AS IT IS ONLY UPDATED OCCASIONALLY. THE MAIN SOURCE OF INFORMATION THAT SHOULD BE USED TO LEARN HOW TO USE THE LIBRARY IS [THE WIKI](https://github.com/chiselverify/chiselverify/wiki).__\n\n## Functional Coverage in Chisel\nThe idea is to implement functional coverage features directly in Chisel. The structure of the system can be seen in the diagram below.\n\n![Structure of the Coverage system](CoverageChisel.png)\n\n#### Coverage Reporter\nThis is the heart of the system. It handles everything from registering the Cover Points to managing the Coverage DataBase. It will also generate the final coverage report. Registering _Cover Points_ together will group them into a same _Cover Group_.\n\n#### Coverage DataBase\nThis DataBase handles the maintenance of the values that were sampled for each of the Cover Point bins. This allows us to know how much of the verification plan was tested. The DataBase also keeps mappings linking _Cover Groups_ to their contained _Cover Points_.\n\n### How to use it\nThe Functional coverage system is compatible with the ChiselTest framework.\n1. The `CoverageReporter` must first be instanciated within a test.\n2. `CoverGroup`s can then be created by using the `register` method of the coverage reporter. This takes as parameter a `List[Cover]`. `Cover` represents either a `CoverPoint` or a `CoverCondition` that contains a `port` that will be sampled, a `portName` that will be shown in the report and either a `List[Bins]`, created from a name and a scala range, or a `List[Condition]`, created from a name and an arbitrary condition function.\n3. `CoverGroups` may also contain a `List[Cross]` which represents a set of hit relations between two ports.\n4. The port must then be manually sampled by calling the `sample` method of the coverage reporter.\n5. Once the test is done, a coverage report can be generated by calling the `printReport` or `report` methods of the coverage reporter.\n\nAn example of this can be found [here](./src/test/scala/examples/leros/AluAccuTester.scala).\n\n### Timed Cross Coverage\n__Idea__: We want to check the relationship between two ports with a delay of a certain amount of cycles.\n__Example__: We have the following situation, imagine we have a device that breaks if:\n- `dut.io.a` takes the value of `1.U` at cycle 1\n- `dut.io.b` takes the value of `1.U` the following cycle\n\nWe want to verify that the above case was tested. This can be done by defining a `TimedCross` between the two points:\n```scala\nval cr = new CoverageReporter(dut)\ncr.register(\n  // Declare CoverPoints\n  cover(\"a\", dut.io.a)(DefaultBin(dut.io.a)),\n  cover(\"b\", dut.io.b)(DefaultBin(dut.io.b)),\n  // Declare timed cross point with a delay of 1 cycle\n  cover(\"timedAB\", dut.io.a, dut.io.b)(Exactly(1))(\n    cross(\"both1\", Seq(1 to 1, 1 to 1))\n  )\n)\n```\n\nUsing that, we can check that we tested the above case in our test suite. This construct can be used to check delay between two cover points.\n\n#### Use\nTo be able to use the timed coverage, stepping the clock must be done through the coverage reporter:\n```scala\ndut.clock.step(nCycles) // Will trigger an exception if used with Timed Cross Coverage\ncr.step(nCycles)        // This works and updates the DataBase\n```\nThis is done in order ensure that the coverage DataBase will always remain synchronized with the DUT's internal clock.\n\n#### Delay Types\nThe current implementation allows for the following special types of timing:\n- `Eventually`: This sees if a cross hit was detected at any point in the next given amount of cycles.\n- `Always`: This only considers a hit if the it was detected every cycle in the next given amount of cycles.\n- `Exactly`: This only considers a hit if it was detected exactly after a given amount of cycles.\n\n### Timed Assertions  \nDelay types can also be used in order to used `Timed Assertions` or `Timed Expect`. These can be used in order to check an assertion, in the form of an arbitrary function, with an added timing argument. We could thus check, for example, that two ports are equal two cycles appart. For example:\n```scala\nAssertTimed(dut, dut.io.a.peek() === dut.io.b.peek(), \"aEqb expected timing is wrong\")(Exactly(2)).join()\n```\nThis can also be done more naturally with the `Expect` interface:\n```scala\nExpectTimed(dut,dut.io.a, dut.io.b.peek().litValue(), \"aEqb expected timing is wrong\")(Exactly(2)).join()\n```\nThese can also be used with a simplyfied syntax, inspired by ScalaTest syntax:\n```scala\n// For Timed Assertions\neventually(2, \"aEqb expected timing is wrong\") { dut.io.a.peek() === dut.io.b.peek() }\nexact(2, \"aEqb expected timing is wrong\") { dut.io.a.peek() === dut.io.b.peek() }\nalways(2, \"aEqb expected timing is wrong\") { dut.io.a.peek() === dut.io.b.peek() }\nnever(2, \"aEqb expected timing is wrong\") { dut.io.a.peek() === dut.io.b.peek() }\n```\n\n### Example use case\nHere is a toy example of how to use the assertion:\n```scala\nalways(9, \"a isn't always less than or equal to one\") { LtEq(dut.io.outB, dut.io.outA) }\nalways(9, \"a isn't always greater than or equal to one\") { GtEq(dut.io.outB, dut.io.outA) }\n```\n\n### Cover Conditions\n__Idea__: A type of coverpoint that can apply arbitrary hit conditions to an arbitrary number of ports.\n```scala\ncover(readableName: String, ports: Data*)(conditions: Condition*)\n// where a condition is declared using the bin function without a range\nbin(name: String, func : Seq[BigInt] =\u003e Boolean)\n```\n__Example__:\n```scala\nval cr = new CoverageReporter(dut)\ncr.register(\n  // Declare CoverPoints\n  cover(\"aAndB\", dut.io.outA, dut.io.outB)(\n    bin(\"aeqb\", { case Seq(a, b) =\u003e a == b })\n  )\n)\n```\n\nBins are thus defined using arbitrary functions of the type `List[BigInt] =\u003e Boolean` which represent different hit conditions. No coverage percentage is given due to cartesian product complexity. Instead, we offer the possibility to use a user-defined \"expected number of hits\" to get a coverage percentage. This looks like the following:\n```scala\nval cr = new CoverageReporter(dut)\ncr.register(\n  // Declare CoverPoints\n  cover(\"aAndB\", dut.io.outA, dut.io.outB)(\n    bin(\"asuptobAtLeast100Times\", condition = { case Seq(a, b) =\u003e a \u003e b }, expectedHits = 100)\n  )\n)\n```\nThe above example results in the following coverage report:\n```\n============ COVERAGE REPORT ============\n============== GROUP ID: 1 ==============\nCOVER_CONDITION NAME: aAndB\nCONDITION aeqb HAS 4 HITS\nCONDITION asuptobAtLeast100 HAS 95 HITS EXPECTED 100 = 95.0%\n=========================================\n=========================================\n```\n\n## Constrained Random Verification (CRV)\nThe CRV package inside this project aims to mimic the functionality of SystemVerilog constraint programming and integrates them into [ChiselTest](https://github.com/ucb-bar/chisel-testers2). It combines a Constraint Satisfactory Problem (CSP) solver with some helper classes to create and use random objects inside your tests. Currently, only the [jacop](https://github.com/radsz/jacop) backend is supported, but in the future other backends can be added.\n\n### Comparison\nThe following is a comparison of how to describe a randomizable packet with SystemVerilog and our proposed framework.\n\n#### SystemVerilog\n```systemverilog\nclass frame_t;\nrand pkt_type ptype;\nrand integer len;\nrandc bit [1:0] no_repeat;\n// Constrain the members\nconstraint legal {\n  len \u003e= 2;\n  len \u003c= 5;\n}\n```\n\n#### CRV w/ jacop backend\n```scala\nclass Frame extends RandObj(new Model) {\n  val pkType: RandVar = rand(0, 3)\n  val len: RandVar = rand(0, 10)\n  val noRepeat: RandVar = rand(0, 1, Cyclic)\n\n  val legal = new ConstraintGroup {\n    len \u003e= 2\n    len \u003c= 5\n  }\n}\n```\n\n### Random Objects\nRandom objects can be created by extending the RandObj trait. This class accepts one parameter which is a Model. A model correspond to a database in which all the random variables and constraints declared inside the RandObj are stored.\n```scala\nclass Frame extends RandObj(new Model)\n```\nA model can be initialized with a seed `new Model(42)`, which allows the user to create reproducible tests.\n\n#### Random Fields  \nRandom fields are defined using the following function: \n```scala\ndef rand(min: Int, max: Int, randType: RandType = Normal)(implicit model: Model)\n```  \n\nA random field can be added to a `RandObj` by declaring a Rand variable.\n```scala\n  val len = rand(0, 10)\n```\n\nRandom-cyclic variable can be added by declaring a `Randc` field inside a `RandObj`. This is done using the `Cyclic RandType` parameter.\n```scala\n  val noRepeat = rand(0, 1, Cyclic)\n```\n\n#### Constraints\nEach variable can have one or multiple constraints. These are defined using constraint operators.\n```scala\nlen \u003e= 2\n```\nIn the previous block of code we are specifying that the variable `len` can only take values that are grater then 2. Each constraint can be assigned to a variable and enabled or disabled at any time during the test.\n```scala\nval lenConstraint = len \u003e 2\n[....]\nlenConstraint.disable()\n[....]\nlenConstraint.enable()\n```\n\nConstraints can also be grouped together in a `ConstraintGroup` and the group itself can be enabled or disabled.\n\n```scala\nval legal = new ConstraintGroup {\n  len \u003e= 2\n  len \u003c= 5\n  payload.size == len\n}\n[...]\nlegal.disable()\n[...]\nlegal.enable()\n```\n\nBy default, constraints and constraint groups are enabled when they are declared.\n\nThe list of operator used to construct constraints is the following: `\u003c`, `\u003c=`, `\u003e`, `\u003e=`,`==`, `div`, `*`, `mod`, `+`, `-`, `\\=`, `^`, `in`, `inside`.\n\nIt is also possible to declare conditional constraints with constructors like `IfCon` and `IfElseCon`.\n```scala\nval constraint1 = IfCon(len == 1) {\n    payload.size == 3\n  } ElseC {\n    payload.size == 10\n  }\n```\n\n### Usage\nAs in SystemVerilog, each random class exposes a method called `randomize()` to solve the constraint problem specified in the class and assign to each random field a random value. The method returns `true` only if the CSP found a set of values that satisfy the current constraints.\n```scala\nval myPacket = new Frame(new Model)\nassert(myPacket.randomize)\n```\n\nOther usage examples can be found in our [backend tests](./src/test/scala/verifyTests/crv/backends/jacop/TestRandJacop.scala).\n\n## Verification of Approximate Designs\nThe `approximation` package within this library serves to simplify the process of verifying that approximate hardware designs satisfy commonly applied error metrics relative to exact counterparts.\n\n#### Metric\nError metrics contained within the library follow a hierarchy of two different characteristics that describe the values they return and which kind of inputs they take. Specifically, `Instantaneous` metrics may be computed on a single sample from two ports, whereas `HistoryBased` metrics require sequences of samples. `Absolute` metrics return non-normalized results, while `Relative` metrics return normalized or somehow relativized results. Inheriting (case) classes must extend one of `Instantaneous` or `HistoryBased` and mixin either `Absolute` or `Relative`.\n\n#### Watcher\nThese elements implement distributed sampling and storage for keeping track of a pair of an approximate port and its related exact port (or an exact reference value) and any number of metrics to track for them. While the main class is abstract and not meant to be used, these elements come in two types:\n- `Tracker`s that can simply sample port values and report on given metrics but lack verification features, thus ignoring any maximum values for the metrics provided.\n- `Constraint`s that extend upon the functionality of the `Tracker` by also allowing for verification, requiring that all provided metrics have maximum values.\n\nIn practice, these two are created using the simple API end-points `track` and `constrain`. They come in two variants: reference-based and port-based. The former are to be used with software models and require passing in expected values when sampled. Contrarily, the latter are to be used with combined DUTs that provide both approximate and exact outputs, thus not requiring but nonetheless accepting reference values when sampled.\n\n#### Error Reporter\nThis is the core of the system. It handles any number of `Tracker`s and/or `Constraints`, sampling of their related ports, reporting on their results, and verifying that their constraints are met.\n\n### How to use it\nThe approximate verification system is closely modeled after our functional coverage tools and remains fully compatible with the ChiselTest framework. Tests may be written in one of two ways depending on whether the exact DUT is implemented as a Chisel `Module` or as a software model.\n\n1. As ChiselTest only allows for testing one Chisel module at a time, it is necessary to first define a top-level module that contains both the approximate and exact DUTs, should the exact DUT be implemented as a Chisel `Module`. Otherwise, skip ahead to step 2.\n2. The `ErrorReporter` must then be instantiated within a test and provided any number of `Tracker`s and/or `Constraints` watching ports of the DUT.\n3. When instantiated, its watchers must be sampled manually by calling the `sample` method of the error reporter. If the exact DUT is a software model, this method must be passed a map of port to reference value pairs with the ports corresponding to ones registered in the watchers. It is not possible to sample a reference-based watcher without passing it a reference value. It is valid to use a combination of reference-based and port-based watchers and selectively override port-based watchers' exact values by passing in expected values.\n4. Once the test is done, an error report can be generated by calling the `report` methods of the error reporter and printing its return value.\n\nAn example of this can be found [here](./src/test/scala/examples/ApproxAdderTest.scala).\n\n## Example Use Cases\n\nWe will explore a handful of use cases to explore verification.\n\n* Leros ALU (basically done)\n* Heap priority queue (from MicroSemi),\n  see also https://www.hackerearth.com/practice/notes/heaps-and-priority-queues/\n* Network-on-chip (in Chisel), see https://github.com/schoeberl/soc-comm\n* Decimation filter from WSA (VHDL code plus testbench given)\n* Approximate adder (Chisel code plus testbench given)\n\n*******************************\n\n### Universal Verification Method (UVM) Examples\nIn the early stages of this project, we explored the possibilty of using UVM to verify Chisel designs. Thus, the [sv](https://github.com/chiselverify/otherverify/tree/main/sv) directory in another repository of ours contains a number of UVM examples.\n\n#### Simple examples\nIn `sv/uvm-simple-examples` a number of simple examples are located. These start with a very basic testbench with no DUT attached, and gradually transition into a complete testbench.\n\n#### Vivado UVM Examples\nThese examples assume that a copy of Xilinx Vivado is installed and present in the PATH. The examples are currently tested only on Linux. The first example is taken from [Vivado Design Suite Tutorial - Logic Simulation](https://www.xilinx.com/support/documentation/sw_manuals/xilinx2020_1/ug937-vivado-design-suite-simulation-tutorial.pdf).\n\n#### Leros ALU\nIn the directory [sv/leros](https://github.com/chiselverify/otherverify/tree/main/sv/leros), the [Leros ALU](./src/main/scala/leros/AluAccuChisel.scala) is tested using UVM, to showcase that Chisel and UVM can work together. This testbench is reused to also test a VHDL implementation of the ALU, to show that UVM is usable on mixed-language designs (when using a mixed-language simulator).\n\nThe VHDL implementaion is run by setting the makefile argument `TOP=top_vhd`.\n\n#### Using the SystemVerilog Direct Programming Interface (DPI) and Java's Native Interface (JNI)\nUsing the SystemVerilog DPI to cosimulate with a golden model described in C is explored in the [sv/leros/scoreboards/scoreboard_dpi.svh](https://github.com/chiselverify/otherverify/tree/main/sv/leros/scoreboards/scoreboard_dpi.svh). The C-model is implemented in [sv/leros/scoreboards/scoreboard.c](https://github.com/chiselverify/otherverify/tree/main/sv/leros/scoreboards/scoreboard.c), and the checking functionality is called from the SystemVerilog code.\n\nImplementing a similar functionality in Scala/Chisel has been explored via the JNI. In the directory [native](https://github.com/chiselverify/otherverify/tree/main/native), the necessary code for a simple Leros tester using the JNI is implemented.\n\nTo use the JNI functionality, first run `make jni` to generate the correct header files and shared libraries. Then, open sbt and type `project native` to access the native project. Then run `sbt test` to test the Leros ALU using a C model called from within Scala. To switch back, type `project chisel-uvm`.\n\n************************************\n\n# Resources\nIf you're interested in learning more about the UVM, we recommend that you explore the [otherverify](https://github.com/chiselverify/otherverify) repository as well as some of the following links:\n* [First steps with UVM](https://www.youtube.com/watch?v=qLr8ayWM_Ww)\n* [UVM Cookbook (requires an account)](https://verificationacademy.com/cookbook/uvm)\n* [ChipVerify.com UVM Tutorials](https://www.chipverify.com/table/uvm/)\n* [Ray Salemi's UVM Primer videos](https://www.youtube.com/watch?v=eeU2zpgXv1A\u0026list=PLigQ6Cc3qFpI_WTgqtDXi_Msk3yRuKGGJ)  \n\n************************************\n\n# Related Work\n\n## Fuzzing \nHere are a few pointers to some interesting documentation around the topic of mutation-based fuzzing:  \n* [American Fuzzy Lop (AFL)](https://github.com/google/AFL)  \n* [Binary fuzzing strategies: what works, what doesn't](https://lcamtuf.blogspot.com/2014/08/binary-fuzzing-strategies-what-works.html)  \n* [AFL \"Whitepaper\"](https://lcamtuf.coredump.cx/afl/technical_details.txt)  \n* [A bit more about american fuzzy lop](https://lcamtuf.blogspot.com/2014/08/a-bit-more-about-american-fuzzy-lop.html)  \n* [The fuzzing book - Mutation-Based Fuzzing](https://www.fuzzingbook.org/html/MutationFuzzer.html)  \n* [RFuzz conference paper](https://people.eecs.berkeley.edu/~ksen/papers/rfuzz.pdf)  \n* [RFuzz](https://github.com/ekiwi/rfuzz)\n\n## CRV\n* [Choco-Solver](https://github.com/chocoteam/choco-solver) Java library for solving CSP problems\n* [QuickCheck](https://hackage.haskell.org/package/QuickCheck) Checker for Haskel, used Lava as example, the inspiration for ScalaCheck\n\n## Testing Frameworks / Simulation Tools\n### Cocotb -- coroutine-based cosimulation library for hardware development in Python\n[Cocotb repository](https://github.com/cocotb/cocotb): cocotb is a coroutine based cosimulation library for writing VHDL and Verilog testbenches in Python.\n\n#### Resources Related to Cocotb\n* Philipp Wagner (FOSSi Foundation, lowRISC) [\"Cocotb: Python-powered hardware verification\"](https://www.youtube.com/watch?v=GUcKJ5zXgPA) (WOSH 2019) (video)\n* Ben Rosser (University of Pennsylvania) [\"Cocotb: a Python-based digital logic verification framework\"](https://indico.cern.ch/event/776422/attachments/1769690/2874927/cocotb_talk.pdf) (CERN 2018) (pdf)\n* Torbjørn Viem Ness (NTNU) [\"Low Power Floating-Point Unit for RISC-V\"](https://brage.bibsys.no/xmlui/bitstream/handle/11250/2564801/19493_FULLTEXT.pdf?sequence=1\u0026isAllowed=y) (2018) [PDF]\n* Andrey Filippov (Elphel) [\"I will not have to learn SystemVerilog\"](https://www.elphel.com/blog/2016/07/i-will-not-have-to-learn-systemverilog/) (2016) [Blog]\n* Chris Higgs (Potential Ventures) \"Applying agile techniques to FPGA development\" [Video](https://www.youtube.com/watch?v=Wi_1bUgHl8s), [Paper](http://www.testandverification.com/conferences/verification-futures/2015-europe/speaker-chris-higgs-potential-ventures/)\n* Chris Higgs (Potential Ventures) [\"Rapid FPGA Verification\"](https://docs.google.com/presentation/d/1U22Y_yyQRAecXXvOKFbHU8p_pGsB-u7iaK57Tw92U_A/embed?start=false\u0026loop=false\u0026delayms=3000) (NMI, February 2014) [Slides]\n* Smith, Andrew Michael; Mayo, Jackson; Armstrong, Robert C.; Schiek, Richard; Sholander, Peter E.; Mei, Ting (Sandia National Lab): [\"Digital/Analog Cosimulation using CocoTB and Xyce\"](https://www.osti.gov/biblio/1488489) (paper)\n\n#### Extension of Cocotb\n* [cocotb-coverage](https://github.com/mciepluc/cocotb-coverage): Extension that enables coverage and constrained random verification\n* Publication in IEEE [Paper](https://ieeexplore.ieee.org/document/7566600)\n* [python-uvm](https://github.com/tpoikela/uvm-python): port of SystemVerilog (SV) Universal Verification Methodology (UVM) 1.2 to Python and cocotb\n\n### Hwt --  Python library for hardware development\n[hwt](https://github.com/Nic30/hwt):  one of the goals of this library is to implement some simulation features similar to UVM\n\n## Not strictly relevant resources\n* [CRAVE: An advanced constrained random verification environment for SystemC](https://ieeexplore.ieee.org/document/6376356)\n* [EnrichingUVM in SystemC with AMS extensions for randomization and functional coverage](https://www.researchgate.net/publication/267437715_Enriching_UVM_in_SystemC_with_AMS_extensions_for_randomization_and_coverage)\n* [Coverage  directed  test  generation  for  functional  verification  using  Bayesian  network](https://www.research.ibm.com/haifa/dept/_svt/papers/simulation/18_2.pdf)\n* [Introducing XCS to Coverage Directed test Generation](https://ieeexplore.ieee.org/document/6114166)\n* [LiveHD](https://github.com/masc-ucsc/livehd) LiveHD is an infrastructure designed for Live Hardware Development. By live, we mean that small changes in the design should have the synthesis and simulation results in a few seconds, as the fast interactive systems usually response in sub-second.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchiselverify%2Fchiselverify","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fchiselverify%2Fchiselverify","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchiselverify%2Fchiselverify/lists"}