Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/norfairking/dekking
https://github.com/norfairking/dekking
Last synced: about 2 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/norfairking/dekking
- Owner: NorfairKing
- License: other
- Created: 2022-11-17T13:55:03.000Z (about 2 years ago)
- Default Branch: master
- Last Pushed: 2024-09-13T17:09:22.000Z (3 months ago)
- Last Synced: 2024-09-14T08:05:08.886Z (3 months ago)
- Language: Haskell
- Homepage: https://cs-syd.eu/posts/2022-12-16-announcing-dekking
- Size: 220 KB
- Stars: 16
- Watchers: 3
- Forks: 3
- Open Issues: 4
-
Metadata Files:
- Readme: README.md
- License: LICENSE.md
Awesome Lists containing this project
README
# Next-gen test coverage reports for Haskell
Dekking is a next-generation coverage report tool for Haskell.
It is implemented as a GHC plugin, as opposed to [HPC](https://hackage.haskell.org/package/hpc), which is built into GHC.
Current status: Used in Prod in all my products.
## Strategy
There are a few pieces of the puzzle.
The relevant programs are:
* `dekking-plugin`:
Modifies the parsed source file within GHC as a source-to-source
transformation plugin.
At compile-time, this plugin also outputs a `.hs.coverables` file which
contains information about which parts of the source file are coverable and
where those pieces are within the source.
The source is transformed such that, when compiled, the result will output
coverage information in `coverage.dat`.
* `ghc`: Compiles the resulting modified source code
* `dekking-report`:
Takes the `*.hs.coverables` files, and any number of `coverage.dat` files,
and produces a machine-readable `report.json` file, as well as human
readable HTML files which can be viewed in a browser.
### Source-to-source transformation
The source-to-source transformation works as follows;
We replace every expression `e` by `adaptValue "identifier for e" e`.
The identifier is generated by `dekking-plugin` at parse-time.
To give an idea of what this looks like, we would transform this
expression:
```
((a + b) * c)
```
into this expression (`f = adaptValue "identifier for e"`):
```
((f a) + (f b)) * (f c)
```
### The value adapter
The `adaptValue` function mentioned above is implemented in the very small `dekking-value` package, in the `Dekking.ValueLevelAdapter` module.
It looks something like this:
``` haskell
{-# NOINLINE adaptValue #-}
adaptValue :: String -> (forall a. a -> a)
adaptValue logStr = unsafePerformIO $ do
hPutStrLn coverageHandle logStr
hFlush coverageHandle
pure id
```
This function uses the _problem_ of `unsafePerformIO`, namely that the IO is only executed once, as a way to make sure that each expression is only marked as covered once.
### Coverables
Each coverable comes with a location, which is a triple of a line number, a
starting column and an ending column.
This location specifies where the coverable can be found in the source code.
The `*.hs.coverables` files are machine-readable JSON files.
### Coverage
The `coverage.dat` files are text files with a line-by-line description of which pieces of the source have been covered.
Each line is split up into five pieces:
```
```
For example:
```
dekking-test-0.0.0.0 Examples.Multi.A 4 1 5
```
### Strategy Overview
### Nix API
Nix support is a strong requirement of the `dekking` project.
A flake has been provided.
The default package contains the following `passthru` attributes:
* `addCoverables`: Add a `coverables` output to a Haskell package.
* `addCoverage`: Add a `coverage` output to a Haskell package.
* `addCoverablesAndCoverage`: both of the above
* `addCoverageReport`: Add a coverage `report` output to a Haskell package, similar to `doCoverage`.
* `compileCoverageReport`: Compile a coverage report (internal, you probably won't need this.)
* `makeCoverageReport`: Produce a coverage report from multiple Haskell packages.
Example usage:
``` nix
{
fuzzy-time-report = dekking.makeCoverageReport {
name = "fuzzy-time-coverage-report";
packages = [
"fuzzy-time"
"fuzzy-time-gen"
];
};
}
```
See the `e2e-test` directory for many more examples.
### Why a source-to-source transformation?
TODO
### Why is there no separate coverage for top-level bindings, patterns, or alternatives?
Only expressions are evaluated, so only expressions can be covered.
Expression coverage also shows you alternative coverage because alternatives
point to an expression.
Top-level bindings are not somehow special either.
They are a code organisation tool that need not have any impact on whether
covering them is more important.
## Why are there no controls to fail when a coverage percentage is not met?
Making automated decisions using a coverage percentage is usually a
shortsighted way to use that number.
If you really want to automate such a thing, you can use the `report.json` file
that `dekking-report` outputs.
## Some part of my code fails to compile with coverage
Because of `RankNTypes` and limitations of `ImpredicativeTypes`, sometimes the source-transformed version of a function does not type-check anymore.
(See `[ref:ThePlanTM]`, `[ref:-XImpredicativeTypes]`, and `[ref:DisablingCoverage]`.)
A common example is Servant's `hoistServerWithContext`, see [ghc ticket 22543](https://gitlab.haskell.org/ghc/ghc/-/issues/22543).
There are three ways to selectively turn off coverage:
1. With an `--exception` for the plugin: `-fplugin-opt=Dekking.Plugin:--exception=My.Module`
2. With a module-level annotation: `{-# ANN module "NOCOVER" #-}`
3. With a function-level annotation: `{-# ANN hoistServerWithContext "NOCOVER" #-}`
## Why not "just" use HPC?
* Strong nix support
* Multi-package coverage reports
* Coupling with GHC
TODO write these out