{"id":20782552,"url":"https://github.com/toktok/hs-tokstyle","last_synced_at":"2025-12-11T23:23:08.007Z","repository":{"id":38380228,"uuid":"68841716","full_name":"TokTok/hs-tokstyle","owner":"TokTok","description":"Style checker for TokTok C projects","archived":false,"fork":false,"pushed_at":"2025-10-05T20:29:10.000Z","size":622,"stargazers_count":3,"open_issues_count":4,"forks_count":5,"subscribers_count":22,"default_branch":"master","last_synced_at":"2025-10-05T21:26:35.083Z","etag":null,"topics":["c","linter","style"],"latest_commit_sha":null,"homepage":"https://toktok.ltd/","language":"Haskell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/TokTok.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2016-09-21T17:32:41.000Z","updated_at":"2025-10-05T19:55:12.000Z","dependencies_parsed_at":"2023-11-08T05:48:01.472Z","dependency_job_id":"37657683-1dc1-438a-bc23-9f37bfe747bf","html_url":"https://github.com/TokTok/hs-tokstyle","commit_stats":null,"previous_names":["toktok/tokstyle"],"tags_count":8,"template":false,"template_full_name":null,"purl":"pkg:github/TokTok/hs-tokstyle","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TokTok%2Fhs-tokstyle","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TokTok%2Fhs-tokstyle/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TokTok%2Fhs-tokstyle/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TokTok%2Fhs-tokstyle/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/TokTok","download_url":"https://codeload.github.com/TokTok/hs-tokstyle/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TokTok%2Fhs-tokstyle/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":278877097,"owners_count":26061380,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-07T02:00:06.786Z","response_time":59,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["c","linter","style"],"created_at":"2024-11-17T14:12:35.046Z","updated_at":"2025-10-08T01:39:22.917Z","avatar_url":"https://github.com/TokTok.png","language":"Haskell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Tokstyle\n\nC style checker for TokTok projects.\n\nThis project uses [cimple](https://hackage.haskell.org/package/cimple), a highly restrictive\nsimplified C dialect used in [c-toxcore](https://github.com/TokTok/c-toxcore). Tokstyle builds on\ntop of an already restrictive grammar to do additional semantic checks.\n\n## Using the tool\n\n### Using the docker image\n\nUsing the built docker image (only contains binaries, so you need an OS around it):\n\n```dockerfile\nFROM toxchat/haskell:hs-tokstyle AS tokstyle\nFROM ubuntu:22.04\n\nCOPY --from=tokstyle /bin/check-cimple /bin/\nWORKDIR /work\nENTRYPOINT [\"/bin/check-cimple\"]\n```\n\nThen, build and run:\n\n```sh-session\n$ docker build -t check-cimple .\n$ docker run --rm -v \"$PWD:/work\" check-cimple myfile.c\n```\n\n### Building from source (cabal)\n\n```sh-session\n$ cabal install tokstyle\n$ check-cimple myfile.c\n```\n\n### Building from source (bazel)\n\n```sh-session\n$ bazel run //hs-tokstyle/tools:check-cimple -- $PWD/myfile.c\n```\n\n## How to write a new linter\n\nIf you want to just get started by copy/pasting an existing one, have a look at one of the many\nexisting linters in `src/Tokstyle/Linter/`. A simple example is `LoggerCalls.hs`. The tutorial below\nwalks you through the implementation of another existing linter, which happens to be the simplest\nlinter we currently have.\n\nRoughly speaking, a linter is a function from `(file path, ast)` to a list of diagnostics\nrepresented simply as `Text`. Typically, we call the linter function `analyse`.\n\n```hs\nanalyse :: (FilePath, [Node (Lexeme Text)]) -\u003e [Text]\n```\n\nMost linters will be using the `TraverseAst` framework which simplifies tree traversals so you can\nmatch over the entire tree, recursively, with just a few lines of code. You don't have to do this,\nbut bear in mind that you will need to manually recurse into top level objects like `PreprocIfdef`\nto even get at all the top level declarations.\n\n`TraverseAst` provides a traversal object called `AstActions` with functions for all kinds of\nelements you might find in the AST. The most commonly used one is `doNode`, which operates on a\nsingle AST node, but if you want to override other functions like `doFile` to handle an entire file\nat once, or `doLexeme` if you only want to inspect the tokens, those can be changed in the\ndefinition of your linter. Each of the `do` functions also has a plural form, e.g. `doNodes`, which\nas a default is simply iterating over the list, but can be used to operate on the list as a whole.\n\nLet's get started with a simple linter: `CompoundInit`, which checks that we don't write the\nfollowing unnecessarily verbose code:\n\n```c\nMyType foo = (MyType){0};\n```\n\nwhen instead, we should simply be writing\n\n```c\nMyType foo = {0};\n```\n\n### Writing the test\n\nWe'll start by writing the test first, in a classical Test-Driven Development style. Create a new\nfile called `test/Tokstyle/Linter/CompoundInitSpec.hs`.\n\n```hs\n{-# LANGUAGE OverloadedStrings #-}\nmodule Tokstyle.Linter.CompoundInitSpec (spec) where\n\nimport           Test.Hspec          (Spec, it, shouldBe)\n\nimport           Tokstyle.Linter     (analyse)\nimport           Tokstyle.LinterSpec (mustParse)\n\nspec :: Spec\nspec = do\n```\n\n- First, some boilerplate: we need `OverloadedStrings` because we'll use string literals for code\n  and expected diagnostics (which are `Text` instead of `String`).\n- We name our module according to what we'll be calling our linter module, suffixed with `Spec`.\n- Then we import a few `Hspec` functions used in the spec below.\n- Then, we need the `analyse` function which dispatches over all linters (we'll add our linter to\n  it, later).\n- Our last import is the `mustParse` function from `LinterSpec`, which fails the test when parsing\n  fails, so we don't need to deal with parse errors. We don't expect parse errors, because we write\n  the C code ourselves and it should be parseable.\n- Finally, the start of our executable specification: `spec`, which is automatically called by\n  `hspec-discover` and hooked up to the test runner.\n\nNow, for our first test, we write one with a piece of code containing the pattern we want to detect.\nIn this case, a function containing the above variable declaration. We write this in the `Spec`\nmonad (using `it` and `shouldBe`).\n\n```hs\n    it \"detects compound literal initialisers\" $ do\n        ast \u003c- mustParse\n            [ \"void f(void) {\"\n            , \"  Foo foo = (Foo){0};\"\n            , \"}\"\n            ]\n        analyse [\"compound-init\"] (\"test.c\", ast)\n            `shouldBe`\n            [ \"test.c:2: don't use compound literals in initialisations; use simple `Type var = {0};` [-Wcompound-init]\"\n            ]\n```\n\n- First, we parse a piece of code given as `[Text]` into an AST. If that fails, the test stops.\n- Otherwise, we continue by calling `analyse` with only the linter we want to test\n  (`\"compound-init\"`) in the enabled linter list. We pass it the `(file path, ast)` tuple. For most\n  linters, the file path won't matter, but some (like the logger related ones) may want to exempt\n  some files. This way, you can test that the exemption works.\n- Finally, we check that the linter gives the diagnostic we expect. We can leave this empty if we\n  don't know what the diagnostic will be, yet, or put an empty string in it to make sure it fails\n  (Test-Driven Development: we start with a failing test and make it pass by writing code later).\n\nTypically, we also write a negative test for the pattern we do want to allow. If the developer sees\nthe above error, they fix it, and then the linter should be silent.\n\n```hs\n    it \"accepts aggregate initialisers\" $ do\n        ast \u003c- mustParse\n            [ \"void f(void) {\"\n            , \"  Foo foo = {0};\"\n            , \"}\"\n            ]\n        analyse [\"compound-init\"] (\"test.c\", ast)\n            `shouldBe` []\n```\n\n### Registering the linter\n\nNow we have our test, which will fail if we run it (using `bazel` below, but you can run it with\n`stack` or build and run with `cabal` as well):\n\n```sh-session\n$ bazel run //hs-tokstyle:testsuite -- --match \"/Tokstyle.Linter.CompoundInit/\"\n...\n  hs-tokstyle/test/Tokstyle/Linter/CompoundInitSpec.hs:18:9:\n  1) Tokstyle.Linter.CompoundInit detects compound literal initialisers\n       expected: [\"test.c:2: don't use compound literals in initialisations; use simple `Type var = {0};` [-Wcompound-init]\"]\n        but got: []\n```\n\nIt fails because there is no linter for `-Wcompound-init`, so we'll write one:\n\n```hs\n{-# LANGUAGE OverloadedStrings #-}\nmodule Tokstyle.Linter.CompoundInit (analyse) where\n\nimport           Data.Text       (Text)\nimport           Language.Cimple (Lexeme (..), Node)\n\nanalyse :: (FilePath, [Node (Lexeme Text)]) -\u003e [Text]\nanalyse _ = []\n```\n\nThis linter doesn't do anything yet, but we'll register it with the linter list anyway. Open\n`src/Tokstyle/Linter.hs`, add a line like\n\n```hs\nimport qualified Tokstyle.Linter.CompoundInit as CompoundInit\n```\n\nto the imports, and then a line like this to the `localLinters` list:\n\n```hs\n    , (\"compound-init\"      , CompoundInit.analyse     )\n```\n\nDon't worry about global linters at this point, those are a more advanced (but to be honest not\nactually more complicated) feature if you need to do whole program analysis.\n\nNow we've created our linter boilerplate and registered it, we can run the test again, and it will\nstill fail. The linter exists, but doesn't return any diagnostics. You can try returning one in the\n`= []` part, e.g. saying `= [\"test.c: oh no!\"]`, and now both tests will fail: one has the wrong\ndiagnostic and the other has a diagnostic when it expects none. Let's write it to actually do\nsomething useful now.\n\n### Traversing the AST\n\nWe'll be using the `TraverseAst` framework here, so we'll need some more imports:\n\n```hs\nimport           Language.Cimple.TraverseAst (AstActions, astActions, doNode,\n                                              traverseAst)\n```\n\nWe also need the AST type constructors to actually perform pattern matching. The Cimple AST uses\nfixpoint types, so we'll need `Fix` and the `NodeF` functor constructors (don't worry about what\nthat means, TL;DR: all the AST node layers have an extra `Fix` in between, useful for recursion\nstrategies, but we're not using those here).\n\n```hs\nimport           Data.Fix                    (Fix (..))\nimport           Language.Cimple             (Lexeme (..), Node, NodeF (..))\n```\n\nAnd finally, we will be using the `Diagnostics.warn` helper function which takes care of formatting\ndiagnostics and adding them to the `State` we'll be using:\n\n```hs\nimport           Control.Monad.State.Strict  (State)\nimport qualified Control.Monad.State.Strict  as State\nimport           Language.Cimple.Diagnostics (warn)\n```\n\nNext, our actual code, starting with a no-op traversal. The default for `astActions` is to simply\ntraverse the entire tree and do nothing. `State [Text]` is the monad each of the `do` actions runs\nin. This can be anything, and more advanced linters may choose to use a record type to contain more\nthan just diagnostics, but for now it's just a list of `Text`.\n\n```hs\nlinter :: AstActions (State [Text]) Text\nlinter = astActions\n```\n\nThe `analyse` function now looks like this:\n\n```hs\nanalyse :: (FilePath, [Node (Lexeme Text)]) -\u003e [Text]\nanalyse = reverse . flip State.execState [] . traverseAst linter\n```\n\nIt creates the linter monad using `traverseAst linter` and then runs it with `State.execState` and\nthe initial state being the empty list `[]` (i.e. no diagnostics). Since `warn` adds diagnostics in\nreverse (because prepending to a linked list is `O(1)` while appending is `O(n)`), we need to\nreverse the final list when returning it.\n\nRunning the test now will still fail, because our linter only does a traversal but doesn't actually\ncheck anything.\n\n### Pattern matching and emitting diagnostics\n\nWe need to extend the `astActions` by overriding some of its functions that by default do nothing:\n\n```hs\nlinter :: AstActions (State [Text]) Text\nlinter = astActions\n    { doNode = \\file node act -\u003e\n        case unFix node of\n            _ -\u003e act\n    }\n```\n\nAs you can see, the `doNode` function gets the current file path being processed, the current node\nbeing visited, and the recursive action. The default `doNode` ignores `file` and `node` and performs\nthe recursive action. So, we have just written the default `doNode` implementation. We use `unFix`\nhere to make the patterns slightly more readable (we still need all the `Fix`es for inner patterns).\n\nNow, let's have a look at what things we might be able to match on by outputting everything this\n`doNode` function sees:\n\n```hs\nimport           Debug.Trace                 (traceShowM)\n...\n        case unFix node of\n            x -\u003e do\n                traceShowM x  -- show the node we're visiting\n                act           -- still want to recurse\n```\n\nRunning the test now will, in addition to failing, print a whole bunch of Haskell expressions, each\nof which can be copy/pasted as a pattern into the `case unFix node of` we wrote (using `...` here to\nskip some uninteresting bits):\n\n```hs\nFunctionDefn Global (Fix (FunctionPrototype (Fix ...\"void\") (L ... \"f\") ...)) (Fix (CompoundStmt ...))\nFunctionPrototype (Fix ...\"void\") (L ... \"f\") ...\nTyStd (L (AlexPn 0 1 1) KwVoid \"void\")\nCompoundStmt [...]\nVarDeclStmt (Fix (VarDecl ...\"Foo\"... (L ... \"foo\") [])) (Just (Fix (CompoundLiteral ...)))\nVarDecl ...\"Foo\"...\nCompoundLiteral ...\n```\n\nOut of all of these, we want the `VarDeclStmt`. Why? Because if we match too deeply, e.g. on a\n`VarDecl`, we don't have the initialiser, or if we match the `CompoundLiteral`, we also match them\nin expressions outside variable initialisers. So next, we copy/paste the entire Haskell expression\ninto the `case`:\n\n```hs\n        case unFix node of\n            VarDeclStmt (Fix (VarDecl ...)) (Just (Fix (CompoundLiteral ...))) -\u003e do\n                traceShowM node\n                -- don't recurse further, there's nothing interesting inside, so no \"act\" here.\n\n            _ -\u003e act  -- recurse for anything not matched\n```\n\nRunning the test again, we can now see we only output the one node we care about. This means a match\nsucceeds. You can also check the second test, the negative test which shouldn't match, to make sure\nonly the first test triggers the `traceShowM` case. You can use the following command lines to run\nspecific tests:\n\n```sh-session\n$ bazel run //hs-tokstyle:testsuite -- \\\n    --match \"/Tokstyle.Linter.CompoundInit/detects compound literal initialiser/\"\n$ bazel run //hs-tokstyle:testsuite -- \\\n    --match \"/Tokstyle.Linter.CompoundInit/accepts aggregate initialisers/\"\n```\n\nThe huge expression we just copied in (simplified above with `...`, which isn't actually valid\nHaskell code, just done to shorten the example here) only exactly matches the one test we wrote. We\nshould simplify and generalise it a bit until it matches everything we want to match and nothing we\ndon't. We don't care about the type or name of the variable, so we use `_` for that part, and we\ndon't care what's inside the `CompoundLiteral`, so we use `{}` to match any `CompoundLiteral`\nregardless of its contents.\n\nFinally, we want to formulate our diagnostic message using `warn`. We pass the currently processed\nfile path, the node to use for source locations, and the diagnostic text.\n\n```hs\n        case unFix node of\n            VarDeclStmt _ (Just (Fix CompoundLiteral{})) -\u003e do\n                warn file node $ \"don't use compound literals in initialisations; \"\n                    \u003c\u003e \"use simple `Type var = {0};`\"\n\n            _ -\u003e act  -- recurse for anything not matched\n```\n\nNow run the test, and it should pass. We're done! You have now written your first linter. The sky's\nthe limit from now on.\n\n### Appendix I: Add files to tokstyle.cabal\n\nThe above tutorial mostly assumes `bazel`, which automatically detects new files. To make a pull\nrequest or to run these with `stack` or `cabal`, you'll need to add your new module names to\n`tokstyle.cabal` in the `library` and `test-suite` sections.\n\n### Appendix II: Some useful tips\n\n- Have a look at other linters to see how they work with diagnostics. E.g. the\n  `Language.Cimple.Pretty` module can be useful for pretty printing nodes as C code.\n- As a performance optimisation, stop recursive traversals whenever you've hit your pattern. If you\n  know there are large subtrees you will never match in, write a pattern to avoid recursing into\n  them (this can save a lot of time if you, for example, only look at top level declarations and\n  don't need to inspect `FunctionDefn`s).\n- The above is also useful if you have a pattern that's always OK in certain contexts, but not in\n  others. You can skip the entire subtree for a node in which it's OK. Example: `LoggerConst.hs`.\n- Use `PatternSynonyms` if you need to match the same pattern many times (for an example, look at\n  `Booleans.hs`).\n- If you need to keep more state during your linter execution, make a `data Linter` with a\n  `HasDiagnostics` instance that tells `warn` how to add diagnostics to your data type. See\n  `DeclaredOnce.hs` for an example.\n- If you write a linter that you don't want to enable by default, add it to the `defaultFlags` list\n  in `tools/check-cimple.hs` as `-Wno-my-linter`. Users can still run it explicitly with\n  `-Wmy-linter`.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftoktok%2Fhs-tokstyle","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftoktok%2Fhs-tokstyle","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftoktok%2Fhs-tokstyle/lists"}