{"id":16659275,"url":"https://github.com/dundalek/stratify","last_synced_at":"2025-04-10T02:29:22.477Z","repository":{"id":250742995,"uuid":"833118643","full_name":"dundalek/stratify","owner":"dundalek","description":"Explore and improve architecture of software, visualize structure and dependencies of codebases, calculate code metrics","archived":false,"fork":false,"pushed_at":"2025-04-02T16:10:41.000Z","size":1785,"stargazers_count":75,"open_issues_count":0,"forks_count":2,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-04-03T00:08:16.966Z","etag":null,"topics":["call-graph","clj-kondo","code-metrics","code-visualization","dependency-graph","graph","graphviz","software-architecture","visualization"],"latest_commit_sha":null,"homepage":"","language":"Clojure","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dundalek.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-07-24T11:45:36.000Z","updated_at":"2025-03-18T08:50:16.000Z","dependencies_parsed_at":"2024-08-13T15:11:55.922Z","dependency_job_id":"6bcc30c1-8aed-4823-9bd6-5332c7c4f9a3","html_url":"https://github.com/dundalek/stratify","commit_stats":null,"previous_names":["dundalek/stratify"],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dundalek%2Fstratify","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dundalek%2Fstratify/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dundalek%2Fstratify/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dundalek%2Fstratify/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dundalek","download_url":"https://codeload.github.com/dundalek/stratify/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248144023,"owners_count":21054865,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["call-graph","clj-kondo","code-metrics","code-visualization","dependency-graph","graph","graphviz","software-architecture","visualization"],"created_at":"2024-10-12T10:24:30.841Z","updated_at":"2025-04-10T02:29:22.452Z","avatar_url":"https://github.com/dundalek.png","language":"Clojure","readme":"# Stratify\n\nStratify is a tool for exploring and improving architecture of software.\nDiscover bits of Stratified Design that are hiding in your code.\nGain big picture understanding to make better decisions how to grow your system.\n\nFeatures and sources:\n\n- Code maps - Visualize structure and dependencies of codebases, supports following sources:\n  - [Clojure/Script](#usage) code\n  - [Graphviz](#graphviz-visualization) - Interactive visualization of outputs produced by other tools  \n    (e.g. Go, JavaSript/TypeScript dependencies or others)\n  - [Architecture maps](#architecture-maps) - Explore C4 models\n  - [Infrastructure maps](#infrastructure-maps) - Infrastructure-as-Code (IaC) using Pulumi or SST\n- [Metrics reports](#metrics-reports) -  Calculate code metrics and generate visual reports\n- [Architecture checks](#architecture-checks) - Enforce architectural constrains, dependency rules, layer violations\n\nVisualization renderers:\n- [DGML](#dgml-renderer) - For visualization it leverages the [code map](https://learn.microsoft.com/en-us/visualstudio/modeling/browse-and-rearrange-code-maps?view=vs-2022) tool from Visual Studio,\nwhich is designed for hierarchical graphs,\nand allows to interactively collapse or expand the amount of shown information.\n- [3D Code City](#3d-code-city-renderer) - Outputs data for use with [CodeCharta](https://codecharta.com) tool to visualize codebase and metrics in 3D view.\n- [3D Code Galaxy](#3d-galaxy-renderer) - Outputs data for use with the [Dep-Tree](https://github.com/gabotechs/dep-tree) visualizer that uses 3D force layout view.\n\n### DGML Renderer\n\nThis is an advantage over static graph rendering tools like Graphviz \nwhich only work for trivial sized graphs,\nbecause for a size of systems encountered in practice it becomes a tangle of lines.\nThat is overwhelming and does not aid much in understanding the structure.\n\n| | |\n| - | - |\n| ![stratify-asami](doc/img/stratify-asami.png) | ![stratify-babashka](doc/img/stratify-babashka.png) |\n| ![stratify-promesa](doc/img/stratify-promesa.png) | ![stratify-jepsen](doc/img/stratify-jepsen.png) |\n\nThe code map tool in Visual Studio uses [DGML](#about-dgml),\nwhich is an XML-based format for representing directed graphs.\nVisualizing a codebase is a two step process:\n1. First, this tool reads Clojure code and outputs a DGML graph.\n2. Then the graph is loaded and visualized using the DGML Editor in Visual Studio.\n\n### 3D Code City Renderer\n\nAdditionally, Clojure code can be extracted to [CodeCharta](https://codecharta.com/) format to visualize it as 3D Code City. In this view code metrics can be mapped to visualization to uncover hotspots or areas that need attention.\n[See details](#3d-code-city).\n\n![Logseq codebase visualized using CodeCharta](doc/img/codecharta-logseq-cropped.avif)\n\n### 3D Galaxy Renderer\n\nThere is also support for the [Dep-Tree](https://github.com/gabotechs/dep-tree) visualizer which uses 3D force layout.\nThe way the nodes are spread into clusters can be used to judge modularity of a codebase.\n[See instructions](#3d-code-galaxy).\n\n![Example visualization of InstandDB](doc/img/dep-tree-instantdb.avif)\n\n### Demos and Talks\n\nWatch the [demo video](https://www.youtube.com/watch?v=8LMrIpxxpDw) which shows several use cases:\n\n- Big picture understanding - Explore a codebase top-down to gain insights about the system structure.\n- Local understanding - Navigate and traverse the call graph to learn details about implementation.\n- Refactoring simulation - Improve structure of a codebase by previewing results of refactoring.\n\n[![Stratify Demo](doc/img/demo-thumbnail.png)](https://www.youtube.com/watch?v=8LMrIpxxpDw)\n\nWatch the [London Clojurians talk](https://www.youtube.com/watch?v=olTNZeKpc2M) that goes into depth and also discusses software architecture in general:\n\n[![Stratify Clojurians talk](doc/img/london-clj-talk-thumbnail.avif)](https://www.youtube.com/watch?v=olTNZeKpc2M)\n\n## Usage\n\nFirst extract DGML graph from source code.\n\n#### Use without installing\n\n```\nclojure -Sdeps '{:deps{io.github.dundalek/stratify{:git/tag\"v0.4.0\":git/sha\"48726c2\"}}}' \\\n        -M -m stratify.main\n```\n\n#### Install by adding alias\n\n`~/.clojure/deps.edn` to `:aliases` section\n\n```clojure\n{:aliases\n {:stratify\n  {:extra-deps {io.github.dundalek/stratify {:git/tag \"v0.4.0\" :git/sha \"48726c2\"}}\n   :main-opts [\"-m\" \"stratify.main\"]\n```\n\nThen run:\n```\nclojure -M:stratify path/to/src -o graph.dgml\n```\n\n#### Troubleshooting\n\nStratify needs Clojure 1.12.\nIn case you get an error `Could not locate clojure/repl/deps__init.class`:\n- Make sure to update Clojure CLI tools to latest version (`clojure --version` should print `Clojure CLI version 1.12.0.1479` or later).\n- Alternatively add `org.clojure/clojure {:mvn/version \"1.12.0\"}`  to `:deps` map explicitly.\n\n\u003cdetails\u003e\n\u003csummary\u003eDetails\u003c/summary\u003e\n\nFull error message:\n\n```\nExecution error (FileNotFoundException) at stratify.main/eval138$loading (main.clj:1).\nCould not locate clojure/repl/deps__init.class, clojure/repl/deps.clj or clojure/repl/deps.cljc on classpath.\n```\n\n`~/.clojure/deps.edn` `:aliases` section with added Clojure 1.12:\n\n```clojure\n{:aliases\n {:stratify\n  {:extra-deps {io.github.dundalek/stratify {:git/tag \"v0.4.0\" :git/sha \"48726c2\"}\n                org.clojure/clojure {:mvn/version \"1.12.0\"}}\n   :main-opts [\"-m\" \"stratify.main\"]\n```\n\n\u003c/details\u003e\n\n#### Options\n\n```\nUsage: stratify \u003coptions\u003e \u003csrc-paths\u003e\n\nOptions:\n      --include-dependencies                Include links to library dependencies\n      --insert-namespace-node \u003clabel\u003e       Group vars mixed among namespaces under a node with a given label\n      --flat-namespaces                     Render flat namespaces instead of a nested hierarchy\n      --coverage-file         \u003cfile\u003e        Include line coverage metric from given Codecov file\n  -o, --out                   \u003cfile\u003e   -    Output file, default \"-\" standard output\n  -f, --from                  \u003cformat\u003e clj  Source format, choices: \"clj\", \"dot\", \"overarch\", \"pulumi\"\n  -h, --help                                Print this help message and exit\n      --metrics                             Calculate and serve namespace metrics report\n  -t, --to                    \u003cformat\u003e dgml Target format, choices: \"codecharta\", \"dgml\"\n```\n\n### Using Visual Studio DGML Editor\n\nOnce you extracted the graph use [Visual Studio](https://visualstudio.microsoft.com/) to visualize it.\n\nA downside is that Visual Studio is Windows-only, but it can be run in a Virtual Machine (VM) and there are VM images provided for developers.\nIt is [sufficient](https://learn.microsoft.com/en-us/visualstudio/modeling/analyze-and-model-your-architecture?view=vs-2022#VersionSupport) to use the free Community edition.\n\n- Run Visual Studio in VM (optional)\n  - [VM images](https://developer.microsoft.com/en-us/windows/downloads/virtual-machines/) for developers in various formats, e.g. for VirtualBox\n  - Visual Studio 2022 Community edition is pre-installed\n- Enable DGML Editor\n  - menu Tools -\u003e Get Tools and Features (opens Visual Studio Installer) -\u003e Individual Components\n    - check DGML Editor\n- Install [DgmlPowerTools 2022](https://marketplace.visualstudio.com/items?itemName=ChrisLovett.DgmlPowerTools2022) extension ([source](https://github.com/clovett/DgmlPowerTools), optional)\n  - provides extra features like neighborhood and butterfly exploration modes\n  - menu Extensions -\u003e Manage Extensions\n\n### 3D Code City\n\n1) Install dependencies\n\n\nInstall [CodeCharta CLI](https://codecharta.com/docs/overview/getting-started#installation) with  `npm i -g codecharta-analysis` to get `ccsh` command.\n\nInstall [tokei](https://github.com/XAMPPRocky/tokei?tab=readme-ov-file#installation) (optional to get line and comment counts).\n\n2) Extract with CLI\n\nRun the extraction command, it creates `output-prefix.cc.json.gz` file.\n````\nclojure -M:stratify -t codecharta -o output-prefix src\n````\n\nAdditionally use `--coverage-file codecov.json` option to include [code coverage](#code-coverage) metrics.\n\n\n3) Visualize  \n\nOpen [CodeCharta Web Studio](https://codecharta.com/visualization/app/index.html)\nand click the open button to load the `.cc.json.gz` file.\n\nSuggested metrics:\n\n- Area - representing size like `loc`, `rloc`\n- Color - representing quality like `graph_betweenness_centrality`, `line_coverage`\n- Height - representing magnitude like `number_of_commits`, `number_of_authors`  \n  (If a source has bad quality metric, the problem is magnified by its height.)\n\n### 3D Code Galaxy\n\n1) Run the extraction command:\n\n````\nclojure -M:stratify -t dep-tree -o output.json src\n````\n\n2) Open the [Dep-Tree visualizer](https://dep-tree.pages.dev/) and load the file to visualize.\n\n### Code coverage\n\nStratify can load test coverage using [Codecov](https://docs.codecov.com/docs/codecov-custom-coverage-format) format.\n\nFor Clojure use [Kaocha](https://github.com/lambdaisland/kaocha) test runner with\n[kaocha-cloverage](https://github.com/lambdaisland/kaocha-cloverage) plugin to collect coverage info.\n\nEnable the `codecov?` option in `tests.edn` to output the codecov file:\n\n```clojure\n#kaocha/v1\n{:plugins [kaocha.plugin/cloverage]\n :cloverage/opts\n {:codecov? true}}\n```\n\nThen use the `--coverage-file` option, nodes in the graph will be colored according to their coverage value.\n```\nclojure -M:stratify --coverage-file target/coverage/codecov.json -o graph.dgml src\n```\n\n![Example graph with code coverage visualization](doc/img/code-coverage-dgml-stratify.avif)\n\nVisualizing code coverage on a directed graph can help to discover a problem when lower layers are poorly tested. \nAs a rule of thumb strive for \"greener\" lower layers, since poor quality on lower layers compounds impact on upper layers.\n\n### Graphviz visualization\n\nTo convert Graphviz `.dot` format to DGML pass the `-f dot` option:\n\n```\nclojure -M:stratify graph.dot -f dot -o graph.dgml\n```\n\nBy default nested hierarchy is created based on segments using `/` as separator.  \nPass the `--flat-namespaces` option for flat nodes without nesting.\n\n##### Extract Go dependencies as Graphviz dot file\n\nUse [goda](https://github.com/loov/goda) to extract dependency graph,\nwhich is then converted to DGML.\n\n```\ngo install github.com/loov/goda@latest\n```\n\nFrom within a Go project:\n\n```\ngoda graph \"./...\" \u003e graph.dot\n```\n\nOr to also include dependencies:\n\n```\ngoda graph \"./:all\" \u003e graph-all.dot\n```\n\nCompare the legibility of a typical graphviz dependency graph to the same graph displayed as hierarchical on the right.\nNote that groups can be further collapsed using the interactive viewer.\n\n![Graphviz vs. layered graph of Lazygit codebase extracted using Goda](doc/img/lazygit-graphviz-vs-layered.avif)\n\n##### JavaScript/TypeScript dependencies visualization\n\nUse [Dependency cruiser](https://github.com/sverweij/dependency-cruiser) to extract JS/TS dependencies as Graphviz dot file:\n\n```\nbunx depcruise src --include-only \"^src\" --output-type dot \u003e graph.dot\n```\n\nTo also include dependencies:\n\n```\nbunx depcruise src --output-type dot \u003e graph.dot\n```\n\n### Architecture maps\n\nView [C4](https://c4model.com) architecture models expressed in [Overarch](https://github.com/soulspace-org/overarch) format.\n\nWhen a model becomes large it can end up overwhelming.\nOverarch lets you choose upfront which parts of your model to render as static diagrams.\nHowever, it can be useful to see the entire model at once,\nexplore it interactively, and drill down to areas of interest as needed.\n\nTo convert an architecture model to DGML use the `-f overarch` option and pass the model directory:\n\n```\nclojure -M:stratify models/banking -f overarch -o banking.dgml\n```\n\nHere is a rendering of the example [banking model](https://github.com/soulspace-org/overarch/blob/0551900472757ca1cf5973f5e598da534d49367e/models/banking/model.edn):\n\n![Overarch Banking model](doc/img/overarch-banking.png)\n\n### Infrastructure maps\n\n[Pulumi](https://www.pulumi.com/) includes builtin graph visualization `pulumi stack graph`.\nIt suffers the same illegibility problem like other solutions based on [Graphviz](#graphviz-visualization).\nStratify can be used an alternative to visualize infrastructure stacks with collapsible levels of detail.\n\n\nExport stack state to JSON:\n```\npulumi stack export --json \u003e state.json\n```\n\nYou can also export preview, useful for visualizing the stack first without deploying:\n```\npulumi preview --show-sames --json \u003e state-preview.json\n```\n\nThen transform the stack state to DGML graph:\n\n```\nclojure -M:stratify -f pulumi -o graph.dgml state.json\n```\n\nSince [SST](https://sst.dev/) internally uses Pulumi, it is possible to also visualize SST stacks.\nRun `sst diagnostic --stage your-stage` to generate `report.zip` which includes `state.json` that can be visualized.\n\nExample visualization of the AWS-based [Voting App](https://github.com/pulumi/examples/blob/master/aws-ts-pern-voting-app/index.ts) example:\n\n![Voting App example infrastructure visualization](doc/img/pulumi-aws-ts-pern-voting-app.avif)\n\n### Metrics reports\n\nUse the  `--metrics` option to calculate code metrics for given source paths and generate a report.\n[Clerk](https://github.com/nextjournal/clerk) is used to start a local web server which renders a notebook as a web page.\nMetrics and charts can be adapted by customizing the [notebook.clj](resources/io/github/dundalek/stratify/notebook.clj).\n\n```\nclojure -M:stratify src --metrics\n```\n\nUse the `-o` / `--out` to generate the report as a HTML file.\nIt can be useful to run periodically on CI and upload the HTML report to a static hosting server.\n\n```\nclojure -M:stratify src --metrics -o report.html\n```\n\n| | |\n| - | - |\n| ![Metrics](doc/img/metrics-table.png) | ![Charts](doc/img/metrics-chart-degrees.png) |\n\n#### Metrics delta\n\nThere is a way to compare metrics between two versions to see how a system changed over time.\n\n1) Extract sources to DGML at different checkouts:\n\n```\ngit checkout commit-a\nclojure -M:stratify -o a.dgml src\n\ngit checkout commit-b\nclojure -M:stratify -o b.dgml src\n```\n\n\n2) Then open the report with:\n\n```\nclojure -M:stratify --metrics-delta a.dgml b.dgml\n```\n\n| | |\n| - | - |\n| ![System metrics diff](doc/img/metrics-delta-top.avif) | ![Element metrics diff](doc/img/metrics-delta-bottom.avif) |\n\n### Architecture checks\n\nThe goal is to be able to define rules for code like architectural constraints, dependency rules, and layer violations.\nIt is inspired by [ArchUnit](https://www.archunit.org) with a difference of using graph queries ([Datalog](https://www.learndatalogtoday.org)) aiming to be mostly programming language agnostic and only needing thin adapters.\n\nCurrently, the feature is not ready yet.\nThere is a work-in-progress namespace [queries.clj](src/io/github/dundalek/stratify/queries.clj) used for experiments in the REPL to demonstrate the approach.\nThe result from code analysis is loaded into in-memory [DataScript](https://github.com/tonsky/datascript) database and queries to check code rules run against it.\nFuture work will be to try to express various rules, identify common patterns, and create more concise helpers.\n\n## About DGML\n\n[DGML](https://en.wikipedia.org/wiki/DGML) stands for Directed Graph Markup Language \n\n- Watch [Overview Video](https://www.youtube.com/watch?v=wIjCdOrZj-I) of the features and how to use the editor showcasing [examples](https://github.com/clovett/dgml).\n- For more details see [Reference](https://learn.microsoft.com/en-us/visualstudio/modeling/directed-graph-markup-language-dgml-reference?view=vs-2022) and [XSD Schema](https://schemas.microsoft.com/vs/2009/dgml/).\n\nAvailable renderers:\n\n- DGML editor in Visual Studio 2022, Windows-only (recommended)\n- [DGMLViewer](https://marketplace.visualstudio.com/items?itemName=coderAllan.vscode-dgmlviewer) plugin for Visual Studio Code, cross-platform\n  - only viewer, no editing\n  - does not seem to work very well, many examples cannot be opened\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdundalek%2Fstratify","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdundalek%2Fstratify","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdundalek%2Fstratify/lists"}