{"id":13581230,"url":"https://github.com/s0rg/decompose","last_synced_at":"2025-04-07T06:04:45.262Z","repository":{"id":188766830,"uuid":"679395781","full_name":"s0rg/decompose","owner":"s0rg","description":"Reverse-engineering tool for docker environments","archived":false,"fork":false,"pushed_at":"2024-12-13T20:21:53.000Z","size":298,"stargazers_count":99,"open_issues_count":3,"forks_count":5,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-03-31T05:02:12.900Z","etag":null,"topics":["cli","developer-tools","diagrams","docker","docker-compose","dotviz","golang-application","reverse-engineering","software-architecture","structurizr-dsl"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/s0rg.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-08-16T18:34:23.000Z","updated_at":"2025-03-19T13:01:48.000Z","dependencies_parsed_at":"2023-11-27T00:27:19.201Z","dependency_job_id":"3bbb3e09-7773-494b-ac3f-67f64430050a","html_url":"https://github.com/s0rg/decompose","commit_stats":{"total_commits":90,"total_committers":3,"mean_commits":30.0,"dds":"0.033333333333333326","last_synced_commit":"2dda43676b855f36bc41ac22cc6f9b4c4baf804f"},"previous_names":["s0rg/decompose"],"tags_count":59,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/s0rg%2Fdecompose","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/s0rg%2Fdecompose/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/s0rg%2Fdecompose/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/s0rg%2Fdecompose/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/s0rg","download_url":"https://codeload.github.com/s0rg/decompose/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247601447,"owners_count":20964864,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cli","developer-tools","diagrams","docker","docker-compose","dotviz","golang-application","reverse-engineering","software-architecture","structurizr-dsl"],"created_at":"2024-08-01T15:01:59.386Z","updated_at":"2025-04-07T06:04:45.244Z","avatar_url":"https://github.com/s0rg.png","language":"Go","readme":"[![License](https://img.shields.io/badge/license-MIT%20License-blue.svg)](https://github.com/s0rg/decompose/blob/master/LICENSE)\n[![FOSSA Status](https://app.fossa.com/api/projects/git%2Bgithub.com%2Fs0rg%2Fdecompose.svg?type=shield)](https://app.fossa.com/projects/git%2Bgithub.com%2Fs0rg%2Fdecompose?ref=badge_shield)\n[![Go Version](https://img.shields.io/github/go-mod/go-version/s0rg/decompose)](go.mod)\n[![Release](https://img.shields.io/github/v/release/s0rg/decompose)](https://github.com/s0rg/decompose/releases/latest)\n[![Mentioned in Awesome Go](https://awesome.re/mentioned-badge.svg)](https://github.com/avelino/awesome-go)\n![Downloads](https://img.shields.io/github/downloads/s0rg/decompose/total.svg)\n\n[![CI](https://github.com/s0rg/decompose/workflows/ci/badge.svg)](https://github.com/s0rg/decompose/actions?query=workflow%3Aci)\n[![Go Report Card](https://goreportcard.com/badge/github.com/s0rg/decompose)](https://goreportcard.com/report/github.com/s0rg/decompose)\n[![Maintainability](https://api.codeclimate.com/v1/badges/1bc7c04689cf612a0f39/maintainability)](https://codeclimate.com/github/s0rg/decompose/maintainability)\n[![Test Coverage](https://api.codeclimate.com/v1/badges/1bc7c04689cf612a0f39/test_coverage)](https://codeclimate.com/github/s0rg/decompose/test_coverage)\n![Issues](https://img.shields.io/github/issues/s0rg/decompose)\n\n# decompose\n\nReverse-engineering tool for docker environments.\n\nTakes all network connections from your docker containers and exports them as:\n\n- [graphviz dot](https://www.graphviz.org/doc/info/lang.html)\n- [structurizr dsl](https://github.com/structurizr/dsl)\n- [compose yaml](https://github.com/compose-spec/compose-spec/blob/master/spec.md)\n- [plant uml](https://github.com/plantuml/plantuml)\n- pseudographical tree\n- json stream\n- statistics - nodes, connections and listen ports counts\n- CSV with columns: `name`, `listen` and `outbounds`\n\n## rationale\n\nI was in need for a tool to visualize and inspect big (more than 470 containers) dockerized legacy system without any\nschemes and having a bare minimum of documentation\n\n## analogs\n\nClosest analogs, i can find, that not suit my needs very well:\n\n- [Red5d/docker-autocompose](https://github.com/Red5d/docker-autocompose) - produces only `compose yaml`\n- [justone/dockviz](https://github.com/justone/dockviz) - produces only `dot`, links and ports are taken\n  from compose configuration (`links` and `ports` sections) directly, therefore can miss some of them\n- [LeoVerto/docker-network-graph](https://github.com/LeoVerto/docker-network-graph) - very same as above, in python\n- [weaveworks/scope](https://github.com/weaveworks/scope) - deprecated, no cli\n\n## features\n\n- os-independent, it uses different strategies to get container connections:\n  - running on **linux as root** is the fastest way and it will work with all types of containers (even `scratch`-based)\n    as it use `nsenter`\n  - running as non-root or on non-linux OS will attempt to run `netsat` inside container, if this fails\n    (i.e. for missing `netstat` binary), no connections for such container will be gathered\n- single-binary, static-compiled unix-way `cli` (all output goes to stdout, progress information to stderr)\n- produces detailed connections graph **with ports**\n- save `json` stream once and process it later in any way you want\n- all output formats are sorted, thus can be placed to any `vcs` to observe changes\n- fast, scans ~470 containers with ~4000 connections in around 5 sec\n- auto-clusterization based on graph topology\n- deep inspection mode, in wich connections between procesess inside containers, also collected and shown\n- unix-sockets connections\n- over 95% test-coverage\n\n## known limitations\n\n- only established and listen connections are listed (but script like [snapshots.sh](examples/snapshots.sh) can beat this)\n- `composer-yaml` is not intended to be working out from the box, it can lack some of crucial information (even in `-full` mode),\n  or may contains cycles between nodes (removing `links` section in services may help), its main purpose is for system overview\n- [gephi](https://github.com/gephi/gephi) fails to load edges from resulting graphviz, this can be fixed by any auto-replacement\n  tool: `sed -i 's/-\u003e/ -\u003e /g' myfile.dot`\n- unix-sockets works only in root mode on linux, this process involves inode matching to find correct connections\n\n## installation\n\n- [binaries / deb / rpm](https://github.com/s0rg/decompose/releases) for Linux, FreeBSD, macOS and Windows\n- [docker image](https://hub.docker.com/r/s0rg/decompose)\n\n## usage\n\n```\ndecompose [flags]\n\n\n-cluster string\n    json file with clusterization rules, or auto:\u003csimilarity\u003e for auto-clustering, similarity is float in (0.0, 1.0] range\n-compress\n    compress graph\n-deep\n    process-based introspection\n-follow string\n    follow only this container by name(s), comma-separated or from @file\n-format string\n    output format: csv, dot, json, puml, sdsl, stat, tree, yaml (default \"json\")\n-help\n    show this help\n-load value\n    load json stream, can be used multiple times\n-local\n    skip external hosts\n-meta string\n    json file with metadata for enrichment\n-no-loops\n    remove connection loops (node to itself) from output\n-no-orphans\n    remove orphaned (not connected) nodes from output\n-out string\n    output: filename or \"-\" for stdout (default \"-\")\n-proto string\n    protocol to scan: tcp,udp,unix or all (default \"all\")\n-silent\n    suppress progress messages in stderr\n-skip-env string\n    environment variables name(s) to skip from output, case-independent, comma-separated\n-version\n    show version\n```\n\n### environment variables:\n\n- `DOCKER_HOST` - connection uri\n- `DOCKER_CERT_PATH` - directory path containing key.pem, cert.pm and ca.pem\n- `DOCKER_TLS_VERIFY` - enable client TLS verification\n- `IN_DOCKER_PROC_ROOT` - for in-docker scenario - root for host-mounted /proc\n\n## json stream format\n\n```go\ntype Item struct {\n    Name       string              `json:\"name\"` // container name\n    IsExternal bool                `json:\"is_external\"` // this host is external\n    Image      *string             `json:\"image,omitempty\"` // docker image (if any)\n    Container  struct{\n        Cmd    []string          `json:\"cmd\"`\n        Env    []string          `json:\"env\"`\n        Labels map[string]string `json:\"labels\"`\n    } `json:\"container\"` // container info\n    Listen     map[string][]{\n        Kind   string            `json:\"kind\"`  // tcp / udp / unix\n        Value  string            `json:\"value\"`\n        Local  bool              `json:\"local\"` // bound to loopback\n    } `json:\"listen\"` // ports with process names\n    Networks   []string            `json:\"networks\"` // network names\n    Tags       []string            `json:\"tags\"` // tags, if meta presents\n    Volumes    []*struct{\n        Type string `json:\"type\"`\n        Src  string `json:\"src\"`\n        Dst  string `json:\"dst\"`\n    } `json:\"volumes\"`           // volumes info, only when '-full'\n    Connected  map[string][]string `json:\"connected\"` // name -\u003e ports slice\n}\n```\n\nSingle node example with full info and metadata filled:\n\n```json\n{\n    \"name\": \"foo-1\",\n    \"is_external\": false,\n    \"image\": \"repo/foo:latest\",\n    \"container\": {\n        \"cmd\": [\n            \"foo\",\n            \"-foo-arg\"\n        ],\n        \"env\": [\n            \"FOO=1\"\n        ],\n        \"labels\": {}\n    },\n    \"listen\": {\"foo\": [\n        {\"kind\": \"tcp\", \"value\": \"80\"}\n    ]},\n    \"networks\": [\"test-net\"],\n    \"tags\": [\"some\"],\n    \"volumes\": [\n        {\n            \"type\": \"volume\",\n            \"src\": \"/var/lib/docker/volumes/foo_1/_data\",\n            \"dst\": \"/data\"\n        },\n        {\n            \"type\": \"bind\",\n            \"src\": \"/path/to/foo.conf\",\n            \"dst\": \"/etc/foo.conf\"\n        }\n    ],\n    \"connected\": {\n        \"bar-1\": [\n            {\"src\": \"foo\", \"dst\": \"[remote]\", \"port\": {\"kind\": \"tcp\", \"value\": \"443\"}}\n        ]\n    }\n}\n```\n\nSee [stream.json](examples/stream.json) for simple stream example.\n\n## metadata format\n\nTo enrich output with detailed descriptions, you can provide additional `json` file, with metadata i.e.:\n\n```json\n{\n    \"foo\": {\n        \"info\": \"info for foo\",\n        \"docs\": \"https://acme.corp/docs/foo\",\n        \"repo\": \"https://git.acme.corp/foo\",\n        \"tags\": [\"some\"]\n    },\n    \"bar\": {\n        \"info\": \"info for bar\",\n        \"tags\": [\"other\", \"not-foo\"]\n    }\n}\n```\n\nUsing this file `decompose` can enrich output with info and additional tags, for every container that match by name with\none of provided keys, like `foo-1` or `bar1` for this example.\n\nSee [csv2meta.py](examples/csv2meta.py) for example how to create such `json` fom csv, and\n[meta.json](examples/meta.json) for metadata sample.\n\n## clusterization\n\n### with rules\n\nYou can join your services into `clusters` by flexible rules, in `dot`, `structurizr` and `stat` output formats.\nExample `json` (order matters):\n\n```json\n[\n    {\n        \"name\": \"cluster-name\",\n        \"weight\": 1,\n        \"if\": \"\u003cexpression\u003e\"\n    },\n    ...\n]\n```\n\nWeight can be omitted, if not specified it equals `1`.\n\nWhere `\u003cexpression\u003e` is [expr dsl](https://expr-lang.org/docs/Language-Definition), having env object `node` with follownig\nfields:\n\n```go\ntype Node struct {\n    Listen     PortMatcher  // port matcher with two methods: `HasAny(...string) bool` and `Has(...string) bool`\n    Name       string       // container name\n    Image      string       // container image\n    Cmd        string       // container cmd\n    Args       []string     // container args\n    Tags       []string     // tags, if meta present\n    IsExternal bool         // external flag\n}\n```\n\nSee: [cluster.json](examples/cluster.json) for detailed example.\n\n### automatic\n\nDecompose provides automatic clusterization option, use `-cluster auto:\u003csimilarity\u003e` to try it out, `similarity` is\na float in `(0.0, 1.0]` range, representing how much similar ports nodes must have to be placed in same cluster\n(`1.0` - must have all ports equal).\n\n## examples\n\nSave full json stream:\n\n```shell\nsudo decompose \u003e nodes-1.json\n```\n\nGet `dot` file:\n\n```shell\ndecompose -format dot \u003e connections.dot\n```\n\nGet tcp and udp connections as `dot`:\n\n```shell\ndecompose -proto tcp,udp -format dot \u003e tcp.dot\n```\n\nMerge graphs from json streams, filter by protocol, skip remote hosts and save as `dot`:\n\n```shell\ndecompose -local -proto tcp -load \"nodes-*.json\" -format dot \u003e graph-merged.dot\n```\n\nLoad json stream, enrich and save as `structurizr dsl`:\n\n```shell\ndecompose -load nodes-1.json -meta metadata.json -format sdsl \u003e workspace.dsl\n```\n\nSave auto-clustered graph, with similarity factor `0.6` as `structurizr dsl`:\n\n```shell\ndecompose -cluster auto:0.6 -format sdsl \u003e workspace.dsl\n```\n\n## example result\n\nScheme taken from [redis-cluster](https://github.com/s0rg/redis-cluster-compose):\n\n![svg](https://github.com/s0rg/redis-cluster-compose/blob/main/redis-cluster.svg) *it may be too heavy to display it with\nbrowser, use `save image as` and open it locally*\n\nSteps to reproduce:\n\n```shell\ngit clone https://github.com/s0rg/redis-cluster-compose.git\ncd redis-cluster-compose\ndocker compose up -d\n```\n\nthen:\n\n```shell\ndecompose -format dot | dot -Tsvg \u003e redis-cluster.svg\n```\n\n## license\n\n[![FOSSA Status](https://app.fossa.com/api/projects/git%2Bgithub.com%2Fs0rg%2Fdecompose.svg?type=large)](https://app.fossa.com/projects/git%2Bgithub.com%2Fs0rg%2Fdecompose?ref=badge_large)\n","funding_links":[],"categories":["Software Packages","Go","软件包","Development","Go Tools","\u003ca name=\"vm\"\u003e\u003c/a\u003eContainerization and virtualization","Container Operations"],"sub_categories":["DevOps Tools","DevOps 工具","Docker","User Interface"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fs0rg%2Fdecompose","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fs0rg%2Fdecompose","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fs0rg%2Fdecompose/lists"}