{"id":22854948,"url":"https://github.com/np-guard/cluster-topology-analyzer","last_synced_at":"2025-04-30T12:17:07.180Z","repository":{"id":38322209,"uuid":"369333280","full_name":"np-guard/cluster-topology-analyzer","owner":"np-guard","description":"A static code analyzer to generate network connection topology for micro-service applications","archived":false,"fork":false,"pushed_at":"2024-12-05T10:20:19.000Z","size":15621,"stargazers_count":16,"open_issues_count":15,"forks_count":4,"subscribers_count":3,"default_branch":"main","last_synced_at":"2024-12-05T11:24:05.377Z","etag":null,"topics":["kubernetes","microsegmentation","network-analysis","network-security","networkpolicy"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/np-guard.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-05-20T20:42:41.000Z","updated_at":"2024-12-05T10:19:04.000Z","dependencies_parsed_at":"2024-02-21T20:26:08.377Z","dependency_job_id":"42899527-dfa0-4b72-b92a-f0ad849acef0","html_url":"https://github.com/np-guard/cluster-topology-analyzer","commit_stats":null,"previous_names":[],"tags_count":20,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/np-guard%2Fcluster-topology-analyzer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/np-guard%2Fcluster-topology-analyzer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/np-guard%2Fcluster-topology-analyzer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/np-guard%2Fcluster-topology-analyzer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/np-guard","download_url":"https://codeload.github.com/np-guard/cluster-topology-analyzer/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":229504578,"owners_count":18083397,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["kubernetes","microsegmentation","network-analysis","network-security","networkpolicy"],"created_at":"2024-12-13T07:09:16.198Z","updated_at":"2024-12-13T07:09:16.828Z","avatar_url":"https://github.com/np-guard.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Shift-left Network Topology Analyzer\n\n## About Topology Analyzer\nThis tool analyzes resource YAMLs of a Kubernetes-based application to extract the required connectivity between its workloads. It looks for network addresses that appear in workload specifications (e.g., in a Deployment's pod-spec environment) and correlates them to known and to predicted network addresses (e.g., expected Service URLs). Optionally, the tool can produce a list of NetworkPolicy resources that limit the workloads' connectivity to nothing but the detected required connectivity.\n\n## Usage\n```\n$ ./bin/net-top -h\nUsage of ./bin/net-top:\n  -dirpath string\n    \tinput directory path (required, can be specified multiple times with different directories)\n  -outputfile string\n    \tfile path to store results\n  -format string\n        output format; must be either \"json\" or \"yaml\" (default \"json\")\n  -netpols\n        whether to synthesize NetworkPolicies to allow only the discovered connections\n  -dnsport int\n        specify DNS port to be used in egress rules of synthesized NetworkPolicies (default 53)\n  -q    runs quietly, reports only severe errors and results\n  -v    runs with more informative messages printed to log\n```\n\n## Algorithm\nThe underlying algorithm for identifying required connectivity works as follows.\n1. Scan the given directories for all YAML files.\n1. In each YAML file identify manifests for [workload resources](https://kubernetes.io/docs/concepts/workloads/controllers/) and [Service resources](https://kubernetes.io/docs/concepts/services-networking/service/#service-resource), as well as [ConfigMap resources](https://kubernetes.io/docs/concepts/configuration/configmap/), [Ingress](https://kubernetes.io/docs/concepts/services-networking/ingress), [Gateway Routes](https://gateway-api.sigs.k8s.io/concepts/api-overview/#route-resources) and [OpenShift Routes](https://docs.openshift.com/container-platform/latest/networking/routes/route-configuration.html).\n1. In each workload resource, identify configuration values that might represent network addresses. This includes strings in containers' `envs`, `args` and `command` fields, as well as references to data in ConfigMaps.\n1. For each target-workload in the list of workload resources:\n    1. Identify all services whose selector matches target-workload\n    1. For each such service:\n        1. Compile a list of possible network addresses that can be used to access this service, e.g., `mysvc`, `mysvc.myns`, `mysvc.myns.svc.cluster.local`.\n        1. Identify all workload resources with a configuration value that matches a value from the list of possible network addresses, possibly with an additional port specifier.\n        1. For each source-workload in the set of identified workloads:\n            1. Add a connection from source-workload to target-workload to the list of identified connections. Add protocol and port information if available.\n\nThe algorithm for synthesizing NetworkPolicies that only allow the required connections and no other connection:\n1. For each workload generate a NetworkPolicy resources as follows:\n    - `metadata.namespace` is set to the workload's namespace (if specified)\n    - `spec.podSelector` is set to the workload pod selector\n    - `spec.policyTypes` is set to `[\"Ingress\", \"Egress\"]`\n    - `spec.ingress` contains one rule for each required connection in which the workload is the target workload. If the Service exposing this workload is of type `LoadBalancer` or `NodePort`, allow ingress from any source. If the service exposing this workload is pointed by an Ingress resource or by a Route resource, allow ingress from any source **within the cluster**.\n    - `spec.egress` contains one rule for each required connection in which the workload is the source workload. If such connections exist, also add a rule to allow egress to UDP port 53 (DNS).\n1. For each **workload namespace** add a *default deny* NetworkPolicy as follows\n    - `metadata.namespace` is set to the workload's namespace \n    - `spec.podSelector` is set to the empty selector (selects all pods in the namespace)\n    - `spec.policyTypes` is set to `[\"Ingress\", \"Egress\"]`\n    - `spec.ingress` contains no rules (allows no ingress)\n    - `spec.egress` contains no rules (allows no egress)\n\n## Assumptions\n\n1. All the relevant application resources (workloads, Services, ConfigMaps) are defined in YAML files under the given directories or their subdirectories\n1. All YAML files can be applied to a Kubernetes cluster as-is using `kubectl apply -f` (i.e., no helm-style templating).\n1. Every workload that needs to connect to a Service, will somehow specify the network address of this Service in its manifest. This can be specified directly in the containers `envs` (see example [here](tests/k8s_guestbook/frontend-deployment.yaml#L25:L28)), or via a ConfigMap (see examples [here](tests/onlineboutique/kubernetes-manifests.yaml#L110:L114) and [here](tests/onlineboutique/kubernetes-manifests.yaml#L270:L272)), or using command-line arguments.\n1. The network addresses of a given Service `\u003csvc\u003e` in Namespace `\u003cns\u003e`, exposing port `\u003cportNum\u003e`, must match this pattern `(http(s)?://)?\u003csvc\u003e(.\u003cns\u003e(.svc.cluster.local)?)?(:\u003cportNum\u003e)?`. Examples for legal network addresses are `wordpress-mysql:3306`, `redis-follower.redis.svc.cluster.local:6379`, `redis-leader.redis`, `http://rating-service`.\n\n## Build the project\nMake sure you have golang 1.22+ on your platform\n\n```shell\ngit clone git@github.com:np-guard/cluster-topology-analyzer.git\ncd cluster-topology-analyzer\nmake\n```\n\n## Run Examples\n\n1. Clone a sample source code repository that you want to scan\n```shell\ngit clone git@github.com:GoogleCloudPlatform/microservices-demo.git $HOME/microservices-demo\n```\n\n2. Point topology analyzer to this sample repo\n```shell\n./bin/net-top -dirpath $HOME/microservices-demo\n```\n3. The tool outputs a list of identified connections. Each connection is defined as a triplet: `\u003csource, target, link\u003e`. The list should look like this:\n```json\n[\n    ...\n    {\n        \"source\": {\n            \"resource\": {\n                \"name\": \"frontend\",\n                \"labels\": {\n                    \"app\": \"frontend\"\n                },\n                \"serviceaccountname\": \"default\",\n                \"filepath\": \"kubernetes-manifests.yaml\",\n                \"kind\": \"Deployment\",\n                \"image\": {\n                    \"id\": \"gcr.io/google-samples/microservices-demo/frontend:v0.2.3\"\n                },\n                \"NetworkAddrs\": [\n                    \"productcatalogservice:3550\",\n                    \"currencyservice:7000\",\n                    \"cartservice:7070\",\n                    \"recommendationservice:8080\",\n                    \"checkoutservice:5050\",\n                    \"adservice:9555\",\n                    \"shippingservice:50051\"\n                ],\n                \"UsedPorts\": [\n                    {\n                        \"port\": 9555,\n                        \"target_port\": 9555\n                    }\n                ]\n            }\n        },\n        \"target\": {\n            \"resource\": {\n                \"name\": \"adservice\",\n                \"labels\": {\n                    \"app\": \"adservice\"\n                },\n                \"serviceaccountname\": \"default\",\n                \"filepath\": \"kubernetes-manifests.yaml\",\n                \"kind\": \"Deployment\",\n                \"image\": {\n                    \"id\": \"gcr.io/google-samples/microservices-demo/adservice:v0.2.3\"\n                },\n                \"NetworkAddrs\": null,\n                \"UsedPorts\": null\n            }\n        },\n        \"link\": {\n            \"resource\": {\n                \"name\": \"adservice\",\n                \"selectors\": [\n                    \"app:adservice\"\n                ],\n                \"type\": \"ClusterIP\",\n                \"filepath\": \"kubernetes-manifests.yaml\",\n                \"kind\": \"Service\",\n                \"network\": [\n                    {\n                        \"port\": 9555,\n                        \"target_port\": 9555\n                    }\n                ]\n            }\n        }\n    }\n]\n```\n4. Produce NetworkPolicies for this sample repo to only allow detected connectivity, and store them in `netpols.json` as a single NetworkPolicyList resource. Run quietly.\n```shell\n./bin/net-top -dirpath $HOME/microservices-demo -netpols -outputfile netpols.json -q\n```\n\n## Golang API\nThe functionality of this tool can be consumed via a [Golang package API](https://pkg.go.dev/github.com/np-guard/cluster-topology-analyzer/pkg/analyzer). The relevant package to import is `github.com/np-guard/cluster-topology-analyzer/pkg/analyzer`.\n\nMain functionality is encapsulated under the `PoliciesSynthesizer`, which exposes six methods:\n* `func (ps *PoliciesSynthesizer) ConnectionsFromFolderPath(dirPath string) ([]*common.Connections, error)` - getting a slice of Connection objects, each representing a required connection in the scanned application.\n* `func (ps *PoliciesSynthesizer) ConnectionsFromFolderPaths(dirPaths []string) ([]*common.Connections, error)` - same as `ConnectionsFromFolderPath()` but allows specifying multiple directories to scan.\n* `func (ps *PoliciesSynthesizer) ConnectionsFromInfos(infos []*resource.Info) ([]*Connections, error)` - same as `ConnectionsFromFolderPath()` but analyzing the K8s resources in a slice of `Info` objects rather than scanning a file-system directory for manifest files.\n* `func (ps *PoliciesSynthesizer) PoliciesFromFolderPath(dirPath string) ([]*networking.NetworkPolicy, error)` - getting a slice of K8s NetworkPolicy objects that limit the allowed connectivity to only the required connections.\n* `func (ps *PoliciesSynthesizer) PoliciesFromFolderPaths(dirPaths []string) ([]*networking.NetworkPolicy, error)` - same as `PoliciesFromFolderPath()` but allows specifying multiple directories to scan.\n* `func (ps *PoliciesSynthesizer) PoliciesFromInfos(infos []*resource.Info) ([]*networking.NetworkPolicy, error)` - same as `PoliciesFromFolderPath()` but analyzing the K8s resources in a slice of `Info` objects rather than scanning a file-system directory for manifest files.\n\nThe example code below extracts required connections from the K8s manifests in the `/tmp/k8s_manifests` directory, and outputs appropriate K8s NetworkPolicies to standard output.\n```golang\npackage main\n\nimport (\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"os\"\n\n\t\"github.com/np-guard/cluster-topology-analyzer/v2/pkg/analyzer\"\n)\n\nfunc main() {\n\tlogger := analyzer.NewDefaultLogger()\n\tsynth := analyzer.NewPoliciesSynthesizer(analyzer.WithLogger(logger))\n\n\tnetpols, err := synth.PoliciesFromFolderPath(\"/tmp/k8s_manifests\")\n\tif err != nil {\n\t\tfmt.Fprintf(os.Stderr, \"Error synthesizing policies: %v\\n\", err)\n\t\tos.Exit(1)\n\t}\n\tbuf, _ := json.MarshalIndent(netpols, \"\", \"    \")\n\tfmt.Printf(\"%v\\n\", string(buf))\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnp-guard%2Fcluster-topology-analyzer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnp-guard%2Fcluster-topology-analyzer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnp-guard%2Fcluster-topology-analyzer/lists"}