{"id":13613855,"url":"https://github.com/salesforce/sloop","last_synced_at":"2025-04-11T11:48:39.180Z","repository":{"id":39579873,"uuid":"207928913","full_name":"salesforce/sloop","owner":"salesforce","description":"Kubernetes History Visualization","archived":false,"fork":false,"pushed_at":"2024-07-26T20:28:08.000Z","size":5554,"stargazers_count":1499,"open_issues_count":56,"forks_count":133,"subscribers_count":28,"default_branch":"master","last_synced_at":"2025-03-27T14:11:08.272Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/salesforce.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE.txt","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":"CODEOWNERS","security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-09-12T00:11:11.000Z","updated_at":"2025-03-24T18:00:41.000Z","dependencies_parsed_at":"2024-06-18T19:52:15.865Z","dependency_job_id":"dc215f6c-0dc4-463c-999d-0dda6f359a86","html_url":"https://github.com/salesforce/sloop","commit_stats":{"total_commits":168,"total_committers":41,"mean_commits":4.097560975609756,"dds":0.875,"last_synced_commit":"81473b2971c704f5a909123f8f8d806ed6c8caf8"},"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/salesforce%2Fsloop","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/salesforce%2Fsloop/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/salesforce%2Fsloop/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/salesforce%2Fsloop/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/salesforce","download_url":"https://codeload.github.com/salesforce/sloop/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247024149,"owners_count":20870940,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T20:00:54.203Z","updated_at":"2025-04-03T15:07:28.619Z","avatar_url":"https://github.com/salesforce.png","language":"Go","readme":"# Sloop - Kubernetes History Visualization\n\n[![Publish Status](https://github.com/salesforce/sloop/workflows/Publish/badge.svg)](https://github.com/salesforce/sloop/actions)\n[![Build Status](https://github.com/salesforce/sloop/workflows/Test/badge.svg)](https://github.com/salesforce/sloop/actions)\n[![Go Report Card](https://goreportcard.com/badge/github.com/salesforce/sloop)](https://goreportcard.com/report/github.com/salesforce/sloop)\n\n\u003cimg src=\"https://github.com/salesforce/sloop/raw/master/other/sloop_logo_color_small_notext.png\"\u003e\n\n---\n\nSloop monitors Kubernetes, recording histories of events and resource state changes\nand providing visualizations to aid in debugging past events.\n\nKey features:\n\n1. Allows you to find and inspect resources that no longer exist (example: discover what host the pod from the previous deployment was using).\n1. Provides timeline displays that show rollouts of related resources in updates to Deployments, ReplicaSets, and StatefulSets.\n1. Helps debug transient and intermittent errors.\n1. Allows you to see changes over time in a Kubernetes application.\n1. Is a self-contained service with no dependencies on distributed storage.\n\n---\n\n## Screenshots\n\n![Screenshot1](other/screenshot1.png?raw=true \"Screenshot 1\")\n\n## Architecture Overview\n\n![Architecture](other/architecture.png?raw=true \"Architecture\")\n\n## Install\n\nSloop can be installed using any of these options:\n\n### Helm Chart\n\nUsers can install sloop by using helm chart now, for instructions refer [helm readme](helm/sloop/README.md)\n\n### Precompiled Binaries\n\nTODO: See the [Releases](https://github.com/salesforce/sloop/releases).\n\n### Build from Source\n\nBuilding Sloop from source needs a working Go environment\nwith the version defined in the [go.mod](./go.mod) file or greater.\n\nSee: https://golang.org/doc/install\n\nClone the sloop repository and build using `make`:\n\n```sh\nmkdir -p $GOPATH/src/github.com/salesforce\ncd $GOPATH/src/github.com/salesforce\ngit clone https://github.com/salesforce/sloop.git\ncd sloop\nmake\n$GOPATH/bin/sloop\n```\n\nWhen complete, you should have a running Sloop version accessing the current context from your kubeConfig. Just point your browser at http://localhost:8080/\n\nOther makefile targets:\n\n- _docker_: Builds a Docker image.\n- _cover_: Runs unit tests with code coverage.\n- _generate_: Updates genny templates for typed table classes.\n- _protobuf_: Generates protobuf code-gen.\n\n### Local Docker Run\n\nTo run from Docker you need to host mount your kubeconfig:\n\n```shell script\nmake docker-snapshot\ndocker run --rm -it -p 8080:8080 -v ~/.kube/:/kube/ -e KUBECONFIG=/kube/config sloop\n```\n\nIn this mode, data is written to a memory-backed volume and is discarded after each run. To preserve the data, you can host-mount /data with something like `-v /data/:/some_path_on_host/`\n\n### Updating webfiles folder\n\nTo reflect any changes to webserver/webfiles, run the following command on terminal while within the webserver directory before submitting a pr:\n\n```shell script\ngo-bindata -pkg webserver -o bindata.go webfiles/\n```\n\nThis will update the bindata folder with your changes to any html, css or javascript files within the directory.\n\n### Local Docker Run and connecting to EKS\n\nThis is very similar to above but abstracts running docker with AWS credentials for connecting to EKS\n\n```shell script\nmake docker\nexport AWS_ACCESS_KEY_ID=\u003caccess_key_id\u003e AWS_SECRET_ACCESS_KEY=\u003csecret_access_key\u003e AWS_SESSION_TOKEN=\u003csession_token\u003e\n./providers/aws/sloop_to_eks.sh \u003ccluster name\u003e\n```\n\nData retention policy stated above still applies in this case.\n\n## Backup \u0026 Restore\n\n\u003e This is an advanced feature. Use with caution.\n\nTo download a backup of the database, navigate to http://localhost:8080/data/backup\n\nTo restore from a backup, start `sloop` with the `-restore-database-file` flag set to the backup file downloaded in the previous step. When restoring, you may also wish to set the `-disable-kube-watch=true` flag to stop new writes from occurring and/or the `-context` flag to restore the database into a different context.\n\n## Memory Consumption\n\nSloop's memory usage can be managed by tweaking several options:\n\n- `badger-use-lsm-only-options` If this flag is set to true, values would be collocated with the LSM tree, with value log largely acting as a write-ahead log only. Recommended value for memory constrained environments: false\n- `badger-keep-l0-in-memory` When this flag is set to true, Level 0 tables are kept in memory. This leads to better performance in writes as well as compactions. Recommended value for memory constrained environments: false\n- `badger-sync-writes` When SyncWrites is true all writes are synced to disk. Setting this to false would achieve better performance, but may cause data loss in case of crash. Recommended value for memory constrained environments: false\n- `badger-vlog-fileIO-mapping` TableLoadingMode indicates which file loading mode should be used for the LSM tree data files. Setting to true would not load the value in memory map. Recommended value for memory constrained environments: true\n\nApart from these flags some other values can be tweaked to fit in the memory constraints. Following are some examples of setups.\n\n- Memory consumption max limit: 1GB\n\n```\n               // 0.5\u003c\u003c20 (524288 bytes = 0.5 Mb)\n               \"badger-max-table-size=524288\",\n               \"badger-number-of-compactors=1\",\n               \"badger-number-of-level-zero-tables=1\",\n               \"badger-number-of-zero-tables-stall=2\",\n```\n\n- Memory consumption max limit: 2GB\n\n```\n               // 16\u003c\u003c20 (16777216 bytes = 16 Mb)\n               \"badger-max-table-size=16777216\",\n               \"badger-number-of-compactors=1\",\n               \"badger-number-of-level-zero-tables=1\",\n               \"badger-number-of-zero-tables-stall=2\",\n```\n\n- Memory consumption max limit: 5GB\n\n```\n               // 32\u003c\u003c20 (33554432 bytes = 32 Mb)\n               \"badger-max-table-size=33554432\",\n               \"badger-number-of-compactors=1\",\n               \"badger-number-of-level-zero-tables=2\",\n               \"badger-number-of-zero-tables-stall=3\",\n```\n\nApart from the above settings, max-disk-mb and max-look-back can be tweaked according to input data and memory constraints.\n\n## Prometheus\n\nSloop uses the [Prometheus](https://prometheus.io/) library to emit metrics, which is very helpful for performance debugging.\n\nIn the root of the repo is a Prometheus config file\n[prometheus.yml](./prometheus.yml).\n\nOn OSX you can install Prometheus with `brew install prometheus`. Then start it from the sloop directory by running `prometheus`\n\nOpen your browser to http://localhost:9090.\n\nAn example of a useful query is [rate(kubewatch_event_count[5m])](\u003chttp://localhost:9090/graph?g0.range_input=1h\u0026g0.expr=rate(kubewatch_event_count%5B1m%5D)\u0026g0.tab=0\u003e)\n\n## Event filtering\n\nEvents can be excluded from Sloop by adding `exclusionRules` to the config file:\n\n```\n{\n  \"defaultNamespace\": \"default\",\n  \"defaultKind\": \"Pod\",\n  \"defaultLookback\": \"1h\",\n  [...]\n  \"exclusionRules\": {\n    \"_all\": [\n      {\"==\": [ { \"var\": \"metadata.namespace\" }, \"kube-system\" ]}\n    ],\n    \"Pod\": [\n      {\"==\": [ { \"var\": \"metadata.name\" }, \"sloop-0\" ]}\n    ],\n    \"Job\": [\n      {\"in\": [ { \"var\": \"metadata.name\" }, [ \"cron1\", \"cron3\" ] ]}\n    ]\n  }\n}`\n\n```\n\nAdding rules can help to reduce resources consumed by Sloop and remove unwanted noise from the UI for events that are of no interest.\n\n### Limiting rules to specific kinds\n\n * Rules under the special key `_all` are evaluated against events for objects of any kind\n * Rules under any other key are evaluated only against objects whose kind matches the key, e.g. `Pod` only applies to pods, `Job` only applies to jobs etc.\n\n### Rule format and supported operations\n\nRules should follow the [JsonLogic](https://jsonlogic.com) format and are evaluated against the json representation of the Kubernetes API object related to the event (see below).\n\nAvailable operators, such as `==` and `in` shown above, are documented [here](https://jsonlogic.com/operations.html).\n\n### Data available to rule logic\n\nKubernetes API conventions for [objects](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md#objects) require the following keys to exist in the json data for all resources, all of which can be referenced in rules:\n\n * `metadata`\n * `spec`\n * `status`\n\nSome commonly useful fields under the `metadata` [object](https://pkg.go.dev/k8s.io/apimachinery/pkg/apis/meta/v1#ObjectMeta) are:\n\n * `name`\n * `namespace`\n * `labels`\n\n#### Type specific data\n\nSome resources contain additional type-specific fields, for example `PersistentVolumeClaimSpec` objects have fields named `selector` and `storageClassName`.\n\nType specific fields for each object and their corresponding keys in the object json representation are documented in the [core API](https://pkg.go.dev/k8s.io/api@v0.27.1/core/v1), e.g. for `PersistentVolumeClaimSpec` objects the documentation is [here](https://pkg.go.dev/k8s.io/api@v0.27.1/core/v1#PersistentVolumeClaimSpec).\n\n## Contributing\n\nRefer to [CONTRIBUTING.md](CONTRIBUTING.md)\u003cbr\u003e\n\n## License\n\nBSD 3-Clause\n","funding_links":[],"categories":["Go","Observability \u0026 Monitoring","Cloud / Docker / Kubernetes","others","Troubleshooting","Tools and Libraries","Observability"],"sub_categories":["Monitoring, Alerts, and Visualization"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsalesforce%2Fsloop","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsalesforce%2Fsloop","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsalesforce%2Fsloop/lists"}