{"id":18098675,"url":"https://github.com/krzko/gcp-anomaly-detector","last_synced_at":"2025-04-13T13:07:59.039Z","repository":{"id":200638841,"uuid":"705902844","full_name":"krzko/gcp-anomaly-detector","owner":"krzko","description":"🔢 Anomaly detection service for Google Cloud Monitoring metrics, utilising statistical analysis to identify significant deviations in time-series data","archived":false,"fork":false,"pushed_at":"2024-01-16T03:05:07.000Z","size":3524,"stargazers_count":4,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-13T13:07:54.472Z","etag":null,"topics":["anomaly","anomaly-detection","gcm","gcp","gcp-monitoring","z-score"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/krzko.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2023-10-16T23:27:51.000Z","updated_at":"2024-12-12T10:08:15.000Z","dependencies_parsed_at":null,"dependency_job_id":"c4208932-87ef-4b59-99ef-08109f73525b","html_url":"https://github.com/krzko/gcp-anomaly-detector","commit_stats":{"total_commits":2,"total_committers":2,"mean_commits":1.0,"dds":0.5,"last_synced_commit":"189ca1f893d1ddb7d8459b44f98e3fb862a4e104"},"previous_names":["krzko/gcp-anomaly-detector"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/krzko%2Fgcp-anomaly-detector","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/krzko%2Fgcp-anomaly-detector/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/krzko%2Fgcp-anomaly-detector/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/krzko%2Fgcp-anomaly-detector/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/krzko","download_url":"https://codeload.github.com/krzko/gcp-anomaly-detector/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248717242,"owners_count":21150389,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["anomaly","anomaly-detection","gcm","gcp","gcp-monitoring","z-score"],"created_at":"2024-10-31T20:12:21.639Z","updated_at":"2025-04-13T13:07:58.989Z","avatar_url":"https://github.com/krzko.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# gcp-anomaly-detector\n\nThis tool is designed to detect anomalies in Google Cloud Monitoring metrics. It fetches historical and recent metrics data, computes baseline statistics, and identifies anomalies based on Z-score analysis. The Z-score represents how many standard deviations an element is from the mean, and a high absolute Z-score indicates a potential anomaly.\n\n## Configuration\n\nThe tool is configured using a YAML file. Here's an example configuration file with explanations for each field:\n\n```yaml\nmetrics:\n  - 'custom.googleapis.com/otel/foo_connection_count'\n  - 'custom.googleapis.com/otel/foo_current_connections'  # List of metric types to monitor\nfilters:\n  custom.googleapis.com/otel/foo_connection_count: 'resource.type=\"generic_task\" AND metric.labels.\"environment\"=\"dev\"'\n  custom.googleapis.com/otel/foo_current_connections\": 'resource.type=\"generic_task\" AND metric.labels.\"environment\"=\"dev\"'  # Filters to apply when fetching metrics\nbaseline_duration: 7  # Baseline duration in days\npolling_time: 60  # Polling time in seconds\nproject_id: foo-bar-dev-1a2b3c  # GCP Project ID\nrecent_duration: 60  # Recent metrics duration in minutes\nz_score_threshold: 3.00  # Z-score threshold for anomaly detection\n\n```\n\n## Usage\n\n1. Create a configuration file following the example above.\n2. Build the tool:\n\n```sh\ngo build\n```\n\n3. Authenticate to Google Cloud\n\n```sh\ngcloud auth login --update-ad\n\nexport GOOGLE_APPLICATION_CREDENTIALS=\"/Users/$USER/.config/gcloud/application_default_credentials.json\"\n```\n\n4. Run the tool:\n\n```sh\n./gcp-anomaly-detector\n```\n\nThe tool will load the configuration, initialise a baseline using historical metrics data, and start polling recent metrics data at the specified interval. It will log the Z-scores for each metric and report any anomalies detected based on the configured Z-score threshold.\n\n## Understanding Z-Score\n\nThe Z-score is a statistical measurement that describes a value's relationship to the mean of a group of values. It is measured in terms of standard deviations from the mean. In this tool, a high absolute Z-score (e.g., 3.0 or -3.0) indicates a potential anomaly.\n\n* A positive Z-score indicates the data point is higher than the mean.\n* A negative Z-score indicates the data point is lower than the mean.\n\nThe `z_score_threshold` in the configuration file determines the Z-score value at which a data point is considered an anomaly. For example, with a `z_score_threshold` of 3.00, any data point with a Z-score of 3.0 or -3.0 and above would be flagged as an anomaly.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkrzko%2Fgcp-anomaly-detector","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkrzko%2Fgcp-anomaly-detector","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkrzko%2Fgcp-anomaly-detector/lists"}