{"id":51322663,"url":"https://github.com/piyushsachdeva/elastic-cost-analyzer","last_synced_at":"2026-07-01T15:02:12.014Z","repository":{"id":362618587,"uuid":"1250713285","full_name":"piyushsachdeva/elastic-cost-analyzer","owner":"piyushsachdeva","description":"AI agent that detects AWS cost spikes, correlates with deployments, posts root-cause Slack alerts — built with Kiro IDE, Amazon Bedrock, and Elastic","archived":false,"fork":false,"pushed_at":"2026-06-05T04:26:43.000Z","size":14038,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-06-05T07:13:41.768Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/piyushsachdeva.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-05-26T22:42:52.000Z","updated_at":"2026-06-05T04:26:47.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/piyushsachdeva/elastic-cost-analyzer","commit_stats":null,"previous_names":["piyushsachdeva/elastic-cost-analyzer"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/piyushsachdeva/elastic-cost-analyzer","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/piyushsachdeva%2Felastic-cost-analyzer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/piyushsachdeva%2Felastic-cost-analyzer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/piyushsachdeva%2Felastic-cost-analyzer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/piyushsachdeva%2Felastic-cost-analyzer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/piyushsachdeva","download_url":"https://codeload.github.com/piyushsachdeva/elastic-cost-analyzer/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/piyushsachdeva%2Felastic-cost-analyzer/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35011257,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-07-01T02:00:05.325Z","response_time":130,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-07-01T15:02:06.392Z","updated_at":"2026-07-01T15:02:12.004Z","avatar_url":"https://github.com/piyushsachdeva.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Cloud Cost Anomaly Agent\n\nAn AI agent that runs daily, detects AWS cost spikes, correlates them with recent\ndeployments, and posts a root-cause Slack alert to `#finops` — automatically.\n\nBuilt with **Amazon Bedrock** (Claude Sonnet 4), **Elastic Cloud Serverless**, and\n**AWS Lambda**. Zero manual triage. ~$3–5/month to run.\n\n---\n\n## How it works\n\n```\nEventBridge (cron 0 8 * * ? *)\n  → Lambda  (thin host — creates agent, calls run())\n      → Bedrock  (Claude Sonnet 4 converse loop, max 20 iterations)\n          ↔ Elasticsearch\n              aws-billing-*          ← Elastic Agent fills automatically\n              deploy-events-*        ← your CI/CD pipeline writes one doc/deploy\n              cost-anomaly-audit-*   ← agent writes after every run\n          → Slack  (Block Kit message, only when anomalies are found)\n      ← Elastic Observability  (OTel traces → Kibana APM, zero instrumentation)\n```\n\nThe agent follows a fixed 7-step reasoning sequence every run:\n\n1. Detect services where today's spend exceeds the 7-day baseline by ≥ 25 %\n2. Pull hourly cost data to pinpoint when the spike started\n3. Look up deployments near the spike start time\n4. Write a one-sentence root cause\n5. Write one actionable fix with an estimated dollar saving\n6. Post a Slack Block Kit message (only if anomalies found)\n7. Write an audit record to Elasticsearch (always — even if Slack fails)\n\n---\n\n## Repository layout\n\n```\nagent.py                  Lambda handler + Bedrock converse loop\ntools/\n  elastic_search.py       Five ES query functions (billing + deploy lookups)\n  slack_notify.py         Slack Block Kit builder\n  audit_writer.py         Audit record writer\n  __init__.py\nscripts/\n  seed_billing.py         Seeds aws-billing-* with 7-day baseline + today's spike\ntests/\n  test_integration.py     8 integration tests — fully mocked, no real AWS calls\nrequirements.txt          Pinned Python deps\nDockerfile                Multi-arch Lambda container image (amd64 + arm64)\nMakefile                  build / test / zip / deploy helpers\n.env.example              Environment variable template\ndemo.md                   Full setup guide + video recording script (start here)\n```\n\n---\n\n## Prerequisites\n\n| Tool | Version | Notes |\n|---|---|---|\n| Python | 3.12+ | matches Lambda runtime |\n| AWS CLI | v2 | configured with credentials |\n| AWS account | — | Bedrock model access needed (free to request) |\n| Elastic Cloud | Serverless | subscribe via AWS Marketplace |\n| Slack workspace | — | incoming webhooks enabled |\n\n---\n\n## Setup\n\nAll setup instructions are in **[`demo.md`](demo.md)** — it covers both the full infrastructure setup guide and the video recording script in a single file. Follow the \"FULL INFRASTRUCTURE SETUP\" section at the top.\n\n| Step | Service | What you create |\n|---|---|---|\n| 1 | Elastic Cloud | Serverless Observability project |\n| 2 | Kibana Dev Tools | `deploy-events-*` + `aws-billing-*` index + seed data |\n| 3 | Kibana Stack Mgmt | API key `cost-anomaly-agent` |\n| 4 | Slack | Incoming webhook → `#finops` |\n| 5 | AWS Secrets Manager | `elastic-creds` + `slack-webhook` secrets |\n| 6 | AWS IAM | `cost-anomaly-agent-lambda-role` + inline policy |\n| 7 | Amazon Bedrock | Claude Sonnet 4.5 model access (use case form) |\n| 8 | AWS Lambda | `cost-anomaly-agent` function (Python 3.12) |\n| 9 | Amazon EventBridge | Daily cron rule (08:00 UTC) |\n| 10 | Verification | End-to-end smoke test |\n\n---\n\n## Local development\n\n### 1. Install dependencies\n\n```bash\npython3 -m venv .venv\nsource .venv/bin/activate\npip install -r requirements.txt\n```\n\n### 2. Run tests (no AWS credentials needed)\n\n```bash\nmake test\n```\n\nExpected: **8 tests pass** in ~2 seconds. All AWS and Elastic calls are mocked.\n\n```\ntests/test_integration.py::TestAgentIntegration::test_full_chain_completes_successfully    PASSED\ntests/test_integration.py::TestAgentIntegration::test_find_spike_services_called_once      PASSED\ntests/test_integration.py::TestAgentIntegration::test_timeseries_called_for_spiked_service PASSED\ntests/test_integration.py::TestAgentIntegration::test_deploy_lookup_called_with_correct_service PASSED\ntests/test_integration.py::TestAgentIntegration::test_slack_message_contains_required_fields    PASSED\ntests/test_integration.py::TestAgentIntegration::test_audit_written_on_success             PASSED\ntests/test_integration.py::TestAgentIntegration::test_no_anomalies_exits_silently          PASSED\ntests/test_integration.py::TestAgentIntegration::test_lambda_handler_returns_200           PASSED\n```\n\n### 3. Build and upload the Lambda zip\n\n```bash\nmake zip\n```\n\nThis produces `cost-anomaly-agent.zip` (≈ 6 MB). Upload it in the Lambda console:\n**Code** tab → **Upload from** → **.zip file** → confirm handler is `agent.lambda_handler`.\n\n### 4. Run locally against real AWS (optional)\n\nCopy `.env.example` to `.env.local`, fill in real values, then:\n\n```bash\nmake run-local\n```\n\nThis builds the Docker image and invokes the Lambda runtime locally. Hits real\nBedrock and real Elasticsearch — use only after infrasetup.md is complete.\n\n---\n\n## Environment variables\n\nAll set on the Lambda function. See `infrasetup.md §8` for where to find each value.\n\n| Variable | Description |\n|---|---|\n| `ELASTIC_SECRET_ARN` | ARN of the `cost-anomaly-agent/elastic-creds` secret |\n| `SLACK_SECRET_ARN` | ARN of the `cost-anomaly-agent/slack-webhook` secret |\n| `AWS_BEDROCK_REGION` | Region where Bedrock is used (e.g. `us-east-1`) |\n| `SPIKE_THRESHOLD_PCT` | Spike threshold percentage, default `25.0` |\n| `AGENT_MAX_ITERATIONS` | Bedrock converse loop limit, default `20` |\n| `BEDROCK_MODEL_ID` | Optional override, default `anthropic.claude-sonnet-4-5` |\n\n---\n\n## Verifying the end-to-end setup\n\nAfter completing infrasetup.md:\n\n**1. Trigger the Lambda manually**\n\nAWS Console → Lambda → `cost-anomaly-agent` → **Test** tab → use event:\n```json\n{\"source\": \"manual-test\"}\n```\n\nExpected response:\n```json\n{\n  \"statusCode\": 200,\n  \"body\": {\n    \"run_id\": \"...\",\n    \"duration_seconds\": 14.2,\n    \"total_tokens\": 4380\n  }\n}\n```\n\n**2. Confirm Slack message**\n\nSlack → `#finops` → new Block Kit message present (only if a cost spike was detected).\n\n**3. Confirm audit record**\n\nKibana → Dev Tools:\n```http\nGET cost-anomaly-audit-*/_search\n{\n  \"sort\": [{\"@timestamp\": {\"order\": \"desc\"}}],\n  \"size\": 1\n}\n```\n\n**4. Confirm OTel trace**\n\nKibana → Observability → APM → Services → `cost-anomaly-agent` → latest transaction.\n\n---\n\n## Cost estimate\n\n| Component | Approximate cost |\n|---|---|\n| Lambda (runs once/day, ~15s) | \u003c $0.01/month |\n| Bedrock Claude Sonnet 4 (~4k tokens/run) | ~$0.12/month |\n| Elastic Cloud Serverless | ~$2–4/month (depends on data volume) |\n| Secrets Manager (2 secrets) | ~$0.80/month |\n| **Total** | **~$3–5/month** |\n\n---\n\n## Makefile targets\n\n| Target | What it does |\n|---|---|\n| `make test` | Run 8 integration tests (no AWS needed) |\n| `make zip` | Build `cost-anomaly-agent.zip` for Lambda upload |\n| `make build` | Build Docker image for native platform |\n| `make build-amd64` | Build Docker image for Lambda x86_64 |\n| `make ecr-push` | Build multi-arch image and push to ECR |\n| `make run-local` | Run agent locally via Docker (needs `.env.local`) |\n| `make clean` | Remove build artefacts |\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpiyushsachdeva%2Felastic-cost-analyzer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpiyushsachdeva%2Felastic-cost-analyzer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpiyushsachdeva%2Felastic-cost-analyzer/lists"}