{"id":29395231,"url":"https://github.com/norandom/observability-k8s","last_synced_at":"2025-12-30T22:13:02.827Z","repository":{"id":301501318,"uuid":"1009448403","full_name":"norandom/observability-k8s","owner":"norandom","description":"Grafana with QuickWit for observability and security logs in Kubernetes (k3s)","archived":false,"fork":false,"pushed_at":"2025-07-05T19:27:23.000Z","size":1628,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-07-05T20:29:59.439Z","etag":null,"topics":["analytics","argocd","conda","dashboard","grafana","javascript","k3s","kubectl","kubernetes","learning-by-doing","loki","node","observablehq","prometheus","python","quickwit","tekton","vector"],"latest_commit_sha":null,"homepage":"https://because-security.atlassian.net/wiki/spaces/AD/pages/613286107/Kubernetes+GitOps+security+and+observability+with+Open-Source+tools+Grafana+Loki+full-text+search+purposeful+dashboards+without+much+programming","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/norandom.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-06-27T06:37:05.000Z","updated_at":"2025-07-05T19:27:26.000Z","dependencies_parsed_at":"2025-06-27T07:41:37.976Z","dependency_job_id":"fdcaf92b-7c5d-4646-8c9f-1f4b7549d7ea","html_url":"https://github.com/norandom/observability-k8s","commit_stats":null,"previous_names":["norandom/observability-k8s"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/norandom/observability-k8s","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/norandom%2Fobservability-k8s","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/norandom%2Fobservability-k8s/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/norandom%2Fobservability-k8s/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/norandom%2Fobservability-k8s/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/norandom","download_url":"https://codeload.github.com/norandom/observability-k8s/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/norandom%2Fobservability-k8s/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":264572250,"owners_count":23630211,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["analytics","argocd","conda","dashboard","grafana","javascript","k3s","kubectl","kubernetes","learning-by-doing","loki","node","observablehq","prometheus","python","quickwit","tekton","vector"],"created_at":"2025-07-10T11:27:17.074Z","updated_at":"2025-12-30T22:13:02.818Z","avatar_url":"https://github.com/norandom.png","language":"Python","readme":"# Observability k8s - speedy insights\n\n**Rapid dashboard development with Kubernetes, GitOps and AI-powered tools**\n\n![AI-Powered Dashboard Development](mydocs/output/ai-development-workflow.png)\n\n**🎬 Interactive Animation Demo: `mydocs/animated-workflow.html`**\n*Open locally to see the AI + Telepresence + Kubernetes workflow in action*\n\n## 🚀 Quick Start: Live Development\n\nExperience **true cloud-native development** - edit locally with your favorite AI tools while running against live Kubernetes data.\n\n### **Core Concept: Mount-and-Develop**\n```bash\n# 1. Mount Observable container filesystem locally via Telepresence\n./scripts/telepresence-observable-connect.sh intercept\n\n# 2. Edit files locally with Claude Code, Gemini CLI, or any IDE\ncode src/security.md  # Your changes sync automatically\n\n# 3. Test against live Kubernetes data instantly\ncurl http://localhost:3000/security  # See your changes immediately\n```\n\n**🎯 Perfect for**: Dashboard development, data analysis, real-time debugging with production-like data\n\n---\n\n## 📊 Dashboard Gallery\n\nExperience **Python data processing** with **Interactive JavaScript visualizations** running on live Kubernetes data.\n\n\u003cdiv align=\"center\"\u003e\n\n### 🏠 [Main Dashboard](docs/dashboards.md#main-dashboard) - System Overview\n[![Main Dashboard](screenshots/home.png)](screenshots/home.png)\n*Real-time system health, service links, and comprehensive metrics overview*\n\n### ⚙️ [Operations Dashboard](docs/dashboards.md#operations-dashboard) - System Monitoring  \n[![Operations Dashboard](screenshots/operations.png)](screenshots/operations.png)\n*Log volume analysis, service activity monitoring, and operational health insights*\n\n### 🛡️ [Security Dashboard](docs/dashboards.md#security-dashboard) - Threat Analysis\n[![Security Dashboard](screenshots/security.png)](screenshots/security.png)\n*Security event monitoring, threat detection, and forensic investigation tools*\n\n\u003c/div\u003e\n\n## 📋 Table of Contents\n\n### **🚀 Development Workflows**\n1. [🎬 Interactive Animation Demo](mydocs/animated-workflow.html) - **See the workflow in action**\n2. [Live Development Setup](#live-development-setup) - **Start here for hands-on development**\n3. [Telepresence Integration](#telepresence-integration) - **Mount remote containers locally**\n4. [AI Tools Integration](#ai-tools-integration) - **Claude Code + Gemini CLI workflows**\n5. [File Mounting Strategies](#file-mounting-strategies) - **Local ↔ Remote file sync**\n\n### **🏗️ Architecture \u0026 Deployment**\n6. [System Architecture](#system-architecture) - **Understanding the stack**\n7. [Architecture Documentation](docs/architecture.md) - **Complete system design**\n8. [GitHub Actions SSH Signing](docs/github-actions-ssh-signing.md) - **Automated workflow setup**\n\n### **📊 Data \u0026 APIs**\n9. [API Endpoints](docs/api-endpoints.md) - **Testing and development**\n10. [Complete Setup Guide](docs/setup.md) - **Full deployment instructions**\n\n### **📖 Documentation**\n11. [Example Usage](docs/examples.md) - **Step-by-step tutorials**\n\n---\n\n## 🚀 Live Development Setup\n\n### **Prerequisites**\n- **Kubernetes cluster** (tested with k3s, kind)\n- **Telepresence** installed locally\n- **AI tools**: Claude Code, Gemini CLI, or your preferred editor\n\n### **1. Quick Cluster Setup**\n```bash\n# Configure cluster IP\nvi config/cluster-config.env  # Set your cluster IP\n\n# Bootstrap entire stack\n./scripts/bootstrap-gitops.sh\n```\n\n### **2. Start Live Development**\n```bash\n# Method 1: Telepresence Intercept (Recommended)\n./scripts/telepresence-observable-connect.sh intercept\n# ✅ Routes cluster traffic to your local machine\n# ✅ Mount remote filesystem locally\n# ✅ Edit with any local tool\n\n# Method 2: Direct Container Access\n./scripts/observable-dashboard-manager.sh quick-edit index.md\n# ✅ Direct file editing in container\n# ✅ Hot reload on save\n```\n\n### **3. AI-Powered Development**\n```bash\n# Option A: Claude Code integration\ncode .  # Open project in VS Code with Claude Code extension\n# Edit files locally, changes sync to cluster automatically\n\n# Option B: Gemini CLI integration  \ngemini chat \"Help me create a security dashboard showing failed login attempts\"\n# Use AI suggestions, implement in local files\n```\n\n## 🔗 Telepresence Integration\n\n### **Advanced Development Workflows**\n\n#### **Traffic Interception**\n```bash\n# Intercept all Observable Framework traffic\n./scripts/telepresence-observable-connect.sh intercept\n\n# Now your local development server handles cluster requests\nnpm run dev  # Your local changes serve cluster traffic\n```\n\n#### **File System Mounting**\n```bash\n# Mount remote container filesystem locally\n./scripts/telepresence-observable-connect.sh local-dev\n\n# Remote files appear in local filesystem\nls -la /app/src/  # See container files locally\n```\n\n#### **Live Sync Workflows**\n```bash\n# Continuous sync mode\n./scripts/telepresence-observable-connect.sh sync\n\n# Edit locally ← AI tools integration\ncode src/security.md\n\n# Changes appear in cluster immediately\ncurl http://observable.k3s.local/security\n```\n\n### **Benefits of Telepresence Development**\n- ✅ **True local development** with cluster data\n- ✅ **AI tool integration** - use Claude Code, Gemini CLI locally\n- ✅ **Instant feedback** - see changes immediately\n- ✅ **Production-like environment** - real Kubernetes data\n- ✅ **Network access** - local tools can call cluster APIs\n\n## 🤖 AI Tools Integration\n\n### **Using Claude Code**\n```bash\n# 1. Start Telepresence intercept\n./scripts/telepresence-observable-connect.sh intercept\n\n# 2. Open project with Claude Code\ncode .\n\n# 3. Edit files with AI assistance\n# Claude Code can:\n# - Analyze live log data from cluster APIs\n# - Generate Observable Plot visualizations\n# - Create Python data loaders\n# - Build markdown dashboards\n```\n\n### **Using Gemini CLI**\n```bash\n# 1. Mount filesystem locally\n./scripts/telepresence-observable-connect.sh local-dev\n\n# 2. Use Gemini for development assistance\ngemini chat \"Create a Python script to analyze Quickwit security logs\"\n\n# 3. Implement suggestions in mounted files\n# Changes sync automatically to cluster\n```\n\n### **File Mounting + AI Workflow**\n```mermaid\ngraph LR\n    A[Local AI Tools\u003cbr/\u003eClaude Code/Gemini CLI] --\u003e B[Local Filesystem\u003cbr/\u003esrc/dashboards/]\n    B --\u003e C[Telepresence Sync] \n    C --\u003e D[Kubernetes Container\u003cbr/\u003e/app/src/]\n    D --\u003e E[Observable Framework\u003cbr/\u003eHot Reload]\n    E --\u003e F[Live Dashboard\u003cbr/\u003eobservable.k3s.local]\n```\n\n## 💾 File Mounting Strategies\n\n### **Strategy 1: Telepresence Volume Mount**\n```bash\n# Best for: Full-featured development with AI tools\n./scripts/telepresence-observable-connect.sh intercept\n\n# Local filesystem mirrors remote container\n/local/project/src/ ↔ /app/src/ (in container)\n```\n\n### **Strategy 2: kubectl cp + Watch**\n```bash\n# Best for: Quick edits and testing\n./scripts/observable-dashboard-manager.sh upload-file dashboard.md\n\n# Watch for changes and auto-sync\nfswatch -o src/ | xargs -n1 ./scripts/sync-to-cluster.sh\n```\n\n### **Strategy 3: Direct Container Development**\n```bash\n# Best for: Container-native development\nkubectl exec -it observable-pod -- /bin/bash\n\n# Use container tools directly\nconda activate observable\npython src/data/loki-logs.py\n```\n\n## 🏗️ System Architecture\n\n### **Complete System Overview**\n![System Architecture](mydocs/output/system-architecture.png)\n*Complete observability stack with GitOps, data processing, and development environment*\n\n### **Data Flow Architecture** \n![Data Flow](mydocs/output/data-flow-architecture.png)\n*Python + JavaScript hybrid architecture for real-time analytics*\n\n### **AI-Powered Development Workflow**\n![AI Development](mydocs/output/ai-development-workflow.png)\n*Telepresence integration with Claude Code and Gemini CLI for live development*\n\n### **CI/CD Pipeline**\n![CI/CD Pipeline](mydocs/output/cicd-pipeline.png)\n*Automated security scanning and deployment pipeline*\n\n### **Core Stack Components**\n- **🐍 Python Data Processing** - Real-time API integration with Loki/Quickwit\n- **📊 Observable Framework** - Interactive JavaScript visualizations\n- **🔄 Live Development** - Telepresence-powered local development\n- **🛡️ Security Focus** - Dedicated security log analysis and threat detection\n- **⚙️ Operations Monitoring** - System health and performance dashboards\n\n### **Development Environment Features**\n- **Live Data Integration** - Real Kubernetes log data\n- **AI Tool Support** - Claude Code, Gemini CLI integration\n- **Hot Reload** - Instant dashboard updates\n- **Remote Debugging** - Local tools with cluster data access\n- **GitOps Deployment** - Automated infrastructure management\n\n[All D2 diagrams →](mydocs/) | [Complete architecture documentation →](docs/architecture.md)\n\n## 🎯 Key Development Scenarios\n\n### **Scenario 1: Security Dashboard Development**\n```bash\n# 1. Start development environment\n./scripts/telepresence-observable-connect.sh intercept\n\n# 2. Create security dashboard with AI assistance\nclaude-code create src/security-advanced.md\n# AI helps analyze Quickwit data and create visualizations\n\n# 3. Test with live data\ncurl http://localhost:3000/security-advanced\n```\n\n### **Scenario 2: Custom Data Loader Development**\n```bash\n# 1. Mount container filesystem\n./scripts/telepresence-observable-connect.sh local-dev\n\n# 2. Develop Python loader locally\ngemini chat \"Help me create a data loader for authentication events\"\n# Edit src/data/auth-events.py locally\n\n# 3. Test against cluster APIs\npython src/data/auth-events.py  # Accesses live Quickwit API\n```\n\n### **Scenario 3: Real-time Dashboard Debugging**\n```bash\n# 1. Intercept traffic for debugging\n./scripts/telepresence-observable-connect.sh intercept\n\n# 2. Debug with local tools\nnpm run dev -- --inspect  # Node.js debugging\n# OR\npython -m pdb src/data/loki-logs.py  # Python debugging\n\n# 3. See results immediately in cluster\n```\n\n## 🔧 Development Tools Available\n\n### **Container Environment**\n- **Python 3.12+** with conda environment\n- **Node.js** with Observable Framework\n- **Git** for version control\n- **curl/wget** for API testing\n- **Standard Unix tools** (vi, nano, bash)\n\n### **Local Integration**\n- **Claude Code** - AI-powered development\n- **Gemini CLI** - Command-line AI assistance  \n- **VS Code** - Full IDE support via Telepresence\n- **Your favorite tools** - Any local editor works\n\n### **APIs Accessible in Development**\n- **Loki API**: `http://loki.k3s.local:3100` - Operational logs\n- **Quickwit API**: `http://quickwit.k3s.local:7280` - Security logs\n- **Prometheus**: `http://prometheus.k3s.local:9090` - Metrics\n- **Observable Framework**: `http://observable.k3s.local` - Dashboard serving\n\n## 📖 Next Steps\n\n1. **[Start developing →](docs/setup.md)** - Complete setup and first dashboard\n2. **[View examples →](docs/examples.md)** - Step-by-step tutorials  \n3. **[Explore APIs →](docs/api-endpoints.md)** - Data sources and integration\n4. **[Learn GitOps →](docs/gitops.md)** - Automated deployment workflows\n\n---\n\n**🎯 This environment provides the perfect blend of local development flexibility with production-like Kubernetes data access. Start developing immediately with your preferred AI tools while leveraging live cluster data!**\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnorandom%2Fobservability-k8s","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnorandom%2Fobservability-k8s","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnorandom%2Fobservability-k8s/lists"}