{"id":21891273,"url":"https://github.com/jakthom/hercules","last_synced_at":"2025-04-15T13:53:13.604Z","repository":{"id":260454332,"uuid":"873715049","full_name":"jakthom/hercules","owner":"jakthom","description":"A DuckDB-powered Prometheus exporter","archived":false,"fork":false,"pushed_at":"2024-12-05T05:27:07.000Z","size":7110,"stargazers_count":38,"open_issues_count":13,"forks_count":2,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-03-28T20:46:21.417Z","etag":null,"topics":["duckdb","metrics","monitoring","observability","observability-data","prometheus","prometheus-exporter","prometheus-metrics","prometheus-utility"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jakthom.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-10-16T15:41:38.000Z","updated_at":"2025-03-28T10:01:07.000Z","dependencies_parsed_at":null,"dependency_job_id":"16b1dd9b-49c3-4ae2-a247-af9b097f9541","html_url":"https://github.com/jakthom/hercules","commit_stats":null,"previous_names":["jakthom/hercules"],"tags_count":6,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jakthom%2Fhercules","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jakthom%2Fhercules/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jakthom%2Fhercules/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jakthom%2Fhercules/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jakthom","download_url":"https://codeload.github.com/jakthom/hercules/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249085474,"owners_count":21210267,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["duckdb","metrics","monitoring","observability","observability-data","prometheus","prometheus-exporter","prometheus-metrics","prometheus-utility"],"created_at":"2024-11-28T12:24:23.088Z","updated_at":"2025-04-15T13:53:13.586Z","avatar_url":"https://github.com/jakthom.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003ch1 align=\"center\"\u003e\n  \u003cimg src=\"./assets/hercules.png\" alt=\"Hercules\" width=\"20%\" style=\"border-radius: 25%\"\u003e\n  \u003cbr\u003e\n  Hercules\n  \u003cbr\u003e\n\u003c/h1\u003e\n\n\u003ch4 align=\"center\"\u003e Write SQL. Get Prometheus Metrics.\u003c/h4\u003e\n\n\u003cdiv align=\"center\"\u003e\n\n![GitHub go.mod Go version](https://img.shields.io/github/go-mod/go-version/jakthom/hercules)\n[![License](https://img.shields.io/badge/License-MIT-blue.svg)](https://opensource.org/licenses/MIT)\n[![test](https://github.com/jakthom/hercules/actions/workflows/test.yml/badge.svg)](https://github.com/jakthom/hercules/actions/workflows/test.yml)\n[![lint](https://github.com/jakthom/hercules/actions/workflows/lint.yml/badge.svg)](https://github.com/jakthom/hercules/actions/workflows/lint.yml)\n\u003c/div\u003e\n\n\n\n# ⚡ Quickstart\n\nLaunching Hercules in a Codespace is the easiest way to get started.\n\nA `make run` from inside your codespace will get things spinning.\n\n[![Launch GitHub Codespace](https://github.com/codespaces/badge.svg)](https://github.com/codespaces/new?hide_repo_select=true\u0026ref=main\u0026repo=873715049)\n\n\n\n\n# Sources\n\nHercules generates Prometheus metrics by querying:\n\n- **Local files** (parquet, json, csv, xlsx, etc)\n- **Object storage** (GCS, S3, Azure Blob)\n- **HTTP endpoints**\n- **Databases** (PostgreSQL, MySQL, SQLite)\n- **Data lakes** (Iceberg, Delta)\n- **Data warehouses** (BigQuery)\n- **Arrow IPC buffers**\n\nSources can be cached and periodically refreshed, or act as views to the underlying data.\n\nMetrics from multiple sources can be materialized using a single exporter.\n\n\n# Metrics\n\n### Definition\n\nMetric definitions are `yml` and use `sql` in a number of supported dialects to aggregate, enrich, and materialize metric values.\n\n\nHercules supports Prometheus **gauges, counters, summaries, and histograms.**\n\n### Enrichment\n\nSources and metrics can be *externally enriched*, leading to more ***thorough***, ***accurate*** (or is it precise?), ***properly-labeled*** metrics.\n\nIntegrate, calculate, enrich, and label on the edge.\n\n\n\n# Macros\n\nMetric definitions can be kept DRY using SQL macros.\n\nMacros are useful for:\n\n- Parsing log lines\n- Reading useragent strings\n- Schematizing unstructured data\n- Enriching results with third-party tooling\n- Tokenizing attributes\n\n\n#  Labels\n\nHercules propagates global labels to all configured metrics. So you don't have to guess where a metric came from.\n\nLabels are propagated from configuration or sourced from environment variables.\n\n\n# Packages\n\nHercules extensions, sources, metrics, and macros can be logically grouped and distributed by the use of **packages**.\n\nExamples can be found in the [hercules-packages](/hercules-packages/) directory.\n\n\n# Bonus\n\n- Calculate prometheus-compatible metrics from geospatial data\n- Coerce unwieldy files to useful statistics using full-text search\n- Use modern [pipe sql syntax](https://research.google/pubs/sql-has-problems-we-can-fix-them-pipe-syntax-in-sql/) or [prql](https://prql-lang.org/) for defining and transforming your metrics\n- You don't need to [start queries with `select`](https://jvns.ca/blog/2019/10/03/sql-queries-don-t-start-with-select/).\n\n\n# Further Resources\n\nMore to come.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjakthom%2Fhercules","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjakthom%2Fhercules","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjakthom%2Fhercules/lists"}