https://github.com/jakthom/hercules
A DuckDB-powered Prometheus exporter
https://github.com/jakthom/hercules
duckdb metrics monitoring observability observability-data prometheus prometheus-exporter prometheus-metrics prometheus-utility
Last synced: 2 months ago
JSON representation
A DuckDB-powered Prometheus exporter
- Host: GitHub
- URL: https://github.com/jakthom/hercules
- Owner: jakthom
- License: mit
- Created: 2024-10-16T15:41:38.000Z (8 months ago)
- Default Branch: main
- Last Pushed: 2024-12-05T05:27:07.000Z (6 months ago)
- Last Synced: 2025-03-28T20:46:21.417Z (3 months ago)
- Topics: duckdb, metrics, monitoring, observability, observability-data, prometheus, prometheus-exporter, prometheus-metrics, prometheus-utility
- Language: Go
- Homepage:
- Size: 6.78 MB
- Stars: 38
- Watchers: 4
- Forks: 2
- Open Issues: 13
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
Hercules
Write SQL. Get Prometheus Metrics.
[](https://opensource.org/licenses/MIT)
[](https://github.com/jakthom/hercules/actions/workflows/test.yml)
[](https://github.com/jakthom/hercules/actions/workflows/lint.yml)
# ⚡ Quickstart
Launching Hercules in a Codespace is the easiest way to get started.
A `make run` from inside your codespace will get things spinning.
[](https://github.com/codespaces/new?hide_repo_select=true&ref=main&repo=873715049)
# Sources
Hercules generates Prometheus metrics by querying:
- **Local files** (parquet, json, csv, xlsx, etc)
- **Object storage** (GCS, S3, Azure Blob)
- **HTTP endpoints**
- **Databases** (PostgreSQL, MySQL, SQLite)
- **Data lakes** (Iceberg, Delta)
- **Data warehouses** (BigQuery)
- **Arrow IPC buffers**
Sources can be cached and periodically refreshed, or act as views to the underlying data.
Metrics from multiple sources can be materialized using a single exporter.
# Metrics
### Definition
Metric definitions are `yml` and use `sql` in a number of supported dialects to aggregate, enrich, and materialize metric values.
Hercules supports Prometheus **gauges, counters, summaries, and histograms.**
### Enrichment
Sources and metrics can be *externally enriched*, leading to more ***thorough***, ***accurate*** (or is it precise?), ***properly-labeled*** metrics.
Integrate, calculate, enrich, and label on the edge.
# Macros
Metric definitions can be kept DRY using SQL macros.
Macros are useful for:
- Parsing log lines
- Reading useragent strings
- Schematizing unstructured data
- Enriching results with third-party tooling
- Tokenizing attributes
# Labels
Hercules propagates global labels to all configured metrics. So you don't have to guess where a metric came from.
Labels are propagated from configuration or sourced from environment variables.
# Packages
Hercules extensions, sources, metrics, and macros can be logically grouped and distributed by the use of **packages**.
Examples can be found in the [hercules-packages](/hercules-packages/) directory.
# Bonus
- Calculate prometheus-compatible metrics from geospatial data
- Coerce unwieldy files to useful statistics using full-text search
- Use modern [pipe sql syntax](https://research.google/pubs/sql-has-problems-we-can-fix-them-pipe-syntax-in-sql/) or [prql](https://prql-lang.org/) for defining and transforming your metrics
- You don't need to [start queries with `select`](https://jvns.ca/blog/2019/10/03/sql-queries-don-t-start-with-select/).
# Further Resources
More to come.