https://github.com/angelerator/nilalytics
Batteries-included, OpenTelemetry-native web & mobile product analytics on DuckLake + your object storage (S3/GCS/R2/Azure). No warehouse, no per-event fees.
https://github.com/angelerator/nilalytics
analytics duckdb ducklake lakehouse mobile-analytics object-storage observability opentelemetry otlp product-analytics quack self-hosted web-analytics
Last synced: 11 days ago
JSON representation
Batteries-included, OpenTelemetry-native web & mobile product analytics on DuckLake + your object storage (S3/GCS/R2/Azure). No warehouse, no per-event fees.
- Host: GitHub
- URL: https://github.com/angelerator/nilalytics
- Owner: Angelerator
- License: apache-2.0
- Created: 2026-07-03T13:09:55.000Z (17 days ago)
- Default Branch: main
- Last Pushed: 2026-07-06T07:27:44.000Z (15 days ago)
- Last Synced: 2026-07-08T19:48:09.832Z (12 days ago)
- Topics: analytics, duckdb, ducklake, lakehouse, mobile-analytics, object-storage, observability, opentelemetry, otlp, product-analytics, quack, self-hosted, web-analytics
- Language: Python
- Homepage: https://angelerator.github.io/nilalytics/
- Size: 92.8 KB
- Stars: 1
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
- Security: docs/security.md
Awesome Lists containing this project
README
# nilalytics
[](https://pypi.org/project/nilalytics/)
[](https://pypi.org/project/nilalytics/)
[](LICENSE)
[](https://angelerator.github.io/nilalytics/)
**The batteries‑included, OpenTelemetry‑native way to run your own web & mobile product analytics — on a DuckLake in your object storage.**
nilalytics collects product events, errors, and performance data over
[OpenTelemetry](https://opentelemetry.io/) (OTLP), stores them in a
[DuckLake](https://ducklake.select/) lakehouse on any major cloud's object
storage, and serves sub‑second reads over DuckDB's [Quack](https://duckdb.org/docs/stable/core_extensions/quack) protocol — no warehouse, no per‑event fees, no data leaving your infrastructure.
- **Web, mobile _and_ backends:** Grafana Faro (web), OpenTelemetry SDKs (mobile + server) → OTLP. Record user actions *and* whether the backend calls behind them succeeded — in one lake.
- **Runs on any cloud:** S3, MinIO, Google Cloud Storage, Cloudflare R2, or Azure / ADLS Gen2.
- **Recommendation‑ready:** a curated per‑user `user_events` table, partitioned **subject › date › person**, so "everything this user did" is one fast read. Look anyone up by email / account id / phone.
- **Subjects & AI usage:** every event is classified — `errors` · `activities` · `ai_usage` · `traceability` — so category reads and LLM/token tracking prune to their own partition.
- **Small files solved:** DuckLake data inlining + compaction keep streaming writes fast and files healthy.
- **Bounded storage:** opt‑in data retention drops old events (with a read‑only dry‑run preview).
- **Batteries included:** funnels, retention, errors, traces, metrics, cross‑device identity.
- **Secure by default:** token‑authenticated ingest, read‑only query authz, a hardened public gateway.
📖 **Full documentation:**
## Install
```bash
pip install nilalytics
```
> Also works with `pipx install nilalytics` or `uv tool install nilalytics`.
> Bleeding edge from source: `pip install git+https://github.com/Angelerator/nilalytics`.
## 60‑second local demo
```bash
# 1. object storage (local MinIO)
minio server .minio-data --address 127.0.0.1:9100 --console-address 127.0.0.1:9101 &
mc alias set nila http://127.0.0.1:9100 minioadmin minioadmin
mc mb --ignore-existing nila/nilalytics
# 2. run the pipeline
nilalytics server & # ingest + Quack catalog
nilalytics gateway & # public ingest gateway (CORS + tokens)
# 3. send + query
nilalytics emit --count 200 --persons 5
nilalytics query report
nilalytics query subject ai_usage 7 # per-subject read (last 7 days)
nilalytics query user --key person0@example.com 3 # one person's activity (last 3 days)
```
## Architecture
```mermaid
flowchart LR
W["Web · Grafana Faro"] -->|OTLP + token| GW
M["Mobile · OpenTelemetry"] -->|OTLP + token| GW
GW["Ingest Gateway
CORS · tokens · TLS"] --> SRV["OTLP server
(duckdb-otlp)"]
B["Backend services · OpenTelemetry"] -->|"OTLP · private net"| SRV
SRV --> LAKE[("DuckLake
DuckDB + Quack catalog
+ Parquet on object storage")]
LAKE --> READ["Reads: DuckDB / DuckDB-WASM
over Quack"]
```
## Acknowledgements
nilalytics stands on the shoulders of giants. Huge thanks to
[DuckDB](https://duckdb.org/), [DuckLake](https://ducklake.select/) and Quack (DuckDB Labs / DuckDB Foundation),
[duckdb‑otlp](https://github.com/smithclay/duckdb-otlp) ([@smithclay](https://github.com/smithclay)),
[OpenTelemetry](https://opentelemetry.io/) (CNCF),
[Grafana Faro](https://github.com/grafana/faro-web-sdk) (Grafana Labs),
and [MinIO](https://min.io/) — plus the projects that inspired it:
[canardstack](https://github.com/smithclay/canardstack),
[icelight](https://github.com/cliftonc/icelight), and
[stratif.io](https://stratif.io/).
Full credits and dependency list: [Acknowledgements](https://angelerator.github.io/nilalytics/acknowledgements/).
## License
[Apache‑2.0](LICENSE) © Angelerator