{"id":50988373,"url":"https://github.com/cantrepro/tsink","last_synced_at":"2026-07-07T22:00:51.980Z","repository":{"id":314513527,"uuid":"1055820632","full_name":"cantrepro/tsink","owner":"cantrepro","description":"A lightweight time-series database written in Rust. Embed it, run it as a server, or scale it as a cluster. ","archived":false,"fork":false,"pushed_at":"2026-06-13T09:19:58.000Z","size":2441,"stargazers_count":312,"open_issues_count":0,"forks_count":10,"subscribers_count":3,"default_branch":"master","last_synced_at":"2026-07-05T11:21:12.239Z","etag":null,"topics":["database","embedded-database","rust","time-series","timeseries","timeseries-database","tsdb"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cantrepro.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"docs/security.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-09-12T21:29:24.000Z","updated_at":"2026-07-05T10:13:06.000Z","dependencies_parsed_at":"2025-09-16T12:03:50.216Z","dependency_job_id":null,"html_url":"https://github.com/cantrepro/tsink","commit_stats":null,"previous_names":["h2337/tsink","cantrepro/tsink"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/cantrepro/tsink","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cantrepro%2Ftsink","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cantrepro%2Ftsink/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cantrepro%2Ftsink/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cantrepro%2Ftsink/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cantrepro","download_url":"https://codeload.github.com/cantrepro/tsink/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cantrepro%2Ftsink/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35243953,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-07-07T02:00:07.222Z","response_time":90,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["database","embedded-database","rust","time-series","timeseries","timeseries-database","tsdb"],"created_at":"2026-06-19T23:00:32.391Z","updated_at":"2026-07-07T22:00:51.974Z","avatar_url":"https://github.com/cantrepro.png","language":"Rust","funding_links":[],"categories":["Rust"],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://raw.githubusercontent.com/h2337/tsink/refs/heads/master/logo.svg\" width=\"220\" height=\"100\" alt=\"tsink logo\"\u003e\u003cbr\u003e\n  A lightweight time-series database written in Rust.\u003cbr\u003e\n  Embed it, run it as a server, or scale it as a cluster.\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://crates.io/crates/tsink\"\u003e\u003cimg src=\"https://img.shields.io/crates/v/tsink.svg\" alt=\"crates.io\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://docs.rs/tsink/latest/tsink\"\u003e\u003cimg src=\"https://img.shields.io/docsrs/tsink.svg\" alt=\"docs.rs\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://pypi.org/project/tsink\"\u003e\u003cimg src=\"https://img.shields.io/pypi/v/tsink.svg\" alt=\"pypi.org\"\u003e\u003c/a\u003e\n  \u003ca href=\"LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/badge/license-MIT-blue.svg\" alt=\"MIT License\"\u003e\u003c/a\u003e\n\u003c/p\u003e\n\n---\n\n## Why tsink?\n\n- **Three deployment modes** — embed the library directly in your Rust or Python application, run a standalone server binary, or form a replicated cluster. Same engine everywhere.\n- **Robust engine** — segmented WAL with crash-safe sync, LSM-style leveled compaction, adaptive delta/XOR/zstd encoding, mmap zero-copy reads, and configurable memory backpressure.\n- **Tiered storage** — hot, warm, and cold tiers with automatic lifecycle management and optional object-store backing.\n- **Drop-in protocol support** — accepts Prometheus remote write/read, InfluxDB line protocol, OTLP, StatsD, and Graphite out of the box.\n- **Built-in PromQL** — query your data with a native PromQL parser and evaluator. No external query layer needed.\n- **Secure by default** — TLS (rustls, no OpenSSL), RBAC with OIDC, multi-tenant isolation, and mTLS between cluster nodes.\n- **Zero external dependencies at runtime** — single static binary for the server; `protoc` is vendored at build time.\n\n---\n\n## Deployment modes\n\n### Embedded library\n\nAdd `tsink` as a dependency and get a full time-series engine in-process — WAL durability, compaction, retention, and queries included.\n\n```rust\nuse tsink::{DataPoint, Row, StorageBuilder, TimestampPrecision};\n\nfn main() -\u003e Result\u003c(), Box\u003cdyn std::error::Error\u003e\u003e {\n    let storage = StorageBuilder::new()\n        .with_data_path(\"./tsink-data\")\n        .with_timestamp_precision(TimestampPrecision::Milliseconds)\n        .build()?;\n\n    storage.insert_rows(\u0026[\n        Row::new(\"cpu_usage\", DataPoint::new(1_700_000_000_000_i64, 42.0)),\n    ])?;\n\n    let points = storage.select(\"cpu_usage\", \u0026[], 1_700_000_000_000, 1_700_000_000_001)?;\n    println!(\"{points:?}\");\n\n    storage.close()?;\n    Ok(())\n}\n```\n\nUniFFI bindings expose the core API as a native Python module:\n\n```python\nfrom tsink import TsinkStorageBuilder, DataPoint, Row, Value\n\nbuilder = TsinkStorageBuilder()\nbuilder.with_data_path(\"./tsink-data\")\ndb = builder.build()\n\ndb.insert_rows([\n    Row(\n        metric=\"cpu_usage\",\n        labels=[],\n        data_point=DataPoint(timestamp=1_700_000_000_000, value=Value.F64(v=42.0)),\n    )\n])\nprint(db.select(\"cpu_usage\", [], 0, 2_000_000_000_000))\n```\n\n### Server mode\n\nA single binary that speaks every major metrics protocol.\n\n```bash\ncargo run -p tsink-server --bin tsink-server --release -- \\\n  --listen 127.0.0.1:9201 \\\n  --data-path ./var/tsink\n```\n\nWrite data with any client you already have:\n\n```bash\n# Prometheus text exposition\ncurl -X POST http://127.0.0.1:9201/api/v1/import/prometheus \\\n  -H 'Content-Type: text/plain' \\\n  -d 'http_requests_total{method=\"GET\"} 1027 1700000000000'\n\n# PromQL query\ncurl 'http://127.0.0.1:9201/api/v1/query?query=http_requests_total'\n```\n\n### Cluster mode\n\nEnable clustering with a flag and scale horizontally. tsink handles shard routing, replication, consistency, hinted handoff, repair, and rebalance automatically.\n\n```bash\ntsink-server \\\n  --listen 0.0.0.0:9201 \\\n  --data-path ./var/tsink \\\n  --cluster-enabled \\\n  --cluster-node-id node-1 \\\n  --cluster-bind 0.0.0.0:9211 \\\n  --cluster-replication-factor 3 \\\n  --cluster-seeds node-2:9212,node-3:9213\n```\n\n---\n\n## Storage engine\n\n| Capability | Details |\n|---|---|\n| **Durability** | Segmented WAL with configurable sync — per-append (crash-safe) or periodic (throughput-optimized). Strict or salvage replay on recovery. |\n| **Compaction** | LSM-style leveled compaction (L0 → L1 → L2) with tombstone-aware merging and atomic segment replacement. |\n| **Tiered storage** | Automatic hot → warm → cold lifecycle with configurable retention windows. Object-store backing for warm/cold tiers. |\n| **Encoding** | Adaptive timestamp codecs (fixed-step, delta-varint, delta-of-delta), Gorilla XOR float compression, and zstd for persisted segments. |\n| **Data types** | float64, bytes, and native Prometheus histograms. |\n| **Memory control** | Configurable memory budget with admission-based backpressure. Cardinality limits on unique series. |\n| **Reads** | mmap-based zero-copy segment reads. Downsampling, aggregation, and regex-capable label matchers built in. |\n\n---\n\n## Ingestion protocols\n\n| Protocol | Endpoint | Notes |\n|---|---|---|\n| Prometheus Remote Write | `POST /api/v1/write` | Snappy-framed protobuf |\n| Prometheus Remote Read | `POST /api/v1/read` | |\n| Prometheus Text Exposition | `POST /api/v1/import/prometheus` | Bulk import |\n| InfluxDB Line Protocol | `POST /write`, `POST /api/v2/write` | v1 and v2 compatible |\n| OTLP HTTP | `POST /v1/metrics` | Protobuf; gauges, sums, histograms, summaries |\n| StatsD | UDP (`--statsd-listen`) | Counter, gauge, timer, set |\n| Graphite | TCP (`--graphite-listen`) | Plaintext protocol |\n\n---\n\n## Clustering \u0026 replication\n\n- **Consistent hash-ring sharding** with configurable shard count\n- **Tunable replication factor** and consistency levels (One / Quorum / All) for writes and reads\n- **Node roles** — dedicated Storage, Query, or Hybrid nodes\n- **Hinted handoff** — queues writes for temporarily unavailable replicas\n- **Digest-based repair** — fingerprint exchange detects and resolves inconsistencies\n- **Online rebalance** — pause, resume, and monitor shard migration\n- **Distributed query fan-out** — concurrent shard-aware reads with merge limits\n- **Cluster-wide snapshots** — coordinated data + control-plane backup and restore\n- **Internal mTLS** — dedicated CA for peer-to-peer traffic\n\n---\n\n## Security \u0026 multi-tenancy\n\n- **TLS** — rustls-based with hot-reloadable certificates\n- **Authentication** — bearer tokens (file or exec-based loading), OIDC JWT validation (RS256, HS256)\n- **RBAC** — roles, service accounts with rotation, and live audit logging\n- **Multi-tenant isolation** — per-tenant policies for write rate, query concurrency, admission budgets, and retention\n- **Secret rotation** — runtime rotation of auth tokens, TLS certs, and mTLS materials with overlap grace periods\n\n---\n\n## Operations\n\n- **`/healthz`** and **`/ready`** — Kubernetes-compatible probes\n- **`/metrics`** — Prometheus-format self-instrumentation\n- **Recording \u0026 alerting rules** — built-in rules engine with configurable evaluation intervals\n- **Rollup policies** — persistent downsampled materialization with automated scheduling\n- **Migration tooling** — backfill, verify, and cutover from Prometheus, VictoriaMetrics, InfluxDB, OTLP, StatsD, and Graphite\n- **Support bundles** — bounded JSON diagnostic snapshots per tenant\n\n---\n\n## Documentation\n\n### Getting started\n\n- [Embedded library guide](docs/embedded-library.md) — using tsink as a Rust dependency, `StorageBuilder` configuration, sync and async APIs, snapshots\n- [Python bindings guide](docs/python-bindings.md) — UniFFI setup, `TsinkStorageBuilder`, type mappings, error handling\n- [Server deployment](docs/server-deployment.md) — running the single-node server binary, CLI flags, environment variables\n- [Cluster setup](docs/cluster-setup.md) — multi-node deployment, peer discovery, shard count, replication factor, consistency levels, node roles\n\n### Architecture \u0026 design\n\n- [Architecture overview](docs/architecture.md) — high-level system design, component interactions, data flow\n- [Storage engine internals](docs/storage-engine.md) — WAL, segments, LSM-style compaction, encoding codecs, mmap reads, write buffer\n- [PromQL implementation](docs/promql.md) — lexer, parser, evaluator, supported functions, aggregations, subqueries\n- [Clustering internals](docs/clustering-internals.md) — consistent hash ring, replication protocol, hinted handoff, digest repair, rebalance, distributed queries\n\n### API \u0026 protocol reference\n\n- [HTTP API reference](docs/http-api.md) — all endpoints, request/response formats, authentication headers, error codes\n- [PromQL reference](docs/promql-reference.md) — function catalogue, operators, vector matching, type coercion rules\n- [Ingestion protocols](docs/ingestion-protocols.md) — Prometheus remote write, InfluxDB line protocol, OTLP, StatsD, Graphite wire formats and endpoints\n- [Configuration reference](docs/configuration.md) — complete list of server, engine, cluster, and security options with defaults\n\n### Features\n\n- [Tiered storage](docs/tiered-storage.md) — hot/warm/cold lifecycle, retention windows, object-store backing\n- [Compaction](docs/compaction.md) — L0/L1/L2 levels, merge strategies, tombstone handling, tuning\n- [Rollups \u0026 downsampling](docs/rollups.md) — rollup policies, materialization scheduling, query integration\n- [Data types \u0026 native histograms](docs/data-types.md) — float64, bytes, native histograms, timestamp precision modes\n- [Exemplars](docs/exemplars.md) — exemplar storage, querying, cardinality limits\n\n### Security \u0026 operations\n\n- [Security model](docs/security.md) — TLS/mTLS setup, RBAC roles, OIDC authentication, audit logging\n- [Multi-tenancy](docs/multi-tenancy.md) — tenant isolation, per-tenant quotas, admission budgets, usage accounting\n- [Secret rotation](docs/secret-rotation.md) — rotating auth tokens, TLS certificates, mTLS materials, grace periods\n- [Monitoring \u0026 observability](docs/monitoring.md) — `/metrics` endpoint, self-instrumentation, health probes, support bundles\n- [Recording \u0026 alerting rules](docs/rules.md) — rule definitions, evaluation intervals, recording rule output\n- [Performance tuning](docs/performance-tuning.md) — memory budgets, compaction tuning, write pipelining, cgroup-aware scheduling\n- [Migration guide](docs/migration.md) — migrating from Prometheus, VictoriaMetrics, InfluxDB; backfill, verify, cutover\n\n---\n\n## License\n\nMIT — see [LICENSE](LICENSE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcantrepro%2Ftsink","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcantrepro%2Ftsink","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcantrepro%2Ftsink/lists"}