{"id":50419094,"url":"https://github.com/tobilg/latlng","last_synced_at":"2026-05-31T07:31:02.307Z","repository":{"id":357080549,"uuid":"1209339352","full_name":"tobilg/latlng","owner":"tobilg","description":null,"archived":false,"fork":false,"pushed_at":"2026-05-11T07:45:44.000Z","size":391,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-11T09:32:59.847Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tobilg.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-04-13T10:26:41.000Z","updated_at":"2026-05-11T07:45:35.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/tobilg/latlng","commit_stats":null,"previous_names":["tobilg/latlng"],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/tobilg/latlng","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tobilg%2Flatlng","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tobilg%2Flatlng/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tobilg%2Flatlng/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tobilg%2Flatlng/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tobilg","download_url":"https://codeload.github.com/tobilg/latlng/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tobilg%2Flatlng/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33723548,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-05-31T02:00:06.040Z","response_time":95,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-05-31T07:31:02.252Z","updated_at":"2026-05-31T07:31:02.300Z","avatar_url":"https://github.com/tobilg.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# latlng\n\n`latlng` is an open source geospatial object engine written in Rust. The repository contains a portable core, native server transports, pluggable storage backends, geofence and webhook eventing, a TypeScript SDK package, and a public browser wasm package built on the same engine.\n\n## Features\n\nImplemented:\n\n- portable Rust core for object storage, spatial indexing, search, metadata fields, and JSON subdocument updates\n- `INTERSECTS` supports real geometry clipping for `OBJECTS` output; non-object outputs ignore `clip`, and clipped results may normalize to GeoJSON\n- native HTTP/JSON server with shared auth, runtime config rewrite, Prometheus metrics, generated OpenAPI v3 docs, and admin endpoints\n- native WebSocket command/event transport with header auth, in-band `auth`, and async subscription streaming\n- native async Cap'n Proto RPC using generated schema bindings and `capnp-rpc`\n- native single-leader follower replication over Cap'n Proto streaming with checksum-based resume/resync\n- in-memory, append-only-file, and SQLite storage backends\n- geofencing with static and roaming geofences, `NODWELL`, channel subscriptions, and durable HTTP POST webhook delivery\n- wasm bindings for `latlng-core` and a browser Web Worker package for in-browser demos and local geospatial workloads\n\nIntentional scope boundaries:\n\n- the CLI covers common query and admin flows, but it is still not a full command-for-command shell\n- replication is native-only and intentionally scoped to single-leader follower mode rather than broader clustering/consensus\n\n## High-Level Architecture\n\nThe same engine is used in three shapes:\n\n1. Embedded Rust library: call `latlng-core` directly with any storage backend.\n2. Native server: expose the engine through HTTP, WebSocket, and Cap'n Proto.\n3. Browser wasm package: compile the portable crates to `wasm32-unknown-unknown` and run the in-memory engine behind a browser Web Worker API.\n\nThe native server keeps `Arc\u003cLatLngNative\u003cS\u003e\u003e` directly, routes request-style synchronous core work through a dedicated bounded native executor, and relies on a portable global control gate plus per-collection cells inside `latlng-core`. In practice that gives native parallel reads, steady-state collection-local concurrency, and explicit backpressure for core request execution without changing the single-threaded wasm behavior model.\n\nAt a high level, requests flow like this:\n\n```text\nHTTP / WebSocket / Cap'n Proto / browser wasm API\n                |\n                v\n         transport adapter\n                |\n                v\n            latlng-core\n         /       |       \\\n        v        v        v\n  latlng-index latlng-geofence latlng-storage\n        |                    /    |     \\\n        v                   v     v      v\n    latlng-geo          memory   AOF   SQLite\n```\n\nMutation events flow separately:\n\n```text\nlatlng-core mutation\n        |\n        v\n  primary log + geofence registry\n      /         |              \\\n     v          v               v\nchannel subs  WS/Capnp     durable webhook outbox\n  live only    streams      -\u003e SQLite queue -\u003e HTTP POST\n```\n\nThe detailed architecture is documented in [docs/architecture.md](docs/architecture.md).\nConfiguration, persistence, and release notes live in [docs/config.md](docs/config.md), [docs/persistence.md](docs/persistence.md), and [docs/release-checklist.md](docs/release-checklist.md).\n\n## Workspace Overview\n\n- `crates/latlng-auth`: shared bearer/JWT validation used by HTTP, WebSocket, and Cap'n Proto\n- `crates/latlng-config`: runtime config model plus JSON/TOML load/save helpers\n- `crates/latlng-platform`: portable lock and mailbox abstractions for native and wasm builds\n- `crates/latlng-geo`: geometry types, bounding boxes, geohash helpers, and JSON path utilities\n- `crates/latlng-index`: spatial index plus filtering, sorting, and output shaping\n- `crates/latlng-storage`: backend trait plus shared persistence contracts\n- `crates/latlng-core`: command engine, collection lifecycle, geofence registration, and server info/config\n- `crates/latlng-geofence`: geofence matching, roaming state, subscriptions, and event generation\n- `crates/latlng-storage-memory`: in-memory backend\n- `crates/latlng-storage-aof`: append-only file backend with compaction, integrity, backup, and restore support\n- `crates/latlng-storage-sqlite`: SQLite backend for embedded/native use\n- `crates/latlng-webhook-queue`: SQLite-backed durable webhook queue materialized from the primary log\n- `crates/latlng-schema`: Cap'n Proto schema plus generated Rust bindings\n- `crates/latlng-capnp`: async Cap'n Proto RPC transport\n- `crates/latlng-http`: HTTP/JSON transport built on `axum`\n- `crates/latlng-ws`: WebSocket event transport\n- `crates/latlng-endpoints`: webhook delivery helpers\n- `crates/latlng-replication`: follower state, replication client/coordinator, and checksum/chunk helpers\n- `crates/latlng-server`: runnable native server binary\n- `tools/latlng-cli`: operational CLI for common query and admin flows\n- `tools/latlng-benchmark`: benchmark harness for writes, queries, geofences, and webhook delivery\n- `tools/latlng-server-benchmark`: black-box localhost benchmark harness for the real `latlng-server` process\n- `packages/sdk`: TypeScript SDK for the HTTP and WebSocket server surfaces\n- `packages/wasm`: public browser-only Web Worker package around the wasm core for demos and local in-browser geospatial workloads\n- `packages/example-wasm`: static Vite site showcasing `@latlng/wasm` for Cloudflare Pages\n\n## Quickstart\n\nThe commands below assume the release binaries `latlng-server` and `latlng-cli`\nare installed and available on `PATH`. Install them with Homebrew, Docker, or a\nGitHub release archive as described below.\n\nStart the native server:\n\n```sh\nlatlng-server\n```\n\nBy default it listens on:\n\n- HTTP: `127.0.0.1:7421`\n\nCap'n Proto is disabled by default. Enable it when native Cap'n Proto clients or\nleader/follower replication are needed:\n\n```sh\nlatlng-server --capnp-enabled=true\n```\n\nThe native server also supports JSON or TOML config files:\n\n```sh\nlatlng-server --config ./latlng.json\n```\n\nThe canonical generated OpenAPI v3 document for the native HTTP server is available at:\n\n- `GET /api-docs`\n\nIt describes the stable native HTTP surface with typed request and response schemas. Diagnostic and replication-management routes are intentionally not part of the stable public API document. Release builds also attach the same generated document as `openapi.json`; locally it can be generated with `latlng-server --print-openapi` or `make openapi`.\n\nThe subscriber mailbox used for channel, WebSocket, Cap'n Proto, and webhook event delivery defaults to `4096` events per subscriber. You can override it with:\n\n```sh\nLATLNG_SUBSCRIBER_QUEUE_CAPACITY=8192 latlng-server\nlatlng-server --subscriber-queue-capacity 8192\n```\n\nThe dedicated native core executor defaults to one worker per available CPU and a bounded queue sized at `threads * 64`. You can override it with:\n\n```sh\nLATLNG_NATIVE_EXECUTOR_THREADS=8 LATLNG_NATIVE_EXECUTOR_QUEUE_LIMIT=512 latlng-server\nlatlng-server --native-executor-threads 8 --native-executor-queue-limit 512\n```\n\nWebhook HTTP delivery uses a per-request timeout that defaults to `5000ms`. You can override it with:\n\n```sh\nLATLNG_WEBHOOK_TIMEOUT_MS=10000 latlng-server\nlatlng-server --webhook-timeout-ms 10000\n```\n\nWebhook delivery concurrency is bounded and defaults to `128` in-flight HTTP deliveries. You can override it with:\n\n```sh\nLATLNG_WEBHOOK_CONCURRENCY_LIMIT=256 latlng-server\nlatlng-server --webhook-concurrency-limit 256\n```\n\nDurable webhook delivery also has queue and retry settings:\n\n```sh\nLATLNG_WEBHOOK_QUEUE_PATH=./data/webhooks.sqlite latlng-server\nLATLNG_WEBHOOK_RETRY_COUNT=8 latlng-server\nLATLNG_WEBHOOK_RETRY_INITIAL_BACKOFF_MS=200 latlng-server\nLATLNG_WEBHOOK_RETRY_MAX_BACKOFF_MS=30000 latlng-server\nLATLNG_WEBHOOK_LEASE_MS=30000 latlng-server\n```\n\nStore and query a point through the CLI:\n\n```sh\nlatlng-cli --base-url http://127.0.0.1:7421 set-point fleet truck-1 52.52 13.405\nlatlng-cli --base-url http://127.0.0.1:7421 get fleet truck-1\nlatlng-cli --base-url http://127.0.0.1:7421 nearby fleet 52.52 13.405 500\nlatlng-cli collection-create fleet\nlatlng-cli fset fleet truck-1 speed 42\nlatlng-cli fget fleet truck-1 speed\nlatlng-cli expire fleet truck-1 300\nlatlng-cli ttl fleet truck-1\nlatlng-cli jset fleet truck-1 properties.status active\nlatlng-cli jget fleet truck-1 properties.status\nlatlng-cli del fleet truck-1\nlatlng-cli timeout set 1.5\nlatlng-cli readonly yes\nlatlng-cli config-rewrite\n```\n\nHook and channel geofences can be created from GeoJSON files. The file may include `properties.collection`, `properties.detect`, `properties.commands`, and `properties.mode`; otherwise pass `--collection`, `--detect`, `--commands`, or `--mode` on the CLI.\n\n```sh\nlatlng-cli hook-set fleet-hook https://example.com/hook --geojson ./geofence.geojson --collection fleet\nlatlng-cli hooks\nlatlng-cli hook-get fleet-hook\nlatlng-cli channel-set fleet-channel --geojson ./geofence.geojson --collection fleet\nlatlng-cli channels\nlatlng-cli channel-del fleet-channel\n```\n\nInspect and maintain an offline AOF file:\n\n```sh\nlatlng-cli aof-verify ./data/appendonly.aof\nlatlng-cli aof-backup ./data/appendonly.aof ./backup/appendonly.backup.json\nlatlng-cli aof-restore ./backup/appendonly.backup.json ./restore/appendonly.aof\n```\n\nOr use plain HTTP:\n\n```sh\ncurl -sS -X POST http://127.0.0.1:7421/collections/fleet/objects/truck-1 \\\n  -H 'content-type: application/json' \\\n  -d '{\"object\":{\"Point\":{\"lat\":52.52,\"lon\":13.405,\"z\":null}}}'\n\ncurl -sS -X POST http://127.0.0.1:7421/collections/fleet/search/nearby \\\n  -H 'content-type: application/json' \\\n  -d '{\"lat\":52.52,\"lon\":13.405,\"meters\":500,\"options\":{}}'\n```\n\nOr use the TypeScript SDK:\n\n```sh\ncd packages/sdk\nnpm install\nnpm run build\n```\n\n```ts\nimport { LatLngClient, point } from \"@latlng/sdk\";\n\nconst client = new LatLngClient({\n  leaderUrl: \"http://127.0.0.1:7421\",\n  token: \"dev-token\",\n});\n\nawait client.setPoint(\"fleet\", \"truck-1\", { lat: 52.52, lon: 13.405 });\nconst object = await client.get(\"fleet\", \"truck-1\");\nconst nearby = await client.nearby(\"fleet\", {\n  lat: 52.52,\n  lon: 13.405,\n  meters: 500,\n});\n```\n\n## Docker\n\nThe repository includes:\n\n- a multi-stage production `Dockerfile`\n- a single-node [docker-compose.yml](docker-compose.yml)\n- a leader/follower [docker-compose.replication.yml](docker-compose.replication.yml)\n- sample mounted configs under [examples/docker](examples/docker)\n\nThe published image is config-file driven and starts with:\n`latlng-server --config /etc/latlng/latlng.toml`.\n\nContainer contract:\n\n| Purpose | Container value | Notes |\n| --- | --- | --- |\n| HTTP, WebSocket, metrics, and API traffic | port `7421` | publish with `-p 7421:7421` |\n| Cap'n Proto RPC and replication traffic | port `7422` | publish only when `capnp_enabled = true` or replication clients need host access |\n| Default config path | `/etc/latlng/latlng.toml` | mount TOML or JSON config here, or override the command / `LATLNG_CONFIG` |\n| Persistent data path | `/var/lib/latlng` | mount this when using AOF persistence or the durable webhook queue |\n| Runtime user | `latlng` | the image runs as a non-root user |\n\nContainer configs should bind to `0.0.0.0`, not `127.0.0.1`, when the port must be\nreachable through Docker port publishing. The sample configs already do this.\n\nBuild the image:\n\n```sh\ndocker build -t latlng-server .\n```\n\nPublished release images are available from Docker Hub as `tobilg/latlng:latest`\nand versioned tags such as `tobilg/latlng:v0.1.3`.\n\nRun a single node from the published image with a mounted config file and persistent\ndata volume:\n\n```sh\ndocker run --rm \\\n  --name latlng \\\n  -p 7421:7421 \\\n  -p 7422:7422 \\\n  -v \"$(pwd)/examples/docker/single-node.toml:/etc/latlng/latlng.toml:ro\" \\\n  -v latlng-data:/var/lib/latlng \\\n  tobilg/latlng:latest\n```\n\nThe bundled single-node example config uses:\n\n- AOF: `/var/lib/latlng/appendonly.aof`\n- webhook queue: `/var/lib/latlng/webhook-queue.sqlite`\n- bearer token: `dev-token`\n\nCheck the HTTP endpoint with the sample bearer token:\n\n```sh\ncurl -sS -H \"Authorization: Bearer dev-token\" http://127.0.0.1:7421/ping\n```\n\nFor an HTTP-only container, omit `-p 7422:7422` and set `capnp_enabled = false`\nin the mounted config.\n\nSingle-node compose:\n\n```sh\ndocker compose up --build\n```\n\nLeader/follower compose:\n\n```sh\ndocker compose -f docker-compose.replication.yml up --build\n```\n\nThat brings up:\n\n- leader HTTP on `127.0.0.1:7421`\n- leader Cap'n Proto on `127.0.0.1:7422`\n- follower HTTP on `127.0.0.1:17421`\n- follower Cap'n Proto on `127.0.0.1:17422`\n\nThe follower example config follows the leader through Docker DNS using:\n\n- `follow_host = \"latlng-leader\"`\n- `follow_port = 7422`\n- `replication_credential = \"replication-secret\"`\n\nIf you want to mount a different config path, either:\n\n- override the command:\n  `docker run ... latlng-server --config /some/other/path.toml`\n- or set `LATLNG_CONFIG=/some/other/path.toml`\n\nConfig precedence is unchanged in containers:\n\n- defaults\n- config file\n- environment variables\n- CLI flags\n\n## Homebrew\n\nmacOS arm64 release binaries are published to the `tobilg/latlng` Homebrew tap:\n\n```sh\nbrew tap tobilg/latlng\nbrew install latlng\n```\n\nThe formula installs `latlng-server` and `latlng-cli`. Its service uses AOF\npersistence by default and listens on `127.0.0.1:7421`.\n\n```sh\nbrew services start latlng\nbrew services stop latlng\n```\n\nDefault Homebrew paths:\n\n| Purpose | Path |\n| --- | --- |\n| Config file | `$(brew --prefix)/etc/latlng/latlng.toml` |\n| AOF and webhook queue data | `$(brew --prefix)/var/latlng` |\n| Service log | `$(brew --prefix)/var/log/latlng/latlng-server.log` |\n\nRun the server manually with the same defaults:\n\n```sh\nlatlng-server --config \"$(brew --prefix)/etc/latlng/latlng.toml\"\n```\n\n## Server Configuration\n\n`latlng-server` reads JSON or TOML config files via `--config` or `LATLNG_CONFIG`.\nThe complete server config option set is:\n\n| Name | Default | Description |\n| --- | --- | --- |\n| `production_mode` | `false` | Enables strict production startup guardrails. |\n| `listen_addr` | `\"127.0.0.1:7421\"` | HTTP listen address. |\n| `capnp_enabled` | `false` | Enables the Cap'n Proto RPC and replication listener. |\n| `capnp_listen_addr` | `\"127.0.0.1:7422\"` | Cap'n Proto listen address. |\n| `server_id` | `\"\u003cgenerated uuid\u003e\"` | Stable server identity used in replication status. |\n| `storage` | `\"memory\"` | Storage backend. Use memory or aof with a path. |\n| `read_only` | `false` | Rejects mutating commands when true. |\n| `command_timeouts` | `{}` | Per-command timeout overrides in seconds. |\n| `subscriber_queue_capacity` | `4096` | Per-subscriber event queue capacity. |\n| `webhook_queue_path` | `null` | SQLite webhook queue path. Defaults near the AOF or current directory. |\n| `webhook_timeout_ms` | `5000` | HTTP timeout for webhook deliveries. |\n| `webhook_concurrency_limit` | `128` | Maximum concurrent webhook delivery attempts. |\n| `webhook_retry_count` | `8` | Maximum webhook retry attempts before dead-lettering. |\n| `webhook_retry_initial_backoff_ms` | `200` | Initial webhook retry backoff. |\n| `webhook_retry_max_backoff_ms` | `30000` | Maximum webhook retry backoff. |\n| `webhook_lease_ms` | `30000` | Webhook job lease duration. |\n| `native_executor_threads` | `\u003cavailable CPU parallelism\u003e` | Native worker thread count for core operations. |\n| `native_executor_queue_limit` | `\u003cnative_executor_threads * 64\u003e` | Native executor queue limit. |\n| `aof_writer_queue_limit` | `4096` | AOF writer queue limit. |\n| `aof_group_commit_delay_ms` | `1` | Maximum AOF group commit delay. |\n| `aof_group_commit_max_requests` | `128` | Maximum requests per AOF commit cycle. |\n| `follow_host` | `null` | Leader host for follower replication. |\n| `follow_port` | `null` | Leader Cap'n Proto port for follower replication. |\n| `replication_credential` | `null` | Dedicated credential for replication streams. |\n| `replication_batch_size` | `512` | Maximum entries per replication stream response. |\n| `replication_reconnect_backoff_ms` | `1000` | Follower reconnect backoff after failures. |\n| `http_cors_enabled` | `false` | Enables HTTP CORS middleware. |\n| `http_cors_allowed_origins` | `[]` | Allowed CORS origins. Avoid `*` with auth. |\n| `http_cors_allowed_methods` | `[\"GET\",\"POST\",\"PUT\",\"DELETE\",\"OPTIONS\"]` | Allowed CORS methods. |\n| `http_cors_allowed_headers` | `[\"authorization\",\"content-type\",\"x-request-id\"]` | Allowed CORS headers. |\n| `http_cors_max_age_seconds` | `null` | Optional CORS preflight cache max-age. |\n| `http_max_body_bytes` | `10485760` | Maximum accepted HTTP request body size. |\n| `http_request_timeout_ms` | `30000` | Maximum HTTP request duration. |\n| `http_rate_limit_enabled` | `false` | Enables a simple global HTTP token-bucket rate limit. |\n| `http_rate_limit_requests_per_second` | `1000` | Global HTTP rate-limit refill rate. |\n| `http_rate_limit_burst` | `1000` | Global HTTP rate-limit burst capacity. |\n| `http_principal_rate_limit_enabled` | `false` | Enables per-principal HTTP token-bucket rate limiting. |\n| `http_principal_rate_limit_requests_per_second` | `100` | Per-principal HTTP rate-limit refill rate. |\n| `http_principal_rate_limit_burst` | `200` | Per-principal HTTP rate-limit burst capacity. |\n| `logging_enabled` | `true` | Enables structured server logging. |\n| `log_format` | `\"compact\"` | Log output format. Values: `compact`, `json`. |\n| `log_level` | `\"info\"` | Tracing filter level. |\n| `log_destination` | `\"stderr\"` | Log destination. Values: `stderr`, `stdout`, `file`, `none`. |\n| `log_file_path` | `null` | Required when log destination is `file`. |\n| `require_auth` | `false` | Rejects unauthenticated requests when true. |\n| `bearer_token` | `null` | Static full-admin bearer token. |\n| `disable_bearer_token` | `false` | Disables static bearer-token authentication even when configured. |\n| `jwt_secret` | `null` | HMAC JWT verification secret. |\n| `jwt_public_key_pem` | `null` | PEM public key for asymmetric JWT validation. |\n| `jwt_issuer` | `null` | Expected JWT issuer. |\n| `jwt_audience` | `null` | Expected JWT audience. |\n| `jwt_algorithm` | `null` | JWT algorithm override. |\n| `jwks_url` | `null` | JWKS endpoint URL. |\n| `jwks_provider_id` | `null` | Provider ID for logs/docs. |\n| `jwks_refresh_interval_seconds` | `300` | JWKS background refresh interval. |\n| `jwks_cache_ttl_seconds` | `3600` | JWKS cache TTL. |\n| `jwks_http_timeout_ms` | `3000` | JWKS HTTP request timeout. |\n| `jwt_leeway_seconds` | `0` | JWT clock-skew leeway. |\n\nUse `latlng-server --print-config-reference` or `latlng-cli config-reference` to inspect the machine-readable reference for the installed binary. Operational guidance and storage config shapes are documented in [docs/config.md](docs/config.md).\n\n## Transport And Auth\n\nHTTP:\n\n- implemented in `latlng-http`\n- supports static bearer token auth, HMAC JWTs, PEM-configured asymmetric JWTs, and JWKS-backed asymmetric JWTs\n- static bearer token remains a full-admin service/dev token unless `disable_bearer_token` is enabled\n- production guardrails can require an auth source with `require_auth`, `LATLNG_REQUIRE_AUTH=1`, or `--require-auth`\n- claims-based authz is collection-scoped and uses the `latlng_permissions` claim\n- `queries:read` and `subscriptions:read` are separate scopes\n- `/metrics` returns Prometheus text exposition and `metrics:read` is separate from `admin:*`\n- See [metrics.md](docs/metrics.md) for the Prometheus metric contract.\n- `latlng-server` currently serves plain HTTP, WebSocket, and Cap'n Proto; production deployments should terminate TLS at an upstream reverse proxy, load balancer, ingress, or service mesh\n- bearer/JWT credentials should only cross trusted networks or TLS-terminated paths\n\nWebSocket:\n\n- implemented in `latlng-ws`\n- supports `auth`, `subscribe`, `psubscribe`, `ping`, and `quit` command envelopes\n- accepts bearer/JWT auth during the upgrade path or through the first `auth` frame\n- enforces `subscriptions:read` separately from request/response query access\n- streams geofence events from the shared registry used by the other transports\n\nCap'n Proto:\n\n- implemented in `latlng-capnp`\n- uses generated schema bindings from `crates/latlng-schema/schema/latlng.capnp`\n- runs on real async `capnp-rpc`, not the previous blocking framed transport\n- uses session auth via the `auth(token)` RPC when bearer/JWT auth is enabled, then enforces the same action-level authz model as the native HTTP routes\n- `timeout`, `configRewrite`, `readonly`, and the other shipped admin RPCs route into the same runtime config model as HTTP\n- also exposes the internal native-only replication stream used by followers\n- disabled by default; enable with `capnp_enabled = true`, `LATLNG_CAPNP_ENABLED=true`, or `--capnp-enabled=true` when Cap'n Proto clients or replication are needed\n\nCLI:\n\n- uses typed `clap` subcommands with `--help` output for command documentation\n- automatically attaches `Authorization: Bearer ...` when `LATLNG_TOKEN` is set\n- can generate HMAC JWT secrets and scoped JWTs for local or self-hosted deployments\n- covers `ping`, `healthz`, `server`, `info`, `collections`, `metrics`, `bounds`, `stats`, `get`, `set-point`, `nearby`, `config-get`, `config-set`, `config-validate`, `config-reference`, `config-rewrite`, `readonly`, `timeout`, `aofshrink`, `aof-verify`, `aof-backup`, and `aof-restore`\n\nCreate a scoped HMAC JWT:\n\n```sh\nlatlng-cli token secret \u003e .latlng-jwt-secret\n```\n\n```toml\nrequire_auth = true\ndisable_bearer_token = true\njwt_secret = \"\u003ccontents of .latlng-jwt-secret\u003e\"\njwt_algorithm = \"HS256\"\njwt_issuer = \"https://id.example.com\"\njwt_audience = \"latlng\"\n```\n\n```sh\nTOKEN=\"$(latlng-cli token create \\\n  --config ./latlng.toml \\\n  --subject dashboard-1 \\\n  --ttl 24h \\\n  --preset dashboard \\\n  --collection 'fleet-*')\"\n\nLATLNG_TOKEN=\"$TOKEN\" latlng-cli collections\nlatlng-cli token verify \"$TOKEN\" --config ./latlng.toml\n```\n\nExternal IdPs can be integrated through JWKS. In that mode the IdP issues access tokens,\nand `latlng-server` verifies them with `jwt_issuer`, `jwt_audience`, `jwt_algorithm`,\nand `jwks_url`. The access token must include `latlng_permissions` or `latlng_admin`.\nSee [docs/auth.md](docs/auth.md#using-an-external-idp-with-jwks).\n\nFull auth/authz documentation, claim examples, and config reference:\n\n- [docs/auth.md](docs/auth.md)\n- [docs/config.md](docs/config.md)\n\nNative query execution:\n\n- `latlng-server` now enables the internal `parallel` query feature by default\n- large `NEARBY`, `WITHIN`, `INTERSECTS`, `SCAN`, and `SEARCH` queries snapshot only their prefiltered candidate set, then run native-only parallel candidate evaluation while preserving deterministic ordering and cursor behavior\n- wasm builds stay on the serial path and do not depend on rayon\n\n## Storage And Eventing\n\nStorage modes:\n\n- default: in-memory\n- AOF server mode: set `LATLNG_AOF_PATH=/path/to/latlng.aof`\n- SQLite: use `latlng-storage-sqlite` directly from embedded/native applications\n- JSON/TOML config files can also select storage mode and auth/runtime settings\n- `require_auth` fails startup when no bearer token or JWT verifier is configured\n- `subscriber_queue_capacity` controls the bounded per-subscriber event mailbox size and defaults to `4096`\n- `native_executor_threads` controls the number of dedicated native core worker threads and defaults to available CPU parallelism\n- `native_executor_queue_limit` controls the bounded native core submission queue and defaults to `native_executor_threads * 64`\n- `webhook_timeout_ms` controls the HTTP request timeout for outbound webhook delivery and defaults to `5000`\n- `webhook_concurrency_limit` controls the maximum number of concurrent outbound webhook deliveries and defaults to `128`\n- `webhook_queue_path` controls the SQLite materialized queue path used by the durable webhook outbox\n- `webhook_retry_count` defaults to `8` retries after the initial attempt\n- `webhook_retry_initial_backoff_ms` defaults to `200`\n- `webhook_retry_max_backoff_ms` defaults to `30000`\n- `webhook_lease_ms` defaults to `30000`\n- `aof_writer_queue_limit` controls the bounded submission queue for the AOF writer thread and defaults to `4096`\n- `aof_group_commit_delay_ms` controls how long the AOF writer waits to coalesce concurrent append requests and defaults to `1`\n- `aof_group_commit_max_requests` caps how many logical append requests can share one durable sync cycle and defaults to `128`\n- `server_id` uniquely identifies the node for replication self-checks and reconnect validation\n- `follow_host` / `follow_port` configure follower mode at startup\n- `replication_credential` configures the dedicated follower-to-leader authentication secret\n- `replication_batch_size` controls how many storage entries are fetched per replication chunk and defaults to `512`\n- `replication_reconnect_backoff_ms` controls follower reconnect delay and defaults to `1000`\n- `http_cors_enabled` enables HTTP CORS; keep it disabled unless browsers need direct access\n- `http_cors_allowed_origins`, `http_cors_allowed_methods`, `http_cors_allowed_headers`, and `http_cors_max_age_seconds` define the CORS policy\n- `http_rate_limit_enabled`, `http_rate_limit_requests_per_second`, and `http_rate_limit_burst` configure a process-global limiter for accidental overload protection\n- `http_principal_rate_limit_enabled`, `http_principal_rate_limit_requests_per_second`, and `http_principal_rate_limit_burst` configure per-principal HTTP buckets for JWT subjects, static bearer service traffic, open access, and anonymous/invalid requests\n- `logging_enabled`, `log_format`, `log_level`, `log_destination`, and `log_file_path` configure structured HTTP, WebSocket, and Cap'n Proto access logs\n- durable webhook recovery across restart requires a durable primary log, so use AOF-backed server storage for that guarantee\n\nYou can set the AOF tuning values in all three configuration layers:\n\n- config file fields:\n  - `aof_writer_queue_limit`\n  - `aof_group_commit_delay_ms`\n  - `aof_group_commit_max_requests`\n- env vars:\n  - `LATLNG_AOF_WRITER_QUEUE_LIMIT`\n  - `LATLNG_AOF_GROUP_COMMIT_DELAY_MS`\n  - `LATLNG_AOF_GROUP_COMMIT_MAX_REQUESTS`\n- CLI flags:\n  - `--aof-writer-queue-limit`\n  - `--aof-group-commit-delay-ms`\n  - `--aof-group-commit-max-requests`\n\nExample config:\n\n```toml\nlisten_addr = \"127.0.0.1:7421\"\ncapnp_enabled = false\ncapnp_listen_addr = \"127.0.0.1:7422\"\n\n[storage]\ntype = \"aof\"\npath = \"/var/lib/latlng/appendonly.aof\"\n\naof_writer_queue_limit = 4096\naof_group_commit_delay_ms = 1\naof_group_commit_max_requests = 128\n```\n\nEquivalent env/CLI overrides:\n\n```sh\nLATLNG_AOF_WRITER_QUEUE_LIMIT=4096 \\\nLATLNG_AOF_GROUP_COMMIT_DELAY_MS=1 \\\nLATLNG_AOF_GROUP_COMMIT_MAX_REQUESTS=128 \\\nlatlng-server \\\n  --aof /var/lib/latlng/appendonly.aof \\\n  --aof-writer-queue-limit 4096 \\\n  --aof-group-commit-delay-ms 1 \\\n  --aof-group-commit-max-requests 128\n```\n\nValidate a config before deployment:\n\n```sh\nlatlng-server --config /etc/latlng/server.toml --check-config\nlatlng-cli config-validate /etc/latlng/server.toml\nlatlng-cli config-reference\n```\n\nEnable browser CORS and JSON access logs:\n\n```toml\nhttp_cors_enabled = true\nhttp_cors_allowed_origins = [\"https://app.example.com\"]\nhttp_cors_allowed_methods = [\"GET\", \"POST\", \"PUT\", \"DELETE\", \"OPTIONS\"]\nhttp_cors_allowed_headers = [\"authorization\", \"content-type\", \"x-request-id\"]\nhttp_cors_max_age_seconds = 600\n\nlogging_enabled = true\nlog_format = \"json\"\nlog_level = \"info\"\nlog_destination = \"file\"\nlog_file_path = \"/var/log/latlng/server.log\"\n```\n\nGeofence and hook behavior:\n\n- channel geofences are registered inside the core engine and exposed through WebSocket and Cap'n Proto streams\n- hook and channel definitions are persisted in the primary log and replayed on restart\n- webhook enqueue intents, retries, acknowledgements, and dead-letter transitions are recorded in the primary log\n- mutating commands and their webhook enqueue intents are persisted in one atomic storage batch on durable backends\n- startup recovery applies primary-log entries incrementally as they are replayed instead of buffering the whole log first\n- `latlng-server` rebuilds a SQLite webhook queue from that log on startup and dispatches due jobs from the queue\n- replication is driven from committed storage entries, not the ephemeral channel/pubsub path\n- followers authenticate with a dedicated replication credential, verify leader identity, and resume from the local last sequence when checksum verification matches\n- checksum mismatch triggers a full local reset/resync from sequence `0`\n- followers are forced into read-only mode while following and reject normal reads until they have caught up once\n- followers do not deliver durable webhooks from replicated log records; webhook dispatch stays leader-local\n- WebSocket and Cap'n Proto subscription streams are wake-driven on native instead of using fixed poll intervals\n- the native webhook outbox is wake-driven too: new work, queue rebuilds, and due retry deadlines wake the dispatcher instead of a steady idle poll\n- request-style native HTTP, WebSocket, and Cap'n Proto core calls use the dedicated native executor; long-lived subscription bridges and background outbox work stay outside that pool\n- outbound webhook requests use the configured `webhook_timeout_ms` timeout and are processed concurrently up to `webhook_concurrency_limit`\n- failed deliveries use exponential backoff and become dead-lettered after `webhook_retry_count` retries\n- webhook delivery is `at-least-once`; payloads and headers include stable event/job IDs for receiver-side deduplication\n- roaming geofences, `ROAM`, and `NODWELL` logic are implemented in the portable geofence layer\n- WebSocket and Cap'n Proto event streams are exercised against the same live server in integration tests to keep payload parity honest\n- subscriber mailboxes are bounded, in-memory queues; when full they drop the oldest events\n- `FLUSHDB` is a full reset: it clears collections, channel geofences, webhook geofences, geofence state, and the durable webhook queue\n- live subscribers stay connected across `FLUSHDB`, but any buffered pre-flush events are discarded so post-flush streams only contain post-flush state\n\nAOF behavior:\n\n- complete but corrupt entries fail startup with a codec error\n- a truncated final AOF frame is ignored during replay so crash-tail recovery can still rebuild the valid prefix\n- when the truncated frame was a batched write, the whole batch is discarded rather than replaying a partial command-plus-webhook tail\n- concurrent AOF appends are funneled through a dedicated writer thread and can share one flush/sync cycle without changing the “success means durable” contract\n- logical compaction preserves current objects, active hooks/channels, and unresolved webhook jobs\n- offline integrity verification reports entry count, sequence range, durable prefix bytes, truncated-tail status, and checksum\n- offline backups are inspectable JSON files with version, source path, timestamp, sequence, and checksum metadata\n- restores refuse to overwrite an existing target unless `--force` is supplied\n\n## Performance Benchmarks\n\nTwo benchmark layers now exist:\n\n- `tools/latlng-benchmark`: in-process engine-level benchmarking for `latlng-core`\n- `tools/latlng-server-benchmark`: black-box localhost benchmarking for the real `latlng-server` binary over HTTP\n\nThe server benchmark tool is manual/local only for now. It is intentionally not wired into standard CI or nightly automation. Benchmark JSON outputs are written into the local `benchmark-results/` directory, which is intentionally gitignored.\nBenchmark binaries are local engineering tools and are intentionally not included in GitHub release binary archives. Release archives contain only `latlng-server` and `latlng-cli`; `openapi.json` is attached separately to GitHub Releases.\n\nBuild and run it with the Makefile entry points:\n\n```sh\nmake bench-server\nmake bench-server-capnp\nmake bench-server-aof\nmake bench-server-tile38\nmake bench-server-compare OLD=benchmark-results/bench-server-memory.json NEW=benchmark-results/bench-server-aof.json\nmake bench-server-compare-capnp OLD=benchmark-results/bench-server-memory.json NEW=benchmark-results/bench-server-capnp.json\nmake bench-server-compare-tile38 OLD=benchmark-results/bench-server-memory.json NEW=benchmark-results/bench-server-tile38.json\n```\n\nUseful overrides:\n\n```sh\nmake bench-server BENCH_FLAGS=\"--warmup-secs 1 --measure-secs 2 --seed-objects 1000 --startup-records 1000\"\nmake bench-server-capnp BENCH_FLAGS=\"--warmup-secs 1 --measure-secs 2 --seed-objects 1000\"\nmake bench-server-aof BENCH_FLAGS=\"--concurrency-list 8,32 --measure-secs 10\"\nmake bench-server-tile38 BENCH_FLAGS=\"--tile38-server-bin /usr/local/bin/tile38-server --scenario get_object_read\"\n```\n\nThe benchmark tool reports:\n\n- throughput in `ops/sec`\n- mean, `p50`, `p95`, and `p99` latency\n- error counts\n- AOF startup replay duration as a separate scenario\n\nLatlng runs default to HTTP; pass `--latlng-transport capnp` or use `make bench-server-capnp` to isolate protocol overhead from core engine work.\nTile38 runs use `tile38-server` by default and write a separate JSON file with `engine: \"tile38\"`.\nThe default Tile38 mode is in-memory-style `--appendonly no`; pass `--tile38-appendonly yes` when comparing Tile38 AOF behavior. The startup replay scenario is latlng-only and is skipped for Tile38 runs. The standard scenario set includes the geofence-heavy `fenced_set_point_write` case.\n\nInterpret the results as local engineering signals for before/after comparison, not as product SLA numbers.\n\n## Verification\n\n```sh\ncargo fmt --all\ncargo clippy --workspace --all-targets\ncargo test --workspace\ncargo test -p latlng-server --test server_smoke\ncargo check --target wasm32-unknown-unknown -p latlng-core --features wasm-bindings\ncd packages/sdk \u0026\u0026 npm ci \u0026\u0026 npm run typecheck \u0026\u0026 npm run build \u0026\u0026 npm run docs:api \u0026\u0026 npm run test\ncd packages/wasm \u0026\u0026 npm ci \u0026\u0026 npm run typecheck \u0026\u0026 npm run build \u0026\u0026 npm run test\ncd packages/example-wasm \u0026\u0026 npm ci \u0026\u0026 npm run typecheck \u0026\u0026 npm run build\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftobilg%2Flatlng","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftobilg%2Flatlng","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftobilg%2Flatlng/lists"}