{"id":49354357,"url":"https://github.com/bravo1goingdark/blipmq","last_synced_at":"2026-04-27T12:06:24.312Z","repository":{"id":304621896,"uuid":"1018831650","full_name":"bravo1goingdark/blipmq","owner":"bravo1goingdark","description":"BlipMQ simplifies real-time communication between microservices and distributed systems with a lightweight, blazing-fast broker that just works — no complex setup, no compromises.","archived":false,"fork":false,"pushed_at":"2026-02-03T13:08:31.000Z","size":6125,"stargazers_count":20,"open_issues_count":0,"forks_count":1,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-02-04T03:42:30.582Z","etag":null,"topics":["broker","message-queue","mpsc","pub-sub-system","rust","spsc-queue","topics"],"latest_commit_sha":null,"homepage":"https://blipmq.dev","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bravo1goingdark.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"docs/SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-07-13T06:21:11.000Z","updated_at":"2026-02-03T13:09:34.000Z","dependencies_parsed_at":"2025-07-14T10:11:52.463Z","dependency_job_id":"c78d1e22-f8ff-432b-b478-a4f31ca0701a","html_url":"https://github.com/bravo1goingdark/blipmq","commit_stats":null,"previous_names":["bravo1goingdark/blipmq"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/bravo1goingdark/blipmq","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bravo1goingdark%2Fblipmq","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bravo1goingdark%2Fblipmq/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bravo1goingdark%2Fblipmq/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bravo1goingdark%2Fblipmq/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bravo1goingdark","download_url":"https://codeload.github.com/bravo1goingdark/blipmq/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bravo1goingdark%2Fblipmq/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32335383,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-26T23:26:28.701Z","status":"online","status_checked_at":"2026-04-27T02:00:06.769Z","response_time":128,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["broker","message-queue","mpsc","pub-sub-system","rust","spsc-queue","topics"],"created_at":"2026-04-27T12:06:23.488Z","updated_at":"2026-04-27T12:06:24.302Z","avatar_url":"https://github.com/bravo1goingdark.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n  \u003cimg src=\"./assets/readme-banner.png\" alt=\"BlipMQ Logo\" width=\"1200\" /\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003cb\u003eBlipMQ\u003c/b\u003e is a lightweight, high-performance message broker written in Rust, designed for durable pub/sub with QoS, WAL, and simple operational ergonomics.\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://blipmq.dev\"\u003e\u003cstrong\u003eWebsite\u003c/strong\u003e\u003c/a\u003e ·\n  \u003ca href=\"https://github.com/bravo1goingdark/blipmq\"\u003e\u003cstrong\u003eGitHub\u003c/strong\u003e\u003c/a\u003e ·\n  \u003ca href=\"https://x.com/blipmq\"\u003e\u003cstrong\u003eTwitter\u003c/strong\u003e\u003c/a\u003e ·\n  \u003ca href=\"https://linkedin.com/company/blipmq\"\u003e\u003cstrong\u003eLinkedIn\u003c/strong\u003e\u003c/a\u003e\n\u003c/p\u003e\n\n# BlipMQ (workspace)\n\nThis repository contains the Rust workspace used to build and evolve the core broker, daemon, and supporting crates.\n\n## Workspace layout\n\n- `blipmqd` – main broker daemon (TCP protocol, WAL, auth, metrics)\n- `core` – core broker logic (topics, queues, QoS, TTL, retry)\n- `net` – TCP server and binary framing\n- `wal` – write-ahead log (append-only, CRC-checked, indexable)\n- `auth` – static API key authentication\n- `metrics` – HTTP metrics endpoint\n- `config` – configuration loader (TOML/YAML + env merge)\n- `bench` – in-process performance benchmark harness\n\n## Running the daemon from source\n\nFrom the workspace root:\n\n```bash\ncargo run -p blipmqd -- --config ./config/blipmq-dev.toml\n```\n\nYou can also rely on the `CONFIG` environment variable instead of `--config`.\n\n## Configuration\n\n`blipmqd` uses `config` to load configuration from:\n\n- an optional TOML or YAML file, and\n- environment variables that override file defaults.\n\nExample minimal TOML config:\n\n```toml\nbind_addr = \"127.0.0.1\"\nport = 7878\n\nmetrics_addr = \"127.0.0.1\"\nmetrics_port = 9090\n\nwal_path = \"blipmq.wal\"\nfsync_policy = \"every_n:1\"        # or \"always\", \"none\", \"interval_ms:50\"\n\nmax_retries = 3\nretry_backoff_ms = 100\n\nallowed_api_keys = [\"dev-key-1\", \"dev-key-2\"]\n```\n\nEnvironment overrides (examples):\n\n- `BIND_ADDR`, `PORT`\n- `METRICS_ADDR`, `METRICS_PORT`\n- `WAL_PATH`, `FSYNC_POLICY`\n- `MAX_RETRIES`, `RETRY_BACKOFF_MS`\n- `ALLOWED_API_KEYS=\"key1,key2,...\"`\n\n## Metrics endpoint\n\n`metrics` exposes a simple HTTP endpoint suitable for Prometheus scraping or basic monitoring:\n\n- Path: `GET /metrics` on `metrics_addr:metrics_port`\n- Response (plain text), for example:\n\n```text\ntopics \u003cn\u003e\nsubscribers \u003cn\u003e\nmessages_published_total \u003cn\u003e\nmessages_delivered_total \u003cn\u003e\nmessages_inflight \u003cn\u003e\nwal_appends_total \u003cn\u003e\nwal_bytes_total \u003cn\u003e\n```\n\nThese values are provided by `corelib::Broker` and `wal::WriteAheadLog`.\n\n## WAL and durability\n\n`wal` implements an append-only log with:\n\n- fixed header and `[id][len][crc32][payload...]` record layout,\n- CRC32 validation for corruption detection,\n- in-memory id→offset index with rebuild on startup,\n- configurable fsync policy via `WalConfig`:\n  - `fsync_every_n`,\n  - `fsync_interval`.\n\n`blipmqd` uses the WAL to:\n\n- durably record QoS1 publishes before fan-out, and\n- replay messages on startup to recover durable topics and queues.\n\n## QoS, TTL, and retry\n\n`core` supports:\n\n- QoS 0 (at-most-once) and QoS 1 (at-least-once),\n- per-message metadata:\n  - `created_at`,\n  - optional TTL,\n  - `delivery_attempts`,\n  - `next_delivery_at`,\n- periodic maintenance:\n  - drops expired messages (TTL),\n  - reschedules unacked QoS1 messages with exponential backoff,\n  - stops after a configurable `max_retries`.\n\n`blipmqd` runs a Tokio-based maintenance loop that calls `Broker::maintenance_tick` on an interval and respects graceful shutdown.\n\n## Graceful shutdown\n\nOn SIGINT/SIGTERM (Ctrl+C), `blipmqd`:\n\n- marks the broker as shutting down (no new publishes),\n- signals the network server and maintenance tasks to stop,\n- waits up to a bounded timeout for in-flight QoS1 messages to drain,\n- flushes the WAL, then exits.\n\nThis provides at-least-once semantics for QoS1 across restarts while avoiding abrupt loss of in-flight messages during orderly shutdowns.\n\n## Benchmark harness (`bench`)\n\nThe `bench` crate provides an in-process benchmark that exercises the `Broker` API directly (with optional WAL) to measure basic throughput and latency:\n\n```bash\ncargo run -p bench -- \\\n  --publishers 4 \\\n  --subscribers 4 \\\n  --topics 8 \\\n  --messages 50000 \\\n  --msg-size 256 \\\n  --qos 1 \\\n  --wal\n```\n\nOptions:\n\n- `--publishers` – number of concurrent publisher tasks\n- `--subscribers` – number of subscriber tasks\n- `--topics` – number of topics used in round-robin\n- `--messages` – messages per publisher\n- `--msg-size` – payload size in bytes (must be ≥ 8)\n- `--qos` – QoS mode (`0` = at-most-once, `1` = at-least-once)\n- `--wal` – enable durable publishes via WAL\n\nThe harness reports:\n\n- total messages received,\n- throughput (msgs/sec),\n- p50 / p95 / p99 latency (µs).\n\nIt is intended as a sanity-check for high-performance behavior rather than a formal benchmark suite.\n\n## Profiling and flamegraphs\n\nYou can profile `blipmqd` using standard Linux tools and `flamegraph`:\n\n```bash\ncargo install flamegraph\n\n# Build a release binary\ncargo build -p blipmqd --release\n\n# Record perf data (may require sudo)\nsudo perf record -F 99 -g -- target/release/blipmqd -- --config ./config/blipmq-dev.toml\n\n# Generate a flamegraph\nsudo flamegraph\n```\n\nHot paths such as publish, frame encode/decode, and WAL append are annotated with `#[inline(always)]` and tracing spans so that perf and `tokio-console` can attribute time accurately.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbravo1goingdark%2Fblipmq","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbravo1goingdark%2Fblipmq","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbravo1goingdark%2Fblipmq/lists"}