{"id":50318736,"url":"https://github.com/josh/jmap2nats","last_synced_at":"2026-05-29T02:02:51.463Z","repository":{"id":358715479,"uuid":"1241885694","full_name":"josh/jmap2nats","owner":"josh","description":"Bridge JMAP email push events to NATS JetStream","archived":false,"fork":false,"pushed_at":"2026-05-18T19:28:02.000Z","size":46,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-18T20:34:44.837Z","etag":null,"topics":["jmap","nats"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/josh.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-05-17T23:46:42.000Z","updated_at":"2026-05-18T19:28:01.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/josh/jmap2nats","commit_stats":null,"previous_names":["josh/jmap2nats"],"tags_count":4,"template":false,"template_full_name":null,"purl":"pkg:github/josh/jmap2nats","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/josh%2Fjmap2nats","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/josh%2Fjmap2nats/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/josh%2Fjmap2nats/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/josh%2Fjmap2nats/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/josh","download_url":"https://codeload.github.com/josh/jmap2nats/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/josh%2Fjmap2nats/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33633468,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-05-29T02:00:06.066Z","response_time":107,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["jmap","nats"],"created_at":"2026-05-29T02:02:48.119Z","updated_at":"2026-05-29T02:02:51.453Z","avatar_url":"https://github.com/josh.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# jmap2nats\n\nA small Go service that subscribes to a JMAP email server's push stream\nand republishes newly-arrived emails onto a NATS JetStream stream.\n\n- **Real-time** via JMAP's `EventSource` (server-sent events). No polling.\n- **HA-safe**: run multiple replicas — every publish carries a `Nats-Msg-Id`\n  set to the JMAP email id, so concurrent publishes from different replicas\n  are deduplicated server-side within the stream's `DuplicateWindow`.\n- **Outage-tolerant**: on boot the service finds the most-recent email\n  already in the stream and re-checks any newer JMAP ids up to\n  `backfill_limit`. Messages are missed only if more than `backfill_limit`\n  arrived during downtime (default 100).\n- **JMAP-faithful** payload: the NATS message body is the JMAP `Email`\n  object as JSON (RFC 8621 field names), with body/attachment bytes\n  externalised to an Object Store and referenced by key.\n\nWorks with any JMAP-compatible mail server. [Fastmail][fastmail] is\nstraightforward to use; create an API token from the Fastmail account\nsettings.\n\n[fastmail]: https://www.fastmail.com/\n\n## Install\n\n```sh\ngo install github.com/josh/jmap2nats@latest\n```\n\nOr clone and `go build .` in the working directory.\n\nYou'll need a reachable NATS server with JetStream enabled. The official\n`nats-server` works out of the box; start it locally with\n`nats-server -js`.\n\n## Configuration\n\nA single JSON file. Path resolution, in order:\n\n1. `-config \u003cpath\u003e` flag.\n2. `JMAP2NATS_CONFIG` env var.\n3. `./jmap2nats.json` in the working directory.\n\n`jmap2nats -print-config` dumps the merged (defaults + your overrides)\nconfig and exits — useful as a starting template. Pass `-verbose` to enable\ndebug-level logging.\n\nExample `jmap2nats.json`:\n\n```json\n{\n  \"jmap\": {\n    \"session_url\": \"https://api.fastmail.com/jmap/session\",\n    \"token_file\": \"/etc/jmap2nats/token\"\n  },\n  \"nats\": {\n    \"url\": \"nats://localhost:4222\"\n  },\n  \"stream\": {\n    \"name\": \"JMAP_EMAILS\",\n    \"subject_prefix\": \"jmap.email\",\n    \"max_age\": \"168h\",\n    \"max_bytes\": \"64MiB\",\n    \"dedup_window\": \"24h\"\n  },\n  \"parts\": {\n    \"bucket\": \"email-parts\",\n    \"max_bytes\": \"960MiB\",\n    \"max_per_part\": \"25MiB\"\n  },\n  \"backfill_limit\": 100\n}\n```\n\n| JSON path                   | Default                 | Notes                                                                                                                                       |\n| --------------------------- | ----------------------- | ------------------------------------------------------------------------------------------------------------------------------------------- |\n| `jmap.session_url`          | (required)              | e.g. `https://api.fastmail.com/jmap/session`                                                                                                |\n| `jmap.token_file`           | (required)              | Path to a file containing the bearer token. Trailing whitespace is trimmed. Keep mode 0400.                                                 |\n| `jmap.account_id`           | primary mail account    | Override if not the session primary.                                                                                                        |\n| `nats.url`                  | `nats://localhost:4222` |                                                                                                                                             |\n| `nats.creds`                | unset                   | Path to a NATS creds file (NGS etc).                                                                                                        |\n| `stream.name`               | `JMAP_EMAILS`           |                                                                                                                                             |\n| `stream.subject_prefix`     | `jmap.email`            | Subject = `\u003cprefix\u003e.\u003caccountId\u003e`.                                                                                                           |\n| `stream.max_age`            | `168h` (1 week)         | Stream `MaxAge`; also TTL on the object store.                                                                                              |\n| `stream.max_bytes`          | `64MiB`                 | Sizes accept `KiB`/`MiB`/`GiB`.                                                                                                             |\n| `stream.dedup_window`       | `24h`                   | Server-side `Nats-Msg-Id` dedup window. Catches concurrent HA publishes; boot-time republishes are gated separately by the high-water mark. |\n| `stream.externally_managed` | `false`                 | Skip create/update; only verify the stream exists. Set true when another operator (e.g. NACK) owns the stream config.                       |\n| `parts.bucket`              | `email-parts`           | Object Store bucket for all body/attachment parts.                                                                                          |\n| `parts.max_bytes`           | `960MiB`                | Bucket cap.                                                                                                                                 |\n| `parts.max_per_part`        | `25MiB`                 | Skip individual parts larger than this.                                                                                                     |\n| `backfill_limit`            | `100`                   | N most-recent emails to re-check on boot.                                                                                                   |\n\nTotal default storage footprint ≈ 1 GiB (64 MiB stream + 960 MiB parts).\n\n## Running\n\n```sh\njmap2nats -config ./jmap2nats.json\n```\n\n`jmap2nats -version` (or `jmap2nats version`) prints the build version\nand exits.\n\nThe service:\n\n1. Authenticates with the JMAP server.\n2. Creates or updates the JetStream stream and object-store bucket.\n3. Re-checks the last N most-recent emails (boot recovery).\n4. Opens a JMAP `EventSource` SSE connection and republishes every newly\n   created email as it arrives.\n\nFor HA, run several copies pointed at the same NATS cluster. Concurrent\npublishes (same JMAP email id from different replicas) are rejected by\nthe stream's `Nats-Msg-Id` dedup window; on boot, each replica reads the\nmost-recent published email per account and skips anything already in the\nstream, so backfill work isn't repeated across restarts.\n\n## Data model\n\nEach email becomes one NATS message:\n\n- **Subject**: `\u003cprefix\u003e.\u003caccountId\u003e` (defaults to `jmap.email.\u003caccountId\u003e`).\n  All emails for an account share one subject; the emailId is carried in\n  the `Nats-Msg-Id` and `Jmap-Email-Id` headers and in the JSON body's\n  `id` field, not in the subject.\n- **`Nats-Msg-Id` header**: the JMAP email id, used for dedup.\n- **Other headers** (filterable without parsing the body):\n  - `Jmap-Account-Id`, `Jmap-Email-Id`, `Jmap-Thread-Id`\n  - `Jmap-From`, `Jmap-To`, `Jmap-Cc`\n  - `Jmap-Subject`\n  - `Jmap-Received-At`, `Jmap-Sent-At` (RFC 3339)\n  - `Jmap-Message-Id`, `Jmap-In-Reply-To`, `Jmap-References`\n  - `Jmap-Mailbox-Ids`, `Jmap-Keywords`\n  - `Jmap-Has-Attachment`, `Jmap-Size`\n- **Body**: the JMAP `Email` object as JSON — same field names as\n  [RFC 8621 §4][rfc8621-4] (`id`, `blobId`, `threadId`, `mailboxIds`,\n  `keywords`, `from`, `to`, `subject`, `receivedAt`, `bodyStructure`,\n  `textBody`, `htmlBody`, `attachments`, …). The `bodyValues` map is\n  omitted — those bytes are in the object store. Each part with a\n  `blobId` carries an extra `objectKey` field pointing into the bucket.\n\nEvery JMAP `EmailBodyPart` with a `blobId` — text bodies, html bodies,\ninline images, attachments — is stored as one object in the\n`email-parts` bucket, keyed `\u003cemailId\u003e/\u003cblobId\u003e`. Parts over\n`parts.max_per_part` are skipped and flagged with `\"skipped\": true,\n\"objectKey\": null` in the JSON; consumers can still fetch them\ndirectly from the JMAP server via the `blobId`.\n\n[rfc8621-4]: https://datatracker.ietf.org/doc/html/rfc8621#section-4\n\n## Consuming with the `nats` CLI\n\n```sh\n# Watch new emails as they arrive\nnats sub 'jmap.email.\u003e'\n\n# Stream info \u0026 storage usage\nnats stream info JMAP_EMAILS\n\n# Replay everything currently in the stream into a new consumer\nnats consumer add JMAP_EMAILS replay --filter 'jmap.email.\u003e' \\\n  --deliver=all --replay=instant --ack=none --pull\nnats consumer next JMAP_EMAILS replay --count 5\n\n# Fetch any body part or attachment by its JMAP blobId\nnats object get email-parts \u003cEMAIL_ID\u003e/\u003cBLOB_ID\u003e --output ./part.bin\n\n# List stored parts\nnats object ls email-parts\n\n# Bucket info \u0026 storage usage\nnats object info email-parts\n```\n\n## Limitations\n\n- Only `created` events are published. JMAP `Email/changes` `updated`\n  and `destroyed` ids are ignored — this is a mail forwarder, not a\n  general state replicator.\n- The boot-time backfill is bounded by `backfill_limit`. If a replica is\n  offline for long enough that more than `backfill_limit` emails arrive\n  before it returns, the oldest of those are not re-checked.\n- One JMAP account per process. To bridge several accounts, run several\n  instances with distinct configs (and ideally distinct subject prefixes\n  if they share a stream).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjosh%2Fjmap2nats","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjosh%2Fjmap2nats","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjosh%2Fjmap2nats/lists"}