{"id":51082988,"url":"https://github.com/kanutocd/cdc-core","last_synced_at":"2026-06-23T20:00:53.437Z","repository":{"id":361746389,"uuid":"1255047315","full_name":"kanutocd/cdc-core","owner":"kanutocd","description":"Database-agnostic Change Data Capture domain primitives for Ruby","archived":false,"fork":false,"pushed_at":"2026-06-09T07:55:12.000Z","size":113,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-06-09T09:25:48.136Z","etag":null,"topics":["cdc","change-data-capture","data-pipeline","domain-events","event-driven","event-pipeline","event-processing","event-sourcing","pipeline","ruby","stream-processing"],"latest_commit_sha":null,"homepage":"https://kanutocd.github.io/cdc-core/","language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kanutocd.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-05-31T10:36:01.000Z","updated_at":"2026-06-09T07:55:20.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/kanutocd/cdc-core","commit_stats":null,"previous_names":["kanutocd/cdc-core"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/kanutocd/cdc-core","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kanutocd%2Fcdc-core","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kanutocd%2Fcdc-core/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kanutocd%2Fcdc-core/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kanutocd%2Fcdc-core/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kanutocd","download_url":"https://codeload.github.com/kanutocd/cdc-core/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kanutocd%2Fcdc-core/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34704748,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-23T02:00:07.161Z","response_time":65,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cdc","change-data-capture","data-pipeline","domain-events","event-driven","event-pipeline","event-processing","event-sourcing","pipeline","ruby","stream-processing"],"created_at":"2026-06-23T20:00:52.632Z","updated_at":"2026-06-23T20:00:53.421Z","avatar_url":"https://github.com/kanutocd.png","language":"Ruby","funding_links":[],"categories":[],"sub_categories":[],"readme":"# cdc-core\n\n[![Gem Version](https://badge.fury.io/rb/cdc-core.svg)](https://badge.fury.io/rb/cdc-core)\n[![CI](https://github.com/kanutocd/cdc-core/workflows/CI/badge.svg)](https://github.com/kanutocd/cdc-core/actions)\n[![Ruby Version](https://img.shields.io/badge/ruby-%3E%3D%203.4-ruby.svg)](https://www.ruby-lang.org/en/)\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)\n\n\n\nShared Change Data Capture vocabulary for Ruby.\n\n`cdc-core` provides immutable, Ractor-safe event objects and processor contracts for building CDC systems. It intentionally does not connect to databases, parse wire protocols, decode PostgreSQL OIDs, run schedulers, or integrate with Rails.\n\n## Requirements\n\n- Ruby 3.4+\n\n## Features\n\n- SourceAdapter normalization contract\n- Immutable `ChangeEvent` objects\n- Transaction grouping via `TransactionEnvelope`\n- Column-level change objects\n- Ordering vocabulary\n- Processor, composite processor, processor chain, and pipeline contracts\n- Event filters\n- Small pipeline orchestration object\n- Router for supported work item shapes\n- Observer hooks and canonical metric names\n- Ractor-safe event and transaction objects\n- RBS signatures\n- YARD-compatible documentation\n- No runtime dependencies\n\n## Ecosystem Position\n\n```text\nupstream source\n      |\n      v\nsource adapter\n      |\n      v\ncdc-core\n      |\n      +--\u003e cdc-parallel       CPU-bound processing\n      |\n      +--\u003e cdc-concurrent     I/O-bound processing\n      |\n      +--\u003e application sinks / processors\n```\n\n`cdc-core` is the shared vocabulary layer. It defines what a change event, transaction, processor, ordering policy, observer notification, and processor result mean without caring where the event came from or how it will be executed.\n\n## Boundary Summary\n\n`cdc-core` is for vocabulary.\n\nRuntime gems are for execution.\n\nSinks are for persistence or side effects.\n\n```text\nsource adapter -\u003e cdc-core vocabulary -\u003e runtime gem -\u003e sink\n```\n\n## Source Adapters\n\nCDC::Core::SourceAdapter defines the normalization contract used to translate source-specific payloads into cdc-core vocabulary objects.\n\nIt translates source-specific payloads into:\n\n- `CDC::Core::ChangeEvent`\n- `CDC::Core::TransactionEnvelope`\n- batches of core work items\n\nThe current PostgreSQL-oriented path is:\n\n```text\npgoutput-client -\u003e pgoutput-parser -\u003e pgoutput-decoder -\u003e source adapter -\u003e cdc-core\n```\n\nThe `pgoutput*` family handles PostgreSQL transport, protocol parsing, and type decoding. The source-adapter boundary is where those source-specific details become generic `cdc-core` objects.\n\nOther adapters can normalize logs, API payloads, application events, or other database streams into the same vocabulary.\n\n## Downstream Runtime Gems\n\n`cdc-parallel` and `cdc-concurrent` are downstream consumers of `cdc-core` events.\n\n### cdc-parallel\n\nUse `cdc-parallel` for heavy CPU-bound processing.\n\nExamples:\n\n- transformations\n- enrichment\n- encoding\n- compression\n- scoring\n- in-memory calculations\n\nIt is the Ractor-oriented runtime path.\n\n### cdc-concurrent\n\nUse `cdc-concurrent` for I/O-heavy processing.\n\nExamples:\n\n- HTTP calls\n- webhook delivery\n- Redis writes\n- search indexing\n- object storage writes\n- database sink writes\n\nIt is the fiber-friendly runtime path.\n\n## Installation\n\n```ruby\ngem 'cdc-core'\n```\n\n```ruby\nrequire 'cdc/core'\n```\n\n## Change Events\n\n```ruby\nevent = CDC::Core::ChangeEvent.new(\n  operation: :update,\n  schema: \"public\",\n  table: \"users\",\n  old_values: { \"email\" =\u003e \"old@example.com\" },\n  new_values: { \"email\" =\u003e \"new@example.com\" },\n  primary_key: { \"id\" =\u003e 7 },\n  transaction_id: 789,\n  commit_lsn: \"0/16B6C50\"\n)\n\nevent.update?\n# =\u003e true\n\nevent.qualified_table_name\n# =\u003e \"public.users\"\n\nevent.changes.map(\u0026:name)\n# =\u003e [\"email\"]\n```\n\n## Transactions\n\n```ruby\ntransaction = CDC::Core::TransactionEnvelope.new(\n  transaction_id: 789,\n  events: [event],\n  commit_lsn: \"0/16B6C50\",\n  committed_at: Time.now.utc\n)\n```\n\nA transaction envelope preserves database transaction boundaries. Runtime gems may use that boundary when they need ordering, batching, or parallel execution decisions.\n\n## Processors\n\n```ruby\nclass AuditProcessor \u003c CDC::Core::Processor\n  def process(event)\n    puts event.to_h\n    CDC::Core::ProcessorResult.success(event)\n  end\nend\n```\n\n## Ractor-safe processor intent\n\n```ruby\nclass AnalyticsProcessor \u003c CDC::Core::Processor\n  ractor_safe!\n\n  def process(event)\n    CDC::Core::ProcessorResult.success(event)\n  end\nend\n\nAnalyticsProcessor.new.ractor_safe?\n# =\u003e true\n```\n\nThis declares intent only. `cdc-core` does not execute processors in Ractors. `cdc-parallel` can use this signal before moving processor work across Ractors.\n\n## Downstream Workflow Primitives\n\n`cdc-core` defines three small workflow primitives. Runtime gems and\napplication-specific integrations can execute these primitives without\ninventing their own composition vocabulary.\n\n### CompositeProcessor\n\nUse `CompositeProcessor` when many independent processors should receive the\nsame input.\n\n```text\nevent\n  ├─ AuditProcessor\n  ├─ AnalyticsProcessor\n  └─ WebhookProcessor\n```\n\n```ruby\nprocessor = CDC::Core::CompositeProcessor.new([\n  AuditProcessor.new,\n  AnalyticsProcessor.new\n])\n\nresults = processor.process(event)\n```\n\n### Pipeline\n\nUse `Pipeline` when one processor should run only after filters match.\n\n```text\nevent\n  ↓\nfilters\n  ↓\nprocessor\n```\n\n```ruby\npipeline = CDC::Core::Pipeline.new(\n  processor: AuditProcessor.new,\n  filters: [\n    CDC::Core::Filter.schema(\"public\"),\n    CDC::Core::Filter.table(\"users\")\n  ]\n)\n\nresult = pipeline.process(event)\n```\n\n### ProcessorChain\n\nUse `ProcessorChain` when each processor depends on the previous processor's\nsuccessful value.\n\n```text\nuser_ids\n  ↓\nLoadUsersProcessor\n  ↓\nusers\n  ↓\nSendNotificationsProcessor\n```\n\n```ruby\nclass LoadUsersProcessor \u003c CDC::Core::Processor\n  def process(user_ids)\n    users = User.where(id: user_ids).to_a\n    CDC::Core::ProcessorResult.success(user_ids, value: users)\n  end\nend\n\nclass SendNotificationsProcessor \u003c CDC::Core::Processor\n  def process(users)\n    users.each { |user| NotificationMailer.notice(user).deliver_later }\n    CDC::Core::ProcessorResult.success(users, value: users.size)\n  end\nend\n\nchain = CDC::Core::ProcessorChain.new([\n  LoadUsersProcessor.new,\n  SendNotificationsProcessor.new\n])\n\nresult = chain.process([1, 2, 3])\nresult.value\n# =\u003e 3\n```\n\n## Non-goals\n\n`cdc-core` does not:\n\n- Connect to PostgreSQL\n- Parse `pgoutput`\n- Decode PostgreSQL values\n- Manage replication slots\n- Implement concrete source adapters\n- Run Ractor pools\n- Run fiber schedulers\n- Persist audit records\n- Integrate with ActiveRecord\n- Publish to Kafka, Redis, HTTP, or other sinks\n\n## Documentation\n\nThe YARD documentation uses `docs/index.md` as its readme and includes the Markdown files under `docs/`.\n\n```text\n--title \"cdc-core API Documentation\"\n--readme docs/index.md\n--markup markdown\n--output-dir doc\nlib/**/*.rb\n-\ndocs/**/*.md\n```\n\n## Development\n\n```bash\nbundle exec rake\nbundle exec rake rbs:validate\nbundle exec yard doc\n```\n\n## License\n\n[MIT](LICENSE.txt)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkanutocd%2Fcdc-core","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkanutocd%2Fcdc-core","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkanutocd%2Fcdc-core/lists"}