{"id":29179714,"url":"https://github.com/trendyol/go-dcp-couchbase","last_synced_at":"2025-07-01T19:06:24.169Z","repository":{"id":180096926,"uuid":"648228154","full_name":"Trendyol/go-dcp-couchbase","owner":"Trendyol","description":"The Go implementation of the Couchbase to Couchbase with DCP.","archived":false,"fork":false,"pushed_at":"2025-06-03T13:07:58.000Z","size":985,"stargazers_count":42,"open_issues_count":5,"forks_count":11,"subscribers_count":12,"default_branch":"master","last_synced_at":"2025-06-03T23:17:50.478Z","etag":null,"topics":["couchbase","dcp","go","golang"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Trendyol.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-06-01T13:42:09.000Z","updated_at":"2025-06-03T13:03:28.000Z","dependencies_parsed_at":"2023-09-27T17:38:40.314Z","dependency_job_id":"4f77fe07-cb9d-4949-a54f-28630f885ab8","html_url":"https://github.com/Trendyol/go-dcp-couchbase","commit_stats":null,"previous_names":["trendyol/go-couchbase-connect-couchbase","trendyol/go-dcp-couchbase"],"tags_count":75,"template":false,"template_full_name":null,"purl":"pkg:github/Trendyol/go-dcp-couchbase","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Trendyol%2Fgo-dcp-couchbase","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Trendyol%2Fgo-dcp-couchbase/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Trendyol%2Fgo-dcp-couchbase/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Trendyol%2Fgo-dcp-couchbase/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Trendyol","download_url":"https://codeload.github.com/Trendyol/go-dcp-couchbase/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Trendyol%2Fgo-dcp-couchbase/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":263021821,"owners_count":23401148,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["couchbase","dcp","go","golang"],"created_at":"2025-07-01T19:06:12.447Z","updated_at":"2025-07-01T19:06:24.142Z","avatar_url":"https://github.com/Trendyol.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Go Dcp Couchbase [![Go Reference](https://pkg.go.dev/badge/github.com/Trendyol/go-dcp-couchbase.svg)](https://pkg.go.dev/github.com/Trendyol/go-dcp-couchbase) [![Go Report Card](https://goreportcard.com/badge/github.com/Trendyol/go-dcp-couchbase)](https://goreportcard.com/report/github.com/Trendyol/go-dcp-couchbase) [![OpenSSF Scorecard](https://api.scorecard.dev/projects/github.com/Trendyol/go-dcp-couchbase/badge)](https://scorecard.dev/viewer/?uri=github.com/Trendyol/go-dcp-couchbase)\n\n**Go Dcp Couchbase** streams documents from Couchbase Database Change Protocol (DCP) and writes to\nCouchbase bucket in near real-time.\n\n## Features\n\n* **Less resource usage** and **higher throughput**.\n* **Update multiple documents** for a DCP event(see [Example](#example)).\n* Handling different DCP events such as **expiration, deletion and mutation**(see [Example](#example)).\n* **Managing batch configurations** such as maximum batch, batch bytes, batch ticker durations.\n* **Scale up and down** by custom membership algorithms(Couchbase, KubernetesHa, Kubernetes StatefulSet or\n  Static, see [examples](https://github.com/Trendyol/go-dcp#examples)).\n* **Easily manageable configurations**.\n\n## Concepts\nGeneral Concept\n![general](docs/couchbase-dcp.png)\n\nMerge at target bucket\n![merge-buckets](docs/couchbase-merge-buckets.png)\n\n\n## Example\n\n[Struct Config](example/struct-config/main.go)\n\n```go\npackage main\n\nimport (\n  \"github.com/Trendyol/go-dcp-couchbase\"\n  \"time\"\n\n  \"github.com/Trendyol/go-dcp-couchbase/config\"\n  dcpConfig \"github.com/Trendyol/go-dcp/config\"\n)\n\nfunc main() {\n  c, err := dcpcouchbase.NewConnectorBuilder(\u0026config.Config{\n    Dcp: dcpConfig.Dcp{\n      Hosts:      []string{\"localhost:8091\"},\n      Username:   \"user\",\n      Password:   \"password\",\n      BucketName: \"dcp-test\",\n      Dcp: dcpConfig.ExternalDcp{\n        Group: dcpConfig.DCPGroup{\n          Name: \"groupName\",\n          Membership: dcpConfig.DCPGroupMembership{\n            RebalanceDelay: 3 * time.Second,\n          },\n        },\n      },\n      Metadata: dcpConfig.Metadata{\n        Config: map[string]string{\n          \"bucket\":     \"dcp-test-meta\",\n          \"scope\":      \"_default\",\n          \"collection\": \"_default\",\n        },\n        Type: \"couchbase\",\n      },\n      Debug: true,\n    },\n    Couchbase: config.Couchbase{\n      Hosts:          []string{\"localhost:8091\"},\n      Username:       \"user\",\n      Password:       \"password\",\n      BucketName:     \"dcp-test-backup\",\n      BatchSizeLimit: 10,\n      RequestTimeout: 10 * time.Second,\n    },\n  }).SetMapper(dcpcouchbase.DefaultMapper).Build()\n  if err != nil {\n    panic(err)\n  }\n\n  defer c.Close()\n  c.Start()\n}\n```\n\n## Configuration\n\n### Dcp Configuration\n\nCheck out on [go-dcp](https://github.com/Trendyol/go-dcp#configuration)\n\n### Couchbase Specific Configuration\n\n| Variable                         | Type          | Required | Default         | Description                                                                                         |                                                           \n|----------------------------------|---------------|----------|-----------------|-----------------------------------------------------------------------------------------------------|\n| `couchbase.hosts`                | []string      | yes      |                 | Couchbase connection urls                                                                           |\n| `couchbase.username`             | string        | yes      |                 | Defines Couchbase username                                                                          |\n| `couchbase.password`             | string        | yes      |                 | Defines Couchbase password                                                                          |\n| `couchbase.bucketName`           | string        | yes      |                 | Defines Couchbase bucket name                                                                       |\n| `couchbase.scopeName`            | string        | no       | _default        | Defines Couchbase scope name                                                                        |\n| `couchbase.collectionName`       | string        | no       | _default        | Defines Couchbase collection name                                                                   |\n| `couchbase.batchSizeLimit`       | int           | no       | 2048            | Maximum message count for batch, if exceed flush will be triggered.                                 |\n| `couchbase.batchTickerDuration`  | time.Duration | no       | 10s             | Batch is being flushed automatically at specific time intervals for long waiting messages in batch. |\n| `couchbase.batchByteSizeLimit`   | int, string   | no       | 10mb            | Maximum size(byte) for batch, if exceed flush will be triggered. `10mb` is default.                 |\n| `couchbase.maxInflightRequests`  | int           | no       | $batchSizeLimit | Maximum request count for Couchbase                                                                 |\n| `couchbase.writePoolSizePerNode` | int           | no       | 1               | Write connection pool size per node                                                                 |\n| `couchbase.requestTimeout`       | time.Duration | no       | 1m              | Maximum request waiting time                                                                        |\n| `couchbase.secureConnection`     | bool          | no       | false           | Enables secure connection.                                                                          |\n| `couchbase.rootCAPath`           | string        | no       | false           | Defines root CA path.                                                                               |\n| `couchbase.connectionBufferSize` | uint          | no       | 20971520        | Defines connectionBufferSize.                                                                       |\n| `couchbase.connectionTimeout`    | time.Duration | no       | 1m              | Defines connectionTimeout.                                                                          |\n\n## Exposed metrics\n\nFor DCP related metrics see [also](https://github.com/Trendyol/go-dcp#exposed-metrics).\n\n## Breaking Changes\n\n| Date taking effect | Date announced    | Change                                               | How to check    |\n|--------------------|-------------------|------------------------------------------------------|-----------------|\n| December 29, 2023  | December 29, 2023 | Mapper first arg changed to `couchbase.EventContext` | Compile project |\n| November 14, 2023  | November 14, 2023 | Creating connector via builder                       | Compile project |\n\n## Contributing\n\nGo Dcp Couchbase is always open for direct contributions. For more information please check\nour [Contribution Guideline document](./CONTRIBUTING.md).\n\n## License\n\nReleased under the [MIT License](LICENSE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftrendyol%2Fgo-dcp-couchbase","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftrendyol%2Fgo-dcp-couchbase","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftrendyol%2Fgo-dcp-couchbase/lists"}