{"id":13630632,"url":"https://github.com/glynnbird/couchreplicate","last_synced_at":"2025-03-16T20:31:34.567Z","repository":{"id":47207068,"uuid":"121215539","full_name":"glynnbird/couchreplicate","owner":"glynnbird","description":"CouchDB and Cloudant replication command-line tool and library","archived":false,"fork":false,"pushed_at":"2024-04-24T08:46:09.000Z","size":801,"stargazers_count":19,"open_issues_count":2,"forks_count":3,"subscribers_count":6,"default_branch":"master","last_synced_at":"2024-04-24T13:48:03.250Z","etag":null,"topics":["cli","couchdb","json","nodejs","replication"],"latest_commit_sha":null,"homepage":"https://medium.com/ibm-watson-data-lab/cloudant-and-couchdb-replication-with-couchreplicate-79ea6e898e6e","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/glynnbird.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-02-12T07:42:07.000Z","updated_at":"2024-06-14T00:11:15.957Z","dependencies_parsed_at":"2024-03-19T10:42:41.555Z","dependency_job_id":"b9c6854a-e449-4d9d-86e1-30215a537ce2","html_url":"https://github.com/glynnbird/couchreplicate","commit_stats":{"total_commits":41,"total_committers":1,"mean_commits":41.0,"dds":0.0,"last_synced_commit":"1e7d792219d05e57b8543cdc25b1d2bca22d05b4"},"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/glynnbird%2Fcouchreplicate","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/glynnbird%2Fcouchreplicate/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/glynnbird%2Fcouchreplicate/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/glynnbird%2Fcouchreplicate/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/glynnbird","download_url":"https://codeload.github.com/glynnbird/couchreplicate/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243830912,"owners_count":20354848,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cli","couchdb","json","nodejs","replication"],"created_at":"2024-08-01T22:01:51.294Z","updated_at":"2025-03-16T20:31:33.872Z","avatar_url":"https://github.com/glynnbird.png","language":"JavaScript","funding_links":[],"categories":["JavaScript","Backup and Migration"],"sub_categories":[],"readme":"# couchreplicate\n\n[![Build Status](https://travis-ci.org/ibm-watson-data-lab/couchreplicate.svg?branch=master)](https://travis-ci.org/ibm-watson-data-lab/couchreplicate) [![npm version](https://badge.fury.io/js/couchreplicate.svg)](https://badge.fury.io/js/couchreplicate)\n\nThis is a command-line tool and library that helps coordinate [Apache CouchDB](http://couchdb.apache.org/)™ or [IBM Cloudant](https://www.ibm.com/cloud/cloudant) replications. It can be used to replicate a single database, multiple databases or an entire cluster from a source instance to a target instance.\n\nIt is written in Node.js and can be installed from `npm` or used programmatically in your own Node.js projects.\n\n![screenshot](img/couchreplicate.gif)\n\n## Pre-requisites\n\n- [Node.js \u0026 npm](https://nodejs.org/en/)\n- one or more source databases in an Apache CouchDB or IBM Cloudant instance\n- a target Apache CouchDB or IBM Cloudant instance\n\n## Installation\n\nInstall with npm\n\n    npm install -g couchreplicate\n\nSupplying `-g` instructs npm to install the library globally. You can find out where global npm modules are installed by running `npm config list`. You can find out which user is used to install global modules by running `npm -g config get user`. I had to repair the permissions of pre-existing libraries by running `chown -R nobody /usr/local/lib/node_modules/*`. \n\n## Usage\n\n### Replicating all databases from one instance to another\n\n    couchreplicate -a -s http://u:p@localhost:5984 -t https://U:P@HOST.cloudant.com\n\nor \n\n    couchreplicate --all --source http://u:p@localhost:5984 --target https://U:P@HOST.cloudant.com\n\nwhere `http://u:p@localhost:5984` is the URL of your source instance and `https://U:P@HOST.cloudant.com` is the URL of your target instance. They must include `admin` credentials.\n\n### Replicating a list of databases\n\n    couchreplicate -d db1,db2,db3 -s http://u:p@localhost:5984 -t https://U:P@HOST.cloudant.com\n\nwhere `db1,db2,db3` is a comma-separated list of database names.\n\n### Replicating a single database\n\n    couchreplicate -d mydb -s http://u:p@localhost:5984 -t https://U:P@HOST.cloudant.com\n\nor \n\n    couchreplicate -s http://u:p@localhost:5984/mydb -t https://U:P@HOST.cloudant.com/mydb\n\nwhere `mydb` is the name of the database. When supplying the source and target database names in the URL, the database names need not match.\n\n### Additionally replicating the _security document\n\nNormal CouchDB replication leaves the `_security` document behind. The `_security` document is used to determine which users have which levels of access to the database. Without a `_security` document specified, only an `_admin` user can read/write/update/delete documents.\n\nIf you pass the `--auth` or `-x` command-line parameter, then *couchreplicate* will copy the `_security` document from the source to the target database at the end of the replication process e.g:\n\n    couchreplicate -s http://u:p@localhost:5984/mydb -t https://U:P@HOST.cloudant.com/mydb --auth\n\n### Generating continuous replications\n\nThe default action is to create one-shot replications that copy data from the source to the target and then finish. If you would like live, continuous replications instead, then add the `--live` or `-l` option to your command-line invocation.\n\ne.g. \n\n    couchreplicate -l -a -s http://u:p@localhost:5984 -t https://U:P@HOST.cloudant.com\n\nSome things to note:\n\n- this tool will only permit you to create fifty live replications\n- as the replications are continuous, they will never reach a \"completed\" state so the tool will remain active forever, or until you kill the process (with Ctrl-C). After killing the process, any running replications will continue to run. See the source cluster's \"replication\" tab in the dashboard.\n\n### Errors during replication\n\nReplication errors can occur and have a multitude of causes. If a replication does not complete successfully, you may see a status like this:\n\n```sh\n cities [▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇——] 97% 21.1s error\n ```\n\n - the document count may not reach 100%\n - the status string at right-hand side will read 'error'\n\n It may be caused by:\n\n - incorrect authentication credentials at the source or target side\n - insufficient privileges - make sure you have \"admin\" credentials for the source and target clusters\n - the target service may have a size restriction on documents. Cloudant instances on the IBM Cloud have a limit of 1MB per API request, so very large JSON documents would not make it\n - the target service may have a API rate limit in place. If the target is very busy, then there may be insufficient capacity left over to service the replication process\n\n### The _replicator database\n\nThe *couchreplicate* tool requires a database called `_replicator` to be present on the source cluster. The tool tries to create a the database if it doesn't already exist at the start of replication job. You can see the current and historical replication jobs in the Cloudant/CouchDB dashboard or by looking at the contents of the `_replicator` database yourself.\n\nNote that even if you `Ctrl-C` out of *couchreplicate*, any running replications will continue to proceed.\n\n## Debugging\n\nIf you need to monitor replications more closely than watching the progress bars, then setting the `DEBUG` environment variable to `couchreplicate` before running the tool will send more status information to the console.\n\ne.g.\n\n    DEBUG=couchreplicate couchreplicate -d mydb -s http://u:p@localhost:5984 -t https://U:P@HOST.cloudant.com\n\n## Command-line reference\n\n- `--source` / `-s` - source URL (required)\n- `--target` / `-t` - target URL (required)\n- `--databases` / `-d` - comma-separated list of database names\n- `--all` / `-a` - replicate all databases\n- `--concurrency` / `-c` - the number of replications to run at once (default = 1)\n- `--auth` / `-x` - also copy the `_security` document during replication\n- `--live` / `-l` - set up continuous replications instead of the default \"one-shot\" replication\n- `--quiet` / `-q` - suppress progress bars\n- `--nomonitor` / `-n` - don't monitor the replications - just start the replication processed and exit. Only works with `--live`/`-l`\n- `--help` / `-h` - show help message\n- `--version` / `-v` - show version number\n\n## Command-line exit codes\n\n- 0 - success\n- 1 - invalid source URL\n- 2 - invalid target URL\n- 3 - `--nomonitor/-n` is only applicable with the `--live/-l` option\n- 4 - no source or target database names supplied\n- 5 - database names supplied in URLs and as other command-line options\n- 6 - runtime error\n\n## Using couchreplicate programmatically\n\nInstall the library into your own Node.js project\n\n    npm install --save couchreplicate\n\nLoad the module into your code\n\n```js\n  const cm = require('couchreplicate')\n```\n\nSet off a single replication:\n\n```js\n  var opts = {\n    source: 'http://u:p@localhost:5984/sourcedb',\n    target: 'https://U:P@HOST.cloudant.com/targetdb',\n    quiet: true,\n    auth: true \n  }\n  cm.migrateDB(opts).then(() =\u003e {\n    console.log('done')\n  })\n```\n\nmultiple replications:\n\n```js\n  var opts = {\n    source: 'http://u:p@localhost:5984',\n    target: 'https://U:P@HOST.cloudant.com',\n    databases: ['animals', 'minerals', 'vegetables'],\n    quiet: false,\n    concurrency: 3\n    auth: true \n  }\n  \n  cm.migrateList(opts).then(() =\u003e {\n    console.log('done')\n  })\n```\n\nor replicate an entire cluster:\n\n```js\n  var opts = {\n    source: 'http://u:p@localhost:5984',\n    target: 'https://U:P@HOST.cloudant.com',\n    quiet: false,\n    concurrency: 3\n    auth: true \n  }\n  \n  cm.migrateAll(opts).then(() =\u003e {\n    console.log('done')\n  })\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fglynnbird%2Fcouchreplicate","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fglynnbird%2Fcouchreplicate","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fglynnbird%2Fcouchreplicate/lists"}