{"id":13400626,"url":"https://github.com/rancher/convoy","last_synced_at":"2025-12-15T00:13:08.109Z","repository":{"id":25606168,"uuid":"29041226","full_name":"rancher/convoy","owner":"rancher","description":"A Docker volume plugin, managing persistent container volumes.","archived":true,"fork":false,"pushed_at":"2021-12-31T12:03:11.000Z","size":4411,"stargazers_count":1308,"open_issues_count":75,"forks_count":135,"subscribers_count":39,"default_branch":"master","last_synced_at":"2024-11-15T10:59:35.312Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rancher.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2015-01-10T00:10:25.000Z","updated_at":"2024-10-17T21:32:05.000Z","dependencies_parsed_at":"2022-07-10T12:16:48.710Z","dependency_job_id":null,"html_url":"https://github.com/rancher/convoy","commit_stats":null,"previous_names":["rancher/rancher-volume","yasker/volmgr","rancherio/volmgr"],"tags_count":40,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rancher%2Fconvoy","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rancher%2Fconvoy/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rancher%2Fconvoy/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rancher%2Fconvoy/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rancher","download_url":"https://codeload.github.com/rancher/convoy/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":234569767,"owners_count":18854133,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-30T19:00:54.041Z","updated_at":"2025-09-28T23:30:29.669Z","avatar_url":"https://github.com/rancher.png","language":"Go","readme":"# Convoy [![Build Status](https://drone8.rancher.io/api/badges/rancher/convoy/status.svg)](https://drone8.rancher.io/rancher/convoy)\n\n## Overview\nConvoy is a Docker volume plugin for a variety of storage back-ends. It supports vendor-specific extensions like snapshots, backups, and restores. It's written in Go and can be deployed as a standalone binary.\n\n[![Convoy_DEMO](https://asciinema.org/a/9y5nbp3h97vyyxnzuax9f568e.png)](https://asciinema.org/a/9y5nbp3h97vyyxnzuax9f568e?autoplay=1\u0026loop=1\u0026size=medium\u0026speed=2)\n\n## Why use Convoy?\nConvoy makes it easy to manage your data in Docker.\nIt provides persistent volumes for Docker containers with support for snapshots, backups, and restores on various back-ends (e.g. device mapper, NFS, EBS).\n\nFor example, you can:\n\n* Migrate volumes between hosts\n* Share the same volumes across hosts\n* Schedule periodic snapshots of volumes\n* Recover a volume from a previous backup\n\n### Supported back-ends\n* Device Mapper\n* Virtual File System (VFS) / Network File System (NFS)\n* Amazon Elastic Block Store (EBS)\n\n## Quick Start Guide\nFirst, make sure Docker 1.8 or above is running.\n```bash\ndocker --version\n```\nIf not, install the latest Docker daemon as follows:\n```bash\ncurl -sSL https://get.docker.com/ | sh\n```\nOnce the right Docker daemon version is running, install and configure the Convoy volume plugin as follows:\n```bash\nwget https://github.com/rancher/convoy/releases/download/v0.5.2/convoy.tar.gz\ntar xvzf convoy.tar.gz\nsudo cp convoy/convoy convoy/convoy-pdata_tools /usr/local/bin/\nsudo mkdir -p /etc/docker/plugins/\nsudo bash -c 'echo \"unix:///var/run/convoy/convoy.sock\" \u003e /etc/docker/plugins/convoy.spec'\n```\nYou can use a file-backed loopback device to test and demo Convoy Device Mapper driver. A loopback device, however, is known to be unstable and should _**not**_ be used in production.\n```bash\ntruncate -s 100G data.vol\ntruncate -s 1G metadata.vol\nsudo losetup /dev/loop5 data.vol\nsudo losetup /dev/loop6 metadata.vol\n```\nOnce the data and metadata devices are set up, you can start the Convoy plugin daemon as follows:\n```bash\nsudo convoy daemon --drivers devicemapper --driver-opts dm.datadev=/dev/loop5 --driver-opts dm.metadatadev=/dev/loop6\n```\nYou can create a Docker container with a convoy volume. As a test, create a file called `/vol1/foo` in the Convoy volume:\n```bash\nsudo docker run -v vol1:/vol1 --volume-driver=convoy ubuntu touch /vol1/foo\n```\nNext, take a snapshot of the convoy volume and backup the snapshot to a local directory: (You can also [make backups to an NFS share or S3 object store](#backup-a-snapshot).)\n```bash\nsudo convoy snapshot create vol1 --name snap1vol1\nsudo mkdir -p /opt/convoy/\nsudo convoy backup create snap1vol1 --dest vfs:///opt/convoy/\n```\nThe `convoy backup` command returns a URL string representing the backup dataset. You can use this URL to recover the volume on another host:\n```bash\nsudo convoy create res1 --backup \u003cbackup_url\u003e\n```\nThe following command creates a new container and mounts the recovered Convoy volume into that container:\n```bash\nsudo docker run -v res1:/res1 --volume-driver=convoy ubuntu ls /res1/foo\n```\nYou should see the recovered file in `/res1/foo`.\n\n## Installation\nEnsure you have Docker 1.8 or above installed.\n\nDownload the latest version of [Convoy][version] and unzip it. Put the binaries in a directory in the execution `$PATH` of sudo and root users (e.g. `/usr/local/bin`).\n\n[version]: https://github.com/rancher/convoy/releases/download/v0.5.2/convoy.tar.gz\n\n```bash\nwget https://github.com/rancher/convoy/releases/download/v0.5.2/convoy.tar.gz\ntar xvzf convoy.tar.gz\nsudo cp convoy/convoy convoy/convoy-pdata_tools /usr/local/bin/\n```\nRun the following commands to set up the Convoy volume plugin for Docker:\n```bash\nsudo mkdir -p /etc/docker/plugins/\nsudo bash -c 'echo \"unix:///var/run/convoy/convoy.sock\" \u003e /etc/docker/plugins/convoy.spec'\n```\n\n## Start Convoy Daemon\n\nYou need to pass different arguments to the Convoy daemon depending on your choice of back-end implementation.\n\n#### Device Mapper\nIf you're running in a production environment with the Device Mapper driver, it's recommended to attach a new, empty block device to the host Convoy is running on.\nThen you can make two partitions on the device using [`dm_dev_partition.sh`][dm_script] to get two block devices ready for the Device Mapper driver. See [Device Mapper Partition Helper][helper] for more details.\n\n[dm_script]: https://raw.githubusercontent.com/rancher/convoy/master/tools/dm_dev_partition.sh\n[helper]: https://github.com/rancher/convoy/blob/master/docs/devicemapper.md#device-mapper-partition-helper\n\nDevice Mapper requires two block devices to create storage pool for all volumes and snapshots. Assuming you have two devices created one data device called `/dev/convoy-vg/data` and the other metadata device called `/dev/convoy-vg/metadata`, then run the following command to start the Convoy daemon:\n```bash\nsudo convoy daemon --drivers devicemapper --driver-opts dm.datadev=/dev/convoy-vg/data --driver-opts dm.metadatadev=/dev/convoy-vg/metadata\n```\n* The default Device Mapper volume size is 100G. You can override it with the `---driver-opts dm.defaultvolumesize` option.\n* You can take a look [here](https://github.com/rancher/convoy/blob/master/docs/devicemapper.md#calculate-the-size-you-need-for-metadata-block-device) if you want to know how much storage should be allocated for the metadata device.\n\n#### NFS\nFirst, mount the NFS share to the root directory used to store volumes. Substitute `\u003cvfs_path\u003e` with the appropriate directory of your choice:\n```bash\nsudo mkdir \u003cvfs_path\u003e\nsudo mount -t nfs \u003cnfs_server\u003e:/path \u003cvfs_path\u003e\n```\nThe NFS-based Convoy daemon can be started as follows:\n```bash\nsudo convoy daemon --drivers vfs --driver-opts vfs.path=\u003cvfs_path\u003e\n```\n\n#### EBS\nMake sure you're running on an EC2 instance and have already [configured AWS credentials](https://github.com/aws/aws-sdk-go#configuring-credentials) correctly.\n```bash\nsudo convoy daemon --drivers ebs\n```\n\n#### DigitalOcean\nMake sure you're running on a DigitalOcean Droplet and that you have the `DO_TOKEN` environment variable set with your key.\n```bash\nsudo convoy daemon --drivers digitalocean\n```\n\n## Volume Commands\n#### Create a Volume\n\nVolumes can be created using the `convoy create` command:\n```bash\nsudo convoy create volume_name\n```\n* Device Mapper: Default volume size is 100G. `--size` [option](https://github.com/rancher/convoy/blob/master/docs/devicemapper.md#create) is supported.\n* EBS: Default volume size is 4G. `--size` and [some other options](https://github.com/rancher/convoy/blob/master/docs/ebs.md#create) are supported.\n\nYou can also create a volume using the [`docker run`](https://github.com/rancher/convoy/blob/master/docs/docker.md#create-container) command. If the volume does not yet exist, a new volume will be created. Otherwise the existing volume will be used.\n```bash\nsudo docker run -it -v test_volume:/test --volume-driver=convoy ubuntu\n```\n\n#### Delete a Volume\n```bash\nsudo convoy delete \u003cvolume_name\u003e\n```\nor\n```bash\nsudo docker rm -v \u003ccontainer_name\u003e\n```\n* NFS, EBS and DigitalOcean: The `-r/--reference` option instructs the `convoy delete` command to only delete the reference to the volume from the current host and leave the underlying files on [NFS server](https://github.com/rancher/convoy/blob/master/docs/vfs.md#delete) or [EBS volume](https://github.com/rancher/convoy/blob/master/docs/ebs.md#delete) unchanged. This is useful when the volume need to be reused later.\n* [`docker rm -v`](https://github.com/rancher/convoy/blob/master/docs/docker.md#delete-container) would be treated as `convoy delete` with `-r/--reference`.\n* If you use `--rm` with `docker run`, all Docker volumes associated with the container would be deleted on container exit with `convoy delete --reference`. See [Docker run reference](https://docs.docker.com/engine/reference/run/) for details.\n\n#### List and Inspect a Volume\n```bash\nsudo convoy list\nsudo convoy inspect vol1\n```\n\n#### Take Snapshot of a Volume\n```bash\nsudo convoy snapshot create vol1 --name snap1vol1\n```\n\n#### Delete a Snapshot\n```bash\nsudo convoy snapshot delete snap1vol1\n```\n* Device Mapper: please make sure you keep [the latest backed-up snapshot](https://github.com/rancher/convoy/blob/master/docs/devicemapper.md#backup-create) for the same volume available to enable the incremental backup mechanism. Convoy needs it to calculate the differences between snapshots.\n\n#### Backup a Snapshot\n* Device Mapper or VFS: You can backup a snapshot to an NFS mount/local directory or an S3 object store:\n```bash\nsudo convoy backup create snap1vol1 --dest vfs:///opt/backup/\n```\nor\n```bash\nsudo convoy backup create snap1vol1 --dest s3://backup-bucket@us-west-2/\n```\nor if you want to use a custom S3 endpoint (like [Minio](https://github.com/minio/minio))\n```bash\nsudo convoy backup --s3-endpoint http://s3.example.com:9000/ create snap1vol1 --dest s3://backup-bucket@us-west-2/\n```\n\nThe backup operation returns a URL string that uniquely identifies the backup dataset.\n```\ns3://backup-bucket@us-west-2/?backup=f98f9ea1-dd6e-4490-8212-6d50df1982ea\\u0026volume=e0d386c5-6a24-446c-8111-1077d10356b0\n```\nIf you're using S3, please make sure you have AWS credentials ready either in `~/.aws/credentials` or as environment variables, as described [here](https://github.com/aws/aws-sdk-go#configuring-credentials). You may need to put credentials in `/root/.aws/credentials` or set up sudo environment variables in order to get S3 credentials to work.\n\n* EBS: `--dest` is [not needed](https://github.com/rancher/convoy/blob/master/docs/ebs.md#backup-create). Just do `convoy backup create snap1vol1`.\n\n#### Restore a Volume from Backup\n```bash\nsudo convoy create res1 --backup \u003curl\u003e\n```\n* EBS: Current host must be in the [same region](https://github.com/rancher/convoy/blob/master/docs/ebs.md#create) of the backup to be restored.\n\n#### Mount a Restored Volume into a Docker Container\nYou can use the standard `docker run` command to mount the restored volume into a Docker container:\n```bash\nsudo docker run -it -v res1:/res1 --volume-driver convoy ubuntu\n```\n\n#### Mount an NFS-Backed Volume on Multiple Servers\nYou can mount an NFS-backed volume on multiple servers. You can use the standard `docker run` command to mount an existing NFS-backed mount into a Docker container. For example, if you have already created an NFS-based volume `vol1` on one host, you can run the following command to mount the existing `vol1` volume into a new container:\n```bash\nsudo docker run -it -v vol1:/vol1 --volume-driver=convoy ubuntu\n```\n## Support and Discussion\nIf you need any help with Convoy, please join us at either our [forum](http://forums.rancher.com/c/convoy) or [#rancher IRC channel](http://webchat.freenode.net/?channels=rancher).\n\nFeel free to submit any bugs, issues, and feature requests to [Convoy Issues](https://github.com/rancher/convoy/issues).\n\n## Contribution\nContribution are welcome! Please take a look at [Development Guide](https://github.com/rancher/convoy/blob/master/docs/development.md) if you want to how to build Convoy from source or running test cases.\n\nWe love to hear new Convoy Driver ideas from you. Implementations are most welcome! Please consider take a look at [enhancement ideas](https://github.com/rancher/convoy/labels/enhancement) if you want contribute.\n\nAnd of course, [bug fixes](https://github.com/rancher/convoy/issues) are always welcome!\n\n## References\n[Convoy Command Line Reference](https://github.com/rancher/convoy/blob/master/docs/cli_reference.md)\n\n[Using Convoy with Docker](https://github.com/rancher/convoy/blob/master/docs/docker.md)\n#### Driver Specific\n[Device Mapper](https://github.com/rancher/convoy/blob/master/docs/devicemapper.md)\n\n[Amazon Elastic Block Store](https://github.com/rancher/convoy/blob/master/docs/ebs.md)\n\n[Virtual File System/Network File System](https://github.com/rancher/convoy/blob/master/docs/vfs.md)\n","funding_links":[],"categories":["Go","Container Operations","Storage \u0026 Data Management","HarmonyOS","Volume management and plugins","Docker","others","Rancher 1.0"],"sub_categories":["Volume Management / Data","Windows Manager","Tools 1.0"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Francher%2Fconvoy","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Francher%2Fconvoy","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Francher%2Fconvoy/lists"}