{"id":13416503,"url":"https://github.com/netvarun/docket","last_synced_at":"2025-03-15T00:31:03.602Z","repository":{"id":26312721,"uuid":"29760781","full_name":"netvarun/docket","owner":"netvarun","description":"Docket - Custom docker registry that allows for lightning fast deploys through bittorrent","archived":false,"fork":false,"pushed_at":"2020-09-02T02:35:17.000Z","size":23,"stargazers_count":704,"open_issues_count":4,"forks_count":37,"subscribers_count":29,"default_branch":"master","last_synced_at":"2024-11-15T11:06:37.375Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/netvarun.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2015-01-24T01:08:12.000Z","updated_at":"2024-11-11T05:25:46.000Z","dependencies_parsed_at":"2022-08-27T09:11:23.797Z","dependency_job_id":null,"html_url":"https://github.com/netvarun/docket","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/netvarun%2Fdocket","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/netvarun%2Fdocket/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/netvarun%2Fdocket/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/netvarun%2Fdocket/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/netvarun","download_url":"https://codeload.github.com/netvarun/docket/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243667681,"owners_count":20328032,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-30T21:00:59.848Z","updated_at":"2025-03-15T00:31:03.190Z","avatar_url":"https://github.com/netvarun.png","language":"Go","readme":"![](http://static2.stuff.co.nz/1318390536/513/5775513_600x400.jpg)\n\n# Docket\n\nDOCKEr + torrenT = Docket\n\nDocket is a custom docker registry that allows for deployments through bittorrent.\nIt allows for lightning fast Docker image deploys across large number of machines.\n\nThis was designed and built in 48 hours as part of the Gopher Gala Golang 48 hour hackathon.\nHence kindly forgive me for the hackish code, and lack of tests.\n\n[On hind sight - container rush or docker flood would have been a better name.]\n\n[Screencast](https://asciinema.org/a/15752)\n\n## Problem Statement\n\nAs containers get more mainstream, server deployments are soon going to be done through a containerization solution like docker [or rocket or lxd].\n\nLarge scale deploys are going to choke your docker registry.\nImagine pulling/deploying a 800mb base image (for example the official perl image) across 200 machines in one go.\nThat's 800*200 = 160GB of data that's going to be deployed and it'll definitely choke your private docker registry (and will take a while pulling in from the public image)\n\n## Solution\n\nThe solution is BitTorrent. Any technology good for distributing music and movies, is usually good for doing server deploys.\nInstead of downloading from one point source, we can now distribute the load across the machines downloading themselves. All of this transparently and automatically handled by the docket client.\n\n## Silly Totally Non-scientific Benchmark\n\nPulling a 800 mb image across 5 machines simultaneously via Docket resulted in a 90% speedup.\n\n## Features\n\n- Written in Golang :)\n- Very easy to use\n- Works along side your private docker registry\n\n## Concepts\n\nDocket constitutes of 3 components:\n\n1. Docket Registry\n\nA REST service that acts as a registry. It receives docker image tarballs from the client, stores metadata into a database,\ncreates torrents out of them and seeds them.\n\n2. Docket Client\n\nThe client itnerface in which the end user will be interacting. Can view available images in the registry, push an image to the registry and pull an image (which triggers a bittorrent deploy) from the registry.\n\n3. Bittorrent Tracker\n\nDocket allows you to BYOT (bring your own tracker), but I suggest installing opentracker\n\nDocket uses ctorrent for the actual downloading of torrents.\n\n## Installation\n\n### Step 0: You need to already have Docker installed\n\n### Step 1: Install ctorrent and opentracker\n\n```bash\n$ sudo apt-get update\n$ sudo apt-get install zlib1g-dev make g++ ctorrent\n$ git clone https://github.com/gebi/libowfat\n$ make\n$ git clone git://erdgeist.org/opentracker\n$ cd opentracker\n$ make\n```\n\n### Step 2: Download the Docket binaries\n\n```bash\n$ wget http://storage.googleapis.com/docket/docket.zip\n$ unzip docket.zip\n```\n\n### Step 3: That's all\n\n\n## Getting Started\n\n### Step 1: Fire up the Tracker\n\n```bash\n$ cd opentracker\n$ ./opentracker 8940\n```\n\n### Step 2: Fire up the Registry\n\n```bash\n$ cd registry\n# Note you need to put an ip address in which other machines can contact the tracker\n# You cannot put in localhost or 127.0.0.1\n$ sudo ./registry --tracker \"10.240.101.85:8940\"\n```\n\n## Usage\n\n### Push an image to the Registry\n\n```bash\n$ cd client\n#Note: You will have to to explicitly mention the tag \":latest\"\n$ sudo ./client -h \"http://10.240.101.85\" push netvarun/test:latest\nFound image:  netvarun/test:latest\nID:  353b94eb357ddb343ebe054ccc80b49bb6d0828522e9f2eff313406363449d17\nRepoTags:  [netvarun/test:latest]\nCreated:  1422145581\nSize:  0\nVirtualSize:  188305056\nParentId:  d7d8be71d422a83c97849c4a8e124fcbe42170d5ce508f339ce52be9954dc3b4\nExporting image to tarball...\nSuccessively exported tarball...\nkey =  image  val =  netvarun/test:latest\nkey =  id  val =  353b94eb357ddb343ebe054ccc80b49bb6d0828522e9f2eff313406363449d17\nkey =  created  val =  1422145581\nSuccessfully uploaded image:  netvarun/test:latest  to the Docket registry.\n```\n\n### List all images available in the Registry\n\n```bash\n$ cd client\n$ sudo ./client -h \"http://10.240.101.85\" images\nnetvarun/test:latest\nperl:latest\n\n```\n\n### Do a deploy\n\n```bash\n$ cd client\n$ sudo ./client -h \"http://10.240.101.85\" pull perl:latest\nDownloading the torrent file for image:  perl:latest\nDownloading from the torrent file...\nMETA INFO\nAnnounce: http://10.240.101.85:8940/announce\nPiece length: 524288\nCreated with: docket-registry\nFILES INFO\n\u003c1\u003e 14f61693dd2db6380755a662d6e4e3583b5214fad9032bd983ce6c70df2144bc_perl_latest.tar [838467072]\nTotal: 799 MB\nFound bit field file; verifying previous state.\nListening on 0.0.0.0:2706\nPress 'h' or '?' for help (display/control client options).\nChecking completed.\n- 0/0/1 [0/1600/0] 0MB,0MB | 0,0K/s | 0,0K E:0,0 Connecting\nEnd of input reached.\nInput channel is now off\n\\ 1/0/2 [1572/1600/1600] 785MB,0MB | 36422,0K/s | 38896,0K E:0,1\nDownload complete.\nTotal time used: 0 minutes.\nSeed for others 0 hours\n| 0/0/2 [1600/1600/1600] 799MB,0MB | 0,0K/s | 14320,0K E:0,1 Connecting\n\nTarball path =  /tmp/docket/14f61693dd2db6380755a662d6e4e3583b5214fad9032bd983ce6c70df2144bc_perl_latest.tar\nExporting image to tarball...\n```\n\n## Reference (Docket Registry)\n\n```bash\n./registry --help\n\nusage: registry [\u003cflags\u003e]\n\nDocket Registry\n\nFlags:\n  --help             Show help.\n  -t, --tracker=\"10.240.101.85:8940\"  \n                     Set host and port of bittorrent tracker. Example: -host 10.240.101.85:8940 Note: This cannot be set to localhost, since this is the tracker in which all the torrents will be created with. They have to be some accessible ip\n                     address from outside\n  -p, --port=\"8000\"  Set port of docket registry.\n  -l, --location=\"/var/local/docket\"  \n                     Set location to save torrents and docker images.\n```\n\n## Reference (Docket Client)\n\n```bash\n./client \n\nusage: client [\u003cflags\u003e] \u003ccommand\u003e [\u003cflags\u003e] [\u003cargs\u003e ...]\n\nDocket Client\n\nFlags:\n  --help             Show help.\n  -h, --host=\"http://127.0.0.1\"  \n                     Set host of docket registry.\n  -p, --port=\"8000\"  Set port of docket registry.\n  -l, --location=\"/tmp/docket\"  \n                     Set location to store torrents and tarballs.\n\nCommands:\n  help [\u003ccommand\u003e]\n    Show help for a command.\n\n  push \u003cpush\u003e\n    Push to the docket registry.\n\n  pull \u003cpull\u003e\n    pull to the docket registry.\n\n  images [\u003cflags\u003e]\n    display images in the docket registry.\n\n```\n\n## Author \n\nSivamani Varun (varun@semantics3.com)\n\n\n## License\n\nMIT License\n\n","funding_links":[],"categories":["Go","Docker Images","Deployment and Infrastructure","others"],"sub_categories":["Registry"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnetvarun%2Fdocket","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnetvarun%2Fdocket","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnetvarun%2Fdocket/lists"}