{"id":13439861,"url":"https://github.com/sheepdog/sheepdog","last_synced_at":"2025-03-20T09:30:48.674Z","repository":{"id":1540765,"uuid":"1854060","full_name":"sheepdog/sheepdog","owner":"sheepdog","description":"Distributed Storage System for QEMU","archived":false,"fork":false,"pushed_at":"2020-12-10T18:52:38.000Z","size":24338,"stargazers_count":988,"open_issues_count":131,"forks_count":263,"subscribers_count":134,"default_branch":"master","last_synced_at":"2024-10-28T02:17:51.867Z","etag":null,"topics":["blockdevice","c","distributed-storage","iscsi","iscsi-target","qemu","qemu-kvm","software-defined-storage","storage","virtualization","vmm"],"latest_commit_sha":null,"homepage":"http://sheepdog.github.io/sheepdog/","language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sheepdog.png","metadata":{"files":{"readme":"README","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"COPYING","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2011-06-06T10:42:34.000Z","updated_at":"2024-09-24T18:56:49.000Z","dependencies_parsed_at":"2022-08-16T13:40:38.658Z","dependency_job_id":null,"html_url":"https://github.com/sheepdog/sheepdog","commit_stats":null,"previous_names":[],"tags_count":80,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sheepdog%2Fsheepdog","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sheepdog%2Fsheepdog/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sheepdog%2Fsheepdog/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sheepdog%2Fsheepdog/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sheepdog","download_url":"https://codeload.github.com/sheepdog/sheepdog/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244585633,"owners_count":20476769,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["blockdevice","c","distributed-storage","iscsi","iscsi-target","qemu","qemu-kvm","software-defined-storage","storage","virtualization","vmm"],"created_at":"2024-07-31T03:01:17.729Z","updated_at":"2025-03-20T09:30:47.946Z","avatar_url":"https://github.com/sheepdog.png","language":"C","funding_links":[],"categories":["C","c"],"sub_categories":[],"readme":"Sheepdog: Distributed Storage System for KVM\n============================================\n\nOverview\n--------\nSheepdog is a distributed storage system for QEMU. It provides\nhighly available block level storage volumes to virtual machines. \nSheepdog supports advanced volume management features such as snapshot,\ncloning, and thin provisioning.\n\nSheepdog is an Open Source software, released under the terms of the\nGNU General Public License version 2. \n\nFor the latest information about Sheepdog, please visit our website at:\n\n   http://sheepdog.github.io/sheepdog/\n\nAnd (recommend for new comers) wiki at:\n   https://github.com/sheepdog/sheepdog/wiki/\n\nRequirements\n------------\n* Three or more x86-64 machines\n* Corosync cluster engine\n\nInstall\n-------\nPlease read the INSTALL file distributed with this package for detailed\ninstructions on installing or compiling from source.\n\nUsage\n-----\n\n* Cluster Management Backends\n\n   Sheepdog uses a cluster management backend to manage membership and broadcast\n   messages to the cluster nodes.\n\n   For now, sheepdog can use local driver (for development on a single box),\n   corosync (the default), zookeeper and Accord.\n\n* Local Driver\n\n   This driver just makes use of UNIX IPC mechanism to manage the membership\n   on a single box, where we start multiple 'sheep' processes to simulate the\n   cluster. It is very easy and fast setup and especially useful to test\n   functionality without involving any other software.\n\n   To set up a 3 node cluster using local driver in one liner bash:\n\n      $ mkdir /path/to/store\n      $ for i in 0 1 2; do sheep -c local /path/to/store/$i -z $i -p 700$i;done\n\n* Configure corosync.\n\n   Nearly every modern Linux distribution has x86_64 corosync binaries pre-built\n   available via their repositories. We recommend you use these packages if they\n   are available on your distribution.\n\n   For debian package based systems:\n\n      $ sudo aptitude install corosync libcorosync-dev\n\n   For RPM package based systems:\n\n      $ sudo yum install corosynclib-devel\n\n   Reference our wiki, the corosync(8) and corosync.conf(5) man page for further\n   details.\n\n* Setup Sheepdog\n   1. Launch sheepdog on each machines of the cluster.\n\n      $ sheep /store_dir\n\n      Notes:\n        /store_dir is a directory to store objects. The directory must\n        be on the filesystem with an xattr support. In case of ext3, you\n        need to add 'user_xattr' to the mount options.\n\n        $ sudo mount -o remount,user_xattr /store_device\n\n   2. Make fs\n\n      $ dog cluster format --copies=3\n\n      --copies specifies the number of default data redundancy. In this case,\n      the replicated data is stored on three machines.\n\n   3. Check cluster state\n\n      Following list shows that Sheepdog is running on 32 nodes.\n\n      $ dog node list\n        Idx\tNode id (FNV-1a) - Host:Port\n      ------------------------------------------------\n        0\t0308164db75cff7e - 10.68.13.15:7000\n      * 1\t03104d8b4315c8e4 - 10.68.13.1:7000\n        2\t0ab18c565bc14aea - 10.68.13.3:7000\n        3\t0c0d27f0ac395f5d - 10.68.13.16:7000\n        4\t127ee4802991f308 - 10.68.13.13:7000\n        5\t135ff2beab2a9809 - 10.68.14.5:7000\n        6\t17bd6240eab65870 - 10.68.14.4:7000\n        7\t1cf35757cbf47d7b - 10.68.13.10:7000\n        8\t1df9580b8960a992 - 10.68.13.11:7000\n        9\t29307d3fa5a04f78 - 10.68.14.12:7000\n        10\t29dcb3474e31d4f3 - 10.68.14.15:7000\n        11\t29e089c98dd2a144 - 10.68.14.16:7000\n        12\t2a118b7e2738f479 - 10.68.13.4:7000\n        13\t3d6aea26ba79d75f - 10.68.13.6:7000\n        14\t42f9444ead801767 - 10.68.14.11:7000\n        15\t562c6f38283d09fe - 10.68.14.2:7000\n        16\t5dd5e540cca1556a - 10.68.14.6:7000\n        17\t6c12a5d10f10e291 - 10.68.14.13:7000\n        18\t6dae1d955ca72d96 - 10.68.13.7:7000\n        19\t711db0f5fa40b412 - 10.68.14.14:7000\n        20\t7c6b95212ee7c085 - 10.68.14.9:7000\n        21\t7d010c31bf11df73 - 10.68.13.2:7000\n        22\t82c43e908b1f3f01 - 10.68.13.12:7000\n        23\t931d2de0aaf61cf5 - 10.68.13.8:7000\n        24\t961d9d391e6021e7 - 10.68.13.14:7000\n        25\t9a3ef6fa1081026c - 10.68.13.9:7000\n        26\tb0b3d300fed8bc26 - 10.68.14.10:7000\n        27\tb0f08fb98c8f5edc - 10.68.14.8:7000\n        28\tb9cc316dc5aba880 - 10.68.13.5:7000\n        29\td9eda1ec29c2eeeb - 10.68.14.7:7000\n        30\te53cebb2617c86fd - 10.68.14.1:7000\n        31\tea46913c4999ccdf - 10.68.14.3:7000\n\n* Create a virtual machine image\n   1. Create a 256 GB virtual machine image of Alice.\n\n      $ qemu-img create sheepdog:Alice 256G\n\n   2. You can also convert from existing KVM images to Sheepdog ones.\n\n      $ qemu-img convert ~/amd64.raw sheepdog:Bob\n\n   3. See Sheepdog images by the following command.\n\n      $ dog vdi list\n        name        id    size    used  shared    creation time  object id\n      --------------------------------------------------------------------\n        Bob          0  2.0 GB  1.6 GB  0.0 MB 2010-03-23 16:16      80000\n        Alice        0  256 GB  0.0 MB  0.0 MB 2010-03-23 16:16      40000\n\n* Boot the virtual machine\n   1. Boot the virtual machine.\n\n      $ qemu-system-x86_64 -hda sheepdog:Alice\n\n   2. Following command checks used images.\n\n      $ dog vm list\n      Name            |Vdi size |Allocated| Shared  | Status\n      ----------------+---------+---------+---------+------------\n      Bob             |   2.0 GB|   1.6 GB|   0.0 MB| running on xx.xx.xx.xx\n      Alice           |   256 GB|   0.0 MB|   0.0 MB| not running\n\n* Snapshot\n   1. Snapshot\n\n      $ qemu-img snapshot -c name sheepdog:Alice\n\n      -c flag is meaningless currently\n\n   2. After getting snapshot, a new virtual machine images are added as a not-\n      current image.\n\n      $ dog vdi list\n        name        id    size    used  shared    creation time  object id\n      --------------------------------------------------------------------\n        Bob          0  2.0 GB  1.6 GB  0.0 MB 2010-03-23 16:16      80000\n        Alice        0  256 GB  0.0 MB  0.0 MB 2010-03-23 16:21      c0000\n      s Alice        1  256 GB  0.0 MB  0.0 MB 2010-03-23 16:16      40000\n\n   3. You can boot from the snapshot image by spcifing tag id\n\n      $ qemu-system-x86_64 -hda sheepdog:Alice:1\n\n* Cloning from the snapshot\n   1. Create a Charlie image as a clone of Alice's image.\n\n      $ qemu-img create -b sheepdog:Alice:1 sheepdog:Charlie\n\n   2. Charlie's image is added to the virtual machine list.\n\n      $ dog vdi list\n        name        id    size    used  shared    creation time  object id\n      --------------------------------------------------------------------\n        Bob          0  2.0 GB  1.6 GB  0.0 MB 2010-03-23 16:16      80000\n        Alice        0  256 GB  0.0 MB  0.0 MB 2010-03-23 16:21      c0000\n      s Alice        1  256 GB  0.0 MB  0.0 MB 2010-03-23 16:16      40000\n        Charlie      0  256 GB  0.0 MB  0.0 MB 2010-03-23 16:23     100000\n\nTest Environment\n----------------\n    - Debian squeeze amd64\n    - Debian lenny amd64\n\n===============================================================================\nCopyright (C) 2009-2011, Nippon Telegraph and Telephone Corporation.\n\nJoin the chat at https://gitter.im/sheepdog/sheepdog\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsheepdog%2Fsheepdog","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsheepdog%2Fsheepdog","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsheepdog%2Fsheepdog/lists"}