{"id":13495633,"url":"https://github.com/ossobv/proxmove","last_synced_at":"2025-04-05T11:13:06.245Z","repository":{"id":38539746,"uuid":"66563898","full_name":"ossobv/proxmove","owner":"ossobv","description":"Migrate virtual machines between different Proxmox VE clusters with minimal downtime","archived":false,"fork":false,"pushed_at":"2023-12-01T11:17:04.000Z","size":566,"stargazers_count":277,"open_issues_count":17,"forks_count":31,"subscribers_count":14,"default_branch":"main","last_synced_at":"2025-03-29T10:09:54.952Z","etag":null,"topics":["cli","proxmox"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ossobv.png","metadata":{"files":{"readme":"README.rst","changelog":"CHANGES.rst","contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2016-08-25T14:15:50.000Z","updated_at":"2025-03-25T13:51:38.000Z","dependencies_parsed_at":"2024-01-08T08:04:06.008Z","dependency_job_id":null,"html_url":"https://github.com/ossobv/proxmove","commit_stats":{"total_commits":111,"total_committers":2,"mean_commits":55.5,"dds":0.06306306306306309,"last_synced_commit":"18fcd8dfed316c70341255c0c3166a8229ac3b85"},"previous_names":[],"tags_count":12,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ossobv%2Fproxmove","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ossobv%2Fproxmove/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ossobv%2Fproxmove/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ossobv%2Fproxmove/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ossobv","download_url":"https://codeload.github.com/ossobv/proxmove/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247325696,"owners_count":20920714,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cli","proxmox"],"created_at":"2024-07-31T19:01:36.602Z","updated_at":"2025-04-05T11:13:06.217Z","avatar_url":"https://github.com/ossobv.png","language":"Python","funding_links":[],"categories":["Python","Tools"],"sub_categories":[],"readme":"|proxmove|\n==========\n\n*The Proxmox VM migrator: migrates VMs between different Proxmox VE clusters.*\n\nMigrating a virtual machine (VM) on a PVE-cluster from one node to\nanother is implemented in the Proxmox Virtual Environment (PVE). But\nmigrating a VM from one PVE-cluster to another is not.\n\nproxmove helps you move VMs between PVE-clusters with minimal hassle.\n*And if you use ZFS, with minimal downtime too.*\n\n\nExample invocation:\n\n.. code-block:: console\n\n    $ proxmove SOURCE_CLUSTER DEST_CLUSTER DEST_NODE DEST_STORAGE VM_NAME1...\n\nBut, to get it to work, you'll need to configure ``~/.proxmoverc``\nfirst. See `Configuration`_.\n\n\nAdditional tips:\n\n- If source and destination filesystems use ZFS, the move is done in two\n  stages, by copying an initial snapshot while the source VM is still\n  up. Combine with ``--wait-before-stop`` for additional control.\n- Use ``--debug``; it doesn't flood your screen, but provides useful clues\n  about what it's doing.\n- If your network bridge is different on the ``DEST_CLUSTER``, use\n  ``--skip-start``; that way *proxmove* \"completes\" successfully when\n  done with the move. (You'll still need to change the bridge before\n  starting the VM obviously.)\n- If *proxmove* detects that a move was in progress, it will\n  interactively attempt a resume. For ZFS to ZFS syncs, it will do\n  *another* initial resync before shutting down the source VM.\n\n\nFull invocation specification (``--help``):\n\n.. code-block::\n\n    usage: proxmove [-c FILENAME] [-n] [--bwlimit MBPS] [--no-verify-ssl]\n                    [--skip-disks] [--skip-start] [--wait-before-stop]\n                    [--ssh-ciphers CIPHERS] [--debug] [--ignore-exists]\n                    [-h] [--version]\n                    source destination nodeid storage vm [vm ...]\n\n    Migrate VMs from one Proxmox cluster to another.\n\n    positional arguments:\n      source                alias of source cluster\n      destination           alias of destination cluster\n      nodeid                node on destination cluster\n      storage               storage on destination node\n      vm                    one or more VMs (guests) to move\n\n    optional arguments:\n      -c FILENAME, --config FILENAME\n                            use alternate configuration inifile\n      -n, --dry-run         stop before doing any writes\n      --bwlimit MBPS        limit bandwidth in Mbit/s\n      --no-verify-ssl       skip ssl verification on the api hosts\n      --skip-disks          do the move, but skip copying of the disks;\n                            implies --skip-start\n      --skip-start          do the move, but do not start the new instance\n      --wait-before-stop    prepare the move, but ask for user\n                            confirmation before shutting down the old\n                            instance (useful if you have to move\n                            networks/IPs)\n      --ssh-ciphers CIPHERS\n                            comma separated list of ssh -c ciphers to\n                            prefer, (aes128-gcm@openssh.com is supposed to\n                            be fast if you have aes on your cpu); set to\n                            \"-\" to use ssh defaults\n\n    debug arguments:\n      --debug               enables extra debug logging\n      --ignore-exists       continue when target VM already exists; allows\n                            moving to same cluster\n\n    other actions:\n      -h, --help            show this help message and exit\n      --version             show program's version number and exit\n\n    Cluster aliases and storage locations should be defined in\n    ~/.proxmoverc (or see -c option). See the example proxmoverc.sample.\n    It requires [pve:CLUSTER_ALIAS] sections for the proxmox \"api\" URL and\n    [storage:CLUSTER_ALIAS:STORAGE_NAME] sections with \"ssh\", \"path\" and\n    \"temp\" settings.\n\n\nExample run\n-----------\n\nFirst you need to configure ``~/.proxmoverc``; see below.\n\nWhen configured, you can do something like this:\n\n.. code-block:: console\n\n    $ proxmove apple-cluster banana-cluster node2 node2-ssd the-vm-to-move\n    12:12:27: Attempt moving apple-cluster\u003ce1400248\u003e =\u003e banana-cluster\u003c6669ad2c\u003e (node 'node2'): the-vm-to-move\n    12:12:27: - source VM the-vm-to-move@node1\u003cqemu/565/running\u003e\n    12:12:27: - storage 'ide2': None,media=cdrom (host=\u003cunknown\u003e, guest=\u003cunknown\u003e)\n    12:12:27: - storage 'virtio0': sharedsan:565/vm-565-disk-1.qcow2,format=qcow2,iops_rd=4000,iops_wr=500,size=50G (host=37.7GiB, guest=50.0GiB)\n    12:12:27: Creating new VM 'the-vm-to-move' on 'banana-cluster', node 'node2'\n    12:12:27: - created new VM 'the-vm-to-move--CREATING' as UPID:node2:00005977:1F4D78F4:57C55C0B:qmcreate:126:user@pve:; waiting for it to show up\n    12:12:34: - created new VM 'the-vm-to-move--CREATING': the-vm-to-move--CREATING@node2\u003cqemu/126/stopped\u003e\n    12:12:34: Stopping VM the-vm-to-move@node1\u003cqemu/565/running\u003e\n    12:12:42: - stopped VM the-vm-to-move@node1\u003cqemu/565/stopped\u003e\n    12:12:42: Ejected (cdrom?) volume 'ide2' (none) added to the-vm-to-move--CREATING@node2\u003cqemu/126/stopped\u003e\n    12:12:42: Begin copy of 'virtio0' (sharedsan:565/vm-565-disk-1.qcow2,format=qcow2,iops_rd=4000,iops_wr=500,size=50G) to local-ssd\n    12:12:42: scp(1) copy from '/pool0/san/images/565/vm-565-disk-1.qcow2' (on sharedsan) to 'root@node2.banana-cluster.com:/node2-ssd/temp/temp-proxmove/vm-126-virtio0'\n    Warning: Permanently added 'node2.banana-cluster.com' (ECDSA) to the list of known hosts.\n    vm-565-disk-1.qcow2   100%   50GB   90.5MB/s   09:26\n    Connection to san.apple-cluster.com closed.\n    12:22:08: Temp data '/node2-ssd/temp/temp-proxmove/vm-126-virtio0' on local-ssd\n    12:22:08: Writing data from temp '/node2-ssd/temp/temp-proxmove/vm-126-virtio0' to '/dev/zvol/node2-ssd/vm-126-virtio0' (on local-ssd)\n        (100.00/100%)\n    Connection to node2.banana-cluster.com closed.\n    12:24:25: Removing temp '/node2-ssd/temp/temp-proxmove/vm-126-virtio0' (on local-ssd)\n    12:24:26: Starting VM the-vm-to-move@node2\u003cqemu/126/stopped\u003e\n    12:24:27: - started VM the-vm-to-move@node2\u003cqemu/126/running\u003e\n    12:24:27: Completed moving apple-cluster\u003ce1400248\u003e =\u003e banana-cluster\u003c6669ad2c\u003e (node 'node2'): the-vm-to-move\n\nBefore, ``the-vm-to-move`` was running on ``apple-cluster`` on ``node1``.\n\nAfterwards, ``the-vm-to-move`` is running on ``banana-cluster`` on ``node2``.\nThe ``the-vm-to-move`` on the ``apple-cluster`` has been stopped and renamed to\n``the-vm-to-move--MIGRATED``.\n\n\nConfiguration\n-------------\n\nSet up the ``~/.proxmoverc`` config file. First you need to define which\nclusters you have. For example *apple-cluster* and *banana-cluster*.\n\n.. code-block:: ini\n\n    ; Example cluster named \"apple-cluster\" with 3 storage devices, one\n    ; shared, and two which exist on a single node only.\n    ;\n    ; The user requires various permissions found in the PVEVMAdmin role (VM\n    ; allocate + audit) and PVEAuditor role (Datastore audit) and PVEPoolAdmin\n    ; (to inspect and create pools). And PVESDNAdmin role for the network conf.\n    ;\n    [pve:apple-cluster]\n    api=https://user@pve:PASSWORD@apple-cluster.com:443\n\n    ; Example cluster named \"banana-cluster\" with 2 storage devices; both\n    ; storage devices exist on the respective nodes only.\n    [pve:banana-cluster]\n    api=https://user@pve:PASSWORD@banana-cluster.com:443\n\nNext, it needs configuration for the storage devices. They are expected\nto be reachable over SSH; both from the caller and from each other\n(using SSH-agent forwarding).\n\nThe following defines two storage devices for the *apple-cluster*, one shared\nand one local to *node1* only.\n\nIf on *sharedsan*, the images are probably called something like\n``/pool0/san/images/VMID/vm-VMID-disk1.qcow2``, while in Proxmox, they are\nreferred to as ``sharedsan:VMID/vm-VMID-disk1.qcow2``.\n\n.. code-block:: ini\n\n    [storage:apple-cluster:sharedsan] ; \"sharedsan\" is available on all nodes\n    ssh=root@san.apple-cluster.com\n    path=/pool0/san/images\n    temp=/pool0/san/private\n\n    [storage:apple-cluster:local@node1] ; local disk on node1 only\n    ssh=root@node1.apple-cluster.com\n    path=/srv/images\n    temp=/srv/temp\n\nIf you use ZFS storage on *banana-cluster*, the storage config could look\nlike this. Disk volumes exist on the ZFS filesystem ``node1-ssd/images``\nand ``node2-ssd/images`` on the nodes *node1* and *node2* respectively.\n\nNote that the ``temp=`` path is always a regular path.\n\n.. code-block:: ini\n\n    [storage:banana-cluster:node1-ssd@node1]\n    ssh=root@node1.banana-cluster.com\n    path=zfs:node1-ssd/images\n    temp=/node1-ssd/temp\n\n    [storage:banana-cluster:node2-ssd@node2]\n    ssh=root@node2.banana-cluster.com\n    path=zfs:node2-ssd/images\n    temp=/node2-ssd/temp\n\nThe config file looks better with indentation. The author suggests this layout:\n\n.. code-block:: ini\n\n    [pve:apple-cluster]\n    ...\n\n      [storage:apple-cluster:sharedsan]\n      ...\n      [storage:apple-cluster:local@node1]\n      ...\n\n    [pve:banana-cluster]\n    ...\n\n      [storage:banana-cluster:node1-ssd@node1]\n      ...\n\n\nDebugging\n---------\n\nIf you run into a ``ResourceException``, you may want to patch proxmoxer 1.0.3\nto show the HTTP error reason as well.\n\n.. code-block:: udiff\n\n    --- proxmoxer/core.py\t2019-04-04 09:13:16.832961589 +0200\n    +++ proxmoxer/core.py\t2019-04-04 09:15:45.434175030 +0200\n    @@ -75,8 +75,10 @@ class ProxmoxResource(ProxmoxResourceBas\n             logger.debug('Status code: %s, output: %s', resp.status_code, resp.content)\n\n             if resp.status_code \u003e= 400:\n    -            raise ResourceException(\"{0} {1}: {2}\".format(resp.status_code, httplib.responses[resp.status_code],\n    -                                                          resp.content))\n    +            raise ResourceException('{0} {1} (\"{2}\"): {3}'.format(\n    +                resp.status_code, httplib.responses[resp.status_code],\n    +                resp.reason,  # reason = textual status_code\n    +                resp.content))\n             elif 200 \u003c= resp.status_code \u003c= 299:\n                 return self._store[\"serializer\"].loads(resp)\n\nIt might reveal a bug (or new feature), like::\n\n    proxmoxer.core.ResourceException:\n      500 Internal Server Error (\"only root can set 'vmgenid' config\"):\n      b'{\"data\":null}'\n\n\nLicense\n-------\n\nproxmove is free software: you can redistribute it and/or modify it under\nthe terms of the GNU General Public License as published by the Free\nSoftware Foundation, version 3 or any later version.\n\n\n.. |proxmove| image:: assets/proxmove_head.png\n    :alt: proxmove\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fossobv%2Fproxmove","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fossobv%2Fproxmove","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fossobv%2Fproxmove/lists"}