{"id":22295905,"url":"https://github.com/mkalioby/partialcopy","last_synced_at":"2025-03-25T22:34:26.036Z","repository":{"id":57450887,"uuid":"215496254","full_name":"mkalioby/partialCopy","owner":"mkalioby","description":"A tool to copy big data to multiple smaller disks","archived":false,"fork":false,"pushed_at":"2019-10-18T07:45:45.000Z","size":22,"stargazers_count":0,"open_issues_count":0,"forks_count":1,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-03-20T17:09:03.133Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mkalioby.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-10-16T08:26:09.000Z","updated_at":"2019-10-18T07:45:46.000Z","dependencies_parsed_at":"2022-09-26T17:31:15.431Z","dependency_job_id":null,"html_url":"https://github.com/mkalioby/partialCopy","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mkalioby%2FpartialCopy","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mkalioby%2FpartialCopy/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mkalioby%2FpartialCopy/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mkalioby%2FpartialCopy/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mkalioby","download_url":"https://codeload.github.com/mkalioby/partialCopy/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245556438,"owners_count":20634881,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-03T17:43:33.105Z","updated_at":"2025-03-25T22:34:26.004Z","avatar_url":"https://github.com/mkalioby.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![PyPI version](https://badge.fury.io/py/partialCopy.svg)](https://badge.fury.io/py/partialCopy)\n\n# partialCopy\nA tool to copy big data to multiple smaller disks\n\n## Motivation\n\nAs the storage becomes larger in big projects, we need to a tool to break large folders (100 TBs) to smaller chunks, so we can migrate to another location or storing it on tapes.\n\n## How does it work?\n\nThe tool finds the best placement for the files and it creates a files list in --save-to directory which can be passed rsync using `--files-from` parameter.\n## Installation\n\n```sudo pip install partialCopy```\n\n## Usage\n```\nusage: pcp.py [-h] [--dest DEST | --dest-size DEST_SIZE] [-s SAVE_TO] [-f]\n              [-n] [-ma MODIFIED_AFTER] [-fp FIND_PARAMS]\n              src\n\npositional arguments:\n  src                   Source Directory\n\noptional arguments:\n  -h, --help            show this help message and exit\n  --dest DEST           Destionation mountpoint\n  --dest-size DEST_SIZE\n                        Destination size, given in bytes or using 1 letter\n                        unit B,K,M,G,T,P\n  -s SAVE_TO, --save-to SAVE_TO\n                        Where to save rsync list,default\n                        '$src/pcp_rsync_list/'\n  -f, --force           Rewrite all lists again\n  -n, --new             Find New Files\n  -ma MODIFIED_AFTER, --modified-after MODIFIED_AFTER\n                        Find files modified after certain time (YYYY-mm-dd)\n  -fp FIND_PARAMS, --find-params FIND_PARAMS\n                        Parameters to find command\n\n```\n## Changes\n* Allowing to provide size rather than mount point.\n* Finds only new files using `-n` parameter.\n* Finds files modified after a certain date using `-ma` parameter.\n* Allows rewriting all lists using `-f` flag.  \n\n\n# Contributors\n* [Zeeshan Ali Shah](https://github.com/zeeshanali)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmkalioby%2Fpartialcopy","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmkalioby%2Fpartialcopy","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmkalioby%2Fpartialcopy/lists"}