{"id":17614018,"url":"https://github.com/w0rmr1d3r/cloner","last_synced_at":"2025-04-29T03:31:02.281Z","repository":{"id":37951549,"uuid":"424339672","full_name":"w0rmr1d3r/cloner","owner":"w0rmr1d3r","description":"A tool to clone efficiently all the repos in an organization","archived":false,"fork":false,"pushed_at":"2024-04-12T09:57:00.000Z","size":414,"stargazers_count":8,"open_issues_count":10,"forks_count":0,"subscribers_count":4,"default_branch":"master","last_synced_at":"2024-04-12T16:13:23.940Z","etag":null,"topics":["clone","git","github","multiprocessing","multithreading","organization","python","python3"],"latest_commit_sha":null,"homepage":"https://pypi.org/project/wr-cloner/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/w0rmr1d3r.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null},"funding":{"github":["w0rmr1d3r"]}},"created_at":"2021-11-03T18:34:38.000Z","updated_at":"2024-04-15T15:08:59.987Z","dependencies_parsed_at":"2023-02-14T00:00:25.034Z","dependency_job_id":"3ef2f3fc-a1c1-4e71-939f-4b2454b79199","html_url":"https://github.com/w0rmr1d3r/cloner","commit_stats":null,"previous_names":[],"tags_count":17,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/w0rmr1d3r%2Fcloner","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/w0rmr1d3r%2Fcloner/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/w0rmr1d3r%2Fcloner/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/w0rmr1d3r%2Fcloner/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/w0rmr1d3r","download_url":"https://codeload.github.com/w0rmr1d3r/cloner/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251426801,"owners_count":21587651,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["clone","git","github","multiprocessing","multithreading","organization","python","python3"],"created_at":"2024-10-22T18:18:48.901Z","updated_at":"2025-04-29T03:31:02.275Z","avatar_url":"https://github.com/w0rmr1d3r.png","language":"Python","funding_links":["https://github.com/sponsors/w0rmr1d3r"],"categories":[],"sub_categories":[],"readme":"# cloner\n\nA tool to clone efficiently all the repos in an organization\n\n[![PyPI](https://img.shields.io/pypi/v/wr-cloner)](https://pypi.org/project/wr-cloner/)\n[![GitHub release (latest by date)](https://img.shields.io/github/v/release/w0rmr1d3r/cloner)](https://github.com/w0rmr1d3r/cloner/releases)\n![PyPI - Python Version](https://img.shields.io/pypi/pyversions/wr-cloner)\n![GitHub last commit](https://img.shields.io/github/last-commit/w0rmr1d3r/cloner)\n[![CI](https://github.com/w0rmr1d3r/cloner/actions/workflows/ci.yml/badge.svg?branch=master)](https://github.com/w0rmr1d3r/cloner/actions/workflows/ci.yml)\n[![CodeQL](https://github.com/w0rmr1d3r/cloner/actions/workflows/codeql.yml/badge.svg?branch=master)](https://github.com/w0rmr1d3r/cloner/actions/workflows/codeql.yml)\n[![OpenSSF Scorecard](https://api.scorecard.dev/projects/github.com/w0rmr1d3r/cloner/badge)](https://scorecard.dev/viewer/?uri=github.com/w0rmr1d3r/cloner)\n[![OpenSSF Best Practices](https://www.bestpractices.dev/projects/9982/badge)](https://www.bestpractices.dev/projects/9982)\n[![PyPi downloads](https://img.shields.io/pypi/dm/wr-cloner?label=PyPi%20downloads)](https://pypistats.org/packages/wr-cloner)\n[![PyPI Downloads](https://static.pepy.tech/badge/wr-cloner/month)](https://pepy.tech/projects/wr-cloner)\n\n## Install\n\n_In PyPi it's called **wr-cloner**, cloner was already taken :sad:_\n\n```bash\npip install wr-cloner\n```\n\n## Usage\n\n_Note: If using cloner after cloning the project and not a PyPi package, add \"python\" at the start.\nThere's an example of that in the examples section._\n\n```text\nUsage: cloner [OPTIONS] GITHUB_ORGANIZATION\n\n  A tool to clone efficiently all the repos in an organization.\n\nOptions:\n  --version                       Show the version and exit.\n  --token TEXT                    GitHub token to read private repos. This\n                                  parameter is needed when cloning from a\n                                  GitHub Enterprise server.\n  --ghe TEXT                      GitHub Enterprise URL. It needs the\n                                  GITHUB_ORGANIZATION parameter to clone repos\n                                  from there and the TOKEN option as well.\n  --threads INTEGER               Number of threads and processes to use. For\n                                  maximum threads and processes on the system,\n                                  use '--max-threads'  [default: 4]\n  --max-threads                   If declared, uses the maximum available\n                                  threads and processes in the system. As per\n                                  physical cores on the system cpu.\n  --logging [ERROR|WARNING|INFO|DEBUG]\n                                  Logging level  [default: INFO]\n  --path TEXT                     Sets a path where to clone the repositories\n                                  (eg: ./another/path/)\n  --git-options TEXT              Add options to the clone command (eg: --git-\n                                  options \"--depth 1\"). By default, clones\n                                  quietly (--quiet).\n  --ignore-archived               If declared, will ignore archived repos when\n                                  cloning.\n  --ignore-template               If declared, will ignore template repos when\n                                  cloning.\n  --ignore-fork                   If declared, will ignore fork repos when\n                                  cloning.\n  --exclude-repos TEXT            Comma separated list of repository names to\n                                  exclude from cloning. Example:\n                                  \"repository1,repository2\".\n  --help                          Show this message and exit.\n```\n\n### Examples\n\n```bash\n# For github.com with 8 threads\ncloner --threads 8 GITHUB_ORGANIZATION\n\n# For github.com with the maximum threads on the system running\ncloner --max-threads GITHUB_ORGANIZATION\n\n# For GHE, default threads\ncloner --ghe GHE_URL --token SUPER_SECURE_TOKEN GITHUB_ORGANIZATION\n\n# Cloning with options\ncloner --git-options \"--depth 1\" GITHUB_ORGANIZATION\n```\n\n### Examples if cloning the repo\n\n```bash\n# Install dependencies with\nmake install\n# Adjust PYTHONPATH\nexport PYTHONPATH=$PYTHONPATH:.\n# Use it\npython cloner GITHUB_ORGANIZATION\n```\n\n## Contributing\n\nCheck the [CONTRIBUTING.md](CONTRIBUTING.md) file.\n\n## Security\n\nFollow the instructions in the [SECURITY.md](SECURITY.md) file.\n\n## License\n\n[MIT](https://github.com/w0rmr1d3r/cloner/blob/master/LICENSE)\n\n## Known Issues\n\n- Multithreading doesn't work to clone repos, since the `os.system` call is 1 for each PID. The splitting is done with\n  multithreading, the cloning with multiprocessing. Same amount of threads and processes.\n- [Windows usage and support](docs/WINDOWS.md)\n- If seeing `RPC failed; curl 92 HTTP/2 stream 5 was not closed cleanly: CANCEL (err 8)` either adjust your git config,\n  as\n  per [this StackOverflow issue](https://stackoverflow.com/questions/59282476/error-rpc-failed-curl-92-http-2-stream-0-was-not-closed-cleanly-protocol-erro).\n  If still persists, either decrease the number of threads using the `--threads` option or adjust the depth of cloning\n  with `--git-options \"--depth 10\"` (10 or any other small number).\n\n## Star History\n\n[![Star History Chart](https://api.star-history.com/svg?repos=w0rmr1d3r/cloner\u0026type=Date)](https://star-history.com/#w0rmr1d3r/cloner\u0026Date)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fw0rmr1d3r%2Fcloner","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fw0rmr1d3r%2Fcloner","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fw0rmr1d3r%2Fcloner/lists"}