{"id":15055537,"url":"https://github.com/lowitea/gitlobster","last_synced_at":"2025-04-10T03:52:40.455Z","repository":{"id":64221204,"uuid":"539184114","full_name":"lowitea/gitlobster","owner":"lowitea","description":"🦞 A tool for full cloning all available repositories from a GitLab instance.","archived":false,"fork":false,"pushed_at":"2025-04-07T10:43:49.000Z","size":741,"stargazers_count":48,"open_issues_count":2,"forks_count":3,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-04-10T03:52:32.197Z","etag":null,"topics":["gitlab","hacktoberfest"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lowitea.png","metadata":{"files":{"readme":"README.RU.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2022-09-20T20:41:10.000Z","updated_at":"2025-04-09T12:20:35.000Z","dependencies_parsed_at":"2024-01-22T11:16:31.236Z","dependency_job_id":"33ee60ec-b298-4040-bf23-3d8c71bf3171","html_url":"https://github.com/lowitea/gitlobster","commit_stats":{"total_commits":177,"total_committers":5,"mean_commits":35.4,"dds":0.3220338983050848,"last_synced_commit":"384e4055f2885bab822407e7c088ee58bce19e5c"},"previous_names":[],"tags_count":11,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lowitea%2Fgitlobster","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lowitea%2Fgitlobster/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lowitea%2Fgitlobster/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lowitea%2Fgitlobster/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lowitea","download_url":"https://codeload.github.com/lowitea/gitlobster/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248154998,"owners_count":21056542,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["gitlab","hacktoberfest"],"created_at":"2024-09-24T21:44:11.863Z","updated_at":"2025-04-10T03:52:40.431Z","avatar_url":"https://github.com/lowitea.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Gitlobster\n\n**_Инструмент полного копирования всех доступных репозиториев с сервера GitLab._**\n\n[![Crates.io](https://img.shields.io/crates/v/gitlobster?style=for-the-badge)](https://crates.io/crates/gitlobster)\n[![Docker Image Version (latest semver)](https://img.shields.io/docker/v/lowitea/gitlobster?sort=semver\u0026label=docker\u0026style=for-the-badge)](https://hub.docker.com/r/lowitea/gitlobster)\n[![GitHub Workflow Status](https://img.shields.io/github/actions/workflow/status/lowitea/gitlobster/integration_test.yml?branch=master\u0026label=integration%20tests\u0026style=for-the-badge)](https://github.com/lowitea/gitlobster/actions)\n[![GitHub Workflow Status](https://img.shields.io/github/actions/workflow/status/lowitea/gitlobster/test.yml?branch=master\u0026label=unit%20tests\u0026style=for-the-badge)](https://github.com/lowitea/gitlobster/actions)\n\n[EN](README.md) | RU\n\n\u003cbr\u003e\n\u003cbr\u003e\n\n\u003cp align=\"center\"\u003e\u003cimg src=\"https://github.com/lowitea/gitlobster/raw/master/logo.png\"\u003e\u003c/p\u003e\n\n\u003cbr\u003e\n\n## Ключевые функции\n\n- Клонирование всех доступных репозиториев\n- Клонирование всех веток каждого репозитория\n- Загрузка всех репозиториев в другой сервер GitLab (или в другую группу)\n- Поддержка скачивания только обновлений (включая скачивание новых репозиториев), после первого полного клонирования\n- Сохранение иерархии групп\n- Поддержка фильтров (в том числе на основе регулярных выражений) для копирования только нужных репозиториев\n\n## Установка\n\n### Docker\n\n```shell\ndocker run --rm -it lowitea/gitlobster:latest --help\n```\n\n### Запуск предварительно собранных бинарных файлов\n\n1. Скачать собранный архив на [странице релизов](https://github.com/lowitea/gitlobster/releases) под свою платформу.\n2. Распаковать архив.\n3. Запустить через консоль файл `gitlobster`.\n\n### Cargo\n\n```shell\ncargo install gitlobster\n```\n\n### Сборка из исходников\n\n```shell\n# клонирование репозитория\ngit clone https://github.com/lowitea/gitlobster.git\n# переход в скачанную директорию\ncd gitlobster\n# сборка\ncargo build --release\n# запуск\n./target/release/gitlobster --help\n```\n\n_Также есть возможность запустить в режиме разработки, без предварительной сборки._\n\n```shell\n# в директории с проектом\ncargo run -- --help\n```\n\n## Использование\n\n### GitLab токен\n\nДля работы программы нужно сгенерировать токен GitLab с правами на чтение API (`read_api`). Если не используется копирование по SSH, также необходимы права на чтение репозиториев (`read_repository`).\n\nЕсли репозитории копируются во второй GitLab, то токен нужен также и для него. Необходимы полные права на API (`api`). Если не используется загрузка по SSH, также необходимы права на запись репозиториев (`write_repository`).\n\nСгенерировать токены можно на [странице настроек](https://gitlab.com/-/profile/personal_access_tokens).\n\n### SSH\n\nЕсли используется копирование через SSH, тогда ssh-ключи должны быть [добавлены](https://gitlab.com/-/profile/keys) в GitLab.\n\n### Копирование всех репозиториев в другой GitLab\n\n```shell\ngitlobster \\\n    --ft=\u003cFETCH_TOKEN\u003e \\\n    --fu=https://gitlab.com/ \\\n    --bt=\u003cUPLOAD_TOKEN\u003e \\\n    --bu=https://gitlab.com/ \\\n    --bg=gitlobster_test/upload\n```\n\n### Скачивание всех репозиториев в локальную папку\n\n```shell\ngitlobster \\\n    --ft=\u003cFETCH_TOKEN\u003e \\\n    --fu=https://gitlab.com/ \\\n    -d out_directory\n```\n\n_Поддерживается одновременное сохранение репозиториев локально и копирование во второй GitLab._\n\n### Использование фильтров и фильтрующих флагов\n\n```shell\ngitlobster \\\n    --ft=\u003cFETCH_TOKEN\u003e \\\n    --fu=https://gitlab.com/ \\\n    --only-owned \\\n    --include=\"^gitlobster_test/download/project_2\" \\\n    --include=\"^gitlobster_test/download/project_1\" \\\n    -d out_directory\n```\n\n_Также поддерживается флаг `--exclude` для скачивания всех репозиториев, кроме тех, что подпадают под шаблон._\n\n_Можно использовать только один из флагов `--exclude` или `--include`. При одновременном использовании этих флагов будет выдана соответствующая ошибка._\n\n### Использование через докер\n\n```shell\ndocker run --rm -it -v $(pwd)/out:/out lowitea/gitlobster:latest \\\n    --ft=\u003cFETCH_TOKEN\u003e \\\n    --fu=https://gitlab.com/ \\\n    --include='^gitlobster_test/example' \\\n    -d /out\n```\n\n### Вызов help\n\n```text\n$ gitlobster --help\n\nA tool for cloning all available repositories in a GitLab instance\n\nUsage: gitlobster [OPTIONS] --fu \u003cFETCH URL\u003e --ft \u003cFETCH TOKEN\u003e\n\nOptions:\n      --fu \u003cFETCH URL\u003e\n          The GitLab instance URL for fetch repositories (example: https://gitlab.local/) [env: GTLBSTR_FETCH_URL=]\n      --ft \u003cFETCH TOKEN\u003e\n          Your personal GitLab token for fetch repositories [env: GTLBSTR_FETCH_TOKEN=]\n      --bu \u003cBACKUP URL\u003e\n          The GitLab instance URL for backup repositories (example: https://backup-gitlab.local/) [env: GTLBSTR_BACKUP_URL=]\n      --bt \u003cBACKUP TOKEN\u003e\n          Your personal GitLab token for backup repositories [env: GTLBSTR_BACKUP_TOKEN=]\n      --bg \u003cBACKUP GROUP\u003e\n          A target created group on backup GitLab for push repositories [env: GTLBSTR_BACKUP_GROUP=]\n  -i, --include \u003cPATTERN\u003e\n          Include regexp patterns (cannot be used together with --exclude flag, may be repeated) [env: GTLBSTR_INCLUDE=]\n  -x, --exclude \u003cPATTERN\u003e\n          Comma separated exclude regexp patterns (cannot be used together with --include flag, may be repeated) [env: GTLBSTR_EXCLUDE=]\n  -d, --dst \u003cDIRECTORY\u003e\n          A destination local folder for save downloaded repositories [env: GTLBSTR_DST=]\n  -v, --verbose...\n          Verbose level (one or more, max four)\n      --dry-run\n          Show all projects to download\n      --objects-per-page \u003cCOUNT\u003e\n          Low-level option, how many projects can fetch in one request [env: GTLBSTR_OBJECTS_PER_PAGE=]\n      --limit \u003cCOUNT\u003e\n          Maximum projects to download [env: GTLBSTR_LIMIT=]\n      --concurrency-limit \u003cLIMIT\u003e\n          Limit concurrency download [env: GTLBSTR_CONCURRENCY_LIMIT=] [default: 21]\n      --only-owned\n          Download projects explicitly owned by user [env: GTLBSTR_ONLY_OWNED=]\n      --only-membership\n          Download only user's projects [env: GTLBSTR_ONLY_MEMBERSHIP=]\n      --group \u003cGROUP\u003e\n          Download projects only in group [env: GTLBSTR_GROUP=]\n      --download-ssh\n          Enable download by ssh instead of http. An authorized ssh key is required [env: GTLBSTR_DOWNLOAD_SSH=]\n      --upload-ssh\n          Enable upload by ssh instead of http. An authorized ssh key is required [env: GTLBSTR_UPLOAD_SSH=]\n      --download-force-http\n          Force download repositories by insecure protocol. Does not work with the download_ssh flag [env: GTLBSTR_DOWNLOAD_FORCE_HTTP=]\n      --download-force-https\n          Force download repositories by secure protocol. Does not work with the download_ssh flag [env: GTLBSTR_DOWNLOAD_FORCE_HTTPS=]\n      --upload-force-http\n          Force upload repositories by insecure protocol. Does not work with the upload_ssh flag [env: GTLBSTR_UPLOAD_FORCE_HTTP=]\n      --upload-force-https\n          Force upload repositories by secure protocol. Does not work with the upload_ssh flag [env: GTLBSTR_UPLOAD_FORCE_HTTPS=]\n      --disable-hierarchy\n          Disable saving the directory hierarchy [env: GTLBSTR_DISABLE_HIERARCHY=]\n      --clear-dst\n          Clear dst path before cloning [env: GTLBSTR_CLEAR_DST=]\n      --only-master\n          Download only default branch [env: GTLBSTR_ONLY_MASTER=]\n      --disable-sync-date\n          Disable adding sync dates in project descriptions [env: GTLBSTR_DISABLE_SYNC_DATE=]\n      --gitlab-timeout \u003cGITLAB_TIMEOUT\u003e\n          Timeout for requests to GitLab instances in seconds [env: GTLBSTR_GITLAB_TIMEOUT=]\n      --continue-on-error\n          Continue downloading if there are any errors [env: GTLBSTR_CONTINUE_ON_ERROR=]\n      --exclude-archived\n          Exclude archived projects\n  -h, --help\n          Print help\n  -V, --version\n          Print version\n```\n\n### Предупреждение\n\nПустые проекты не будут перенесены. Так как, их невозможно склонировать через git, пока в этих проектах не будет создано ни одной ветки ([issue 37](https://github.com/lowitea/gitlobster/issues/37)).\n\n## Аналоги\n\n- [gitlabber](https://github.com/ezbz/gitlabber)\n- [ghorg](https://github.com/gabrie30/ghorg)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flowitea%2Fgitlobster","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flowitea%2Fgitlobster","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flowitea%2Fgitlobster/lists"}