Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/gaultier/gitlab-clone-all
Clone locally all git projects from Gitlab.
https://github.com/gaultier/gitlab-clone-all
git gitlab rust
Last synced: about 2 months ago
JSON representation
Clone locally all git projects from Gitlab.
- Host: GitHub
- URL: https://github.com/gaultier/gitlab-clone-all
- Owner: gaultier
- Created: 2021-11-25T16:00:55.000Z (about 3 years ago)
- Default Branch: master
- Last Pushed: 2022-05-06T06:24:49.000Z (over 2 years ago)
- Last Synced: 2024-03-17T22:10:50.936Z (9 months ago)
- Topics: git, gitlab, rust
- Language: Rust
- Homepage:
- Size: 73.2 KB
- Stars: 0
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# gitlab-clone-all
Clone locally all git projects from Gitlab. This is handy for example to search locally with `ripgrep` very quickly, or hack on projects.
Every project is cloned concurrently for maximum performance and the memory usage remains under 70MiB even with thousands of big projects.
## Usage
*The api token is optional. Without it, only publicly accessible repositories can be cloned. Go to https://gitlab.custom.com/-/profile/personal_access_tokens to create one with `read_api` and `read_repository` access.*
Usage:
```
Clone all git repositories from gitlab
USAGE:
gitlab-clone-all [OPTIONS]
OPTIONS:
-a, --api-token [default: ]
-c, --clone-method [default: https] [possible values: https, ssh]
-d, --directory Root directory where to clone all the projects [default: .]
-h, --help Print help information
-u, --url [default: gitlab.com]
```
Build:
```sh
# Requires rust, libgit2 & openssl e.g. `brew install openssl libgit2 rust`
$ cargo build --release
```
Simple usage (the exact output will be different for you):
```sh
$ ./target/release/gitlab-clone-all --directory=/tmp
...
✓ youlysses/pmm-theme (2.5 KB, 6 objects)
✓ dkrikun/someproj (531 B, 5 objects)
✓ naggie/averclock (37.8 KB, 268 objects)
✓ rocksoniko/easy (305 B, 3 objects)
✓ diverops/hello-again (678 B, 6 objects)
✓ leberwurscht/teardownwalls (268.7 KB, 932 objects)
✓ hcs/hcs_utils (250.2 KB, 858 objects)
✓ alessioalex/pushover (428.8 KB, 738 objects)
✓ thanhtam1612/xdpm2010 (1.1 KB, 12 objects)
✓ brad_richards/math-stuff (671.5 KB, 1415 objects)
...
Successfully cloned: 299/300 (1.7 GB)
Duration: 270.068645291s
```
Complex usage:
```sh
$ ./target/release/gitlab-clone-all --api-token --clone-method=ssh --directory=/tmp/ --url=gitlab.custom.com
```
## Development
```sh
# Adapt for your platform
$ brew install rust openssl libgit2
# Optional verbose logs
$ export RUST_LOG=debug
$ cargo run -- --api-token="$GITLAB_API_TOKEN" --directory=/tmp/$(date +%s) --clone-method=ssh
```
## Docker
*Due to limitations of git/Gitlab/the current capabilities of the project, running the app in Docker won't work if authentication is required without extra work (that's because you need to copy your ssh private key into the container).*
```sh
$ docker build -t gitlab-clone-all:latest .
$ docker run -it gitlab-clone-all:latest gitlab-clone-all --directory=/tmp/
```
## Roadmap
- [ ] Retrying
- [ ] `--me` option to only clone my repositories
- [ ] Stop if no project could be fetched from the Gitlab API at all
- [ ] Max network rate CLI option