https://github.com/samrocketman/cloneable
A Java CLI utility which lists available GitHub repositories under a user or organization.
https://github.com/samrocketman/cloneable
backups github-backup
Last synced: 8 months ago
JSON representation
A Java CLI utility which lists available GitHub repositories under a user or organization.
- Host: GitHub
- URL: https://github.com/samrocketman/cloneable
- Owner: samrocketman
- License: apache-2.0
- Created: 2020-05-09T10:00:43.000Z (over 5 years ago)
- Default Branch: main
- Last Pushed: 2025-03-27T00:41:49.000Z (9 months ago)
- Last Synced: 2025-03-27T01:32:52.197Z (9 months ago)
- Topics: backups, github-backup
- Language: Groovy
- Size: 116 KB
- Stars: 7
- Watchers: 2
- Forks: 0
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: LICENSE
Awesome Lists containing this project
README
# Cloneable :octocat: GitHub Backups
- :arrow_forward: Download `cloneable.jar` from [GitHub releases
page][download].
- :white_check_mark: Supports GitHub App or personal access token (PAT) for
querying and cloning.
- :white_check_mark: Clone over HTTP with app or PAT via [`GIT_ASKPASS`
backend][git-credentials].
- :white_check_mark: Query projects over HTTP and clone using SSH private key.
- :white_check_mark: Flexibly query projects based on age, contents, and other
factors. See CLI Documentation below for a full list.
- :white_check_mark: Minimal dependencies: Java, `/bin/sh`, and standard
utilities; GNU or BSD coreutils, and busybox supported.
A simple backup solution for offsite backups of GitHub repository source code.
Cloneable is a Java-based CLI application which queries GitHub and reports
cloneable projects. It only does one thing: lists repositories. Other Linux
utilities can be used in conjunction with cloneable to create and update a local
backup of GitHub. Useful for personal backups of GitHub with minimal
dependencies.
Cloneable utilizes [GitHub API v4 GraphQL][v4]. If you need a more thorough
backup solution, check out [other solutions on GitHub][other-backup].
# Download
Binary download is available in [GitHub releases][download].
# Setup
In order to call cloneable, it is recommended install the wrapper shell script
along with the bash competion for it.
java -jar cloneable.jar --print-cli-script > /usr/local/bin/cloneable
chmod 755 /usr/local/bin/cloneable
Install bash completion script for interactive auto-complete of options.
cloneable --print-bash-completion > /etc/bash_completion.d/cloneable_completion
You need to also set up one of two methods for authentication to query GraphQL
APIs.
### GitHub App authentication
Create and install a GitHub app. Recommended permissions is repository
`contents:readonly` for private repositories or no scopes for public
repositories.
Set environment variables with GitHub App authentication.
export CLONEABLE_GITHUB_APP_ID=1234
export CLONEABLE_GITHUB_APP_KEY=/path/to/private_key
### GitHub Personal Access Tokens
Requires a [GitHub personal access token][github-token] with scopes:
- `repo` and `read:org` for an organization with private repositories.
- `repo` for a user with private repositories.
- No scopes required for a user or organization with public repositories only.
Setup authentication.
export GITHUB_TOKEN=
# Example Usage
Example usage covers HTTP and SSH cloning. The following methods work with both
App and personal access token auth.
### Backup over HTTP
Cloning over HTTP requires backend authentication via `GIT_ASKPASS`. See
[git-credentials][git-credentials] for details.
Install a `GIT_ASKPASS` script set up for cloning.
cloneable -o your-org --print-askpass-script | /bin/sh
# Or alternate location instead of /tmp/askpass
cloneable -o integralads --print-askpass-script | \
CLONEABLE_ASKPASS_LOCATION=/alternate/askpass /bin/sh
The `GIT_ASKPASS` script will default to always auth against `your-org`. If you
want to clone from another associated org for the GitHub App, you can set
environment variable `CLONEABLE_OWNER`.
Clone all repositories. You can pass any cloneable arguments.
export GIT_ASKPASS=/tmp/askpass
cloneable -o your-org --http --skip-local-bare-repos \
--print-clone-script | /bin/sh -x
export CLONEABLE_OWNER=another-org
cloneable --http --skip-local-bare-repos \
--print-clone-script | /bin/sh -x
Update clones using HTTP.
cloneable --http --print-update-script | /bin/sh -x
### Backup over SSH
Assuming your already have an SSH clone key loaded within ssh-agent. You can
clone and update repositories using SSH.
cloneable --url --skip-local-bare-repos \
--print-clone-script | /bin/sh -x
Update clones using SSH.
cloneable --url --print-update-script | /bin/sh -x
### Other usage
List repository names under a user or organization.
```
export GITHUB_TOKEN
read -ersp token: GITHUB_TOKEN
cloneable --owner samrocketman
```
Show samrocketman repositories which have a `jenkins` topic.
```bash
cloneable --owner samrocketman --match-topics jenkins
```
# Build Jar
./gradlew clean jar
Run tests
./gradlew clean check
Recent development done in the following environment.
Pop!_OS 22.04 LTS
Linux 6.9.3-76060903-generic x86_64
GNU bash, version 5.1.16(1)-release (x86_64-pc-linux-gnu)
Gradle 8.13
Build time: 2025-02-25 09:22:14 UTC
Revision: 073314332697ba45c16c0a0ce1891fa6794179ff
Kotlin: 2.0.21
Groovy: 3.0.22
Ant: Apache Ant(TM) version 1.10.15 compiled on August 25 2024
Launcher JVM: 17.0.12 (Ubuntu 17.0.12+7-Ubuntu-1ubuntu222.04)
Daemon JVM: /usr/lib/jvm/java-17-openjdk-amd64 (no JDK specified, using current Java home)
OS: Linux 6.9.3-76060903-generic amd64
openjdk 17.0.12 2024-07-16
# CLI Documentation
The following options are available from `java -jar cloneable.jar --help`.
```
Usage: cloneable [-abdefhipPsuV] [--http] [--owner-is-user]
[--print-askpass-script] [--print-auth-token]
[--print-bash-completion] [--print-cli-script]
[--print-clone-script] [--print-update-script]
[--after=] [-B=]
[--before=] [-g=] [-k=]
[-o=] [-t=] [-E=]...
[-F=]... [-m=]...
Description:
Gets a list of repositories if given a GitHub user or GitHub organization.
The primary purpose of this program is to facilitate listing GitHub
repositories and cloning them for an offline backup.
Website: https://github.com/samrocketman/cloneable
Example Usage:
Set up GitHub App authentication.
export CLONEABLE_GITHUB_APP_ID=1234
export CLONEABLE_GITHUB_APP_KEY=/path/to/private_key
Or you can use a GitHub personal access token.
export GITHUB_TOKEN=
Create CLI integration with bash completion.
java -jar cloneable.jar --print-cli-script > /usr/local/bin/cloneable
chmod 755 /usr/local/bin/cloneable
cloneable --print-bash-completion > /etc/bash_completion.
d/cloneable_completion
Install GIT_ASKPASS script when using GitHub App auth.
cloneable -o your-org --print-askpass-script | /bin/bash
HTTP mirror clones using GIT_ASKPASS.
export GIT_ASKPASS=/tmp/askpass
cloneable -o your-org --http \
--skip-local-bare-repos --print-clone-script \
| /bin/sh
export CLONEABLE_OWNER=another-org
cloneable --http --skip-local-bare-repos --print-clone-script \
| /bin/sh
Update HTTP mirrors.
cloneable --print-update-script | /bin/sh -x
Environment Variables:
CLONEABLE_CACHE_KEY
RSA private key used to encrypt cache. By default,
CLONEABLE_GITHUB_APP_KEY is used.
CLONEABLE_CACHE_PATH
Directory where persisted encrypted cache and lockfile get stored.
CLONEABLE_DEBUG
Same as --debug, enables debug mode showing more information.
CLONEABLE_GITHUB_APP_ID
GitHub App ID used in GitHub App authentication.
CLONEABLE_GITHUB_APP_KEY
GitHub App private key used in GitHub App authentication.
GITHUB_GRAPHQL_URL
GitHub API endpoint for GraphQL. Default: https://api.github.com/graphql
GITHUB_API_URL
GitHub API endpoint for REST. Default: https://api.github.com/
GITHUB_TOKEN
GitHub personal token for API communication when GitHub App authentication
is not available.
Options:
-a, --skip-archived-repos If a repository is archived, then it will be
skipped.
--after=
Find all repositories updated after the given
timeframe. Must be a positive integer followed
by one of: d, m, y. For example, 1y will find
all repositories updated after 1 year ago; or
within the past 12m or 12 months. If d, m, or y
is not provided then d will be assumed.
-b, --skip-local-bare-repos
If a bare repository directory exists locally (the
name of the repo ending with '.git'), then it
will not be printed out. This is useful for
cloning only missing repositories.
-B, --branch= If using -F or -E options, then choose the Git
branch to search for files. The default branch
will be queried if this option is not specified.
--before=
Find all repositories updated before the given
timeframe. Must be a positive integer followed
by one of: d, m, y. For example, 1y will find
all repositories updated before 1 year ago. If
d, m, or y is not provided then d will be
assumed.
-d, --debug Prints out stack traces.
-e, --skip-empty-repos If a repository does not contain any Git commits,
then it will be skipped.
-E, --exclude-repos-with=
Match any repository containing which does NOT
contain the file at the repository root. -E can
be specified more than once to exclude a
repository containing any or all of the listed
files.
-f, --skip-forked-repos If a repository is a fork from another user or
organization, then it will be skipped.
-F, --contains=
Match any repository containing any file at the
repository root. -F can be specified more than
once to match any one of multiple files.
-g, --github-app-id=
GitHub App app ID associated with GitHub App
private key.
-h, --help Show this help message and exit.
--http Prints HTTP clone URL instead of repository name.
Also dictates output of --print-update-script
and --print-clone-script
-i, --inverse-search Inverse the search results. For example, instead
of skipping repositories it will match
repositories. Adding the option
--inverse-search along side --skip-archived will
find all archived repositories. If you provide
multiple options then the inverse finds any
match. For example, adding --inverse-search
with options --skip-archived and --skip-empty
will find BOTH empty or archived repositories.
-k, --github-app-key=
GitHub App private key.
-m, --match-topics=
Require all repositories to have one of the listed
topics. -m can be specified multiple times.
-o, --owner= GitHub account or organization for querying a list
of projects.
--owner-is-user If using GitHub App auth this flag indicates app
installation is for a user instead of
organization.
-p, --skip-private-repos If a repository is private, then it will be
skipped.
-P, --skip-public-repos If a repository is public, then it will be skipped.
--print-askpass-script Print a GIT_ASKPASS script meant for cloning HTTP
repositories provided by --http option. Pipe
into /bin/sh.
--print-auth-token Print GitHub token generated by GitHub app
installation used for cloning.
--print-bash-completion
Print bash-completion script meant to help with
auto-complete on interactive command line. e.g.
redirect stdout to '/etc/bash_completion.d'.
--print-cli-script Prints 'cloneable' CLI wrapper script. e.g.
redirect stdout to '/usr/local/bin/cloneable'.
--print-clone-script Prints a bash script meant for updating HTTP
clones created by app auth. Pipe into /bin/sh
--print-update-script Prints a bash script meant for updating HTTP
clones created by app auth. Pipe into /bin/sh
-s, --skip-source-repos If a repository is **not** a fork from another
user or organization, then it will be skipped.
-t, --github-token= GitHub personal access token or file containing a
token. Falls back to checking GITHUB_TOKEN
environment variable.
-u, --url Prints out SSH clone URL instead of repository
name.
-V, --version Print version information and exit.
```
[download]: https://github.com/samrocketman/cloneable/releases
[git-credentials]: https://git-scm.com/docs/gitcredentials
[github-token]: https://help.github.com/en/github/authenticating-to-github/creating-a-personal-access-token-for-the-command-line
[other-backup]: https://github.com/topics/github-backup
[v4]: https://developer.github.com/v4/