Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

https://github.com/supermarsx/autogitpull

Automatic Git Puller and Monitor
https://github.com/supermarsx/autogitpull

automation autoupdate bot cli cross-platform git git-pull headless repository-management synchronization terminal tui

Last synced: about 1 month ago
JSON representation

Automatic Git Puller and Monitor

Awesome Lists containing this project

README 

          
icon_128



# autogitpull



Automatic Git Puller & Monitor



Tested and working on MacOS, Ubuntu and Windows



`autogitpull` scans a directory of Git repositories, pulls updates on a schedule

and shows progress either in an interactive TUI or with plain console output.



image



## Features



- Periodic scanning and automatic `git pull`

 - Optional CLI mode that prints logs without the TUI

- Very lightweight, low resource usage

- YAML or JSON configuration files

- Detailed logging with resource tracking

- Throttling and CPU/memory limits

- Automatically resumes after system sleep/hibernation



## Usage



`autogitpull  [--include-private] [--show-skipped] [--show-notgit] [--show-version] [--version] [--interval ] [--refresh-rate ] [--cpu-poll ] [--mem-poll ] [--thread-poll ] [--log-dir ] [--log-file ] [--max-log-size ] [--include-dir ] [--ignore ] [--recursive] [--max-depth ] [--log-level ] [--verbose] [--concurrency ] [--threads ] [--single-thread] [--max-threads ] [--cpu-percent ] [--cpu-cores ] [--mem-limit ] [--check-only] [--no-hash-check] [--no-cpu-tracker] [--no-mem-tracker] [--no-thread-tracker] [--net-tracker] [--download-limit ] [--upload-limit ] [--disk-limit ] [--total-traffic-limit ] [--cli] [--single-run] [--silent] [--force-pull] [--remove-lock] [--hard-reset] [--confirm-reset] [--confirm-alert] [--sudo-su] [--debug-memory] [--dump-state] [--dump-large ] [--attach ] [--background ] [--reattach ] [--persist[=name]] [--help]`



### TLDR usage tips



- For minimum memory footprint use `--single-thread`, trade off on performance/speed.

- To override and discard local changes every time, use `--force-pull`; uncommitted work is erased as repositories reset to remote.

- To only sync the latest repos you're working on use `--updated-since` 6h, to only sync repos updated in the last 6 hours.

- To only show dates from repos that have been synced during the current session use `--session-dates-only`.



Repositories with uncommitted changes are skipped by default to avoid losing work. Use `--force-pull` (alias: `--discard-dirty`) to reset such repositories to the remote state, permanently deleting their uncommitted changes.



### Usage options



Most options have single-letter shorthands. Run `autogitpull --help` for a complete list.

The full catalogue of flags with their default values is documented in

[`docs/cli_options.md`](docs/cli_options.md).



#### Basics

- `--include-private` (`-p`) – Include private repositories.

- `--root` (`-o`) `` – Root folder of repositories.

- `--interval` (`-i`) `` – Delay between scans.

- `--refresh-rate` (`-r`) `` – TUI refresh rate.

- `--recursive` (`-e`) – Scan subdirectories recursively.

- `--max-depth` (`-D`) `` – Limit recursive scan depth.

- `--include-dir` `` – Additional directory to scan (repeatable).

- `--ignore` (`-I`) `` – Directory to ignore (repeatable).

- `--single-run` (`-u`) – Run a single scan cycle and exit.

- `--single-repo` (`-S`) – Only monitor the specified root repo.

- `--rescan-new` (`-w`) `` – Rescan for new repos periodically.

- `--wait-empty` (`-W`) `[n]` – Keep retrying when no repos are found, up to an optional limit.

- `--dont-skip-timeouts` – Retry repositories that timeout.

- `--keep-first-valid` – Keep valid repos from the first scan even if missing.

- `--updated-since` `` – Only sync repos updated recently.

- `--help` (`-h`) – Show this message.



#### Display

- `--show-skipped` (`-k`) – Show skipped repositories.

- `--show-notgit` – Show non-git directories.

- `--show-version` (`-v`) – Display program version in TUI.

- `--version` (`-V`) – Print program version and exit.

- `--show-runtime` – Display elapsed runtime.

- `--show-repo-count` (`-Z`) – Display number of repositories.

- `--show-commit-date` (`-T`) – Display last commit time.

- `--show-commit-author` (`-U`) – Display last commit author.

- `--show-pull-author` – Show author when pull succeeds.

- `--session-dates-only` – Only show dates for repos pulled this session.

- `--hide-date-time` – Hide date/time line in TUI.

- `--hide-header` (`-H`) – Hide status header.

- `--row-order` `` – Row ordering (alpha/reverse).

- `--color` `` – Override status color.

- `--no-colors` (`-C`) – Disable ANSI colors.

- `--censor-names` – Mask repository names in output.

- `--censor-char` `` – Character used for masking.



#### Config

- `--auto-config` – Auto detect YAML or JSON config.

- `--auto-reload-config` – Reload config when the file changes (disabled by default).

- `--rerun-last` – Reuse args from `.autogitpull.config`.

- `--save-args` – Save args to config file.

- `--enable-history[=]` – Enable command history (default `.autogitpull.config`).

- `--config-yaml` (`-y`) `` – Load options from YAML file.

- `--config-json` (`-j`) `` – Load options from JSON file.



#### Authentication

- `--credential-file` `` – Read username and password from file.

- `--ssh-public-key` `` – Path to SSH public key.

- `--ssh-private-key` `` – Path to SSH private key.



#### Process

- `--cli` (`-c`) – Use console output.

- `--silent` (`-s`) – Disable console output.

- `--attach` (`-A`) `` – Attach to daemon and show status.

- `--background` (`-b`) `` – Run in background with attach name.

- `--reattach` (`-B`) `` – Reattach to background process.

- `--persist` (`-P`) `[name]` – Keep running after exit; optional run name.

- `--respawn-limit` `` – Respawn limit within minutes.

- `--max-runtime` `` – Exit after given runtime.

- `--pull-timeout` (`-O`) `` – Network operation timeout.

- `--exit-on-timeout` – Terminate worker if a poll exceeds the timeout.

- `--print-skipped` – Print skipped repositories once.

- `--keep-first` – Keep repositories validated on the first scan.



#### Daemon management

On macOS and Linux, `autogitpull` can run as a background service via

`launchd` or `systemd`:



- `--install-daemon` – install the service unit.

- `--uninstall-daemon` – remove the service unit.

- `--start-daemon` / `--stop-daemon` – control the service.

- `--daemon-status` – check whether it is installed and running.



#### Logging

- `--log-dir` (`-d`) `` – Directory for pull logs.

- `--log-file` (`-l`) `` – File for general logs.

- `--max-log-size` `` – Rotate `--log-file` when over this size.

- `--log-level` (`-L`) `` – Set log verbosity.

- `--verbose` (`-g`) – Shortcut for DEBUG logging.

- `--debug-memory` (`-m`) – Log memory usage each scan.

- `--dump-state` – Dump container state when large.

- `--dump-large` `` – Dump threshold for `--dump-state`.

- `--syslog` – Log to syslog.

- `--syslog-facility` `` – Syslog facility.



#### Concurrency

- `--concurrency` (`-n`) `` – Number of worker threads.

- `--threads` (`-t`) `` – Alias for `--concurrency`.

- `--single-thread` (`-q`) – Run using a single worker thread.

- `--max-threads` (`-M`) `` – Cap the scanning worker threads.



#### Resource limits

- `--cpu-percent` (`-E`) `` – Approximate CPU usage limit.

- `--cpu-cores` `` – Set CPU affinity mask.

- `--mem-limit` (`-Y`) `` – Abort if memory exceeds this amount.

- `--download-limit` `` – Limit total download rate.

- `--upload-limit` `` – Limit total upload rate.

- `--disk-limit` `` – Limit disk throughput.

- `--total-traffic-limit` `` – Stop after transferring this much data.



#### Tracking

- `--cpu-poll` `` – CPU polling interval.

- `--mem-poll` `` – Memory polling interval.

- `--thread-poll` `` – Thread count interval.

- `--no-cpu-tracker` (`-X`) – Disable CPU usage tracker.

- `--no-mem-tracker` – Disable memory usage tracker.

- `--no-thread-tracker` – Disable thread tracker.

- `--net-tracker` – Track network usage.

- `--vmem` – Show virtual memory usage.



#### Actions

- `--check-only` (`-x`) – Only check for updates.

- `--no-hash-check` (`-N`) – Always pull without hash check.

- `--force-pull` (`-f`) – Reset repos to remote state, losing uncommitted changes and untracked files.

- `--discard-dirty` – Alias for `--force-pull`; same data loss.

- `--install-daemon` – Install background daemon.

- `--uninstall-daemon` – Uninstall background daemon.

- `--daemon-config` `` – Config file for daemon install.

- `--install-service` – Install system service.

- `--uninstall-service` – Uninstall system service.

- `--start-service` – Start installed service.

- `--stop-service` – Stop installed service.

- `--force-stop-service` – Force stop service.

- `--restart-service` – Restart service.

- `--force-restart-service` – Force restart service.

- `--service-status` – Check if the service exists and is running.

- `--start-daemon` – Start daemon unit.

- `--stop-daemon` – Stop daemon unit.

- `--force-stop-daemon` – Force stop daemon.

- `--restart-daemon` – Restart daemon unit.

- `--force-restart-daemon` – Force restart daemon.

- `--daemon-status` – Check if the daemon exists and is running.

- `--service-config` `` – Config file for service install.

- `--remove-lock` (`-R`) – Remove directory lock file and exit.

- `--kill-all` – Terminate running instance and exit.

- `--list-services` – List installed service units.

- `--list-daemons` – Alias for `--list-services`.

- `--ignore-lock` – Don't create or check lock file.

- `--hard-reset` – Remove autogitpull logs, configs, and lock files (cannot be undone).

- `--confirm-reset` – Confirm `--hard-reset`.

- `--confirm-alert` – Confirm unsafe interval or force pull.

- `--sudo-su` – Suppress confirmation alerts.



By default, repositories whose `origin` remote does not point to GitHub or require authentication are skipped during scanning. Use `--include-private` to include them. Skipped repositories are hidden from the TUI unless `--show-skipped` is also provided.



Provide `--log-dir ` to store pull logs for each repository. After every pull operation the log is written to a timestamped file inside this directory and its location is shown in the TUI. Use `--log-file ` to append high level messages to the given file.



### Persistent mode



Run with `--persist` to automatically restart `autogitpull` whenever the main

worker exits. This keeps the application alive if it is terminated while the

computer is under heavy load or resumes from sleep or hibernation. Unhandled

errors from the worker are caught and logged so the monitor can restart the

process cleanly. Optionally specify a name like `--persist=myrun` to tag the

instance. The `--respawn-limit` option controls how many restarts are allowed

within a time window (set it to `0` for no limit).



### YAML configuration



Frequently used options can be stored in a YAML file and loaded with `--config-yaml `.

Keys match the long option names without the leading dashes. Boolean flags should be set to `true` or `false`.

Arguments provided on the command line override values from the YAML file. See `examples/example-config.yaml` and `examples/example-config.json` for complete examples.



### JSON configuration



Settings can also be provided in JSON format and loaded with `--config-json `.

The keys mirror the long command line options without the leading dashes. Values from the command line override those from the JSON file. See `examples/example-config.json` for a complete example.



### Per-repository settings



Configuration files may include a `repositories` section that maps repository paths to option overrides. Keys inside each repository entry correspond to long command line options without the leading dashes. The old format that places repository paths at the top level is still supported.



YAML example:



```yaml

root: /home/user/repos

repositories:

  /home/user/repos/foo:

    force-pull: true

    download-limit: 100

```



JSON example:



```json

{

  "root": "/home/user/repos",

  "repositories": {

    "/home/user/repos/foo": {

      "force-pull": true,

      "download-limit": 100

    }

  }

}

```



## Build requirements



This tool relies on [libgit2](https://libgit2.org/). The helper scripts

`scripts/install_deps.sh` (Linux/macOS) and `scripts/install_deps.bat` (Windows) automatically

download and install `libgit2` when needed. You can also run `make deps` on

Unix-like systems to invoke the installer. The build requires the development

package (named `libgit2-dev` on Debian/Ubuntu). The Makefile tries to link

statically but falls back to dynamic linking when static libraries are

unavailable. If you prefer to install the library yourself, follow the

instructions below.



Running the unit tests (`make test`) additionally requires the development

headers and libraries for `yaml-cpp` and

[nlohmann/json](https://github.com/nlohmann/json). The same installer scripts

(`scripts/install_deps.sh` or `scripts/install_deps.bat`) will install these packages along with

`libgit2`.



### Installing libgit2 on Linux



```

sudo apt-get update && sudo apt-get install -y libgit2-dev     # Debian/Ubuntu

sudo yum install -y libgit2-devel                              # RHEL/Fedora

```



### Installing libgit2 on Windows



#### MSVC (Visual Studio)



Use [vcpkg](https://github.com/microsoft/vcpkg):



```

git clone https://github.com/microsoft/vcpkg

cd vcpkg && bootstrap-vcpkg.bat

vcpkg\vcpkg install libgit2

```



Ensure the resulting `installed` folder is on your `LIB` and `INCLUDE`

paths when compiling with `compile-cl.bat`.



#### MinGW



Run `scripts/install_libgit2_mingw.bat` to build libgit2 and yaml-cpp natively

with MinGW. The script installs the static libraries and headers under the

`libs` directory and also downloads the header-only `nlohmann-json` library to

complete the dependencies.



`scripts/compile-cl.bat` expects a vcpkg installation while `scripts/compile.bat` uses the

library produced by `scripts/install_libgit2_mingw.bat` and will call it

automatically if `libs/libgit2_install` is missing. When linking with MinGW,

additional Windows system libraries are required. `scripts/compile.bat` now attempts

to install MinGW through Chocolatey if `g++` is not found and already

includes `winhttp`, `ole32`, `rpcrt4` and `crypt32` so that the build

succeeds without manual tweaks.



## Building



### Using the provided scripts



Run `make` (Linux/macOS), `scripts/compile.bat` (MinGW) or `scripts/compile-cl.bat` (MSVC) to

build the project. `scripts/compile.bat` invokes `scripts/install_libgit2_mingw.bat` when

`libgit2` is missing. All helper scripts place the resulting executable in the `dist/`

directory as `dist/autogitpull` (or `dist/autogitpull.exe` on Windows).

For an extra-small Windows binary run `scripts/compile-compress.bat` which

reuses `compile.bat` with size optimizations and then compresses the result

with UPX.



The repository also ships with `scripts/compile.sh` for Unix-like environments which

will attempt to install a C++ compiler if one isn't present. Windows users get

`scripts/compile.bat` (MinGW) and `scripts/compile-cl.bat` (MSVC) along with

`scripts/install_deps.bat`, `scripts/install_libgit2_mingw.bat` and `scripts/run.bat`.



Clean up intermediate files with `make clean`. Dedicated cleanup scripts are also

available under `scripts/` as `scripts/clean.sh` (Unix-like systems) and `scripts/clean.bat`

(Windows) to remove the generated binary, object files and the `build` and `dist` directories.



### Debug builds for leak analysis



The scripts `scripts/compile-debug.sh`, `scripts/compile-debug.bat` and `scripts/compile-debug-cl.bat`

compile the program with AddressSanitizer and debug information enabled. They

produce `dist/autogitpull_debug` (or `dist/autogitpull_debug.exe` on Windows). Use these

builds when running leak detection tools:



```bash

scripts/compile-debug.sh      # Linux/macOS

scripts/compile-debug.bat       # MinGW

scripts/compile-debug-cl.bat    # MSVC

```



### Manual compilation



If you prefer to build without the helper scripts, the following commands show

the bare minimum required to compile the program.



On Linux with `g++`:



```bash

g++ -std=c++20 autogitpull.cpp git_utils.cpp tui.cpp logger.cpp \

    $(pkg-config --cflags libgit2) \

    $(pkg-config --static --libs libgit2 2>/dev/null || pkg-config --libs libgit2) \

    -pthread -o dist/autogitpull

```



On macOS with `clang++`:



```bash

clang++ -std=c++20 autogitpull.cpp git_utils.cpp tui.cpp logger.cpp $(pkg-config --cflags --libs libgit2) -pthread -o dist/autogitpull

```



macOS builds automatically fall back to the header-only implementation

`include/thread_compat.hpp` when the system libc++ lacks `std::jthread`.



On Windows with MSVC's `cl`:



```batch

cl /std:c++20 /EHsc /MT /Ipath\to\libgit2\include autogitpull.cpp git_utils.cpp tui.cpp logger.cpp ^

    /link /LIBPATH:path\to\libgit2\lib git2.lib /Fedist\autogitpull.exe

```



These commands mirror what the scripts do internally.



### Building with CMake



Alternatively, configure the project with CMake:



```bash

cmake -S . -B build

cmake --build build

```



The resulting executable will appear in the `dist/` directory.



### Running tests



Unit tests use [Catch2](https://github.com/catchorg/Catch2). If the library is

not installed, CMake will automatically download it using `FetchContent`.

`make test` requires the development packages for `libgit2`, `yaml-cpp`,

`nlohmann-json` and `zlib`. Use `scripts/install_deps.sh` (Linux/macOS) or

`scripts/install_deps.bat` (Windows) to install them before configuring and

building the tests.

Once the dependencies are in place, run `ctest`:



```bash

make test

```



This command generates a `build` directory (if missing), compiles the tests and

executes them through CMake's `ctest` driver.



### Leak test



To run the memory leak regression test you need both the `valgrind` tool and the

`libgit2-dev` package installed. After building the tests with `make test`, run:



```bash

valgrind ./build/memory_leak_test

```



### Icon generation



Run `scripts/generate_icons.sh` (Linux/macOS) or `scripts/generate_icons.bat`

(Windows) to create platform icons from `graphics/icon.png`. If ImageMagick's

`magick` command is missing, the Unix script attempts to install the

`imagemagick` package using `apt`, `dnf`, or `yum`. Windows relies on

`winget` to install ImageMagick automatically. Windows builds embed the

generated `icon.ico` and macOS uses `icon.icns`.



## Linting



The project uses `clang-format` and `cpplint` (configured via `CPPLINT.cfg`) to

enforce a consistent code style. Run `make lint` before committing to ensure

formatting and style rules pass:



```bash

make lint

```



The CI workflow also executes this command and will fail on formatting or lint errors.



`scripts/install_deps.sh` and `scripts/install_deps.bat` automatically install

`cpplint` if it is missing.



### Status labels



When the program starts, each repository is listed with the **Pending** status

until it is checked for the first time. Once a scan begins the status switches

to **Checking** and later reflects the pull result.



### Versioning



autogitpull uses a rolling release model. Each push to the `main` branch

triggers the CI workflow that tags the commit with a date-based string such as

`2025.07.31-1`. The `--version` flag prints this tag so the program is always

identified by the latest CI release.



## Production requirements



- **Git** must be available in your `PATH` for libgit2 to interact with repositories.

- Network access is required to contact remote Git servers when pulling updates.

- The application prints ANSI color codes; on Windows run it in a terminal that

  supports color (e.g. Windows Terminal or recent PowerShell).



## Licensing



autogitpull is licensed under the MIT license (see `LICENSE`). The project

bundles the license texts for its third-party dependencies under

the `licenses/` directory, including `libgit2.txt`, `yaml-cpp.txt`,

`nlohmann-json.txt` and `zlib.txt`.