https://github.com/julian-corbet/dotkeeper
P2P file sync with git history — embedded Syncthing + staggered git auto-backup across any number of machines
https://github.com/julian-corbet/dotkeeper
dotfiles git golang p2p-sync self-hosted syncthing
Last synced: about 2 months ago
JSON representation
P2P file sync with git history — embedded Syncthing + staggered git auto-backup across any number of machines
- Host: GitHub
- URL: https://github.com/julian-corbet/dotkeeper
- Owner: julian-corbet
- License: agpl-3.0
- Created: 2026-05-05T11:51:50.000Z (2 months ago)
- Default Branch: main
- Last Pushed: 2026-05-17T17:37:02.000Z (about 2 months ago)
- Last Synced: 2026-05-17T19:55:46.534Z (about 2 months ago)
- Topics: dotfiles, git, golang, p2p-sync, self-hosted, syncthing
- Language: Go
- Homepage: https://dotkeeper.corbet.ch
- Size: 1.1 MB
- Stars: 1
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
- Codeowners: .github/CODEOWNERS
- Security: SECURITY.md
Awesome Lists containing this project
README
# dotkeeper
[](https://github.com/julian-corbet/dotkeeper/releases/latest)
[](https://github.com/julian-corbet/dotkeeper/actions/workflows/ci.yml)
[](LICENSE)
[](https://pkg.go.dev/github.com/julian-corbet/dotkeeper)
[](https://www.bestpractices.dev/projects/12584)
[](https://syncthing.net/)
Sync your repos and dotfiles across machines. Close the laptop, open the desktop, keep working.
## What it does
dotkeeper keeps your code, configs, and dotfiles in sync across however many machines you use. It combines **embedded Syncthing** for real-time P2P file sync with **staggered git backup** for history and rollback. You declare what you want synced in plain TOML files; a reconciler loop continuously converges the live system to match. Single binary, no external dependencies beyond git.
Each local repo copy you want managed gets a `.dotkeeper.toml` at its root. That file is local machine state: dotkeeper writes Git and Syncthing excludes for it, and it must not be committed or synced. It says which machines should receive the repo, how git backup should run, and which Syncthing folder backs it. Per-machine settings — your machine's name, slot number, peer roster, defaults, and which directories to scan for repos — live in `~/.config/dotkeeper/machine.toml`. You edit files; dotkeeper handles the rest.
## Why it exists
Working across multiple machines is awkward when code, dotfiles, and editor configs drift. Git gives rollback and auditability, but only after commits and pushes. Syncthing gives real-time P2P sync, but not git history. dotkeeper wires the two together with a declarative config model and little ongoing maintenance.
Repo-specific dotkeeper metadata stays out of Git and out of Syncthing. Public project history stays clean; private topology can be generated by Nix/Home Manager or created locally with `dotkeeper track`.
## How it works
dotkeeper runs as a daemon. At its core is a reconcile loop:
```
desired = read_config() # machine.toml + local .dotkeeper.toml files
observed = query_state() # Syncthing REST API + git + filesystem
actions = diff(desired, observed)
for action in actions:
apply(action)
```
`diff()` is a pure function. Given the same inputs it always returns the same list of actions. `apply()` is where side effects happen — adding a Syncthing folder, auto-committing a dirty repo, pushing a backup — and every action is idempotent. The daemon triggers reconcile on inotify events (file changes land in milliseconds), on a periodic timer (default five minutes, safety net), and on explicit `dotkeeper reconcile` invocations.
The mesh is P2P — no central hub. Every peer reaches every other peer directly. Each machine also runs staggered git backup: machines are assigned slot numbers (0, 1, 2, ...) and commit at different offsets within the configured interval, so no two machines ever race to push the same branch.
dotkeeper v1.0 introduced a **multi-transport** layer. Two ways to move a change between peers coexist: Syncthing's BEP gossip (the universal fallback, present in every install) and `git push` over SSH (used when a peer-resolution path is configured — currently Tailscale; mDNS and static-hub variants are planned). A cost-model picks per change based on the change's size: small text payloads typically route via git, larger payloads via Syncthing. The model learns from observed transfer durations, so the per-change crossover is empirical, not hardcoded. See [`dotkeeper transport list`](#commands) for what's available on a given host.
`.git/` directories are excluded from Syncthing sync. Each machine keeps its own independent git history; histories converge via the chosen git transport or via the configured remote, never through syncing raw git objects. This keeps the Syncthing mesh fast and avoids the class of problems that come from syncing `.git/` directly.
## Quick start
### Install
**Arch Linux (AUR):**
```bash
paru -S dotkeeper-bin # pre-built binary
paru -S dotkeeper-git # builds from main HEAD
```
**macOS / Linux (Homebrew):**
```bash
brew tap julian-corbet/dotkeeper
brew install dotkeeper
```
**From source — use `make`, not `go build` directly:**
```bash
git clone https://github.com/julian-corbet/dotkeeper.git
cd dotkeeper
make build && make install
```
Or download a pre-built binary from [Releases](https://github.com/julian-corbet/dotkeeper/releases).
> **Why `make build` and not `go build ./...`?**
>
> dotkeeper embeds Syncthing as a library. Syncthing's `lib/api` package
> expects generated web-GUI assets (`gui.files.go`) that dotkeeper has no
> use for and does not ship. A plain `go build ./cmd/dotkeeper` therefore
> fails with `undefined: auto.Assets`. Pass `-tags noassets` to skip the
> assets path:
>
> ```bash
> go build -tags noassets ./cmd/dotkeeper # build the daemon
> go test -tags noassets ./... # run the test suite
> go install -tags noassets github.com/julian-corbet/dotkeeper/cmd/dotkeeper@latest
> ```
>
> The `Makefile`, `Dockerfile`, CI workflows, and `release.yml` all set
> this tag automatically — when contributing, prefer `make build`/`make
> test` so the tag is never forgotten. If your IDE or linter shows the
> `auto.Assets` error, configure it to pass `-tags noassets` to gopls.
### First machine
```bash
# Generate Syncthing identity, write initial state.toml, scaffold machine.toml.
dotkeeper init
# Print this machine's Syncthing device ID — you'll need it when adding peers.
dotkeeper identity
# Edit machine.toml to set your name, slot, peers, and scan roots.
$EDITOR ~/.config/dotkeeper/machine.toml
```
If `machine.toml` is generated by Home Manager or another declarative system,
run `dotkeeper init --no-service` after activation. dotkeeper preserves the
generated file and only creates runtime Syncthing identity/state.
A minimal `machine.toml` looks like:
```toml
schema_version = 2
name = "desktop"
slot = 0
[discovery]
scan_roots = [
"~/Documents/GitHub",
"~/.config/nvim",
]
[[peers]]
name = "laptop"
device_id = ""
learned_at = 2026-01-01T00:00:00Z
```
For each repo you want to manage, run `dotkeeper track`:
```bash
dotkeeper track ~/Documents/GitHub/my-project
```
That writes a local `.dotkeeper.toml`, adds dotkeeper/Syncthing control files to `.git/info/exclude`, and writes a managed `.stignore` so those files do not cross the mesh.
```toml
# ~/Documents/GitHub/my-project/.dotkeeper.toml
schema_version = 2
[repo]
name = "my-project"
added = "2026-01-01T00:00:00Z"
added_by = "desktop"
[sync]
syncthing_folder_id = "my-project-a1b2c3"
share_with = ["desktop", "laptop"]
```
Then run the daemon:
```bash
dotkeeper start # or: systemctl --user start dotkeeper-syncthing.service
```
dotkeeper walks your scan roots, finds local `.dotkeeper.toml` files, and starts managing those repos.
### Each additional machine
```bash
# On the new machine:
dotkeeper init
dotkeeper identity # copy this device ID
# On an existing machine, add the peer declaratively in machine.toml
# or imperatively:
dotkeeper peer add laptop
dotkeeper reconcile
# Back on the new machine, materialize local repo configs with Nix/Home Manager
# or run dotkeeper track for the repo paths this machine should manage:
dotkeeper track ~/Documents/GitHub/my-project
dotkeeper reconcile
```
Because `.dotkeeper.toml` is intentionally not synced, each machine needs its own local copy. Nix/Home Manager is the intended low-friction path for repeatable personal topology; `dotkeeper track` is the portable manual path.
### Nix / Home Manager topology
For declarative setups, keep the public dotkeeper program generic and put your
private sync topology in your own flake. A practical pattern is denylist-first:
scan a small set of roots for Git repos, generate local `.dotkeeper.toml` files
for everything discovered, and list only the exceptions.
```nix
{
scanRoots = [
"~/Documents/GitHub"
"~/.config"
];
dontSync = {
all = [
"~/Documents/GitHub/dotkeeper"
"~/Documents/GitHub/archive"
"~/Documents/GitHub/example-rag/workspace"
];
laptop = [
"~/Documents/GitHub/video-renderer"
];
};
}
```
Whole denied repos are left unmanaged on that machine. Denied paths inside a
managed repo should be emitted as `[sync].ignore` patterns, so generated data can
stay local while the repo itself still syncs. See
[`docs/examples/home-manager-denylist.nix`](docs/examples/home-manager-denylist.nix)
for a full generic activation sketch.
## Commands
| Command | Purpose |
|---------|---------|
| `dotkeeper init` | First-run setup: generate Syncthing identity, write `state.toml`, scaffold `machine.toml` |
| `dotkeeper start` | Run the daemon (invoked by systemd) |
| `dotkeeper reconcile` | Force a reconcile pass now |
| `dotkeeper status` | Snapshot: last reconcile time, tracked repos, mesh peers, pending work |
| `dotkeeper identity` | Print this machine's Syncthing device ID and name |
| `dotkeeper peer add/list/remove` | Manage imperative peers in `state.toml`; declarative peers can live in `machine.toml` |
| `dotkeeper track ` | Bootstrap local `.dotkeeper.toml`, local excludes, and tracked override state |
| `dotkeeper untrack ` | Deregister a tracked override |
| `dotkeeper doctor` | Run self-diagnostic health checks; `--json` for machine-readable output |
| `dotkeeper conflict list` | List Syncthing sync-conflict files across all managed folders |
| `dotkeeper conflict resolve-all` | Batch auto-resolve trivial conflicts (dedup + text merge) |
| `dotkeeper conflict keep ` | Delete the conflict variant, keep the current file |
| `dotkeeper conflict accept ` | Replace current file with the conflict variant and commit |
| `dotkeeper transport list` | List configured transports and which are currently available |
| `dotkeeper transport status [peer]` | Per-peer reachability + learned cost-model parameters |
| `dotkeeper bare-init [--peer=NAME] [--host=USER@ADDR]` | Configure peer-side git repos for direct `git push` (sets `receive.denyCurrentBranch=updateInstead`) |
| `dotkeeper version` | Print dotkeeper version |
## Architecture
- [docs/architecture.md](docs/architecture.md) — full walkthrough: reconciler model, config file roles, discovery, command reference
- [ADR 0001](docs/adr/0001-per-repo-config.md) — why local per-repo `.dotkeeper.toml` is authoritative
- [ADR 0002](docs/adr/0002-machine-state-split.md) — `machine.toml` vs `state.toml`: declarative vs tool-owned
- [ADR 0003](docs/adr/0003-reconciler-loop.md) — pure `Diff(desired, observed) → Plan` reconciler
- [ADR 0004](docs/adr/0004-scan-root-discovery.md) — how repos are discovered without `dotkeeper add`
- [ADR 0005](docs/adr/0005-state-toml-locking.md) — cross-process state.toml locking
- [ADR 0006](docs/adr/0006-syncthing-v2-migration.md) — Syncthing v2 migration and the pseudo-version pin
## Status
**v1.0.0** is the current release. It adds the multi-transport framework (Manager, cost-model routing, GitSSHTransport, Tailscale resolver, `dotkeeper transport` + `bare-init` CLIs) on top of the v0.9-era performance and correctness improvements (auto-pause for idle folders, smart rescan with cross-platform filesystem classification, the v0.9.5 schedule-drift detector that pushed steady-state CPU into the single digits). See [CHANGELOG.md](CHANGELOG.md) for full release notes.
## Port isolation
dotkeeper runs its embedded Syncthing on separate ports so it does not conflict with a system-installed Syncthing instance.
| Resource | System Syncthing | dotkeeper |
|----------|-----------------|-----------|
| API | 127.0.0.1:8384 | 127.0.0.1:18384 |
| Sync | :22000 | :12000 |
| Discovery | :21027 | :11027 |
| Config | `~/.config/syncthing` | `~/.local/share/dotkeeper/syncthing` |
## Requirements
- **Go 1.26+** (build only)
- **git**
- **Service manager** — systemd (Linux), launchd (macOS), or cron (BSD/fallback)
- Internet access for cross-network sync (LAN-only also works)
## License
Copyright (C) 2026 Julian Corbet
This program is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License as published by the Free Software Foundation, version 3.
See [LICENSE](LICENSE) for the full text.
Syncthing (embedded) is licensed under the Mozilla Public License 2.0.