{"id":48598378,"url":"https://github.com/ardiloot/pubgate","last_synced_at":"2026-04-08T21:04:56.261Z","repository":{"id":348778120,"uuid":"1181626215","full_name":"ardiloot/pubgate","owner":"ardiloot","description":"Safe bidirectional sync between internal and public git repos","archived":false,"fork":false,"pushed_at":"2026-04-02T19:38:17.000Z","size":114,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-04-03T06:40:29.957Z","etag":null,"topics":["cli","git","open-source","python","sync"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ardiloot.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-03-14T12:01:16.000Z","updated_at":"2026-04-02T19:38:20.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/ardiloot/pubgate","commit_stats":null,"previous_names":["ardiloot/pubgate"],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/ardiloot/pubgate","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ardiloot%2Fpubgate","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ardiloot%2Fpubgate/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ardiloot%2Fpubgate/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ardiloot%2Fpubgate/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ardiloot","download_url":"https://codeload.github.com/ardiloot/pubgate/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ardiloot%2Fpubgate/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31573800,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-08T14:31:17.711Z","status":"ssl_error","status_checked_at":"2026-04-08T14:31:17.202Z","response_time":54,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cli","git","open-source","python","sync"],"created_at":"2026-04-08T21:04:52.222Z","updated_at":"2026-04-08T21:04:56.253Z","avatar_url":"https://github.com/ardiloot.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# pubgate\n\n[![Pre-commit](https://github.com/ardiloot/pubgate/actions/workflows/pre-commit.yml/badge.svg)](https://github.com/ardiloot/pubgate/actions/workflows/pre-commit.yml)\n[![Pytest](https://github.com/ardiloot/pubgate/actions/workflows/pytest.yml/badge.svg)](https://github.com/ardiloot/pubgate/actions/workflows/pytest.yml)\n\nSafe bidirectional sync between internal and public git repos.\n\n## The Problem\n\nYou have an internal repo with proprietary code and you want to open-source parts of it. This creates two ongoing problems:\n\n1. **Leak risk.** Internal files, code sections, and commit history must never reach the public repo. Standard git tools (fork, merge, cherry-pick) all carry or expose internal commits, and a single misstep exposes everything. `git filter-repo` can strip history once, but rewrites SHAs on every run, breaking external clones and making it unusable for continuous sync.\n\n2. **Silent divergence.** Once two repos exist, they drift apart. Public contributions arrive on one side, internal development continues on the other, and without a disciplined process the repos become increasingly hard to reconcile: patches stop applying, filtered content falls out of sync, and nobody notices until it's a project to fix.\n\n## What It Does\n\npubgate prepares branches for review. You create and merge PRs on your git host (GitHub, GitLab, etc.). It handles both directions:\n\n- Stage changes behind an internal leak-review PR gate\n- Push reviewed content to a PR branch on the public repo\n- Absorb public contributions back into internal main with three-way merge\n\nFiltering is mechanical: built-in ignore patterns exclude common internal/private/secret file naming conventions out of the box, `BEGIN-INTERNAL` / `END-INTERNAL` markers strip sections from individual files, and `pubgate.toml` is always excluded automatically. Custom `ignore` patterns in the config replace the defaults.\n\n**Core principle:** the public repo is always an exact filtered copy of internal, never an independent fork. If external contributions arrive mid-cycle, `publish` bases the public PR on the last absorbed commit; git's three-way merge preserves them or surfaces conflicts. Divergence stays controlled and bounded.\n\nThe two workflows:\n\n\u003ctable\u003e\n\u003ctr\u003e\n\u003ctd valign=\"top\" width=\"50%\"\u003e\u003cstrong\u003eMaking changes public: absorb → stage → publish\u003c/strong\u003e\u003cbr\u003e\u003cem\u003eabsorb is recommended first if the public repo has unabsorbed changes, but stage and publish proceed either way\u003c/em\u003e\n\n```mermaid\n%%{init: {\"flowchart\": {\"useMaxWidth\": false, \"nodeSpacing\": 25, \"rankSpacing\": 30, \"padding\": 10}}}%%\nflowchart TD\n    A[\"🔒 main (internal)\"]\n    A --\u003e|\"stage\"| B[\"🔒 pubgate/stage (internal)\"]\n    B --\u003e|\"PR · leak review\"| C[\"🔒 pubgate/public-approved (internal)\"]\n    C ==\u003e|\"publish\"| D[\"🌐 pubgate/publish (public)\"]\n    D --\u003e|\"PR · publish check\"| E[\"🌐 main (public)\"]\n\n    style A fill:#2d5a2d,stroke:#4a4,color:#fff\n    style B fill:#1a3a1a,stroke:#4a4,color:#ccc\n    style C fill:#2d5a2d,stroke:#4a4,color:#fff\n    style D fill:#1a3a5a,stroke:#48f,color:#ccc\n    style E fill:#2d3a5a,stroke:#48f,color:#fff\n```\n\n\u003c/td\u003e\n\u003ctd valign=\"top\" width=\"50%\"\u003e\u003cstrong\u003eIncorporating public contributions: absorb\u003c/strong\u003e\u003cbr\u003e\u003cem\u003eRun when the public repo has external contributions\u003c/em\u003e\n\n```mermaid\n%%{init: {\"flowchart\": {\"useMaxWidth\": false, \"nodeSpacing\": 25, \"rankSpacing\": 30, \"padding\": 10}}}%%\nflowchart TD\n    E2[\"🌐 main (public)\"]\n    E2 --\u003e|\"absorb\"| F[\"🔒 pubgate/absorb (internal)\"]\n    F --\u003e|\"PR · merge review\"| A2[\"🔒 main (internal)\"]\n\n    style E2 fill:#2d3a5a,stroke:#48f,color:#fff\n    style F fill:#1a3a1a,stroke:#4a4,color:#ccc\n    style A2 fill:#2d5a2d,stroke:#4a4,color:#fff\n```\n\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/table\u003e\n\n## Getting Started\n\n### Prerequisites\n\n- Python 3.10+ and `git` CLI\n- An existing internal repo with an `origin` remote\n- An existing public repo with at least one commit (e.g. a README created during repo setup)\n- *(Optional)* [`gh` CLI](https://cli.github.com/) authenticated via `gh auth login`. Enables automatic PR creation for GitHub-hosted repos. Without it, pubgate logs the manual steps instead.\n- *(Optional)* [`az` CLI](https://learn.microsoft.com/en-us/cli/azure/) with the `azure-devops` extension, authenticated via `az login`. Enables automatic PR creation for Azure DevOps-hosted repos. The extension is installed automatically if missing. Without it, pubgate logs the manual steps instead.\n- *(Optional)* [Git LFS](https://git-lfs.com/) if your repo uses LFS-tracked files. pubgate auto-detects LFS and handles pointer files automatically. Without it, LFS-specific operations are silently skipped.\n- A clean worktree on `main`, synced with `origin` (no uncommitted changes, no unpushed commits)\n\n### Setup\n\n1. Install:\n   ```bash\n   pip install pubgate\n   ```\n2. Create `pubgate.toml` in repo root:\n   ```toml\n   public_url = \"https://github.com/you/public-repo.git\"\n   ```\n   Built-in ignore patterns cover common conventions (`.internal/*`, `internal/*`, `*-internal.*`, `*.internal.*`, `*_internal.*`, `*-private.*`, `*.private.*`, `*_private.*`, `*.secret`, `*.secrets`). To override them, set `ignore` explicitly (see [Configuration](#configuration)).\n3. Optionally, mark internal-only sections in files (in addition to ignore patterns, you can hide parts of individual files). Three comment styles are supported:\n   ```python\n   # BEGIN-INTERNAL\n   secret_stuff()\n   # END-INTERNAL\n   ```\n   ```javascript\n   // BEGIN-INTERNAL\n   secretStuff();\n   // END-INTERNAL\n   ```\n   ```html\n   \u003c!-- BEGIN-INTERNAL --\u003e\n   \u003cdiv class=\"secret\"\u003e...\u003c/div\u003e\n   \u003c!-- END-INTERNAL --\u003e\n   ```\n   Markers must be properly paired. Nested, unclosed, or orphan `END-INTERNAL` markers cause an error. After scrubbing, a residual check catches any surviving markers that were not removed.\n4. Commit and push your changes to `main` (direct push or via PR). `pubgate absorb` requires a clean worktree synced with `origin`.\n5. Initialize tracking:\n   ```bash\n   pubgate absorb\n   ```\n   On first run, this records the current public repo HEAD as the starting point for future syncs. It creates a PR branch that records this baseline in a tracking file (`.pubgate-absorbed`). Create a PR from that branch into `main` on your git host and merge it.\n6. After your first `pubgate stage` run creates the `pubgate/public-approved` branch, protect it on your git host: require pull requests (no direct pushes) and optionally require approvals. This ensures content only reaches the approved branch through reviewed PRs — the leak-review gate.\n\n## Workflow\n\n### Making changes public: absorb → stage → publish\n\npubgate prepares branches for review. When a supported CLI is available (`gh` for GitHub, `az` for Azure DevOps) and the remote is a recognized host, pubgate automatically creates or updates the PR for each branch it pushes. Otherwise it logs the manual steps. Use `--no-pr` to skip automatic PR creation. After merging changes into internal `main` through your normal PR process:\n\n1. Recommended: run `pubgate absorb` if the public repo has unabsorbed changes, then merge the absorb PR. This isn't required (`stage` and `publish` proceed either way) but it keeps the public snapshot clean.\n2. Run `pubgate stage`. This creates a `pubgate/stage` branch with the filtered snapshot for leak review.\n3. Review the PR from `pubgate/stage` → `pubgate/public-approved` (created automatically on GitHub/Azure DevOps, or create it manually on other hosts). This is the leak-review gate. Review it to ensure no internal code is exposed. Merge when satisfied.\n4. Run `pubgate publish`. This delivers the reviewed content to the public repo as a `pubgate/publish` branch.\n5. Review the PR from `pubgate/publish` → `main` on the public repo (created automatically on GitHub/Azure DevOps, or create it manually). Merge after CI passes.\n\n### Incorporating public contributions: absorb\n\nRun when the public repo has external contributions that need to be brought into the internal repo.\n\n1. Run `pubgate absorb`. This creates a `pubgate/absorb` branch with the merged public changes for review.\n2. Review the PR from `pubgate/absorb` → `main` (created automatically on GitHub/Azure DevOps, or create it manually on other hosts).\n3. Resolve conflicts if any.\n4. Merge the PR.\n\n## Branches and State Tracking\n\n**Branches**\n\nMaking changes public (absorb → stage → publish):\n- **`main`** (internal): internal development branch (protected)\n- **`pubgate/stage`** (internal): branch for leak review: filtered internal content → `pubgate/public-approved`\n- **`pubgate/public-approved`** (internal): holds reviewed staged content approved for publication; created automatically on first `stage` if it doesn't exist. **Protect this branch** on your git host — require PRs (no direct pushes), and optionally require approvals. This is the leak-review gate: only content that passes PR review should land here.\n- **`pubgate/publish`** (public): branch for publish review: reviewed content → public `main`\n- **`main`** (public): public-facing branch (protected)\n\nIncorporating public contributions (absorb):\n- **`pubgate/absorb`** (internal): branch for merge review: public changes → internal `main`\n\n**State files**\n\n- `.pubgate-absorbed` (on `main`): tracks which public commit was last absorbed\n- `.pubgate-staged` (on `pubgate/public-approved`): tracks which internal commit was last staged\n\nCreated and updated automatically.\n\n## CLI Reference\n\n| Command | What it does |\n|---------|-------------|\n| `pubgate stage` | Build a filtered snapshot of internal code and create a branch for leak review |\n| `pubgate publish` | Push reviewed content to a PR branch on the public repo |\n| `pubgate absorb` | Merge public contributions into an internal branch for review |\n| `pubgate status` | Show sync status of absorb, stage, and publish (read-only, fetches remotes) |\n\nFlags `--dry-run`, `--force`, and `--no-pr` apply to `absorb`, `stage`, and `publish` (not `status`). Flags come after the command; `--repo-dir` comes before it.\n\n| Flag | Position | Description |\n|------|----------|-------------|\n| `--dry-run` | after command | Show planned actions without writing branches or files. Still syncs with remotes to ensure accurate plans. Example: `pubgate stage --dry-run` |\n| `--force` | after command | Overwrite an existing PR branch from a previous run whose PR was not yet merged. Without this flag, pubgate errors out if the PR branch already exists. Force-push is blocked on protected branches (`main`, `pubgate/public-approved`, and public `main`). Example: `pubgate absorb --force` |\n| `--no-pr` | after command | Skip automatic PR creation even when a supported CLI (`gh`/`az`) is available. pubgate will still push the branch and log manual steps. Example: `pubgate stage --no-pr` |\n| `--repo-dir` | before command | Run pubgate against a specific repo path instead of the current directory. Example: `pubgate --repo-dir /path/to/repo stage` |\n\n## Configuration\n\nFull `pubgate.toml` example (all fields shown with defaults, only `public_url` is required for first-time setup when the remote doesn't already exist):\n\n```toml\n# Internal repo\ninternal_main_branch = \"main\"\ninternal_approved_branch = \"pubgate/public-approved\"\ninternal_absorb_branch = \"pubgate/absorb\"\ninternal_stage_branch = \"pubgate/stage\"\n\n# Public repo (public_url is required if the git remote isn't already configured)\npublic_url = \"https://github.com/you/public-repo.git\"\npublic_remote = \"public-remote\"\npublic_main_branch = \"main\"\npublic_publish_branch = \"pubgate/publish\"\n\n# State tracking\nabsorb_state_file = \".pubgate-absorbed\"\nstage_state_file = \".pubgate-staged\"\n\n# Filtering (fnmatch syntax; patterns match against both full path and basename)\n# These override the built-in defaults. Omit to use the defaults:\n#   .internal/*  internal/*  *-internal.*  *.internal.*  *_internal.*\n#   *-private.*  *.private.*  *_private.*  *.secret  *.secrets\nignore = [\n    \".internal/*\",\n    \"*-internal.*\",\n    \"*.internal.*\",\n    \"*.secret\",\n]\n```\n\n## Edge Cases\n\n- **Binary files**: included as-is in staged snapshots (`BEGIN-INTERNAL` markers inside binaries are not processed); during absorb, binary modifications take the public version and are flagged for manual review.\n- **Git LFS files**: LFS pointers pass through all pipelines without modification. LFS files are treated as binary (never merged, never scrubbed for internal markers). pubgate runs `git lfs fetch`/`push` automatically during absorb and publish. Use ignore patterns in `pubgate.toml` to exclude sensitive LFS files from publication. If LFS is not installed, these operations are silently skipped.\n- **Renames on public repo**: the new path is copied in; the old file is kept locally and flagged for review.\n- **Deletions on public repo**: deleted files are kept locally and flagged for review in the absorb PR.\n- **Merge conflicts**: absorb uses three-way merge. Conflicts produce standard git conflict markers (`\u003c\u003c\u003c\u003c\u003c\u003c\u003c`/`=======`/`\u003e\u003e\u003e\u003e\u003e\u003e\u003e`) for manual resolution.\n- **Sync artifacts**: absorb excludes both state files (`.pubgate-absorbed`, `.pubgate-staged`) from the diff (they are sync artifacts, not external contributions). When only state files changed since the last absorb, the resulting PR only updates `.pubgate-absorbed` (tracking-only).\n- **Empty files after scrubbing**: files that become empty after removing `BEGIN-INTERNAL` blocks are still included in the staged snapshot.\n- **External contribution between stage and publish**: if someone pushes to the public repo after you stage but before you publish, `publish` still proceeds: it bases the public PR on the last absorbed commit, and git's three-way merge preserves external contributions or surfaces conflicts in the public PR. For a clean snapshot, run `absorb` → merge absorb PR → `stage` → merge stage PR → `publish`.\n- **Stale branch cleanup**: after you merge a PR and its source branch is auto-deleted on the server, pubgate automatically prunes the stale local branch on the next run. No manual cleanup needed.\n- **Commit messages**: absorb commit messages list the public commits being absorbed (safe, they are already public). Stage commit messages list the internal commits since the last stage (safe, stays on the internal repo; useful context for the leak reviewer).\n- **Repeated publish without absorb**: if you publish multiple times without running `absorb` between cycles, each publish PR is based on the same absorbed commit. This produces a trivially resolvable merge conflict on `.pubgate-staged` in the public PR (take the newer value). Running `absorb` between cycles avoids this.\n- **Do not edit the pubgate PR branch directly**: the `pubgate/publish` branch must only contain content produced by `publish`. Manual edits to this branch before merging will be silently overwritten by the next publish cycle (they are not detected as external contributions). If published content needs a fix, make the change in the internal repo and re-run `stage` → `publish`.\n\n## Troubleshooting\n\n| Error | Cause | Fix |\n|-------|-------|-----|\n| \"working tree is not clean\" | Dirty worktree | Commit or stash your changes |\n| \"expected branch 'main', currently on '...'\" | Not on the main branch | Run `git checkout main` |\n| \"HEAD is detached\" | Detached HEAD state | Run `git checkout main` |\n| \"unpushed commit(s)\" | Local `main` is ahead of origin | Push your commits or reset |\n| \"behind\" | Local `main` is behind origin | Run `git pull --rebase` |\n| \"diverged\" | Local `main` has diverged from origin | Reconcile manually (rebase or reset) |\n| \"branch '...' already exists\" | Previous PR not merged | Merge the PR, or use `--force` to overwrite |\n| \"no absorb state found\" | First run, or absorb not yet done | Run `pubgate absorb` to create initial baseline |\n| \"no stage state found\" | Stage PR not merged | Run `pubgate stage` and merge the internal PR |\n| \"has no 'main' branch\" | Public repo is empty (no commits) | Push at least one commit to the public repo (e.g. add a README) |\n\n## Example: Full First-Time Walkthrough\n\n```bash\n# 1. Clone your internal repo and cd into it\ngit clone git@internal-host:you/internal-repo.git\ncd internal-repo\n\n# 2. Create pubgate.toml (built-in ignore patterns cover common conventions)\ncat \u003e pubgate.toml \u003c\u003c 'EOF'\npublic_url = \"https://github.com/you/public-repo.git\"\nEOF\ngit add pubgate.toml \u0026\u0026 git commit -m \"Add pubgate config\" \u0026\u0026 git push\n\n# 3. Bootstrap - records the public repo's current HEAD as baseline\npubgate absorb\n# Output: pushes pubgate/absorb branch\n# → If gh/az CLI is set up, a PR is created automatically\n# → Otherwise, go to your git host, create PR: pubgate/absorb → main\n# → Merge the PR\n\n# 4. Stage staged content (filters out internal files and scrubs markers)\npubgate stage\n# Output: pushes pubgate/stage branch\n# → PR: pubgate/stage → pubgate/public-approved (auto-created on GitHub/Azure DevOps)\n# → Review for leaks, merge it\n\n# 5. Publish to public repo\npubgate publish\n# Output: pushes pubgate/publish to the public remote\n# → PR: pubgate/publish → main on the public repo (auto-created on GitHub/Azure DevOps)\n# → Merge after CI passes\n\n# Done! For future syncs: absorb (if needed) → stage → publish.\n```\n\n## Development\n\nRequires Python 3.10+ and [uv](https://docs.astral.sh/uv/).\n\nFor design decisions and detailed command specifications, see [SPEC.md](SPEC.md).\n\n```bash\nuv sync                         # install dependencies\nuv run pre-commit install       # set up pre-commit hooks (ruff, ty, etc.)\nuv run pytest                   # run tests (-n auto for parallel)\nuv run pre-commit run -a        # run all linting \u0026 formatting checks\n```\n\nTests create temporary git repos locally. No network access needed.\n\n## License\n\nMIT. See [LICENSE](LICENSE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fardiloot%2Fpubgate","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fardiloot%2Fpubgate","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fardiloot%2Fpubgate/lists"}