{"id":13567549,"url":"https://github.com/icy/git_xy","last_synced_at":"2025-10-28T22:44:51.339Z","repository":{"id":139397406,"uuid":"269400182","full_name":"icy/git_xy","owner":"icy","description":"Handy way to manage (sub)module in multiple repositories. Designed for lazy engineers. This project is written for educational purpose only. Use them at your own risk.","archived":false,"fork":false,"pushed_at":"2021-10-08T07:20:28.000Z","size":183,"stargazers_count":6,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-07-03T11:01:45.159Z","etag":null,"topics":["code-sharing","git","micro-nightmare","submodule","subtree","xyproblem"],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/icy.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-06-04T15:45:26.000Z","updated_at":"2024-01-14T00:01:22.000Z","dependencies_parsed_at":null,"dependency_job_id":"f17e8a22-cac9-4f0d-9087-37f8a25b5b3a","html_url":"https://github.com/icy/git_xy","commit_stats":null,"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"purl":"pkg:github/icy/git_xy","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/icy%2Fgit_xy","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/icy%2Fgit_xy/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/icy%2Fgit_xy/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/icy%2Fgit_xy/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/icy","download_url":"https://codeload.github.com/icy/git_xy/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/icy%2Fgit_xy/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":281527355,"owners_count":26516845,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-28T02:00:06.022Z","response_time":60,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["code-sharing","git","micro-nightmare","submodule","subtree","xyproblem"],"created_at":"2024-08-01T13:02:34.388Z","updated_at":"2025-10-28T22:44:51.306Z","avatar_url":"https://github.com/icy.png","language":"Shell","funding_links":[],"categories":["Shell"],"sub_categories":[],"readme":"## Description\n\n**WARNING**:\nThis project is written for educational purpose only. Use them at your own risk.\n\n`git_xy` helps to synchronize sub directories between git repositories\nsemi-automatically, and may generate pull requests on `github` if\nchanges are detected on the destination repository. It works like\n`rsync`, but for git repositories.\n\n`git_xy` reads a list of source/destination specifications from a\nconfiguration file, and for each of them, `git_xy` fetches changes\nfrom the source repository and synchronizes them to destination path\n(thanks to `rsync`). It finally generates commit and creates new PR\n(pull request) if necessary.\nSee more details in [How it works](#how-it-works).\n\n## TOC\n\n* [Description](#description)\n* [Usage](#usage)\n  * [Installation](#installation)\n  * [Configuration](#configuration)\n  * [Invocation](#invocation)\n  * [Sample Prs on Github](#sample-prs-on-github)\n  * [Environment variables](#environment-variables)\n  * [Hooks](#hooks)\n  * [How it works](#how-it-works)\n* [TODO](#todo)\n* [Why](#why)\n  * [Native support](#native-support)\n  * [Meta repository](#meta-repository)\n  * [Very contraint tools](#very-constraint-tools)\n  * [Back-and-forth tools](#back-and-forth-tools)\n* [Resources](#resources)\n* [Author. License](#author-license)\n\n## Usage\n\n### Installation\n\n`git_xy` is a Bash4 script. It requires some additional tools on system:\n\n* GNU tools: `awk`, `rsync`, `bash`, `git`, `grep`, `sed`\n* Github command tool for PR creation: https://github.com/cli/cli/releases\n\nThe main program `git_xy` can be installed anywhere on your search path\n\n```\n$ sudo wget -O /usr/local/bin/git_xy \\\n    https://github.com/icy/git_xy/raw/ng/git_xy.sh\n\n$ sudo chmod 755 /usr/local/bin/git_xy\n```\n\n### Configuration\n\nConfiguration consists of source/destination specification in the following\nformat:\n\n```\nsrc_repo src_branch src_path   dst_repo dst_branch dest_path [pr_base] [rsync_options]\n```\nwhich reads in the order\n\n* `src_repo`, `src_branch`, `src_path`: The source repository, branch and path\n* `dst_repo`, `dst_branch`, `dst_path`: The destination repository, branch and path.\n* `pr_base` is optional and is used to specify where you want the PR arrives.\n  By default, it's the upstream repository.\n* `rsync_options` (optional): Any options for `rsync` command. The first option\n  should start with `-`.\n\nSee examples in [git_xy.config-sample.txt](git_xy.config-sample.txt).\n\n```\ngit@github.com:icy/pacapt ng lib/ git@github.com:icy/pacapt master lib/\n```\n\n### Invocation\n\nNow execute the script\n\n```\nGIT_XY_CONFIG=\"git_xy.config-sample.txt\" ./git_xy.sh\n```\n\nthe script will fetch changes in `lib` directory from branch `ng`\nin the `pacapt` repository,\nand update the same repository on another branch `master`.\nIf changes are detected, a new branch will be created and/or\nsome pull request will be generated.\n\nIf you provide `GIT_XY_CONFIG=\"-\"`, the script will read from `STDIN`.\n\n### Sample Prs on Github\n\nThese PRs are generated by using [sample configuration file](git_xy.config-sample.txt).\n\nGenerated by the latest version of the script:\n\n* https://github.com/icyfork/pacapt/pull/18 (File synchronization)\n* https://github.com/icyfork/pacapt/pull/17\n* https://github.com/icyfork/pacapt/pull/14\n* https://github.com/icyfork/pacapt/pull/15 (same configuration,\n  but when `GIT_XY_REVERSE=yes`)\n\nGenerated by some older versions of the script:\n\n* https://github.com/icyfork/pacapt/pull/10\n* https://github.com/icyfork/pacapt/pull/1\n* https://github.com/icy/pacapt/pull/140\n* https://github.com/icy/pacapt/pull/139\n\n### Environment variables\n\n* `GIT_XY_ENV_FILE`: Where you define some configuration for your\n  script. In this file you can define/do whatever you want before\n  the script starts. Please note that you may want to use `export`\n  to make your variables available to subsequent processes.\n* `GIT_XY_CONFIG`: Path to the configuration file. When it is empty and/or\n  not specified, the value `git_xy.config` is used. You can also use `-`\n  to specify `stdin` as the input source of configurations.\n* `GIT_XY_PUSH_OPTIONS`: Options used by `git push` command when\n  new commit is generated by the script. Default to empty string.\n* `GIT_XY_SET_OPTIONS`: Options used by `set` command. See `man set`\n  for details. For example, if you want to turn on debug mode,\n  use `GIT_XY_SET_OPTIONS=-x`. Please use this variable with care;\n  and please note that the option `-u` and `+e` will be always enforced.\n* `GIT_XY_HOOKS`: Specify a list of post-commit hooks. By default,\n  it is `gh`. See [Hooks](#hooks) for details\n* `GIT_XY_REVERSE`: When the value is `yes`, the synchronization direction\n  is reversed (dst becomes src, src become dst). This is useful when\n  you want to fetch something from the downstream.\n  The `pr_base` option is not changed when the option is `yes`.\n* `D_GIT_SYNC`: Where the script fetches remote repositories.\n  It's a hard-coded string `$HOME/.local/share/git_xy/`\n  here `$HOME` is your home folder.\n\n### Hooks\n\nA hook is a predefined method which would be executed when new commit\nis generated. A hook is expected to return zero code.\n\n* `gh`: Generate a Github pull request when new commit is created.\n\n### How it works\n\nNothing magic, it's a wrapper of `git clone, rsync and git commit`:)\nLet's say we have configuration file\n\n```\nsrc_repo src_branch src_path dst_repo dst_branch dst_path\n```\n\nthe script will do as below\n\n* Create a clone of the `src_repo` in `~/.local/share/git_xy/src_repo`\n  (The actual folder name is a bit different to avoid some special characters\n  in the user input.)\n* Check out the existing branch `src_branch`\n* Create a clone of the `dest_repo` in `~/.local/share/git_xy/dst_repo`\n* Check out the existing branch `dst_branch`\n* Create new branch from `dst_branch` (if neccessary).\n  This branch is specially used for PR creation.\n* Use `rsync` to synchronize the contents of the `src_path` and `dst_path`.\n  On the local machine where the script runs, it's a variant of the command\n  `rsync -ra SRC DST` here\n  `SRC` is `~/.local/share/git_xy/src_repo/src_path` and\n  `DST` is `~/.local/share/git_xy/dst_repo/dst_path`\n* Generate new commit\n* If specified, execute the hook to generate new pull request on Github\n\nWell, it's so easy right? It's an automation support of your handy commands.\n\n## TODO\n\n- [ ] Add tests and automation support for the project\n- [ ] Provide a link to the original source\n- [ ] More `POSIX` ;)\n- [ ] Sometimes we only need to create a PR without generate any commit\n- [ ] Re-use existing branch to generate new PR\n\nDone\n\n- [x] Read configuration file from `STDIN`\n- [x] `hook/gh`: Return zero if a PR already exists\n- [x] Support file synchronization...\n- [x] Add option to reverse the synchronization (dst becomes src and vice versa)\n- [x] Better error reporting\n- [x] Handle `--delete` option\n- [x] Create a hook script to create pull requests\n- [x] Gather multiple sub-folder in a single PR\n- [x] Re-use existing `git_xy` branch\n- [x] Better hook to handle where PRs will be created\n- [x] Add some information from the last commit of the source repository\n- [x] Make sure the top/root directory is not used (we allow that)\n- [x] Allow a repository to update itself\n\n## Why\n\nThere are many tools trying to solve the code-sharing problem:\n\n### Native support\n\n* `git submodule`:\n  Create pointers to some commit hashes on the upstream repositories, and\n  check out the upstream repository as sub-directory of the current\n  repository upon request.\n  It's you and your team who watch changes on upstream and fetch them\n  manually. More submodules to watch, more issues to handle.\n* `git subtree`:\n  Quite similar to `git submodule`, but it doesn't provide fragile pointer.\n  Instead, it fetches all upstream commits and creates some merge points\n  in the current repository.\n  This way it's more stable than `git submodule`,\n  when you always have a copy of the upstream code in your repository.\n  You get what you don't really mean: duplication of commits,\n  bigger size, confused/noisy commit messages,\n  and you have to learn how to merge (really?)\n  Good reading: https://www.atlassian.com/git/tutorials/git-subtree.\n\n  Git-subtree original experimental project is found\n  here https://github.com/apenwarr/git-subtree.\n\nLooked like `git submodule` requires you to understand C programming language,\nwhile `git subtree` is kind of Python which hides pointers from your laptop:D\n\n### Meta-repository\n\n* https://github.com/esrlabs/josh: Acting likes `nginx`-reverse proxy, \n  `josh` \"combines the advantages of a monorepo with those of multirepo setups \n  by leveraging a blazingly-fast, incremental, and reversible implementation \n  of git history filtering\". Discussing on HN:\n  https://news.ycombinator.com/item?id=27844363\n* https://github.com/zephyrproject-rtos/west:\n  This tool is origined from Zephyr-RTOS project, and inspired by\n  google repo tool and `git submodule`. \n  \"West lets you manage multiple Git repositories under a single \n  directory using a single file, called the west manifest file, or manifest for short.\"\n  By default, it uses Zephyr's manifest file\n  (https://github.com/zephyrproject-rtos/zephyr/blob/master/west.yml) but it\n  doesn't really care much, that means the tool can be used by other project too.\n  I've found this tool on July 2nd 2021. I'll get back with some reviews.\n* https://github.com/ingydotnet/git-subrepo:\n  Another `git-slave`-liked project, which helps to manage multiple\n  small repositories. It uses `git work-tree`, and it helps to generate\n  pull/push/merge command in sub repositories by using the same command.\n  Forget your `git command`, as you have to learn to use this\n  new wrapper for all sub commands\n  (commit, pull, merge...) Written completely in Bash.\n* https://github.com/twosigma/git-meta:\n  Another `git-slave` which uses `git-submodule` to create\n  the meta repository. Another NodeJs tool\n* https://github.com/mateodelnorte/meta:\n  Similar to `git-slave`, which creates a meta repository\n  that includes multiple small repositories. You adapt both mono/micro\n  repository idea. Written in NodeJs...\n* https://sourceforge.net/projects/gitslave/:\n  The project is still on `sourceforge`, looked like it's not maintained.\n  The idea is to have meta project which handles multiple repository.\n  The basic tutorial is here:\n  http://gitslave.sourceforge.net/tutorial-basic.html.\n\n### Very constraint tools\n\n* https://github.com/teambit/bit (npm only):\n  Use this if your current Linux kernel is written in NodeJs.\n* https://github.com/lerna/lerna (javascript only):\n  Use this if your current Linux kernel is written in Javascript.\n* https://gerrit.googlesource.com/git-repo/ (Android only?):\n  Use this if your server is running on Android.\n* https://github.com/microsoft/VFSForGit (sic, Windows only):\n  Use this if you are running Linux inside a VM inside a Windows host.\n\n### Back-and-forth tools\n\n* https://github.com/splitsh/lite:\n  Split a repository to read-only standalone repositories\n* https://github.com/unravelin/tomono:\n  You hate micro-`***` enough and you just want a big repository\n  that includes all your small repositories. This tool helps you.\n\nWell, there are too many tools...\nWhat I really need is a simple way to pull changes from some repository\nto another repository, generates some pull request for reviewing,\nand the downstream maintainer will decide what they would do next.\n\nMorever, this process should be done automatically when the upstream\nrepository is updated. Human intervention is not the right way when\nthere are just 100 or 500 repositories because of the raise of the\nmicro-repository `design` (if any) :D\n\n## Resources\n\n* A monorepo misconception - atomic cross-project commits: \n  https://www.snellman.net/blog/archive/2021-07-21-monorepo-atomic/, \n  https://news.ycombinator.com/item?id=27903282\n\n## Author. License\n\nThe script is written by Ky-Anh Huynh.\nThe work is released under a MIT license.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ficy%2Fgit_xy","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ficy%2Fgit_xy","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ficy%2Fgit_xy/lists"}