{"id":19259060,"url":"https://github.com/r3drun3/github-content-sync","last_synced_at":"2025-04-21T16:30:39.267Z","repository":{"id":178368094,"uuid":"661710234","full_name":"R3DRUN3/github-content-sync","owner":"R3DRUN3","description":"check for file differences in github repo folders 🔎 📁","archived":false,"fork":false,"pushed_at":"2025-04-02T07:50:39.000Z","size":95,"stargazers_count":18,"open_issues_count":9,"forks_count":2,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-04-21T10:03:10.800Z","etag":null,"topics":["api-rest","github","golang"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/R3DRUN3.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2023-07-03T13:24:41.000Z","updated_at":"2024-11-26T18:07:09.000Z","dependencies_parsed_at":"2023-11-09T16:27:25.537Z","dependency_job_id":null,"html_url":"https://github.com/R3DRUN3/github-content-sync","commit_stats":null,"previous_names":["r3drun3/github-content-sync"],"tags_count":13,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/R3DRUN3%2Fgithub-content-sync","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/R3DRUN3%2Fgithub-content-sync/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/R3DRUN3%2Fgithub-content-sync/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/R3DRUN3%2Fgithub-content-sync/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/R3DRUN3","download_url":"https://codeload.github.com/R3DRUN3/github-content-sync/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250090650,"owners_count":21373225,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["api-rest","github","golang"],"created_at":"2024-11-09T19:15:19.296Z","updated_at":"2025-04-21T16:30:38.926Z","avatar_url":"https://github.com/R3DRUN3.png","language":"Go","readme":"# GITHUB CONTENT SYNC 🔎 📁\n[![goaction](https://github.com/R3DRUN3/github-content-sync/actions/workflows/goaction.yaml/badge.svg)](https://github.com/R3DRUN3/github-content-sync/actions/workflows/goaction.yaml)\n[![goreleaser](https://github.com/R3DRUN3/github-content-sync/actions/workflows/release.yaml/badge.svg)](https://github.com/R3DRUN3/github-content-sync/actions/workflows/release.yaml)\n[![oci](https://github.com/R3DRUN3/github-content-sync/actions/workflows/oci.yaml/badge.svg)](https://github.com/R3DRUN3/github-content-sync/actions/workflows/oci.yaml)  \n[![Latest Release](https://img.shields.io/github/release/R3DRUN3/github-content-sync.svg)](https://github.com/R3DRUN3/github-content-sync/releases/latest)\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)\n[![Go Report Card](https://goreportcard.com/badge/github.com/r3drun3/github-content-sync)](https://goreportcard.com/report/github.com/r3drun3/github-content-sync)  \n\nThe *Github Content Sync* tool is a command-line script written in *Go* that allows you to compare the contents of *two folders* within a GitHub repository.  \nIt helps identify files difference between the two folders.  \n\n\u003cbr/\u003e\n\nBasically, if `A` and `B` are the two folders, the tool will output:  \n- files present in `A` but not in `B`\n- files present in `B` but not in `A`\n- files present in both `A` and `B` but with newer commits in `A`  \n\n\u003e [!NOTE]  \n\u003e You can also do cross-branches comparison by specifing the branches for both directories.  \n\n## Purpose\n\nThis tool has been specifically developed to assist the *Special Interest Groups (SIGs)* responsible for *glossary* management within the [CNCF](https://github.com/cncf).  \nThe purpose of the tool is to facilitate the comparison of folder contents within a GitHub repository.  \n**This was specifically meant for those repo that contain documentation in various languages** (divided into different folders) and you need a fast way to know the deltas:  \nIn this case, usually the reference folder and \"*source of truth*\" is the \"*english*\" one (for a real world example take a look at [this repo](https://github.com/cncf/glossary/tree/main/content), for a test playground we use [this one](https://github.com/R3DRUN3/content-sync-tester)).  \nGenerally, it can be useful in scenarios where you have two folders within a repository and you want to identify the differences between them, such as missing files or files with newer commits.  \n## Arguments\n\nThe script requires the following environment variables to be set:\n- `REPO_URL`: The URL of the GitHub repository to analyze. [MANDATORY]\n- `REPO_FOLDER_1`: The name of the reference folder (source of truth, or folder `A`). [MANDATORY]\n- `REPO_FOLDER_2`: The name of the second folder to compare to the reference folder (folder `B`). [MANDATORY]\n- `TOKEN`: An access token with appropriate permissions to *read* and *open issues* on the target repo. [MANDATORY]\n- `FOLDER_1_BRANCH`: The branch for the first folder. If not specified, the default is main [OPTIONAL]\n- `FOLDER_2_BRANCH`: The branch for the second folder. If not specified, the default is main [OPTIONAL]\n- `OPEN_ISSUE`: If set to `true`, this specify that the script needs to open a \"*synchronization issue*\" on the target repo, specifying the folder differences. [OPTIONAL]  \nThe opened issues are structured like [this one](https://github.com/R3DRUN3/content-sync-tester/issues/29).\n- `MULTIPLE_ISSUES`: If `OPEN_ISSUE` is set to `true` and this var is also set to `true`, the script will create multiple issues, one for every file difference. [OPTIONAL]  \n\n\n\u003e [!WARNING]  \n\u003e Be careful when setting the `MULTIPLE_ISSUES` var to *true*: if you execute this script against two folders with many files, it will create many issues on your target repo.  \n\n\n## How it works\n\nThe script performs the following steps:\n1. Checks the presence of the required environment variables and their values.\n1. Creates a GitHub client using the provided access token.\n1. Retrieve the content of the two specified folders via the Github client object.\n1. Compares the contents of the two specified folders within the repository.\n1. Prints the files that are present in the first folder but not in the second folder.\n1. Prints the files with newer commits in the first folder compared to the same files in the second folder.\n1. Prints the files that are present in the second folder but not in the first folder.\n2. If `OPEN_ISSUE` env var is present and set to `true`, opens a \"synchronization issue\" on the target repo.  \n## Examples\n\nYou can run this utility in many ways:  \n\n### As an Executable\nDownload the [release](https://github.com/R3DRUN3/github-content-sync/releases/) that you want and run it:\n\n```shell\n\nexport REPO_URL=https://github.com/R3DRUN3/content-sync-tester\nexport REPO_FOLDER_1=en\nexport REPO_FOLDER_2=it\nexport TOKEN=\u003cyour-github-token-here\u003e\n\n./github-content-sync\n```\n\n\nOutput:\n```console\n   __   __ _____   _ __  _ __   ___      __   _    _  __ _____   ___   _  __ _____      ___  _  __   _  __   __\n ,'_/  / //_  _/  /// / /// /  / o.)   ,'_/ ,' \\  / |/ //_  _/  / _/  / |/ //_  _/    ,' _/ | |/,'  / |/ / ,'_/\n/ /_n / /  / /   / ` / / U /  / o \\   / /_ / o | / || /  / /   / _/  / || /  / /     _\\ `.  | ,'   / || / / /_\n|__,'/_/  /_/   /_n_/  \\_,'  /___,'   |__/ |_,' /_/|_/  /_/   /___/ /_/|_/  /_/     /___,' /_/    /_/|_/  |__/\n\n[ ALL ENVIRONMENT VARIABLES ARE CONFIGURED ]\n[ TARGET REPO URL:  https://github.com/R3DRUN3/content-sync-tester ]\n\n[ FILES PRESENT IN en BUT NOT IN it ]\nnot_present_in_it.md\nnot_present_in_it_2.md\ntest.md\n\n\n[ FILES PRESENT IN BOTH en AND it WITH NEWER COMMITS IN en ]\ndoc2.md\nlast.md\n\n\n ___ ___ ___ ___ ___ ___ ___ ___ ___ ___ ___ ___ ___ ___ ___ ___ ___ ___ ___ ___ ___ ___ ___ ___ ___ ___ ___ ___\n/__//__//__//__//__//__//__//__//__//__//__//__//__//__//__//__//__//__//__//__//__//__//__//__//__//__//__//__/\n```  \n\n### With Docker (Local Build)\nThis repo also contain a Dockerfile so you can launch the script as a docker container.  \nClone the repo locally and buil the image:  \n```console\ngit clone https://github.com/r3drun3/github-content-sync \\\n\u0026\u0026 cd github-content-sync \\\n\u0026\u0026 docker build -t github-content-sync:latest .\n```  \n\nRun the docker container (change env vars accordingly):  \n```console\ndocker run -it --rm -e REPO_URL=https://github.com/cncf/glossary -e REPO_FOLDER_1=content/en -e REPO_FOLDER_2=content/it -e TOKEN=\u003cyour-github-token-here\u003e github-content-sync:latest\n```  \n\n\n### With Docker (Github Packages)\nAlternatively, this repo already contains an action to publish the script's OCI image to [Github Packages](https://github.com/features/packages).  \nPull the version that you want: \n```console\ndocker pull ghcr.io/r3drun3/github-content-sync:1.5.0 \n```  \n\nRun the docker container (change env vars accordingly):  \n```console\ndocker run -it --rm -e REPO_URL=https://github.com/cncf/glossary -e REPO_FOLDER_1=content/en -e REPO_FOLDER_2=content/it -e TOKEN=\u003cyour-github-token-here\u003e ghcr.io/r3drun3/github-content-sync:1.5.0\n```  \n\n### Run via Github Action\nThe script in this repo can also executed inside a  *Github action*, for an example take a look at the [goaction](https://github.com/R3DRUN3/github-content-sync/actions/workflows/goaction.yaml)  Github Action associated to this repo.  \n\n\n## Development and Debug\nFor development and debug I suggest the use of the [VS Code](https://code.visualstudio.com/) IDE.  \nIn order to debug the script locally, you can create the `.vscode/launch.json` file with the following structure:  \n```json\n{\n    \"version\": \"0.2.0\",\n    \"configurations\": [\n      {\n        \"name\": \"Launch\",\n        \"type\": \"go\",\n        \"request\": \"launch\",\n        \"mode\": \"auto\",\n        \"program\": \"${workspaceFolder}/main.go\",\n        \"env\": {\n            \"REPO_URL\": \"\u003cyour-github-repo-target-url\u003e\",\n            \"REPO_FOLDER_1\": \"\u003cpath-of-the-reference-folder-inside-target-repo\u003e\",\n            \"REPO_FOLDER_2\": \"\u003cpath-of-the-folder-to-compare-to-the-reference\u003e\",\n            \"TOKEN\": \"\u003cyour-github-token-here\u003e\",\n            \"OPEN_ISSUE\": \"false\",\n            \"MULTIPLE_ISSUES\": \"false\"\n        }\n      }\n    ]\n  }\n```  \n\n\n\n## Improvements and Next Steps\n\n- It can be useful to maybe add the possibility of comparing multiple folders at the same time, not just 2.\n\n\n## License\n\nThis script is released under the [MIT License](https://opensource.org/license/mit/).  \nFeel free to modify and distribute it as per your needs.  \n\n\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fr3drun3%2Fgithub-content-sync","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fr3drun3%2Fgithub-content-sync","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fr3drun3%2Fgithub-content-sync/lists"}