{"id":13777791,"url":"https://github.com/voxpupuli/modulesync","last_synced_at":"2025-05-11T11:34:11.556Z","repository":{"id":18017871,"uuid":"21045921","full_name":"voxpupuli/modulesync","owner":"voxpupuli","description":"Synchronize common files across your Git repositories.","archived":false,"fork":false,"pushed_at":"2025-03-16T18:28:37.000Z","size":647,"stargazers_count":107,"open_issues_count":33,"forks_count":49,"subscribers_count":110,"default_branch":"master","last_synced_at":"2025-04-29T21:23:14.079Z","etag":null,"topics":["hacktoberfest"],"latest_commit_sha":null,"homepage":"","language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/voxpupuli.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"open_collective":"vox-pupuli","github":"voxpupuli"}},"created_at":"2014-06-20T17:15:24.000Z","updated_at":"2025-04-15T08:56:05.000Z","dependencies_parsed_at":"2022-08-20T23:40:40.296Z","dependency_job_id":"df412c1c-b0a3-4094-9b4b-de42bdbd3c8b","html_url":"https://github.com/voxpupuli/modulesync","commit_stats":{"total_commits":505,"total_committers":51,"mean_commits":9.901960784313726,"dds":0.7346534653465346,"last_synced_commit":"aca023770fc59020db1b446d6631964c56884c27"},"previous_names":[],"tags_count":32,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/voxpupuli%2Fmodulesync","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/voxpupuli%2Fmodulesync/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/voxpupuli%2Fmodulesync/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/voxpupuli%2Fmodulesync/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/voxpupuli","download_url":"https://codeload.github.com/voxpupuli/modulesync/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253102594,"owners_count":21854499,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["hacktoberfest"],"created_at":"2024-08-03T18:00:48.793Z","updated_at":"2025-05-11T11:34:11.499Z","avatar_url":"https://github.com/voxpupuli.png","language":"Ruby","funding_links":["https://opencollective.com/vox-pupuli","https://github.com/sponsors/voxpupuli"],"categories":["Development","Ruby","Modules","Module Management"],"sub_categories":["Time Series","Modules Management"],"readme":"ModuleSync\n===========\n\n[![License](https://img.shields.io/github/license/voxpupuli/modulesync.svg)](https://github.com/voxpupuli/modulesync/blob/master/LICENSE)\n[![Test](https://github.com/voxpupuli/modulesync/actions/workflows/ci.yml/badge.svg)](https://github.com/voxpupuli/modulesync/actions/workflows/ci.yml)\n[![codecov](https://codecov.io/gh/voxpupuli/modulesync/branch/master/graph/badge.svg?token=Mypkl78hvK)](https://codecov.io/gh/voxpupuli/modulesync)\n[![Release](https://github.com/voxpupuli/modulesync/actions/workflows/release.yml/badge.svg)](https://github.com/voxpupuli/modulesync/actions/workflows/release.yml)\n[![RubyGem Version](https://img.shields.io/gem/v/modulesync.svg)](https://rubygems.org/gems/modulesync)\n[![RubyGem Downloads](https://img.shields.io/gem/dt/modulesync.svg)](https://rubygems.org/gems/modulesync)\n[![Donated by Puppet Inc](https://img.shields.io/badge/donated%20by-Puppet%20Inc-fb7047.svg)](#transfer-notice)\n\nTable of Contents\n-----------------\n\n1. [Usage TLDR](#usage-tldr)\n2. [Overview](#overview)\n3. [How it works](#how-it-works)\n4. [Installing](#installing)\n5. [Workflow](#workflow)\n6. [The Templates](#the-templates)\n\nUsage TLDR\n----------\n\n```\ngem install modulesync\nmsync --help\n```\n\nOverview\n--------\n\nModuleSync was written as a simple script with ERB templates to help the\nPuppet Labs module engineers manage the zoo of Puppet modules on GitHub, and\nhas now been restructured and generalized to be used within other\norganizations. Puppet modules within an organization tend to have a number of\nmeta-files that are identical or very similar between modules, such as the\n`Gemfile`, `.travis.yml`, `.gitignore`, or `spec_helper.rb`. If a file needs to\nchange in one module, it likely needs to change in the same way in every other\nmodule that the organization manages.\n\nOne approach to this problem is to use sed in a bash for loop on the modules to\nmake a single change across every module. This approach falls short if there is\na single file that is purposefully different than the others, but still needs\nto be managed. Moreover, this approach does not help if two files are\nstructured differently but need to be changed with the same meaning; for\ninstance, if the .travis.yml of one module uses a list of environments to\ninclude, and another uses a matrix of includes with a list of excludes, adding\na test environment to both modules will require entirely different approaches.\n\nModuleSync provides the advantage of defining a standard template for each\nfile to follow, so it becomes clear what a file is supposed to look like. Two\nfiles with the same semantics should also have the same syntax. A difference\nbetween two files should have clear reasons, and old cruft should not be left\nin files of one module as the files of another module march forward.\n\nAnother advantage of ModuleSync is the ability to run in no-op mode, which\nmakes local changes and shows the diffs, but does not make permanent changes in\nthe remote repository.\n\nHow It Works\n------------\n\nModuleSync is a gem that uses the GitHub workflow to clone, update, and push module\nrepositories. It expects to be activated from a directory containing\nconfiguration for modulesync and the modules, or you can pass it the location\nof this configuration directory. [The configuration for the Puppet Labs\nmodules](https://github.com/puppetlabs/modulesync_configs), can be used as an\nexample for your own configuration. The configuration directory contains a\ndirectory called moduleroot which mirrors the structure of a module. The files\nin the moduleroot are ERB templates, and MUST be named after the target file,\nwith `.erb.` appended. The templates are\nrendered using values from a file called `config_defaults.yml` in the root (not\nmoduleroot) of the configuration directory. The default values can be\noverridden or extended by adding a file called `.sync.yml` to the module itself.\nThis allows us to, for example, have a set of \"required\" gems that are added\nto all Gemfiles, and a set of \"optional\" gems that a single module might add.\n\nWithin the templates, values can be accessed in the `@configs` hash, which is\nmerged from the values under the keys `:global` and the target file name (no\n`.erb` suffix).\n\nThe list of modules to manage is in `managed_modules.yml` in the configuration\ndirectory. This lists just the names of the modules to be managed.\n\nModuleSync can be called from the command line with parameters to change the\nbranch you're working on or the remote to clone from and push to. You can also\ndefine these parameters in a file named `modulesync.yml` in the configuration\ndirectory.\n\nInstalling\n----------\n\n```\ngem install modulesync\n```\n\nFor developers:\n\n```\ngem build modulesync.gemspec\ngem install modulesync-*.gem\n```\n\nWorkflow\n--------\n\n### Default mode\n\nWith no additional arguments, ModuleSync clones modules from the puppetlabs\ngithub organization and pushes to the master branch.\n\n#### Make changes\n\nMake changes to a file in the moduleroot. For sanity's sake you should commit\nand push these changes, but in this mode the update will be rendered from the\nstate of the files locally.\n\n#### Dry-run\n\nDo a dry-run to see what files will be changed, added and removed. This clones\nthe modules to `modules/\u003cnamespace\u003e-\u003cmodulename\u003e` in the current working\ndirectory, or if the modules are already cloned, does an effective `git fetch\norigin; git checkout master; git reset --hard origin/master` on the modules.\nDon't run modulesync if the current working directory contains a modules/\ndirectory with changes you want to keep. The dry-run makes local changes there,\nbut does not commit or push changes. It is still destructive in that it\noverwrites local changes.\n\n```\nmsync update --noop\n```\n\n#### Offline support\n\nThe --offline flag was added to allow a user to disable git support within\nmsync. One reason for this is because the user wants to control git commands\nexternal to msync.  Note, when using this command, msync assumes you have\ncreate the folder structure and git repositories correctly. If not, msync will\nfail to update correctly.\n\n```\nmsync update --offline\n```\n\n#### Damage mode\n\nMake changes for real and push them back to master. This operates on the\npre-cloned modules from the dry-run or clones them fresh if the modules aren't\nfound.\n\n```\nmsync update -m \"Commit message\"\n```\n\nAmend the commit if changes are needed.\n\n```\nmsync update --amend\n```\n\nFor most workflows you will need to force-push an amended commit. Not required\nfor gerrit.\n\n```\nmsync update --amend --force\n```\n\n#### Automating Updates\n\nYou can install a pre-push git hook to automatically clone, update, and push\nmodules upon pushing changes to the configuration directory. This does not\ninclude a noop mode.\n\n```\nmsync hook activate\n```\n\nIf you have activated the hook but want to make changes to the configuration\ndirectory (such as changes to `managed_modules.yml` or `modulesync.yml`) without\ntouching the modules, you can deactivate the hook.\n\n```\nmsync hook deactivate\n```\n\n#### Submitting PRs/MRs to GitHub or GitLab\n\nYou can have modulesync submit Pull Requests on GitHub or Merge Requests on\nGitLab automatically with the `--pr` CLI option.\n\n```\nmsync update --pr\n```\n\nIn order for GitHub PRs or GitLab MRs to work, you must either provide\nthe `GITHUB_TOKEN` or `GITLAB_TOKEN` environment variables,\nor set them per repository in `managed_modules.yml`, using the `github` or\n`gitlab` keys respectively.\n\nFor GitHub Enterprise and self-hosted GitLab instances you also need to set the\n`GITHUB_BASE_URL` or `GITLAB_BASE_URL` environment variables, or specify the\n`base_url` parameter in `modulesync.yml`:\n\n```yaml\n---\nrepo1:\n  github:\n    token: 'EXAMPLE_TOKEN'\n    base_url: 'https://api.github.com/'\n\nrepo2:\n  gitlab:\n    token: 'EXAMPLE_TOKEN'\n    base_url: 'https://git.example.com/api/v4'\n```\n\nThen:\n\n* Set the PR/MR title with `--pr-title` or in `modulesync.yml` with the\n  `pr_title` attribute.\n* Assign labels to the PR/MR with `--pr-labels` or in `modulesync.yml` with\n  the `pr_labels` attribute. **NOTE:** `pr_labels` should be a list. When\n  using the `--pr-labels` CLI option, you should use a comma separated list.\n* Set the target branch with `--pr-target-branch` or in `modulesync.yml` with\n  the `pr_target_branch` attribute.\n\nMore details for GitHub:\n\n* Octokit [`api_endpoint`](https://github.com/octokit/octokit.rb#interacting-with-the-githubcom-apis-in-github-enterprise)\n\n### Using Forks and Non-master branches\n\nThe default functionality is to run ModuleSync on the puppetlabs modules, but\nyou can use this on your own organization's modules. This functionality also\napplies if you want to work on a fork of the puppetlabs modules or work on a\nnon-master branch of any organization's modules. ModuleSync does not support\ncloning from one remote and pushing to another, you are expected to fork\nmanually. It does not support automating pull requests.\n\n#### Dry-run\n\nIf you dry-run before doing the live update, you need to specify what namespace\nto clone from because the live update will not re-clone if the modules are\nalready cloned. The namespace should be your fork, not the upstream module (if\nworking on a fork).\n\n```\nmsync update -n puppetlabs --noop\n```\n\n#### Damage mode\n\nYou don't technically need to specify the namespace if the modules are already\ncloned from the dry-run, but it doesn't hurt. You do need to specify the\nnamespace if the modules are not pre-cloned. You need to specify a branch to\npush to if you are not pushing to master.\n\n```\nmsync update -n puppetlabs -b sync_branch -m \"Commit message\"\n```\n\n#### Configuring ModuleSync defaults\n\nIf you're not using the puppetlabs modules or only ever pushing to a fork of\nthem, then specifying the namespace and branch every time you use ModuleSync\nprobably seems excessive. You can create a file called modulesync.yml in the\nconfiguration directory that provides these arguments automatically. This file\nhas a form such as:\n\n```yaml\n---\nnamespace: mygithubusername\nbranch: modulesyncbranch\n```\n\nThen you can run ModuleSync without extra arguments:\n\n```\nmsync update --noop\nmsync update -m \"Commit message\"\n```\n\nAvailable parameters for modulesync.yml\n\n* `git_base` : The default URL to git clone from (Default: 'git@github.com:')\n* `namespace` : Namespace of the projects to manage (Default: 'puppetlabs').\n  This value can be overridden in the module name (e.g. 'namespace/mod') or by\n  using the `namespace` key for the module in `managed_modules.yml`.\n* `branch` : Branch to push to (Default: 'master')\n* `remote_branch` : Remote branch to push to (Default: Same value as branch)\n* `message` : Commit message to apply to updated modules.\n* `pre_commit_script` : A script to be run before commiting (e.g. 'contrib/myfooscript.sh')\n* `pr_title` : The title to use when submitting PRs/MRs to GitHub or GitLab.\n* `pr_labels` : A list of labels to assign PRs/MRs created on GitHub or GitLab.\n\n##### Example\n\n###### GitHub\n\n```yaml\n---\nnamespace: MySuperOrganization\nbranch: modulesyncbranch\npr_title: \"Updates to module template files via modulesync\"\npr_labels:\n  - TOOLING\n  - MAINTENANCE\n  - MODULESYNC\n```\n\n###### GitLab\n\n```yaml\n---\ngit_base: 'user@gitlab.example.com:'\nnamespace: MySuperOrganization\nbranch: modulesyncbranch\n```\n\n###### Gerrit\n\n```yaml\n---\nnamespace: stackforge\ngit_base:  ssh://jdoe@review.openstack.org:29418/\nbranch: msync_foo\nremote_branch: refs/publish/master/msync_foo\npre_commit_script: openstack-commit-msg-hook.sh\n```\n\n#### Filtering Repositories\n\nIf you only want to sync some of the repositories in your managed_modules.yml, use the `-f` flag to filter by a regex:\n\n```\nmsync update -f augeas -m \"Commit message\"    # acts only on the augeas module\nmsync update -f puppet-a..o \"Commit message\"\n```\n\nIf you want to skip syncing some of the repositories in your managed_modules.yml, use the `-x` flag to filter by a regex:\n\n```\nmsync update -x augeas -m \"Commit message\"    # acts on all modules except the augeas module\nmsync update -x puppet-a..o \"Commit message\"\n```\n\nIf no `-f` is specified, all repository are processed, if no `-x` is specified no repository is skipped. If a repository matches both `-f` and `-x` it is skipped.\n\n#### Pushing to a different remote branch\n\nIf you want to push the modified branch to a different remote branch, you can use the -r flag:\n\n```\nmsync update -r master_new -m \"Commit message\"\n```\n\n#### Automating updates\n\nIf you install a git hook, you need to tell it what remote and branch to push\nto. This may not work properly if you already have the modules cloned from a\ndifferent remote. The hook will also look in modulesync.yml for default\narguments.\n\n```\nmsync hook activate -n puppetlabs -b sync_branch\n```\n\n#### Updating metadata.json\n\nModulesync can optionally bump the minor version in `metadata.json` for each\nmodified modules if you add the `--bump` flag to the command line:\n\n```\nmsync update -m \"Commit message\" --bump\n```\n\n#### Tagging repositories\n\nIf you wish to tag the modified repositories with the newly bumped version,\nyou can do so by using the `--tag` flag:\n\n```\nmsync update -m \"Commit message\" --bump --tag\n```\n\n#### Setting the tag pattern\n\nYou can also set the format of the tag to be used (`printf`-formatted)\nby setting the `tag_pattern` option:\n\n```\nmsync update -m \"Commit message\" --bump --tag --tag_pattern 'v%s'\n```\n\nThe default for the tag pattern is `%s`.\n\n#### Updating the CHANGELOG\n\nWhen bumping the version in `metadata.json`, modulesync can let you\nupdating `CHANGELOG.md` in each modified repository.\n\nThis is one by using the `--changelog` flag:\n\n```\nmsync update -m \"Commit message\" --bump --changelog\n```\n\nThis flag will cause the `CHANGELOG.md` file to be updated with the\ncurrent date, bumped (minor) version, and commit message.\n\nIf `CHANGELOG.md` is absent in the repository, nothing will happen.\n\n\n#### Working with templates\n\nAs mentioned, files in the moduleroot directory must be ERB templates (they must have an .erb extension, or they will be ignored). These files have direct access to @configs hash, which gets values from config_defaults.yml file and from the module being processed:\n\n```erb\n\u003c%= @configs[:git_base] %\u003e\n\u003c%= @configs[:namespace] %\u003e\n\u003c%= @configs[:puppet_module] %\u003e\n```\n\nAlternatively some meta data is passed to the template. This will allow you to add custom Ruby extensions inside the\ntemplate, reading other files from the module, to make the template system more adaptive.\n\n```erb\nmodule: \u003c%= @metadata[:module_name] %\u003e\ntarget: \u003c%= @metadata[:target_file] %\u003e\nworkdir: \u003c%= @metadata[:workdir] %\u003e\n```\n\nWill result in something like:\n\n```\nmodule: puppet-test\ntarget: modules/github-org/puppet-test/test\nworkdir: modules/github-org/puppet-test\n```\n\nThe Templates\n-------------\n\nSee [Puppet's modulesync\\_configs](https://github.com/puppetlabs/modulesync_configs) and [Vox Pupuli's modulesync\\_config](https://github.com/voxpupuli/modulesync_config)\nrepositories for different templates currently in use.\n\n## Transfer Notice\n\nThis plugin was originally authored by [Puppet Inc](http://puppet.com).\nThe maintainer preferred that Vox Pupuli take ownership of the module for future improvement and maintenance.\nExisting pull requests and issues were transferred over, please fork and continue to contribute at https://github.com/voxpupuli/modulesync.\n\nPreviously: https://github.com/puppetlabs/modulesync\n\n## License\n\nThis gem is licensed under the Apache-2 license.\n\n## Release information\n\nTo make a new release, please do:\n* update the version in the gemspec file\n* Install gems with `bundle install --with release --path .vendor`\n* generate the changelog with `bundle exec rake changelog`\n* Check if the new version matches the closed issues/PRs in the changelog\n* Create a PR with it\n* After it got merged, push a tag. GitHub actions will do the actual release to rubygems and GitHub Packages\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvoxpupuli%2Fmodulesync","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvoxpupuli%2Fmodulesync","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvoxpupuli%2Fmodulesync/lists"}