Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/voxpupuli/librarian-puppet


https://github.com/voxpupuli/librarian-puppet

hacktoberfest

Last synced: about 1 month ago
JSON representation

Awesome Lists containing this project

README 

        
# Librarian-puppet



[![License](https://img.shields.io/github/license/voxpupuli/librarian-puppet.svg)](https://github.com/voxpupuli/librarian-puppet/blob/master/LICENSE)

[![Test](https://github.com/voxpupuli/librarian-puppet/actions/workflows/test.yml/badge.svg)](https://github.com/voxpupuli/librarian-puppet/actions/workflows/test.yml)

[![Release](https://github.com/voxpupuli/librarian-puppet/actions/workflows/release.yml/badge.svg)](https://github.com/voxpupuli/librarian-puppet/actions/workflows/release.yml)

[![RubyGem Version](https://img.shields.io/gem/v/librarian-puppet.svg)](https://rubygems.org/gems/librarian-puppet)

[![RubyGem Downloads](https://img.shields.io/gem/dt/librarian-puppet.svg)](https://rubygems.org/gems/librarian-puppet)

[![Donated by Tim Sharpe](https://img.shields.io/badge/donated%20by-Tim%20Sharpe-fb7047.svg)](#transfer-notice)



## Introduction



Librarian-puppet is a bundler for your puppet infrastructure.  You can use

librarian-puppet to manage the puppet modules your infrastructure depends on,

whether the modules come from the [Puppet Forge](https://forge.puppet.com/),

Git repositories or just a path.



* Librarian-puppet can reuse the dependencies listed in your `Modulefile` or `metadata.json`

* Forge modules can be installed from [Puppetlabs Forge](https://forge.puppet.com/) or an internal Forge such as [Pulp](http://www.pulpproject.org/)

* Git modules can be installed from a branch, tag or specific commit, optionally using a path inside the repository

* Modules can be installed from GitHub using tarballs, without needing Git installed

* Modules can be installed from a filesystem path

* Module dependencies are resolved transitively without needing to list all the modules explicitly



Librarian-puppet manages your `modules/` directory for you based on your

`Puppetfile`.  Your `Puppetfile` becomes the authoritative source for what

modules you require and at what version, tag or branch.



Once using Librarian-puppet you should not modify the contents of your `modules`

directory.  The individual modules' repos should be updated, tagged with a new

release and the version bumped in your Puppetfile.



It is based on [Librarian](https://github.com/applicationsonline/librarian), a

framework for writing bundlers, which are tools that resolve, fetch, install,

and isolate a project's dependencies.



## Versions



Librarian-Puppet 5.0.0 and newer requires Ruby >= 2.7 and Puppet >= 6. Use version 4.0.1 if you need support for Puppet 6 or Ruby 2.6 or earlier.



Librarian-Puppet 4.0.0 and newer requires Ruby >= 2.5 and Puppet >= 5. Use version 3.0.1 is you need support for Puppet 3 or Puppet 4, or Ruby 2.4 or earlier.



Librarian-Puppet 3.0.0 and newer requires Ruby >= 2.0. Use version 2.2.4 if you need support for Puppet 3.7 or earlier, or Ruby 1.9 or earlier. Note that [Puppet 4.10 and newer require Ruby 2.1](https://puppet.com/docs/puppet/4.10/system_requirements.html#prerequisites) or newer.



Librarian-Puppet 2.0.0 and newer requires Ruby >= 1.9 and uses Puppet Forge API v3. For Ruby 1.8 use 1.5.0.



See the [Changelog](CHANGELOG.md) for more details.



## The Puppetfile



Every Puppet repository that uses Librarian-puppet may have a file named

`Puppetfile`, `metadata.json` or `Modulefile` in the root directory of that repository.

The full specification

for which modules your puppet infrastructure repository depends goes in here.



### Simple usage



If no Puppetfile is present, `librarian-puppet` will download all the dependencies

listed in your `metadata.json` or `Modulefile` from the Puppet Forge,

as if the Puppetfile contained



    forge "https://forgeapi.puppet.com"



    metadata



### Example Puppetfile



    forge "https://forgeapi.puppet.com"



    mod 'puppetlabs-razor'

    mod 'puppetlabs-ntp', "0.0.3"



    mod 'puppetlabs-apt',

      :git => "git://github.com/puppetlabs/puppetlabs-apt.git"



    mod 'puppetlabs-stdlib',

      :git => "git://github.com/puppetlabs/puppetlabs-stdlib.git"



    mod 'puppetlabs-apache', '0.6.0',

      :github_tarball => 'puppetlabs/puppetlabs-apache'



    mod 'acme-mymodule', :path => './some_folder'



    exclusion 'acme-bad_module'



### Recursive module dependency resolution



When fetching a module all dependencies specified in its

`Modulefile`, `metadata.json` and `Puppetfile` will be resolved and installed.



### Puppetfile Breakdown



    forge "https://forgeapi.puppet.com"



This declares that we want to use the official Puppet Labs Forge as our default

source when pulling down modules.  If you run your own local forge, you may

want to change this.



    metadata



Download all the dependencies listed in your `metadata.json` or `Modulefile` from the Puppet Forge.



    mod 'puppetlabs-razor'



Pull in the latest version of the Puppet Labs Razor module from the default

source.



    mod 'puppetlabs-ntp', "0.0.3"



Pull in version 0.0.3 of the Puppet Labs NTP module from the default source.



    mod 'puppetlabs-apt',

      :git => "git://github.com/puppetlabs/puppetlabs-apt.git"



Our puppet infrastructure repository depends on the `apt` module from the

Puppet Labs GitHub repos and checks out the `master` branch.



    mod 'puppetlabs-apt',

      :git => "git://github.com/puppetlabs/puppetlabs-apt.git",

      :ref => '0.0.3'



Our puppet infrastructure repository depends on the `apt` module from the

Puppet Labs GitHub repos and checks out a tag of `0.0.3`.



    mod 'puppetlabs-apt',

      :git => "git://github.com/puppetlabs/puppetlabs-apt.git",

      :ref => 'feature/master/dans_refactor'



Our puppet infrastructure repository depends on the `apt` module from the

Puppet Labs GitHub repos and checks out the `dans_refactor` branch.



When using a Git source, we do not have to use a `:ref =>`.

If we do not, then librarian-puppet will assume we meant the `master` branch.



If we use a `:ref =>`, we can use anything that Git will recognize as a ref.

This includes any branch name, tag name, SHA, or SHA unique prefix. If we use a

branch, we can later ask Librarian-puppet to update the module by fetching the

most recent version of the module from that same branch.



Note that Librarian-puppet recognizes the [r10k Puppetfile's](https://github.com/puppetlabs/r10k/blob/master/doc/puppetfile.mkd) additional

options, `:tag`, `:commit`, and `:branch`, but only as aliases for `:ref`.

That is, there is no implementation of r10k's optimizations around fetching

these different types of git objects.



The Git source also supports a `:path =>` option. If we use the path option,

Librarian-puppet will navigate down into the Git repository and only use the

specified subdirectory. Some people have the habit of having a single repository

with many modules in it. If we need a module from such a repository, we can

use the `:path =>` option here to help Librarian-puppet drill down and find the

module subdirectory.



    mod 'puppetlabs-apt',

      :git => "git://github.com/fake/puppet-modules.git",

      :path => "modules/apt"



Our puppet infrastructure repository depends on the `apt` module, which we have

stored as a directory under our `puppet-modules` git repos.



    mod 'puppetlabs-apache', '0.6.0',

      :github_tarball => 'puppetlabs/puppetlabs-apache'



Our puppet infrastructure repository depends on the `puppetlabs-apache` module,

to be downloaded from GitHub tarball.



    mod 'acme-mymodule', :path => './some_folder'



Our puppet infrastructure repository depends on the `acme-mymodule` module,

which is already in the filesystem.



    exclusion 'acme-bad_module'



Exclude the module `acme-bad_module` from resolution and installation.



## How to Use



Install librarian-puppet:



    $ gem install librarian-puppet



Prepare your puppet infrastructure repository:



    $ cd ~/path/to/puppet-inf-repos

    $ (git) rm -rf modules

    $ librarian-puppet init



Librarian-puppet takes over your `modules/` directory, and will always

reinstall (if missing) the modules listed the `Puppetfile.lock` into your

`modules/` directory, therefore you do not need your `modules/` directory to be

tracked in Git.



Librarian-puppet uses a `.tmp/` directory for tempfiles and caches. You should

not track this directory in Git.



Running `librarian-puppet init` will create a skeleton Puppetfile for you as

well as adding `tmp/` and `modules/` to your `.gitignore`.



    $ librarian-puppet install [--clean] [--verbose]



This command looks at each `mod` declaration and fetches the module from the

source specified.  This command writes the complete resolution into

`Puppetfile.lock` and then copies all of the fetched modules into your

`modules/` directory, overwriting whatever was there before.



Librarian-puppet support both v1 and v3 of the Puppet Forge API.

Specify a specific API version when installing modules:



    $ librarian-puppet install --use-v1-api # use the v1 API

    $ librarian-puppet install --no-use-v1-api # use the v3 API; this is the default



Get an overview of your `Puppetfile.lock` with:



    $ librarian-puppet show



Inspect the details of specific resolved dependencies with:



    $ librarian-puppet show NAME1 [NAME2, ...]



Find out which dependencies are outdated and may be updated:



    $ librarian-puppet outdated [--verbose]



Update the version of a dependency:



    $ librarian-puppet update apt [--verbose]

    $ git diff Puppetfile.lock

    $ git add Puppetfile.lock

    $ git commit -m "bumped the version of apt up to 0.0.4."



## Configuration



Configuration comes from three sources with the following highest-to-lowest

precedence:



* The local config (`./.librarian/puppet/config`)

* The environment

* The global config (`~/.librarian/puppet/config`)



You can inspect the final configuration with:



    $ librarian-puppet config



You can find out where a particular key is set with:



    $ librarian-puppet config KEY



You can set a key at the global level with:



    $ librarian-puppet config KEY VALUE --global



And remove it with:



    $ librarian-puppet config KEY --global --delete



You can set a key at the local level with:



    $ librarian-puppet config KEY VALUE --local



And remove it with:



    $ librarian-puppet config KEY --local --delete



You cannot set or delete environment-level config keys with the CLI.



Configuration set at either the global or local level will affect subsequent

invocations of `librarian-puppet`. Configurations set at the environment level are

not saved and will not affect subsequent invocations of `librarian-puppet`.



You can pass a config at the environment level by taking the original config key

and transforming it: replace hyphens (`-`) with underscores (`_`) and periods

(`.`) with doubled underscores (`__`), uppercase, and finally prefix with

`LIBRARIAN_PUPPET_`. For example, to pass a config in the environment for the key

`part-one.part-two`, set the environment variable

`LIBRARIAN_PUPPET_PART_ONE__PART_TWO`.



Configuration affects how various commands operate.



* The `path` config sets the directory to install to. If a relative

  path, it is relative to the directory containing the `Puppetfile`. The

  equivalent environment variable is `LIBRARIAN_PUPPET_PATH`.



* The `tmp` config sets the cache directory for librarian. If a relative

  path, it is relative to the directory containing the `Puppetfile`. The

  equivalent environment variable is `LIBRARIAN_PUPPET_TMP`.



Configuration can be set by passing specific options to other commands.



* The `path` config can be set at the local level by passing the `--path` option

  to the `install` command. It can be unset at the local level by passing the

  `--no-path` option to the `install` command. Note that if this is set at the

  environment or global level then, even if `--no-path` is given as an option,

  the environment or global config will be used.



## Rsync Option



The default convergence strategy between the cache and the module directory is

to execute an `rm -r` on the module directory and just `cp -r` from the cache.

This causes the module to be removed from the module path every time librarian

puppet updates, regardless of whether the content has changed. This can cause

some problems in environments with lots of change. The problem arises when the

module directory gets removed while Puppet is trying to read files inside it.

The `puppet master` process will lose its CWD and the catalog will fail to

compile. To avoid this, you can use `rsync` to implement a more conservative

convergence strategy. This will use `rsync` with the `-avz` and `--delete`

flags instead of a `rm -r` and `cp -r`. To use this feature, just set the

`rsync` configuration setting to `true`.



    $ librarian-puppet config rsync true --global



Alternatively, using an environment variable:



    LIBRARIAN_PUPPET_RSYNC='true'



Note that the directories will still be purged if you run librarian-puppet with

the --clean or --destructive flags.



## How to Contribute



 * Pull requests please.

 * Bonus points for feature branches.



## Reporting Issues



Bug reports to the github issue tracker please.

Please include:



 * Relevant `Puppetfile` and `Puppetfile.lock` files

 * Version of ruby, librarian-puppet, and puppet

 * What distro

 * Please run the `librarian-puppet` commands in verbose mode by using the

  `--verbose` flag, and include the verbose output in the bug report as well.



## Transfer Notice



This plugin was originally authored by [Tim Sharpe](https://github.com/rodjek).

The maintainer preferred that [Vox Pupuli](https://voxpupuli.org/) take ownership of the module for future improvement and maintenance.

Existing pull requests and issues were transferred, please fork and continue to contribute [here](https://github.com/voxpupuli/librarian-puppet).



## License

Please see the [LICENSE](https://github.com/voxpupuli/librarian-puppet/blob/master/LICENSE)

file.