{"id":15022754,"url":"https://github.com/puppetlabs/vanagon","last_synced_at":"2025-04-08T01:36:52.225Z","repository":{"id":23294400,"uuid":"26653553","full_name":"puppetlabs/vanagon","owner":"puppetlabs","description":"All of your packages will fit into this van with this one simple trick.","archived":false,"fork":false,"pushed_at":"2025-02-04T02:16:00.000Z","size":2370,"stargazers_count":27,"open_issues_count":7,"forks_count":77,"subscribers_count":146,"default_branch":"main","last_synced_at":"2025-04-01T00:34:56.687Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/puppetlabs.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"docs/CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":"CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2014-11-14T19:17:18.000Z","updated_at":"2024-09-13T12:14:37.000Z","dependencies_parsed_at":"2023-01-14T13:21:18.448Z","dependency_job_id":"f53a0e5e-a730-4518-98c4-44dbd1226e4e","html_url":"https://github.com/puppetlabs/vanagon","commit_stats":{"total_commits":1238,"total_committers":90,"mean_commits":"13.755555555555556","dds":0.7810985460420032,"last_synced_commit":"701cccd065d8ebac128fe262873c94020be983db"},"previous_names":[],"tags_count":152,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/puppetlabs%2Fvanagon","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/puppetlabs%2Fvanagon/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/puppetlabs%2Fvanagon/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/puppetlabs%2Fvanagon/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/puppetlabs","download_url":"https://codeload.github.com/puppetlabs/vanagon/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247761051,"owners_count":20991531,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-09-24T19:58:21.032Z","updated_at":"2025-04-08T01:36:52.208Z","avatar_url":"https://github.com/puppetlabs.png","language":"Ruby","funding_links":[],"categories":[],"sub_categories":[],"readme":"![Build Status](https://travis-ci.org/puppetlabs/vanagon.svg?branch=master)\n# The Vanagon Project\n\n\u003c!-- MarkdownTOC --\u003e\n\n- [What is Vanagon?](#what-is-vanagon)\n  - [How is it pronounced?](#how-is-it-pronounced)\n- [Runtime Requirements](#runtime-requirements)\n  - [Local Host:](#local-host)\n  - [Remote Build Target:](#remote-build-target)\n- [Configuration and Usage](#configuration-and-usage)\n  - [Overview](#overview)\n  - [`build` usage](#build-usage)\n  - [`inspect` usage](#inspect-usage)\n- [Engines](#engines)\n  - [Amazon Ec2](#amazon-ec2)\n- [Contributing](#contributing)\n- [License](#license)\n- [Maintainers](#maintainers)\n\n\u003c!-- /MarkdownTOC --\u003e\n\n\n## What is Vanagon?\n\nVanagon is a tool to build a single package out of a project, which can itself\ncontain one or more components. This tooling is being used to develop the\npuppet-agent package, which contains components such as openssl, ruby, and\naugeas among others. For a simple example, please see the project in the\n[`examples/`](examples) directory.\n\nVanagon builds up a Makefile and packaging files (specfile for RPM,\ncontrol/rules/etc for DEB) and copies them to a remote host, where make can be\ninvoked to build all of the components and make a package of the contents.\n\n### How is it pronounced?\n\nVanagon (/ˈvænəgɪn/) sounds like \"van again.\" It does not sound like \"van wagon\" or \"van and gone.\"\n\n## Runtime Requirements\n\nVanagon carries two sets of requirements: requirements for the local host where Vanagon is run, and for the remote target where compilation and packaging happens.\n\nAlso, Vanagon ships with a number of engines which may include additional optional dependencies if you wish to use them. These engines are currently considered experimental, and receive less attention than the `Hardware` or `vmpooler` engines do. If you find a bug in these engines, [please open a ticket](https://tickets.puppetlabs.com/browse/PA) and let us know.\n\n### Local Host:\n\n- [Ruby](https://www.ruby-lang.org/en/) (Ruby 2.3.x is the miniumum supported version)\n- [fustigit](https://github.com/mckern/fustigit)\n- [ruby-git](https://github.com/schacon/ruby-git)\n- [docopt](https://github.com/docopt/docopt.rb)\n- The command line tool `ssh` ([homepage](https://www.openssh.com/)) available on the local `${PATH}` (any modern version should suffice)\n- The command line tool `rsync` ([homepage](https://rsync.samba.org/)) available on the local `${PATH}` (At least rsync 2.6.x)\n- The command line tool `git` ([homepage](https://git-scm.com/)) available on the local `${PATH}` (Vanagon is tested against Git version 1.8.x but should work with any newer version)\n\n#### Optional requirements\n\n- [AWS SDK for Ruby](https://github.com/aws/aws-sdk-ruby), if you're using the EC2 engine\n- [Docker](https://www.docker.com/community-edition), if you're using the Docker engine\n\n### Remote Build Target:\n\n**Note:** package installation \u0026 builder configuration for the remote target can be customized in the `Platform` configuration that defines target provisioning instructions.\n\n- GNU Make ([homepage](https://www.gnu.org/software/make/)) (Vanagon specifically targets the feature set provided by [GNU Make 3.81](http://git.savannah.gnu.org/cgit/make.git/tree/NEWS?h=3.81\u0026id=776d8b7bc2ff83f8ebf5d357ec89e3bbe6d83962) but newer versions are known to work -- older versions are specifically known to **not** work!)\n- Bash ([homepage](https://www.gnu.org/software/bash/)) is required by the Makefiles that Vanagon generates\n- An ssh server ([homepage](https://www.openssh.com/)) is required by most engines\n- The command line tool `rsync` ([homepage](https://rsync.samba.org/)) (At least rsync 2.6.x)\n\n## Configuration and Usage\n\nVanagon won't be much use without a project to build. Beyond that, you must\ndefine any platforms you want to build for. Vanagon ships with some simple\nbinaries to use, but the one you probably care about is named 'build'.\n\n### Overview\n\nVanagon is broken down into three core ideas: the project, the component and\nthe platform. The project contains one or more components and is built for a\nplatform. As a quick example, if I had a ruby app and wanted to package it, the\nproject would probably contain a component for ruby and a component for my app.\nIf I wanted to build it for debian wheezy, I would define a platform called\nwheezy and build my project against it.\n\nFor more detailed examples of the DSLs available, please see the\n[examples](examples) directory and the YARD documentation for Vanagon.\n\n### CLI commands\n\nThe vanagon command line tool contains these commands: \n\n* `build`               build a package given a project and platform\n* `build_host_info`     print information about build hosts\n* `build_requirements`  print external packages required to build project\n* `completion`          outputs path to tab completion script\n* `inspect`             a build dry-run, printing lots of information about the build\n* `list`                shows a list of available projects and platforms\n* `render`              create local versions of packaging artifacts for project\n* `sign`                sign a package\n* `ship`                upload a package to a distribution server\n* `help`                print this help\n\nCommands are called with a git-like pattern of `vanagon \u003csubcommand\u003e`. \nFor example: `vanagon list`\n\n### CLI changes and deprecations (from version 0.16.0)\n\nPrior to 0.16.0, the vanagon command line contained these commands\n\n* `build`\n* `build_host_info`\n* `build_requirements`\n* `inspect`\n* `render`\n* `repo`\n* `ship`\n* `sign`\n\nWith the exception of `repo`, which calls `packaging` methods, the remaining commands\nhave been moved to a git-like pattern of `vanagon \u003csubcommand\u003e`. `vangon build` replaced `build`, `vanagon ship` replaced `ship` and so forth.\n\nThe older calling usage is deprecated.\n\n### `vanagon build` usage\nThe build command has positional arguments and position independent flags.\n\n#### Positional arguments\n\n##### project name\nThe name of the project to build; a file named `\u003cproject_name\u003e.rb` must be\npresent under `configs/projects` in the working directory.\n\n##### platform name\nThe name of the target platform to build `\u003cproject_name\u003e` against; a file named\n`\u003cplatform_name\u003e.rb` must be present under `configs/platforms` in the working\ndirectory. This can also be a comma separated list of platforms such as `platform1,platform2`;\nnote that there are no spaces after the comma.\n\n##### target host \u003coptional\u003e\nTarget host is an optional argument to override the host selection. Instead of using\na random VM collected from the pooler (Vanagon's default build engine), the build will\nattempt connect to the target host over SSH as the `root` user.\n\nIf building on multiple platforms, multiple targets can also be specified using\na comma separated list such as `host1,host2` (note that there are no spaces after\nthe comma). If less targets are specified than platforms, the default engine\n(`pooler`) will be used for platforms without a target. If more targets are specified\nthan platforms, the extra platforms will be ignored.\n\nBuild machines should be cleaned between builds.\n\n#### Flagged arguments\n\n**Note:** command flags currently can be used anywhere in the command. Recommended usages\nis to use flagged arguments before the positional arguments.\n\n##### -w DIR, --workdir DIR\nSpecifies a directory on the local host where the sources should be placed and builds performed.\nDefaults to a temporary directory created with Ruby's `Dir.mktmpdir` method.\n\n##### -r DIR, --remote-workdir DIR\nExplicitly specify a directory on the remote target to place sources and perform\nbuilds. Components can then be rebuilt manually on the build host for faster iteration. Sources may not be correctly updated if this directory already exists.\nDefaults to a temporary directory created by running `mktemp -d` on the remote target.\n\n##### -c DIR, --configdir DIR\nSpecifies where project configuration is found. Defaults to $pwd/configs.\n\n##### -e ENGINE, --engine ENGINE\nChoose a different virtualization engine to use to select the build target.\nCurrently supported engines are:\n* `base` - Pure ssh backend; no teardown currently defined\n* `local` - Build on the local machine; platform name must match the local machine\n* `docker` - Builds in a docker container\n* `pooler` - Selects a vm from Puppet's internal vmpooler to build on\n* `hardware` - Build on a specific taget and lock it in redis\n* `ec2` - Build on a specific AWS instance.\n* `ABS` - Selects a vm from Puppet's internal Always-be-scheduling service to build on\n\n#### Flags\n\n**Note:** command flags can be used anywhere in the command.\n\n##### -p, --preserve\nIndicates that the host used for building the project should be left intact\nafter the build instead of destroyed. The host is usually destroyed after a\nsuccessful build, or left after a failed build.\n\n##### -h, --help\nDisplay command-line help.\n\n\n#### Environment variables\n\n##### `VANAGON_SSH_KEY`\nA full path on disk for a private ssh key to be used in ssh and rsync\ncommunications. This will be used instead of whatever defaults are configured\nin .ssh/config.\n\n##### `VANAGON_SSH_AGENT`\nWhen set, Vanagon will forward the ssh authentication agent connection.\n\n##### `VMPOOLER_TOKEN`\nUsed in conjunction with the pooler engine, this is a token to pass to the\nvmpooler to access the API. Without this token, the default lifetime of vms\nwill be much shorter.\n\n##### `LOCK_MANAGER_HOST`\nThe name of the host where redis is running. Redis is used to handle a lock\nwhen using the hardware engine. It defaults to *redis*, with no domain.\n\n##### `LOCK_MANAGER_PORT`\nPort of the system where redis is running. Defaults to *6379*.\n\n##### `VANAGON_USE_MIRRORS`\nControls whether component sources are downloaded directly from upstream URLs\nor from configured mirrors. Most Puppet projects using Vanagon default to\nfetching components from internal mirrors. Set this variable to `n` or `false`\nwhen building outside of the Puppet private network to download directly from\nupstream sources.\n\n##### `VANAGON_RETRY_COUNT`\nSome phases of compilation support retries. The default value is *1* but\nsetting to any integer value greater than 1 will causes these components\nto retry operations on failure until the `VANAGON_RETRY_COUNT` limit is reached.\n\n##### `VANAGON_TIMEOUT`\nSome phases of compilation can take an indeterminate (but substantial) amount of\ntime. The default value is *7200* seconds(120 minutes) but setting to any\ninteger value these components to fail after the `VANAGON_TIMEOUT` count is reached.\nNote that this value is expected to be in seconds.\n\n##### `VANAGON_FORCE_SIGNING`\nBy default, Vanagon does not fail if extra files signing fails, it just logs an\nerror and continues building the package. This is unwanted behavior in\nenvironments where we expect a hard failure when signing cannot proceed. To\nforce Vanagon to fail if extra files signing fails, ensure this variable is set\nbefore starting a build.\n\n#### Example usage\n`vanagon build --preserve puppet-agent el-6-i386` will build the puppet-agent project\non the el-6-i386 platform and leave the host intact afterward.\n\n`vanagon build --engine=docker puppet-agent el-6-i386` will build the puppet-agent\nproject on the el-6-i386 platform using the docker engine (the platform must\nhave a docker\\_image defined in its config).\n\n---\n\n### `vanagon inspect` usage\n\nThe `inspect` command has positional arguments and position independent flags. It\nmirrors the `build` command, but exits with success after loading and interpolating\nall of the components in the given project. No attempt is made to actually build\nthe given project; instead, a JSON formatted array of hashes is returned and printed\nto `stdout`. This JSON array can be further processed by external tooling, such as `jq`.\n\n#### Positional arguments\n\n##### project name\nThe name of the project to build, and a file named \\\u003cproject\\_name\\\u003e.rb must be\npresent in configs/projects in the working directory.\n\n##### platform name\nThe name of the platform to build against, and a file named\n\\\u003cplatform\\_name\\\u003e.rb must be present in configs/platforms in the working\ndirectory.\n\nPlatform can also be a comma separated list of platforms such as platform1,platform2.\n\n#### Flagged arguments\n\n**Note:** command flags currently can be used anywhere in the command. Recommended usages\nis to use flagged arguments before the positional arguments.\n\n##### -w DIR, --workdir DIR\nSpecifies a directory where the sources should be placed and builds performed.\nDefaults to a temporary directory created with Ruby's Dir.mktmpdir.\n\n##### -c DIR, --configdir DIR \u003coptional\u003e\nSpecifies where project configuration is found. Defaults to $pwd/configs.\n\n##### -e ENGINE, --engine ENGINE\nChoose a different virtualization engine to use to select the build target.\nEngines are respected, but only insofar as components and projects are\nrendered -- the `vanagon inspect` command performs no compilation.\n\nSupported engines are the same as the `vanagon build` command.\n\n#### Flags\n\n##### -h, --help\nDisplay command-line help.\n\n#### Environment variables\n\nEnvironment variables are respected, but only insofar as components and projects are\nrendered -- the `vanagon inspect` command has no behavior to alter.\n\nSupported environment variables are the same as the `vanagon build` command.\n\n#### Example usage\n`vanagon inspect puppet-agent el-6-i386` will load the puppet-agent project\non the el-6-i386 platform and print the resulting list of dependencies,\nbuild-time configuration, environment variables, and expected artifacts.\n\n---\n\nEngines\n---\n\n### Amazon Ec2\n\nNote: If you have the `aws_ami` setup Vanagon will default to the ec2 engine.\n\nTo use the ec2 engine you should have your credentials set either via your `~/.aws/credentials` or environment variables.\nAfter this you can setup your `configs/platforms/\u003cplatform\u003e.rb` to use your\nami, instance type, and key_name to setup the instance.\n\nA simple one looks like this\n\n```ruby\n# configs/platforms/el-7-x86_64.rb\nplatform \"el-7-x86_64\" do |plat|\n    plat.aws_ami \"your-ami-id-here\" # You must set this\n    plat.aws_instance_type \"t2.small\" # Defaults to t1.micro\n    plat.aws_key_name \"vanagon\" # this is the default but you can use whichever\n    plat.aws_user_data \u003c\u003c-eos\n#cloud-config\n    runcmds:\n        - echo #{my_ssh_key} \u003e /root/.ssh/authorized_keys # Most amis block you from logging in as root.\n    eos\n\n### Rest of your code here\n\nend\n```\n\n### ABS (internal)\nWhen using the ABS engine, there is a variety of ways you can specify your token: \n- the environment variable ABS_TOKEN\n- or vanagon token file ~/.vanagon-token (note this is the same file read by the pooler engine)\n- or [vmfloaty](https://github.com/puppetlabs/vmfloaty)'s config file ~/.vmfloaty.yml\n\nIn order to modify or track the VM via floaty or bit-bar you can optionally add the vmpooler token (different from the ABS token) via\n- VMPOOLER_TOKEN\n- or as a second line to the file ~/.vanagon-token\n- or by using the existing mechanism in floaty using a vmpooler_fallback\n\nContributing\n---\nWe'd love to get contributions from you! Once you are up and running, take a look at the\n[Contribution Documents](docs/CONTRIBUTING.md) to see how to get your changes merged\nin.\n\nLicense\n---\nSee [LICENSE](LICENSE) file.\n\n## Maintainers\nSee [MAINTAINERS](MAINTAINERS) file.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpuppetlabs%2Fvanagon","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpuppetlabs%2Fvanagon","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpuppetlabs%2Fvanagon/lists"}