{"id":22511046,"url":"https://github.com/rjayroach/prepd","last_synced_at":"2025-03-28T00:40:31.436Z","repository":{"id":56888700,"uuid":"147282573","full_name":"rjayroach/prepd","owner":"rjayroach","description":null,"archived":false,"fork":false,"pushed_at":"2018-09-18T08:18:49.000Z","size":19,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-03-02T22:34:24.442Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rjayroach.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-09-04T03:20:59.000Z","updated_at":"2018-09-18T08:18:51.000Z","dependencies_parsed_at":"2022-08-20T16:00:07.646Z","dependency_job_id":null,"html_url":"https://github.com/rjayroach/prepd","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rjayroach%2Fprepd","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rjayroach%2Fprepd/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rjayroach%2Fprepd/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rjayroach%2Fprepd/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rjayroach","download_url":"https://codeload.github.com/rjayroach/prepd/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245949553,"owners_count":20698916,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-07T02:08:10.338Z","updated_at":"2025-03-28T00:40:31.391Z","avatar_url":"https://github.com/rjayroach.png","language":"Shell","readme":"# Prepd\n\nPrepd - A Production Ready Environment for Project Development\n\nOne of the core principles of Agile Development is delivering viewable results\nto the business from Week 1. Too often product developement begins with the\napplication software, while the infrastructure to deploy into is addressed as\nand when it is needed.\n\nThankfully, many web application products get to market on similar,\nif not identical, infrastructure. However setting up this infastructure takes time,\nis error prone and typically is non-repeatable ending up as a unique snowflake.\n\nTo avoid this, many development teams turn to a PaaS service such as Heroku.\nThis has limitations and only addresses the final deployment infrastructure.\n\nPrepd aims to address this by providing a 'convention over configruation' approach\nto provisioning infrastructure. From local developer machines (vagrant running linux\non the developer's laptop) to staging and production running a docker swarm cluster.\n\nWith microservices becoming a common application development strategy, prepd\naims to make it dead simple to build and deploy a microservice based application.\nBeginning with the end in mind, Prepd offers a simple, conventional way to provision\nall this infrastructure, including CI workflow, secrets managment, 12-factor apps\n\nAgile Development requires 'near production' infrastructure to be in place from Day 1.\nUsing Prepd, makes that possible quickly and easily without resorting to a PaaS provider.\n\n## Focus\n\nThe focus of Prepd is on enabling developers to build and deploy applications following current\nindustry best practices with as little effort as possible. Being flexible and configurable\nfor the wide variety of application deployment strategies is currently a secondary goal to\ngetting something up and running. Therefore, choices are made:\n\n1. Infrastructure is provisioned via:\n..* Vagrantfile on local machines for development and a local cluster\n..* Terraform plans for clutser infrastructure exclusively on AWS\n2. Ansible is the automation tool used to configure the infrastructure for application deployment\n3. Docker conatainer deployment is currently the only method for deploying applications\n4. The development environment currently supports:\n..* Postgres and Redis for data storage\n..* Rails and Ember for application development\n\nA future goal for Prepd is to enable more application types and tool support\n\n# What is a Production Ready Environment?\n\nIt takes a lot of services tuned to work together to make smoothly running infrastructure\n\n## Networking\n- Domain names figured out and DNS running on Route53 etc\n- Ability to programatically change and update DNS\n- SSL certs are already installed so we do TLS from the beginning on all publicly available infrastructure\n- Load Balancing is setup, configured and running in at least staging and production, but also possible in development\n\n## Development Pipeline Required Services\n\nPrepd provisions and configures the infrastructure and provides a tool to deploy applications into the infrastructure.\nHowever, certain aspects of the pipeline are expected to be provided outside of Prepd, which are:\n\n- Continuous Integration\n- Container Build and Store\n\n### Continuous Integration\n\nCI is expected to be setup and configured as part of an automated deploy process from the outset of the project.\nHere is an example overview of using CircleCI to test a Rails API application\n\n- Create an account on CircleCI and link it to your GitHub account. Authorize CircleCI to access the account\n- Add the Rails API repository as a project on CircleCI. If using rails-templates a circle.yml project already exists\n- Configure slack notifications for when a build completes\n\n### Container Build and Store\n\nA container repository that also builds containers is expected to be provided.\nHere is an example overview of using quay.io to build a Rails API application container\n\n- Create an account on quay.io and link it to your GitHub account. Authorize quay.io to access the account\n- Add the Rails API repository as a docker repository on quay.io\n- Create a trigger to build the container when there is a push on a certain branch of the GitHub repository\n\nPrepd provides ansible playbooks that invoke docker compose to deploy the container from quay.io to the target infrastructure\n\n## Application Services (TODO)\n\nPrepd will be augmented to provide playbooks for the default Application Group as well as Terraform plans that provide:\n\n- Communication Services, e.g. SMTP, SNS (Push), Slack webhooks, Twilio, etc\n- Logging in both local/development and in staging/production with ELK\n- Monitoring/alert service (Prometheus)\n- Additional common 3rd party services as needed\n\n## Swarm Load Balancing\n- network overlays\n- load balancing between micro services\n- manage cluster scaling with compose/swarm mode/ansible or some combination thereof\n\n\n# Installation\n\nPrepd is a ruby gem. It also requires software on the local laptop, including VirtualBox, Vagrant and Ansible\n\n```bash\ngem install prepd\n```\n\n## Automated Installation of Dependencies (TODO)\n\nWith the gem installed, navigate to it's directory and run bootstrap.sh to install dependencies\n\n```bash\nbundle cd prepd\n./bootstrap.sh\n```\n\nThis will:\n\n- Install ansible\n- Clone the ansible-roles repository\n- Run ansible to install Virtualbox and Vagrant\n\n## Manual Installation of Dependencies\n\n### Ansible\n\nTested with version 2.2.0\n\n#### Install on MacOS\n\nIf planning to install on a clean machine:\n1. Wipe Mac: http://support.apple.com/kb/PH13871  OR http://support.apple.com/en-us/HT201376\n2. Create New User with Admin rights\n3. Enable ssh\n\n\n```bash\nsudo systemsetup -f -setremotelogin on\n```\n\n\nInstall Homebrew:\n\n```bash\nruby -e \"$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)\"\n```\n\nInstall ansible\n\n```bash\nbrew install ansible\n```\n\n\nInstall python with zlib and ssl support\n\n```bash\nxcode-select --install\nbrew install openssl\nbrew link openssl --force\nbrew uninstall python\nbrew install python --with-brewed-openssl\nsudo easy_install pip\nsudo pip install -U ansible\nsudo pip install -U setuptools cryptography markupsafe\nsudo pip install -U ansible boto\n```\n\n#### Install on Ubuntu\n\n```bash\napt-get install ansible\n```\n\n#### Copy ssh key\n\n```bash\nscp ~/.ssh/id_rsa.pub user@host:~/.ssh/authorized_keys\nssh user@host 'mkdir ~/.ssh \u0026\u0026 curl https://github.com/user.keys -o ~/.ssh/authorized_keys'\nssh -A user@host\n# wget https://raw.githubusercontent.com/rjayroach/prepd/master/install.sh\nbash \u003c(curl -s https://raw.githubusercontent.com/rjayroach/prepd/master/install.sh)\n```\n\n\n### VirtualBox\n\nInstall VirtualBox from [here](https://www.virtualbox.org/wiki/Downloads)\n\n### Vagrant\n\nInstall Vagrant from [here](https://www.vagrantup.com/docs/installation/)\n\n```bash\nvagrant plugin install vagrant-vbguest      # keep your VirtualBox Guest Additions up to date\nvagrant plugin install vagrant-cachier      # caches guest packages\nvagrant plugin install vagrant-hostmanager  # updates /etc/hosts file when machines go up/down\n```\n\n#### vagrant-hostmanager\nThis plugin automatically updates the host's /etc/hosts file when vagrant machines go up/down\n\nIn order to do that it needs sudo password or sudo priviledges.\nTo avoid being asked for the password every time the hosts file is updated,\n[enable passwordless sudo](https://github.com/devopsgroup-io/vagrant-hostmanager#passwordless-sudo)\nfor the specific command that hostmanager uses to update the hosts file\n\n\n# Prepd Actors\n\nA Client may have multiples projects. Applications share common infrastructure that is defined by the Project\n\n- Client: An organization with one or more projects, e.g Acme Corp\n- Project: A definition of infrastructure provided for one or more applications\n- Application: A logical group of deployable repositories, e.g. a Rails API server and an Ember web client\n\n\n## Projects\n\n- A project is comprised of Infrastructure Environments (IE) and Application Groups (AG)\n- Infrastructure Environemnts are defined separately for each environment\n- Application Groups are deployed into one or more Infrastructure EnvironmentS\n\n## Infrastructure Environments\n\nInfrastructure is either Vagrant machines for development and local environments or EC2 instances for staging and production\n\nLocal, Staging and Production Environments use a Docker swarm network to manage applicaiton groups\n\n- local: virtual machines running on laptop via vagrant whose primary purpose is application development\n- development: primary purpose is also application development, but the infrastructure is deployed in the cloud (AWS)\n- staging: a mirror of production in every way with the possible exception of reduced or part-time resources\n- production: production ;-)\n\n## Applications\n\nApplications are the content that actually gets deployed. The entire purpose of prepd is to provide a consistent\nand easy to manage infrastructure for each environment into which the application will be deployed.\n\n\n# Usage\n\n## New Client\n\nThis overview assumes a complete greenfield, e.g. that no infrastructure exists, no applications exist or even 3rd\nparty service have been setup. To start from zero, then:\n\n- Create a new GH Organization\n- Create an AWS Account and two IAM Groups: Administrators and ReadOnlyAdministrators\n- Create a CI Account and give it access to the GH Organization\n- Create a Docker Private Repository account and give it access to the GH Organization\n- Create the project in prepd\n\nThe first four items are outside the scope of this document.\n\n```ruby\nprepd\nc = Client.create(name: 'Acme')\n```\n\n## New Project\n- create a GH repo for the project\n- create an IAM user for project_name-terraform and download the AWS credentials CSV\n- create an IAM user for project_name-ansible and download the AWS credentials CSV\n- use prepd to create the project using the repo_url and path names (tf_creds and ansible_creds) to CSV files\n\n```ruby\nc = Client.find_by(name: 'Acme')\nc.projects.new(name: 'widget', repo_url: 'git@github.com:my_git_hub_account/widget.git')\nc.tf_creds = 'Users/dude/aws/widget-terraform.csv'\nc.ansible_creds = 'Users/dude/aws/widget-ansible.csv'\nc.save\n```\n\n## New Application\n\nView the [lego README.md](https://github.com/rjayroach/lego) on creating micro serivce applications with Rails and Ember\n\n## Bring Up the Machine\n\n```ruby\ncd ~/prepd/acme/widget\nvagrant up\nvagrant ssh\n```\n\n\n# Credentials\n\n## Project Credentials\nPrepd will create the following credential (hidden) files in project_root:\n\n- .boto: AWS IAM credentials that give read only access to Ansible\n- .developer.yml: Developer’s git account (and other account) details\n- .terraform-vars.txt: AWS IAM credentials that give full access to CRUD AWS resources\n- .vault-password.txt: a UUID used to encrypt and decrypt ansible vault files\n- .id_rsa.pub: the public key uploaded to AWS as the primary key pair for accessing EC2 instances\n- .id_rsa: the private key\n\n- terraform will use project_root/id_rsa.pub to upload key_material to AWS for the machine key\n- config-development.yml checks the project_root and: 1) if .boto exists link it, 2) if id_rsa and id_rsa.pub exist then link them\n- the developer can then do ssh-add which will auto load ~/.ssh/id_rsa to login or run ansible\n\n\n## Transfer Credentials to New Machine\n\nThe prepd gem can encrypt the credentials using gpg which must be installed on the host machine\n\nThe encrypted credentials are written to and read from the user's home directory so that they are not accidentally\ncommitted to the project repository\n\n### Encrypt\n\n```ruby\nprepd\nc = Client.find_by(name: 'Acme')\np = c.projects.find_by(name: 'widget')\np.encrypt\n```\n\nThis will create a tar file containing the various project credentials. It will then invoke gpg to encrypt the archive.\nThe credentials will be placed in the project's data directory\n\nYou will be prompted for a passphrase to enter twice. After doing that send the file by email or other mechanism\n\n### Decrypt\n\nOn the target machine, use prepd to decrypt the file and place it in the correct directory\n\n- Clone the project repository\n- Place the gpg tar file in the project's data directory\n- Run prepd. It will expect to find the credentials file in the project's data directory\n\n```ruby\nprepd\nc = Client.find_by(name: 'Acme')\np = c.projects.find_by(name: 'widget')\np.decrypt\n```\n\n## Authorization\n\nIf giving a developer access to the machine for development only (not terraform or ansible) then add their public key to the\ninstance’s ~/.ssh/authorized_keys. The developer uses ssh-agent forwarding to access the machine from the VM\n\n\n# Development\n\nAfter checking out the repo, run `bin/setup` to install dependencies. You can also run `bin/console` for an interactive prompt that will allow you to experiment.\n\nTo install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).\n\n# Contributing\n\nBug reports and pull requests are welcome on GitHub at https://github.com/[USERNAME]/prepd. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [Contributor Covenant](http://contributor-covenant.org) code of conduct.\n\n\n# License\n\nThe gem is available as open source under the terms of the [MIT License](http://opensource.org/licenses/MIT).\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frjayroach%2Fprepd","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frjayroach%2Fprepd","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frjayroach%2Fprepd/lists"}