https://github.com/nemonik/hands-on-DevOps
A hands-on DevOps course covering the culture, methods and repeated practices of modern software development involving Packer, Vagrant, VirtualBox, Ansible, Kubernetes, K3s, MetalLB, Traefik, Docker-Compose, Docker, Taiga, GitLab, Drone CI, SonarQube, Selenium, InSpec, Alpine 3.10, Ubuntu-bionic, CentOS 7...
https://github.com/nemonik/hands-on-DevOps
ansible ansible-automation continuous-delivery continuous-integration devops docker docker-image drone gitlab go hands-on-devops-course inspec kubernetes pipeline plantuml selenium sonarqube taiga vagrant virtual-machine
Last synced: about 1 year ago
JSON representation
A hands-on DevOps course covering the culture, methods and repeated practices of modern software development involving Packer, Vagrant, VirtualBox, Ansible, Kubernetes, K3s, MetalLB, Traefik, Docker-Compose, Docker, Taiga, GitLab, Drone CI, SonarQube, Selenium, InSpec, Alpine 3.10, Ubuntu-bionic, CentOS 7...
- Host: GitHub
- URL: https://github.com/nemonik/hands-on-DevOps
- Owner: nemonik
- License: bsd-3-clause
- Archived: true
- Created: 2018-11-30T23:07:02.000Z (over 7 years ago)
- Default Branch: master
- Last Pushed: 2021-09-05T22:34:04.000Z (over 4 years ago)
- Last Synced: 2024-10-01T06:24:06.907Z (over 1 year ago)
- Topics: ansible, ansible-automation, continuous-delivery, continuous-integration, devops, docker, docker-image, drone, gitlab, go, hands-on-devops-course, inspec, kubernetes, pipeline, plantuml, selenium, sonarqube, taiga, vagrant, virtual-machine
- Language: Shell
- Homepage:
- Size: 76.5 MB
- Stars: 370
- Watchers: 19
- Forks: 154
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- awesome-starred - nemonik/hands-on-DevOps - A hands-on DevOps course covering the culture, methods and repeated practices of modern software development involving Packer, Vagrant, VirtualBox, Ansible, Kubernetes, K3s, MetalLB, Traefik, Docker-C (go)
README
[](./LICENSE)
[](https://github.com/nemonik/hands-on-DevOps/network/members)
[](https://github.com/nemonik/hands-on-DevOps/stargazers)
[](https://twitter.com/intent/tweet?text=I%27ve%20found%20a%20nifty%20hands-on%20DevOps%20course%20developed%20by%20%40nemonik%20at%20https%3A%2F%2Fgithub.com%2Fnemonik%2Fhands-on-DevOps)
# 1. Preface
This Vagrant-based version of my Hands-on DevOps class is DEPRECATED. I suggest you hop over to the new version of this class that significantly modifies the factory's configuration... It is very likely portions of this class IaC will fail at this time. I cannot create the new content while also supporting the prior. The new class can be found at https://github.com/nemonik/hands-on-DevOps-gen2.
# 2. DevOps
A hands-on DevOps course covering the culture, methods and repeated practices of modern software development involving Vagrant, VirtualBox, Ansible, Kubernetes, K3s, MetalLB, Traefik, Docker-Compose, Docker, Taiga, GitLab, Drone CI, SonarQube, Selenium, InSpec, Alpine 3.10, Ubuntu-bionic, CentOS 7...
A reveal.js presentation written to accompany this course can found at [https://nemonik.github.io/hands-on-DevOps/](https://nemonik.github.io/hands-on-DevOps/).
This course will
1. Discuss DevOps,
2. Have you spin up a DevOps toolchain and development environment, and then
3. Author two applications and their accompanying pipelines, the first a continuous integration (CI) and the second a continuous delivery (CD) pipeline.
After this course, you will
1. Be able to describe and have hands-on experience DevOps methods and repeated practices (e.g., use of Agile methods, configuration management, build automation, test automation and deployment automation orchestrated under a CICD orchestrator), and why it matters;
2. Address challenges transitioning to DevOps methods and repeated practices;
3. Have had hands-on experience with Infrastructure as Code( Vagrant and Ansible ) to provision and configure an entire DevOps Factory (i.e. a toolchain and development environment) on VirtualBox including Docker Registry, Taiga, GitLab, Drone CI, and SonarQube;
4. Have had hands-on experience authoring code to include authoring and running automated tests in a CICD pipeline all under Configuration Management to ensure an application follows style, adheres to good coding practices, builds, identify security issues, and functions as expected;
5. Have had hands-on experience with
1. using Infrastructure as Code (IaC) in Vagrant and Ansible;
2. creating and using Kanban board in Taiga;
3. code configuration in git and GitLab;
4. authoring code in Go;
5. using style checkers and linters;
6. authoring a Makefile;
7. various commands in Docker (e.g., building a container image, pushing a container into a registry, creating and running a container);
8. authoring a pipeline for Drone CI;
9. using Sonar Scanner CLI to perform static analysis;
10. authoring security test in InSpec;
11. author an automated functional test in Selenium;
12. authoring a dynamic security test in OWASP Zap; and
13. using container platform to author and scale services;
6. Have had hands-on experience authoring code to include authoring and running automated tests in a CICD pipeline all under Configuration Management to ensure an application follows style, adheres to good coding practices, builds, identify security issues, and functions as expected.
We will be spending most of the course hands-on working with the tools and in the Unix command line making methods and repeated practices of DevOps happen, so as to grow an understanding of how DevOps actually works. Although, not necessary I would encourage you to pick up a free PDF of [The Linux Command Line by William Shotts](http://linuxcommand.org/tlcl.php) if you are no familiar wit the Linux command line.
Don't fixate on the tools used, nor the apps we develop in the course of learning how and why. How and why is far more important. This course like DevOps is not about tools although we'll be using them. You'll spend far more time writing code. (Or at the very least cutting-and-pasting code.)
# 3. Author
- Michael Joseph Walsh [mjwalsh@mitre.org](mailto:mjwalsh@mitre.org), [walsh@nemonik.com](mailto:walsh@nemonik.com)
# 4. Copyright and license
See the [License file](LICENCE) at the root of this project.
# 5. Prerequisites
The following skills would be useful in following along but aren't strictly necessary.
What you should bring:
- Managing Linux or Unix-like systems would be tremendously helpful, but not necessary, as we will be living largely within the terminal.
- A basic understanding of Vagrant, Docker, and Ansible would also be helpful, but not necessary.
# 6. Table of Contents
- [1. Preface](#1-preface)
- [2. DevOps](#2-devops)
- [3. Author](#3-author)
- [4. Copyright and license](#4-copyright-and-license)
- [5. Prerequisites](#5-prerequisites)
- [6. Table of Contents](#6-table-of-contents)
- [7. DevOps unpacked](#7-devops-unpacked)
- [7.1. What is DevOps?](#71-what-is-devops)
- [7.2. What DevOps is not](#72-what-devops-is-not)
- [7.3. To succeed at DevOps you must](#73-to-succeed-at-devops-you-must)
- [7.4. If your effort doesn't](#74-if-your-effort-doesnt)
- [7.5. Conway's Law states](#75-conways-law-states)
- [7.6. DevOps is really about](#76-devops-is-really-about)
- [7.7. What is DevOps culture?](#77-what-is-devops-culture)
- [7.7.1. We were taught the requisite skills as children](#771-we-were-taught-the-requisite-skills-as-children)
- [7.7.2. Maintaining relationships is your most important skill](#772-maintaining-relationships-is-your-most-important-skill)
- [7.7.3. Be quick... Be slow to...](#773-be-quick-be-slow-to)
- [7.7.4. The pressures of social media](#774-the-pressures-of-social-media)
- [7.8. How is DevOps related to the Agile?](#78-how-is-devops-related-to-the-agile)
- [7.9. How do they differ?](#79-how-do-they-differ)
- [7.10. Why?](#710-why)
- [7.11. What are the principles of DevOps?](#711-what-are-the-principles-of-devops)
- [7.12. Much of this is achieved](#712-much-of-this-is-achieved)
- [7.13. What is Continuous Integration (CI)?](#713-what-is-continuous-integration-ci)
- [7.14. How?](#714-how)
- [7.15. CI best practices](#715-ci-best-practices)
- [7.15.1. Utilize a Configuration Management System](#7151-utilize-a-configuration-management-system)
- [7.15.2. Automate the build](#7152-automate-the-build)
- [7.15.3. Employ one or more CI services/orchestrators](#7153-employ-one-or-more-ci-servicesorchestrators)
- [7.15.4. Make builds self-testing](#7154-make-builds-self-testing)
- [7.15.5. Never commit broken](#7155-never-commit-broken)
- [7.15.6. Stakeholders are expected to pre-flight new code](#7156-stakeholders-are-expected-to-pre-flight-new-code)
- [7.15.7. The CI service/orchestrator provides feedback](#7157-the-ci-serviceorchestrator-provides-feedback)
- [7.16. What is Continuous Delivery?](#716-what-is-continuous-delivery)
- [7.16.1. Extending Continuous Integration (CI)](#7161-extending-continuous-integration-ci)
- [7.16.2. Consistency](#7162-consistency)
- [7.17. But wait. What's a pipeline?](#717-but-wait-whats-a-pipeline)
- [7.18. How is a pipeline manifested?](#718-how-is-a-pipeline-manifested)
- [7.19. What underlines all of this?](#719-what-underlines-all-of-this)
- [7.20. But really why do we automate err. code?](#720-but-really-why-do-we-automate-err-code)
- [7.20.1. Why do I mention Larry Wall?](#7201-why-do-i-mention-larry-wall)
- [7.20.2. Laziness](#7202-laziness)
- [7.20.3. Impatience](#7203-impatience)
- [7.20.4. Hubris](#7204-hubris)
- [7.20.5. We automate for](#7205-we-automate-for)
- [7.21. Monitoring](#721-monitoring)
- [7.21.1. The _primary_ metric](#7211-the-_primary_-metric)
- [7.21.2. An understanding of performance](#7212-an-understanding-of-performance)
- [7.21.3. Establish a baseline performance](#7213-establish-a-baseline-performance)
- [7.21.4. Set reaction thresholds](#7214-set-reaction-thresholds)
- [7.21.5. Reacting](#7215-reacting)
- [7.21.6. Gaps in CICD](#7216-gaps-in-cicd)
- [7.21.7. Eliminating waste](#7217-eliminating-waste)
- [7.22. Crawl, walk, run](#722-crawl-walk-run)
- [7.22.1. Ultimately, DevOps is Goal](#7221-ultimately-devops-is-goal)
- [8. Reading list](#8-reading-list)
- [9. Now the hands-on part](#9-now-the-hands-on-part)
- [9.1. Configuring environmental variables](#91-configuring-environmental-variables)
- [9.2. VirtualBox](#92-virtualbox)
- [9.2.1. Installing VirtualBox](#921-installing-virtualbox)
- [9.3. Git Bash](#93-git-bash)
- [9.3.1. Installing Git Bash](#931-installing-git-bash)
- [9.4. Retrieve the course material](#94-retrieve-the-course-material)
- [9.5. Infrastructure as code (IaC)](#95-infrastructure-as-code-iac)
- [9.5.1. Hashicorp Packer](#951-hashicorp-packer)
- [9.5.1.1. Packer document and source](#9511-packer-document-and-source)
- [9.5.1.2. Installing Packer](#9512-installing-packer)
- [9.5.1.3. Packer project explained](#9513-packer-project-explained)
- [9.5.1.4. Packer execution](#9514-packer-execution)
- [9.5.2. Vagrant](#952-vagrant)
- [9.5.2.1. Vagrant documentation and source](#9521-vagrant-documentation-and-source)
- [9.5.2.2. Installing Vagrant](#9522-installing-vagrant)
- [9.5.2.3. The Vagrantfile explained](#9523-the-vagrantfile-explained)
- [9.5.2.3.1. Modelines](#95231-modelines)
- [9.5.2.3.2. Setting extra variables for Ansible roles](#95232-setting-extra-variables-for-ansible-roles)
- [9.5.2.3.3. Automatically installing and removing the necessary Vagrant plugins](#95233-automatically-installing-and-removing-the-necessary-vagrant-plugins)
- [9.5.2.3.4. Inserting Proxy setting via host environmental variables](#95234-inserting-proxy-setting-via-host-environmental-variables)
- [9.5.2.3.5. Inserting enterprise CA certificates](#95235-inserting-enterprise-ca-certificates)
- [9.5.2.3.6. Auto-generate the Ansible inventory file](#95236-auto-generate-the-ansible-inventory-file)
- [9.5.2.3.7. Mounting the project folder into each vagrant](#95237-mounting-the-project-folder-into-each-vagrant)
- [9.5.2.3.8. Build a Vagrant Box](#95238-build-a-vagrant-box)
- [9.5.2.3.9. Configuring the Kubernetes cluster vagrant(s)](#95239-configuring-the-kubernetes-cluster-vagrants)
- [9.5.2.3.10. Provisioning and configuring the *development* vagrant](#952310-provisioning-and-configuring-the-development-vagrant)
- [9.5.3. Ansible](#953-ansible)
- [9.5.3.1. Playbooks](#9531-playbooks)
- [9.5.3.1.1. The `master` vagrant's playbook](#95311-the-master-vagrants-playbook)
- [9.5.3.1.2. The `worker` vagrant's playbook](#95312-the-worker-vagrants-playbook)
- [9.5.3.1.3. The `development` playbooks](#95313-the-development-playbooks)
- [9.5.3.2. Roles](#9532-roles)
- [9.5.3.3. Bringing all the vagrants up](#9533-bringing-all-the-vagrants-up)
- [9.6. The cloud-native technologies underlying the tools](#96-the-cloud-native-technologies-underlying-the-tools)
- [9.6.1. Docker image and containers](#961-docker-image-and-containers)
- [9.6.2. Docker-compose](#962-docker-compose)
- [9.6.3. Kubernetes](#963-kubernetes)
- [9.6.3.1. K3s, light-weight Kubernetes](#9631-k3s-light-weight-kubernetes)
- [9.6.3.2. Kubectl, the Kubernetes command-line tool](#9632-kubectl-the-kubernetes-command-line-tool)
- [9.6.3.3. Kubernetes-Dashboard](#9633-kubernetes-dashboard)
- [9.7. The long-running tools](#97-the-long-running-tools)
- [9.7.1. Taiga, an example of Agile project management software](#971-taiga-an-example-of-agile-project-management-software)
- [9.7.1.1. Documentation, source, container image](#9711-documentation-source-container-image)
- [9.7.1.2. URL, Username and password](#9712-url-username-and-password)
- [9.7.2. GitLab CE, an example of configuration management software](#972-gitlab-ce-an-example-of-configuration-management-software)
- [9.7.2.1. Documentation, source, container image](#9721-documentation-source-container-image)
- [9.7.2.2. URL, Username and password](#9722-url-username-and-password)
- [9.7.2.3. Notes in regards to the class GitLab](#9723-notes-in-regards-to-the-class-gitlab)
- [9.7.3. Drone CI, an example of CICD orchestrator](#973-drone-ci-an-example-of-cicd-orchestrator)
- [9.7.3.1. Documentation, source, container image](#9731-documentation-source-container-image)
- [9.7.3.2. URL, Username and password](#9732-url-username-and-password)
- [9.7.4. SonarQube, an example of a platform for the inspection of code quality](#974-sonarqube-an-example-of-a-platform-for-the-inspection-of-code-quality)
- [9.7.4.1. Documentation, source, container image](#9741-documentation-source-container-image)
- [9.7.4.2. URL, Username and password](#9742-url-username-and-password)
- [9.7.5. PlantUML Server, an example of light-weight documentation](#975-plantuml-server-an-example-of-light-weight-documentation)
- [9.7.5.1. Documentation, source, container image](#9751-documentation-source-container-image)
- [9.7.5.2. URL](#9752-url)
- [9.8. Golang _helloworld_ project](#98-golang-_helloworld_-project)
- [9.8.1. Create the project's backlog](#981-create-the-projects-backlog)
- [9.8.2. Create the project in GitLab](#982-create-the-project-in-gitlab)
- [9.8.3. Setup the project on the _development_ Vagrant](#983-setup-the-project-on-the-_development_-vagrant)
- [9.8.4. Author the application](#984-author-the-application)
- [9.8.5. Align source code with Go coding standards](#985-align-source-code-with-go-coding-standards)
- [9.8.6. Lint your code](#986-lint-your-code)
- [9.8.7. Build the application](#987-build-the-application)
- [9.8.8. Run your application](#988-run-your-application)
- [9.8.9. Author the unit tests](#989-author-the-unit-tests)
- [9.8.10. Automate the build (i.e., write the Makefile)](#9810-automate-the-build-ie-write-the-makefile)
- [9.8.11. Author Drone-based Continuous Integration](#9811-author-drone-based-continuous-integration)
- [9.8.11.1. Configure Drone to execute your pipeline](#98111-configure-drone-to-execute-your-pipeline)
- [9.8.11.2. Trigger the build](#98112-trigger-the-build)
- [9.8.12. The completed source for *helloworld*](#9812-the-completed-source-for-helloworld)
- [9.9. Golang *helloworld-web* project](#99-golang-helloworld-web-project)
- [9.9.1. Create the project's backlog](#991-create-the-projects-backlog)
- [9.9.2. Create the project in GitLab](#992-create-the-project-in-gitlab)
- [9.9.3. Setup the project on the _development_ Vagrant](#993-setup-the-project-on-the-_development_-vagrant)
- [9.9.4. Author the application](#994-author-the-application)
- [9.9.5. Build and run the application](#995-build-and-run-the-application)
- [9.9.6. Run gometalinter.v2 on application](#996-run-gometalinterv2-on-application)
- [9.9.7. Fix the application](#997-fix-the-application)
- [9.9.8. Author unit tests](#998-author-unit-tests)
- [9.9.9. Perform static analysis (i.e., sonar-scanner) on the command line](#999-perform-static-analysis-ie-sonar-scanner-on-the-command-line)
- [9.9.9.1. Optionally, register your app in SonarQube](#9991-optionally-register-your-app-in-sonarqube)
- [9.9.9.2. Install the SonarGo plugin](#9992-install-the-sonargo-plugin)
- [9.9.9.3. Perform static analysis (run *sonar-scanner*) on the command line](#9993-perform-static-analysis-run-sonar-scanner-on-the-command-line)
- [9.9.10. Automate the build (i.e., write the Makefile)](#9910-automate-the-build-ie-write-the-makefile)
- [9.9.11. Dockerize the application](#9911-dockerize-the-application)
- [9.9.12. Run the Docker container](#9912-run-the-docker-container)
- [9.9.12.1. Option 1](#99121-option-1)
- [9.9.12.2. Option 2](#99122-option-2)
- [9.9.13. Push the container image to the private Docker registry](#9913-push-the-container-image-to-the-private-docker-registry)
- [9.9.14. Configure Drone to execute your CICD pipeline](#9914-configure-drone-to-execute-your-cicd-pipeline)
- [9.9.15. Add Static Analysis (*SonarQube*) step to pipeline](#9915-add-static-analysis-sonarqube-step-to-pipeline)
- [9.9.16. Add the build step to the pipeline](#9916-add-the-build-step-to-the-pipeline)
- [9.9.17. Add container image publish step to pipeline](#9917-add-container-image-publish-step-to-pipeline)
- [9.9.18. Add container deploy step to pipeline](#9918-add-container-deploy-step-to-pipeline)
- [9.9.19. Add compliance and policy automation (InSpec) test to the pipeline](#9919-add-compliance-and-policy-automation-inspec-test-to-the-pipeline)
- [9.9.19.1. First author an InSpec test](#99191-first-author-an-inspec-test)
- [9.9.19.2. Execute your test](#99192-execute-your-test)
- [9.9.19.3. Add InSpec to the pipeline](#99193-add-inspec-to-the-pipeline)
- [9.9.19.4. Viewing the results in Heimdall-lite](#99194-viewing-the-results-in-heimdall-lite)
- [9.9.20. Add automated functional test to pipeline](#9920-add-automated-functional-test-to-pipeline)
- [9.9.20.1. Run the *helloworld-web* application](#99201-run-the-helloworld-web-application)
- [9.9.20.2. Pull and run Selenium Firefox Standalone](#99202-pull-and-run-selenium-firefox-standalone)
- [9.9.20.3. Create our test automation](#99203-create-our-test-automation)
- [9.9.20.4. Enable `Trusted` for the repository in Drone](#99204-enable-trusted-for-the-repository-in-drone)
- [9.9.20.5. Add a *selenium* step to the pipeline](#99205-add-a-selenium-step-to-the-pipeline)
- [9.9.21. Add DAST step (*OWASP ZAP*) to pipeline](#9921-add-dast-step-owasp-zap-to-pipeline)
- [9.9.22. All the source for *helloworld-web*](#9922-all-the-source-for-helloworld-web)
- [9.10. Additional best practices to consider around securing containerized applications](#910-additional-best-practices-to-consider-around-securing-containerized-applications)
- [9.11. Microservices](#911-microservices)
- [9.11.1. What's cloud-native?](#9111-whats-cloud-native)
- [9.11.2. Let's create a microservice](#9112-lets-create-a-microservice)
- [9.11.2.1. Modify the helloworld-web application](#91121-modify-the-helloworld-web-application)
- [9.11.2.2. Create a Kubernetes manifest for the microservice](#91122-create-a-kubernetes-manifest-for-the-microservice)
- [9.11.2.3. Deploy your application](#91123-deploy-your-application)
- [9.11.2.4. Test your microservice](#91124-test-your-microservice)
- [9.11.2.5. Scale your microservice](#91125-scale-your-microservice)
- [9.12. Using what you've learned](#912-using-what-youve-learned)
- [9.13. Shoo away your vagrants](#913-shoo-away-your-vagrants)
- [9.14. That's it](#914-thats-it)
# 7. DevOps unpacked
## 7.1. What is DevOps?
DevOps (a clipped compound of the words *development* and *operations*) is a software development methodology with an emphasis on a reliable release pipeline, automation, and stronger collaboration across all stakeholders with the goal of delivery of value in close alignment with business objectives into the hands of users (i.e., production) more efficiently and effectively.
*Ops* in DevOps gathers up every IT operation stakeholders (i.e., cybersecurity, testing, DB admin, infrastructure and operations practitioners -- essentially, any stakeholder not commonly thought of as directly part of the development team in the system development life cycle).
Yeah, that's the formal definition.
In the opening sentences of _Security Engineering: : A Guide to Building Dependable Distributed Systems — Third Edition_, author Ross Anderson defines what a security engineer is
> Security engineering is about building systems to remain dependable in the face of malice, error, or mischance. As a discipline, it focuses on the tools, processes, and methods needed to design, implement, and test complete systems, and to adapt existing systems as their environment evolve.
The words _security engineering_ could be replaced in the opening sentence with each one of the various stakeholders (e.g., development, quality assurance, technology operations).
The point I'm after is everyone is in it to collectively deliver dependable software.
Also, there is no need to overload the _DevOps_ term -- To _Dev wildcard (i.e., *) Ops_ to include your pet interest(s), such as, _security_, _test_, _whatever_... to form _DevSecOps_, _DevTestOps_, _DevWhateverOps_... _DevOps_ has you covered.
## 7.2. What DevOps is not
About the tools.
There are countless vendors out there, who want to sell you their crummy tool.
## 7.3. To succeed at DevOps you must
__Combine software development and information technology operations in the systems development life cycle__ with __a focus on collaboration across the life cycle to deliver features, fixes, and updates frequently in close alignment with business objectives__.
If the effort cannot combine both Dev and Ops in collaboration with this focus the effort will most certainly fail.
## 7.4. If your effort doesn't
__grok (i.e, Understand intuitively) what DevOps is in practice__ and have performed the __necessary analysis of the existing culture and a strategy for how to affect a change__ the effort again will likely fail.
I say this because the culture is the largest influencer over the success of both Agile and DevOps and ultimately the path taken (i.e., plans made.)
## 7.5. Conway's Law states
> Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization's communication structure.
From ["How Do Committees Invent?"](http://www.melconway.com/Home/Conways_Law.html)
Followed with
> Ways must be found to reward design managers for keeping their organizations lean and flexible.
This was written over 50 years ago.
If your communication structure is broke, so shall your systems be.
## 7.6. DevOps is really about
Providing the culture, methods and repeated practices to permit stakeholders to collaborate.
## 7.7. What is DevOps culture?
> **culture** noun \ *ˈkəl-chər* \
> the set of shared attitudes, values, goals, and practices that characterizes an institution or organization
I love when a word means precisely what you need it to mean.
With the stakeholders sharing the same attitudes, values, goals, using the same tools, methods and repeated practices for their particular discipline you have ***DevOps Culture***.
### 7.7.1. We were taught the requisite skills as children
### 7.7.2. Maintaining relationships is your most important skill
### 7.7.3. Be quick... Be slow to...
### 7.7.4. The pressures of social media
## 7.8. How is DevOps related to the Agile?
Agile Software Development is an umbrella term for a set of methods and practices based on the [values](http://www.agilealliance.org/agile101/the-agile-manifesto/) and [principles](http://www.agilealliance.org/agile101/12-principles-behind-the-agile-manifesto/) expressed in the Agile Manifesto.
For Agile, solutions evolve through collaboration between self-organizing, cross-functional teams utilizing the appropriate practices for their context.
DevOps builds on this.
## 7.9. How do they differ?
While DevOps extends Agile methods and practices by adding communication and collaboration between
- development,
- security,
- quality assurance, and
- technology operations
functionaries as stakeholders into the broader effort to ensure software systems are delivered in a reliable, low-risk manner.
## 7.10. Why?
In Agile Software Development, there is rarely an integration of these individuals outside the immediate application development team with members of technology operations (e.g., network engineers, administrators, testers, security engineers.)
## 7.11. What are the principles of DevOps?
As DevOps matures, several principles have emerged, namely the necessity for product teams to:
- Apply holistic thinking to solve problems,
- Develop and test against production-like environments,
- Deploy with repeatable and reliable processes,
- Remove the drudgery and uncertainty through automation,
- Validate and monitor operational quality, and
- Provide rapid, automated feedback to the stakeholders
## 7.12. Much of this is achieved
Through the repeated practices of Continuous Integration (CI) and Continuous Delivery (CD) often conflated into simply "CI/CD" or "CICD".
WARNING: After tools, CICD is the next (**albeit mistakenly**) thing thought to be the totality of DevOps.
## 7.13. What is Continuous Integration (CI)?
It is a repeated Agile software development practice lifted specifically from Extreme programming, where members of a development team frequently integrate their work to detect integration issues as quickly as possible thereby shifting discovery of issues "left" (i.e., early) in the software release.
## 7.14. How?
Each integration is orchestrated through a CI service/orchestrator (e.g., Jenkins CI, Drone CI, GitLab Runners, Concourse CI) that essentially assembles a build, runs unit and integration tests every time a predetermined trigger has been met; and then reports with immediate feedback.
## 7.15. CI best practices
### 7.15.1. Utilize a Configuration Management System
For the software's source code, where the mainline (i.e., master branch) is the most recent working version, past releases held in branches, and new features not yet merged into the mainline branch worked in their own branches.
### 7.15.2. Automate the build
By accompanying build automation (e.g., Gradle, Apache Maven, Make) alongside the source code.
### 7.15.3. Employ one or more CI services/orchestrators
To perform source code analysis via automating formal code inspection and assessment.
### 7.15.4. Make builds self-testing
In other words, ingrain testing by including unit and integration tests (e.g., Spock, JUnit, Mockito, SOAPUI, go package *Testing*) with the source code to be executed by the build automation to be executed by the CI service.
### 7.15.5. Never commit broken
Or untested source code to the CMS mainline or otherwise risk breaking a build.
### 7.15.6. Stakeholders are expected to pre-flight new code
Prior to committing source code in their own workspace.
### 7.15.7. The CI service/orchestrator provides feedback
On the success or fail of a build integration to all its stakeholders.
## 7.16. What is Continuous Delivery?
It is a repeated software development practice of providing a rapid, reliable, low-risk product delivery achieved through automating all facets of building, testing, and deploying software.
### 7.16.1. Extending Continuous Integration (CI)
With additional stages/steps aimed to provide ongoing validation that a newly assembled software build meets all desired requirements and thereby is releasable.
### 7.16.2. Consistency
Is achieved through delivering applications into production via individual repeatable pipelines of ingrained system configuration management and testing
## 7.17. But wait. What's a pipeline?
A pipeline automates the various stages/steps (e.g., Static Application Security Testing (SAST), build, unit testing, Dynamic Application Security Testing (DAST), secure configuration acceptance compliance, integration, function and non-functional testing, delivery, and deployment) to enforce quality conformance.
## 7.18. How is a pipeline manifested?
Each delivery pipeline is manifested as **Pipeline as Code** (i.e., software automation) accompanying the application's source code in its version control repository.
## 7.19. What underlines all of this?
I and the community of practice argue DevOps will struggle without ubiquitous access to shared pools of software configurable system resources and higher-level services that can be rapidly provisioned (i.e., cloud).
Although, it is actually possible to [DevOps on mainframes](https://www.youtube.com/watch?v=eMS97X5ZTGc) The video is in the contect of continuous delivery, but read between the lines.
## 7.20. But really why do we automate err. code?
In 2001, I think Larry Wall in his 1st edition of *Programming Perl* book put it best with "We will encourage you to develop the three great virtues of a programmer:
laziness,
impatience, and
hubris."
The second edition of the same book provided definitions for these terms
### 7.20.1. Why do I mention Larry Wall?
Well...
> _Once you have established yourself as an icon in your field it is important that you pay tribute to some of the great legends that came before you. This kind of gesture will create the illusion that you’re still humble and serve as a preemptive strike against anyone who has noticed what a callus and delusional ass you have become._
>
> From the opening monologue to the Blue Man Group’s _I Feel Love_ https://www.youtube.com/watch?v=8vBKI3ya-l0
I kid, but in all serious the sentiment of this seminal, nearly twenty-something year-old book still holds true.
Let me explain.
### 7.20.2. Laziness
> The quality that makes you go to great effort to reduce overall energy expenditure. It makes you write labor-saving programs that other people will find useful, and document what you wrote so you don't have to answer so many questions about it. Hence, the first great virtue of a programmer._ (p.609)
### 7.20.3. Impatience
> The anger you feel when the computer is being lazy. This makes you write programs that don't just react to your needs, but actually anticipate them. Or at least pretend to. Hence, the second great virtue of a programmer._ (p.608)
### 7.20.4. Hubris
> Excessive pride, the sort of thing Zeus zaps you for. Also, the quality that makes you write (and maintain) programs that other people won't want to say bad things about. Hence, the third great virtue of a programmer._ (p.607)
### 7.20.5. We automate for
- Faster, coordinated, repeatable, and therefore more reliable deployments.
- Discover bugs sooner. Shifting their discovery left in the process.
- To accelerates the feedback loop between Dev and Ops (Again, Ops is everyone not typically considered part of the development team.)
- Reduce tribal knowledge, where one group or person holds the keys to how things get done. Yep, this is about making us all replaceable.
- Reduce shadow IT (i.e., hardware or software within an enterprise that is not supported by IT. Just waiting for its day to explode.)
## 7.21. Monitoring
Once deployed, the work is done, right?
So, that improvements can be gauged and anomalies detected.A development team's work is not complete once a product leaves CICD and enters production; especially, under DevOps where the development team includes members of ops (e.g., security and technology operations).
### 7.21.1. The _primary_ metric
Is working software, but this is not the only, measurement. The key to successful DevOps is knowing how well the methodology and the software it produces are performing. Is the software truely dependable?
### 7.21.2. An understanding of performance
Is achieved by collecting and analyzing data produced by environments used for CICD and production.
### 7.21.3. Establish a baseline performance
So, that improvements can be gauged and anomalies detected.
### 7.21.4. Set reaction thresholds
To formulate and prioritize reactions weighting factors, such as, the frequency at which an anomaly arises and who is impacted.
### 7.21.5. Reacting
Could be as simple as operations instructing users through training to not do something that triggers the anomaly, or more ideally, result in an issue being entered into the product's backlog culminating in the development team delivering a fix into production.
### 7.21.6. Gaps in CICD
Are surfaces through monitoring resulting in for example additional testing for an issue discovered in prodcuction.
Yep. News flash. DevOps will not entirely stop all bugs or vulnerabilities from making it into production, but this was never the point.
### 7.21.7. Eliminating waste
Through re-scoping of requirements, re-prioritizing of a backlog, or the deprecation of unused features. Again, all surfaced through monitoring.
## 7.22. Crawl, walk, run
### 7.22.1. Ultimately, DevOps is Goal
- With DevOps one does not simply hit the ground running.
- One must first crawl, walk and then ultimately run as you embrace the necessary culture change, methods, and repeated practices.
- Collaboration and automation are expected to continually improve so to achieve more frequent and more reliable releases.
# 8. Reading list
**AntiPatterns: Refactoring Software, Architectures, and Projects in Crisis**
William J. Brown, Raphael C. Malveau, Hays W. "Skip" McCormick, and Thomas J. Mowbray
ISBN: 978-0-471-19713-3
Apr 1998
**Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation (Addison-Wesley Signature Series (Fowler))**
David Farley and Jez Humble
ISBN-13: 978-0321601919
August 2010
**The DevOps Handbook: How to Create World-Class Agility, Reliability, and Security in Technology Organizations**
Gene Kim Jez Humble, Patrick Debois, and John Willis
ISBN-13: 978-1942788003
October 2016
**Accelerate: The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations**
Nicole Forsgren PhD, Jez Humble, and Gene Kim
ISBN-13: 978-1942788331
March 27, 2018
**Site Reliability Engineering: How Google Runs Production Systems 1st Edition**
Betsy Beyer, Chris Jones, Jennifer Petoff, and Niall Richard Murphy
ISBN-13: 978-1491929124
April 16, 2016
Also, available online at https://landing.google.com/sre/book/index.html
**Release It!: Design and Deploy Production-Ready Software 2nd Edition**
Michael T. Nygard
ISBN-13: 978-1680502398
January 18, 2018
**The SPEED of TRUST: The One Thing That Changes Everything**
Stephen M .R. Covey
ISBN-13: 978-1416549000
February 5, 2008
The gist of the book can be found at SlideShare https://www.slideshare.net/nileshchamoli/the-speed-of-trust-13205957
**RELATIONSHIP TRUST: The 13 Behaviors of High-Trust Leaders Mini Session**
Franklin Covey Co.
https://archive.franklincovey.com/facilitator/minisessions/handouts/13_Behaviors_MiniSession_Handout.pdf
**How to Deal With Difficult People**
Ujjwal Sinha
Oct 25, 2014
The SlideShare can be found here https://www.slideshare.net/abhiujjwal/how-2-deal-wid-diiclt-ppl
**Leadership Secrets of the Rouge Warrior: A Commando's Guide to Success**
Richard Marcinko w/ John Weisman
ISBN-13: 978-0671545154
June 1, 1996
**Security Engineering: A Guide To Building Dependable Distributed Systems**
Ross Anderson
ISBN-13: 978-0470068526
April 14, 2008
The second edition of this book can be downloaded in whole from https://www.cl.cam.ac.uk/~rja14/book.html and Mr Anderson has released chapters from his 3rd edition under development.
**How Do Committees Invent?**
Melvin E. Conway
Copyright 1968, F. D. Thompson Publications, Inc.
http://www.melconway.com/Home/Conways_Law.html
**The Pragmatic Programmer: Your Journey To Mastery, 20th Anniversary Edition (2nd Edition)**
David Thomas and Andrew Hunt
ISBN-13: 978-0135957059
September 23, 2019
# 9. Now the hands-on part
In this class, you will spin up a development and toolchain environment.
**NOTE**
- This class makes use of **NOTE** sections to call out things that are important to know or to drop a few tidbits. Reading these notes may save you some aggravation.
- This diagrams used in this document are authored in PlantUML. Each diagram is authored in a PlantUML domain-specific language (DSL) and can be found in a separate file in the [plantuml folder](plantuml) in the root of the project. The class project makes use of [GitHub worflow](.github/worflows/main.yml) to render each file into a scalable vector graphic (SVG) placing each in the [diagrams folder](diagrams) also at the root of the class project.
[PlantUML source for this diagram](plantuml/deployment.puml)
## 9.1. Configuring environmental variables
If your environment makes use of an HTTP proxy or SSL inspection, you will need to configure environment variables for this class.
**On Mac OS X or \*NIX environments**
The following `set_env.sh` BASH script is included in the root of the project and can be used to configure the UNIX environment variables, but must be adjusted for your specific environment.
```bash
#!/usr/bin/env bash
# Copyright (C) 2019 Michael Joseph Walsh - All Rights Reserved
# You may use, distribute and modify this code under the
# terms of the the license.
#
# You should have received a copy of the license with
# this file. If not, please email
# run in shell via
#
# ```
# . ./set_env.sh
# ```
#
# will set proxy setting to the the hard-cded value on line 36.
# Modify for your environment.
#
# ```
# . ./set_env.sh no_proxy
# ```
#
# will unset all proxy related environmental variables.
set_proxy=true
if [ $# -ne 0 ]; then
args=("$@")
if [[ $args[1] = "no_proxy" ]]; then
set_proxy=false
fi
fi
if [[ $set_proxy = true ]]; then
export PROXY=http://gatekeeper.mitre.org:80
echo "Setting proxy environment varaibles to $PROXY"
export proxy=$PROXY
export HTTP_PROXY=$PROXY
export http_proxy=$PROXY
export HTTPS_PROXT=$PROXY
export https_proxy=$PROXY
export ALL_PROXY=$PROXY
export NO_PROXY="127.0.0.1,localhost,.mitre.org,.local,192.168.0.9,192.168.0.10,192.168.0.11,192.168.0.206,192.168.0.10,192.168.0.202,192.168.0.203,192.168.0.204,192.168.0.205"
export no_proxy=$NO_PROXY
else
echo "Unsetting proxy environment varaibles"
unset PROXY
unset proxy
unset HTTP_PROXY
unset http_proxy
unset HTTPS_PROXY
unset https_proxy
unset NO_PROXY
unset no_proxy
unset ALL_PROXY
fi
export CA_CERTIFICATES=http://employeeshare.mitre.org/m/mjwalsh/transfer/MITRE%20BA%20ROOT.crt,http://employeeshare.mitre.org/m/mjwalsh/transfer/MITRE%20BA%20NPE%20CA-3%281%29.crt
echo "Setting CA_CERTIFICATES environment variable to $CA_CERTIFICATES"
export VAGRANT_ALLOW_PLUGIN_SOURCE_ERRORS=0
echo "Setting VAGRANT_ALLOW_PLUGIN_SOURCE_ERRORS to $VAGRANT_ALLOW_PLUGIN_SOURCE_ERRORS"
# Force the use of the vagrant cacert.pem file
echo "unsetting CURL_CA_BUNDLE and SSL_CERT_FILE environment variables"
unset CURL_CA_BUNDLE
unset SSL_CERT_FILE
```
When in the root of the project, the script can be executed in the terminal session via
```bash
. ./set_env.sh
```
If you're doing this class on your MITRE Life cycle running OS X while on the MITRE network you will **NOT** want to set any proxy related environmental variable. So, you will want to execute this script in this manner
```bash
. ./set_env.sh no_proxy
```
If you have no HTTP proxy and no SSL inspection to be concerned about (such as running the class off of MITRE's corporate network), the alternative is to execute `unset.sh` BASH script to unset all these values:
```bash
#!/usr/bin/env bash
# Copyright (C) 2019 Michael Joseph Walsh - All Rights Reserved
# You may use, distribute and modify this code under the
# terms of the the license.
#
# You should have received a copy of the license with
# this file. If not, please email
# run in shell via
#
# ```
# . ./unset.sh
# ```
unset no_proxy
unset NO_PROXY
unset ALL_PROXY
unset PROXY
unset proxy
unset https_proxy
unset http_proxy
unset HTTP_PROXY
unset HTTPS_PROXY
unset ftp_proxy
unset FTP_PROXY
unset ca_certificates
unset CA_CERTIFICATES
```
Execute in terminal session via
```bash
. ./unset.sh
```
**On Windows**
If you are on Windows perform the following to set environmental variable adjusting for your environment:
1. In the Windows taskbar, enter `edit the system environment variables` into `Search Windows` and select the icon with the corresponding name.
3. The `Systems Property` window will likely open in the background, so you will likely need to go find it and bring it forward.
4. In the `Systems Property`'s `Advanced` tab select `Environment Variables...` button.
5. In `Environment Variables` windows that opens, under `User variables for...` press `New ...` to open a `New User Variable` window, enter each `Variable Name` and and its respective `Value` for each pair in the table below
| Variable Name | Value |
|------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------|
| proxy | http://gatekeeper.mitre.org:80 |
| http_proxy | http://gatekeeper.mitre.org:80 |
| https_proxy | http://gatekeeper.mitre.org:80 |
| no_proxy | 127.0.0.1,localhost,.mitre.org,.local,192.168.0.9,192.168.0.10,192.168.0.11,192.168.0.206,192.168.0.10,192.168.0.202,192.168.0.203,192.168.0.204,192.168.0.205 |
| CA_CERTIFICATES | http://pki.mitre.org/MITRE%20BA%20Root.crt,http://pki.mitre.org/MITRE%20BA%20NPE%20CA-3%281%29.crt |
| VAGRANT_ALLOW_PLUGIN_SOURCE_ERRORS | 0 |
If you're on MITRE Institute Lab PC you will want to set all of these variables.
If you're doing this class on your MITRE Life cycle running Windows (I have yet to verify a Windows MITRE Life cycle.) you will likely **NOT** want to set the `proxy`, `http_proxy`, `https_proxy`, `no_proxy` or any proxy related environmental variables. You will only need to set the `CA_CERTIFICATES` and `VAGRANT_ALLOW_PLUGIN_SOURCE_ERRORS` environment variables.
**NOTE**
- The certificate URLs need to be encoded for parentheses to work.
- On Windows, you may inadvertently cut-and-paste blank space characters (e.g., tabs, spaces) and the subsequent Ansible automation may fail.
## 9.2. VirtualBox
You will need to install VirtualBox, a general-purpose full virtualizer for x86 hardware.
The class has been verified to work with VirtualBox 6.1.14. Newer version may or may not work.
### 9.2.1. Installing VirtualBox
For the MITRE Institute class when I teach it, it is assumed VirtualBox is installed, but below are the instructions for installing it on Windows 10.
1. Open your browser to
2. Click `Windows hosts` link under `VirtualBox 6.1.14 platform packages`.
3. Find and click the installer to install.
You will also need to turn off `Hyper-V`, `Virtual Machine Platform`, `Windows Sandbox` and `Windows Subsystem for Linux` if installed.
1. Click Windows `Start` and then type `turn Windows features on or off` into the search bar.
2. Select the icon with the corresponding name.
3. This will open the `Windows Features` page and then unselect the `Hyper-V`, `Virtual Machine Platform`, `Windows Hypervisor Platform`, `Windows Sandbox` and `Windows Subsystem for Linux` checkboxes if enabled and then click `Okay`.
The same site has the Mac OS X download. The installation is less involved.
If you're using Linux use your package manager. For example, to install on Arch Linux one would use `sudo pacman -Syu virtualbox`.
**NOTE**
- On Windows 10, if you are running into a VirtualBox that is partially working -- Generally, just flaking out. -- check to make sure you have turned off all the features listed above.
## 9.3. Git Bash
Git Bash is `git` packaged for Windows with bash (a command-line shell) and a collection of other, separate *NIX utilities, such as, `ssh`, `scp`, `cat`, `find` and others compiled for Windows.
### 9.3.1. Installing Git Bash
If you are on Windows, you'll need to install `git`.
1. Download from https://git-scm.com/download/win
2. Click the installer.
3. Click `next` until you reach the `Configuring the line ending conversions` page select `Checkout as, commit Unix-style line endings`.
4. Then `next`, `next`, `next`...
5. Don't open git-bash from the final window as it will not have the environmental variables set. Go onto step-6.
6. On the Windows task bar, enter `git` into `Search Windows` then select `Git Bash`. Use `Git Bash` instead of `Command` or `Powershell`.
On OS X, `git` can be installed via [Homebrew](https://brew.sh/) or you can install the Git client directly .
## 9.4. Retrieve the course material
If you are reading this on paper and have nothing else, you only have a small portion of the class material. You will need to download the class project containing all the automation to spin up a DevOps toolchain and development virtual machines, etc.
In a shell, for the purposes of the MITRE Institute class, this means in `Git Bash`, clone the project from [https://github.com/nemonik/hands-on-DevOps.git](https://github.com/nemonik/hands-on-DevOps.git) via git like so:
```
git -c http.sslVerify=false clone https://github.com/nemonik/hands-on-DevOps.git
```
Output will resemble (i.e., will not be precisely the same):
```
Cloning into 'hands-on-DevOps'...
remote: Enumerating objects: 52, done.
remote: Counting objects: 100% (52/52), done.
remote: Compressing objects: 100% (39/39), done.
remote: Total 5236 (delta 21), reused 28 (delta 11), pack-reused 5184
Receiving objects: 100% (5236/5236), 75.60 MiB | 13.40 MiB/s, done.
Resolving deltas: 100% (1578/1578), done.
```
## 9.5. Infrastructure as code (IaC)
This class uses Infrastructure as code (IaC) to set up the class environment (i.e., two or more virtual machines that will later be referred to as "vagrants".) IaC is the process of provisioning, and configuring (i.e., managing) computer systems through code, rather than directly manipulating the systems by hand (i.e., through manual processes).
This class uses Vagrant and Ansible IaC frameworks and the following sections will unpack each.
### 9.5.1. Hashicorp Packer
This class uses Packer, a command-line utility that can be used for many [use cases](https://www.terraform.io/intro/use-cases.html), but I primarily use to build EC2 and VirtualBox machine images. In the case of VirtualBox to create Vagrant box. In this class, if you select to use the `nemonik/alpine310` base box in the `ConfigurationVars` module defined in `configuration_vars.rb` file at the root of the project, then Terraform will be invoked to create the `nemonik/alpine310` box, and then this image will be used to create the `nemonik/devops_alpine310` box on which all the vagrants will be based. If you select `ubuntu/bionic64` or `centos/7` these boxes will be retrieved from Hashicorp and then used to build their respective boxes: `nemonik/devops_bionic64` and `nemonik/devops_7` on top of which the class' vagrants are built.
#### 9.5.1.1. Packer document and source
Packer's documetnation can be found at [https://packer.io/docs/](https://packer.io/docs/).
Its canonical (i.e., authoritative) source can be found at [https://github.com/hashicorp/terraform](https://github.com/hashicorp/terraform).
#### 9.5.1.2. Installing Packer
Retrieved the installationation executable for your machine's operating system from here
https://releases.hashicorp.com/packer/1.5.1/
If you're on OS X you want [https://releases.hashicorp.com/packer/1.5.1/packer_1.5.1_darwin_amd64.zip](https://releases.hashicorp.com/packer/1.5.1/packer_1.5.1_darwin_amd64.zip) and if you're on Windows you want [https://releases.hashicorp.com/packer/1.5.1/packer_1.5.1_windows_amd64.zip](https://releases.hashicorp.com/packer/1.5.1/packer_1.5.1_windows_amd64.zip).
You may also be able to install Packer via your Linux operating system's package manager.
**Installing on OS X**
To install on OS X simply unpack and copy to `/user/local/bin/packer` as `root` for example
```bash
cd ~/Downloads
unzip ~/Downloads/packer_1.5.1_darwin_amd64.zip
Archive: /Users/mjwalsh/Downloads/packer_1.5.1_darwin_amd64.zip
inflating: packer
sudo cp ~/Downloads/packer /usr/local/bin
packer version
Packer v1.5.1
```
**Installing on Windows**
To install on Windows:
1. Right click the downloaded zip file.
2. Extract all.
3. Enter `C:\Program Files\packer_1.5.1_windows_amd64` as the path.
4. Extract.
5. Continue (Give admin permission.)
6. In the Windows taskbar, enter `env` into `Search Windows` and select `edit the system environment variables`.
7. In the `Systems Property`'s `Advanced` tab select `Environment Variables...` button.
8. In `Environment Variables` windows that opens, in `User variables for...` select the Path variable then select`Edit ...` to open a `Edit environment variable` window.
9. Select `new` and enter `C:\Program Files\packer_1.5.1_windows_amd64`.
#### 9.5.1.3. Packer project explained
The terraform project to build `nemonik/alpine310` box is found in `box/packer-alpine310` looks like so:
[PlantUML source for this diagram](plantuml/packer-alpine310.puml)
The folders and files have the following purpose:
- `alpine310.json` - is the packer template used to orchestrate the creation of the `nemonik_alpine310.box`. It defines the number of CPUs, the memory, the ISO used to install the VM's OS, the root password, the automation to install the OS, the scripts to call to further configure, and the box to result
- `build_box.sh` - is a bash shell script used to run the Packer build
- `configs` - a folder, holds the Vagrantfile template used in post processing the VM
- `vagrantfile.tpl` - is the template used in post processing the VM
- `http` - a folder, holds the `answer` file `wget` into the VM
- `answer`- holds all the answers provide to `setup-alpine` to configure the VM
- `isos` - a folder,is a cache that holds the ISOs downloaded to install Alpine, so that they don't have to be repeatedly downloaded
- `nemonik_alpine310.box` - the vagrant box created by Packer.
- `packer_cache` - a folder, is a packer cache used to speed production of the box
- `remove_box.sh` - a bash script used to remove the `nemonik_alpine310.box` from Vagrant
- `configure` - a folder, holding the shells script used to configure the VM
- `configure.sh` - the script to configure the VM past what is directed in `builders`' `boot_command` in `alpine310.json`
The bulk of the work is in `alpine310.json`, `http/answer` and `scripts/configure.sh`.
The contents of the `alpine310.json`:
```
{
"description": "Build Alpine 3.10 x86_64 vagrant box",
"variables": {
"vm_name": "alpine-3.10.0-x86_64",
"cpus": "1",
"memory": "1024",
"disk_size": "61440",
"iso_local_url": "isos/x86_64/alpine-virt-3.10.0-x86_64.iso",
"iso_download_url": "http://dl-cdn.alpinelinux.org/alpine/v3.10/releases/x86_64/alpine-virt-3.10.0-x86_64.iso",
"iso_checksum": "b3d8fe65c2777edcbc30b52cde7f5ae21dff8ecda612d5fe7b10d5c23cda40c4",
"iso_checksum_type": "sha256",
"root_password": "vagrant",
"ssh_username": "root",
"ssh_password": "vagrant"
},
"provisioners": [
{
"type": "shell",
"execute_command": "/bin/sh -ux '{{.Path}}'",
"script": "scripts/configure.sh"
}
],
"builders": [
{
"type": "virtualbox-iso",
"headless": false,
"vm_name": "{{user `vm_name`}}",
"format": "ova",
"guest_os_type": "Linux26_64",
"guest_additions_mode": "disable",
"disk_size": "{{user `disk_size`}}",
"iso_urls": [
"{{user `iso_local_url`}}",
"{{user `iso_download_url`}}"
],
"iso_checksum": "{{user `iso_checksum`}}",
"iso_checksum_type": "{{user `iso_checksum_type`}}",
"http_directory": "http",
"communicator": "ssh",
"ssh_username": "{{user `ssh_username`}}",
"ssh_password": "{{user `ssh_password`}}",
"ssh_wait_timeout": "10m",
"shutdown_command": "/sbin/poweroff",
"boot_wait": "30s",
"boot_command": [
"root",
"ifconfig eth0 up && udhcpc -i eth0",
"wget http://{{ .HTTPIP }}:{{ .HTTPPort }}/answers",
"setup-alpine -f $PWD/answers",
"{{user `root_password`}}",
"{{user `root_password`}}",
"",
"",
"",
"",
"y",
"",
"",
"",
"",
"mount /dev/sda2 /mnt",
"echo 'PermitRootLogin yes' >> /mnt/etc/ssh/sshd_config",
"umount /dev/sda2",
"sync",
"reboot"
],
"hard_drive_interface": "sata",
"vboxmanage": [
[
"modifyvm",
"{{.Name}}",
"--memory",
"{{user `memory`}}"
],
[
"modifyvm",
"{{.Name}}",
"--cpus",
"{{user `cpus`}}"
],
[
"modifyvm",
"{{.Name}}",
"--rtcuseutc",
"on"
]
]
}
],
"post-processors": [
[
{
"type": "vagrant",
"vagrantfile_template": "configs/vagrantfile.tpl",
"output": "nemonik_alpine310.box"
}
]
]
}
```
Packer's `builders`' `boot_command`'s execution is very time sensitive, so you may need to add additional ``s if the build was to fail for you.
#### 9.5.1.4. Packer execution
Along the way a modal will pop up asking you "Do you want the application “packer” to accept incoming network connections?". Give your approval otherwise Packer won't be able to do its job. The command-line output of Packer when creating the nemonik_alpine310.box will resemble:
```packer
INFO: Creating nemonik/devops_alpine310 box...
setting PROXYOPTS to none in http/answers
building nemonik_alpine310.box via packer
virtualbox-iso: output will be in this color.
==> virtualbox-iso: Retrieving ISO
==> virtualbox-iso: Trying isos/x86_64/alpine-virt-3.10.0-x86_64.iso
==> virtualbox-iso: Trying isos/x86_64/alpine-virt-3.10.0-x86_64.iso?checksum=sha256%3Ab3d8fe65c2777edcbc30b52cde7f5ae21dff8ecda612d5fe7b10d5c23cda40c4
==> virtualbox-iso: isos/x86_64/alpine-virt-3.10.0-x86_64.iso?checksum=sha256%3Ab3d8fe65c2777edcbc30b52cde7f5ae21dff8ecda612d5fe7b10d5c23cda40c4 => /Users/mjwalsh/Development/workspace/DevOps_class/alpine-playground/box/packer-alpine310/packer_cache/0172be7009f3c62b985d67dec5495188c7596bcd.iso
==> virtualbox-iso: Starting HTTP server on port 8922
==> virtualbox-iso: Creating virtual machine...
==> virtualbox-iso: Creating hard drive...
==> virtualbox-iso: Creating forwarded port mapping for communicator (SSH, WinRM, etc) (host port 3684)
==> virtualbox-iso: Executing custom VBoxManage commands...
virtualbox-iso: Executing: modifyvm alpine-3.10.0-x86_64 --memory 1024
virtualbox-iso: Executing: modifyvm alpine-3.10.0-x86_64 --cpus 1
virtualbox-iso: Executing: modifyvm alpine-3.10.0-x86_64 --rtcuseutc on
==> virtualbox-iso: Starting the virtual machine...
==> virtualbox-iso: Waiting 30s for boot...
==> virtualbox-iso: Typing the boot command...
```
A little VirtualBox window for the VM will also pop up, so that you can monitor progress as Alpine is installed. At some point Alpine will reboot and Packer will continue outputting:
```packer
==> virtualbox-iso: Using ssh communicator to connect: 127.0.0.1
==> virtualbox-iso: Waiting for SSH to become available...
==> virtualbox-iso: Provisioning with shell script: scripts/configure.sh
==> virtualbox-iso: + rc-update -u
virtualbox-iso: * Caching service dependencies ... [ ok ]
==> virtualbox-iso: + setup-apkcache
==> virtualbox-iso: + apk update
virtualbox-iso: Enter apk cache directory (or '?' or 'none') [/var/cache/apk]: fetch http://dl-cdn.alpinelinux.org/alpine/v3.10/main/x86_64/APKINDEX.tar.gz
virtualbox-iso: v3.10.4-14-g975b6a3945 [http://dl-cdn.alpinelinux.org/alpine/v3.10/main/]
virtualbox-iso: OK: 5669 distinct packages available
==> virtualbox-iso: + apk upgrade -U --available
virtualbox-iso: OK: 87 MiB in 52 packages
==> virtualbox-iso: + apk add -U bash bash-completion sudo curl
virtualbox-iso: (1/10) Installing readline (8.0.0-r0)
virtualbox-iso: (2/10) Installing bash (5.0.0-r0)
virtualbox-iso: Executing bash-5.0.0-r0.post-install
virtualbox-iso: (3/10) Installing bash-completion (2.8-r0)
virtualbox-iso: (4/10) Installing openrc-bash-completion (0.41.2-r1)
virtualbox-iso: (5/10) Installing ca-certificates (20190108-r0)
virtualbox-iso: (6/10) Installing nghttp2-libs (1.39.2-r0)
virtualbox-iso: (7/10) Installing libcurl (7.66.0-r0)
virtualbox-iso: (8/10) Installing curl (7.66.0-r0)
virtualbox-iso: (9/10) Installing kmod-bash-completion (24-r1)
virtualbox-iso: (10/10) Installing sudo (1.8.27-r2)
virtualbox-iso: Executing busybox-1.30.1-r3.trigger
virtualbox-iso: Executing ca-certificates-20190108-r0.trigger
virtualbox-iso: OK: 94 MiB in 62 packages
==> virtualbox-iso: + echo http://dl-cdn.alpinelinux.org/alpine/v3.10/community
==> virtualbox-iso: + apk add -U virtualbox-guest-additions virtualbox-guest-modules-virt
virtualbox-iso: fetch http://dl-cdn.alpinelinux.org/alpine/v3.10/community/x86_64/APKINDEX.tar.gz
virtualbox-iso: (1/3) Installing virtualbox-guest-additions (6.0.8-r1)
virtualbox-iso: Executing virtualbox-guest-additions-6.0.8-r1.pre-install
virtualbox-iso: (2/3) Installing virtualbox-guest-modules-virt (4.19.98-r0)
virtualbox-iso: (3/3) Installing virtualbox-guest-additions-openrc (6.0.8-r1)
virtualbox-iso: Executing busybox-1.30.1-r3.trigger
virtualbox-iso: Executing kmod-24-r1.trigger
virtualbox-iso: OK: 98 MiB in 65 packages
==> virtualbox-iso: + rc-update add virtualbox-guest-additions
virtualbox-iso: * service virtualbox-guest-additions added to runlevel default
==> virtualbox-iso: + echo vboxsf
==> virtualbox-iso: + echo 'UseDNS no'
==> virtualbox-iso: + adduser -D vagrant
==> virtualbox-iso: + chpasswd
==> virtualbox-iso: + echo vagrant:vagrant
==> virtualbox-iso: chpasswd: password for 'vagrant' changed
==> virtualbox-iso: + mkdir -pm 700 /home/vagrant/.ssh
==> virtualbox-iso: + chown -R vagrant:vagrant /home/vagrant/.ssh
==> virtualbox-iso: + wget -O /home/vagrant/.ssh/authorized_keys https://raw.githubusercontent.com/mitchellh/vagrant/master/keys/vagrant.pub
==> virtualbox-iso: Connecting to raw.githubusercontent.com (151.101.0.133:443)
==> virtualbox-iso: authorized_keys 100% |********************************| 409 0:00:00 ETA
==> virtualbox-iso: + chmod 644 /home/vagrant/.ssh/authorized_keys
==> virtualbox-iso: + adduser vagrant wheel
==> virtualbox-iso: + echo 'Defaults exempt_group=wheel'
==> virtualbox-iso: + echo '%wheel ALL=NOPASSWD:ALL'
==> virtualbox-iso: + sed -i.bak 's@/ash$@/bash@g' /etc/passwd
==> virtualbox-iso: + rm /etc/passwd.bak
==> virtualbox-iso: + echo
==> virtualbox-iso: + rm -Rf /var/cache/apk/APKINDEX.6456cd1a.tar.gz /var/cache/apk/APKINDEX.d8b2a6f4.tar.gz /var/cache/apk/bash-5.0.0-r0.045599ea.apk /var/cache/apk/bash-completion-2.8-r0.c0f500d3.apk /var/cache/apk/ca-certificates-20190108-r0.c0ec382f.apk /var/cache/apk/curl-7.66.0-r0.86434890.apk /var/cache/apk/installed /var/cache/apk/kmod-bash-completion-24-r1.ea84cd3e.apk /var/cache/apk/libcurl-7.66.0-r0.ae6524b8.apk /var/cache/apk/nghttp2-libs-1.39.2-r0.6f35d6f8.apk /var/cache/apk/openrc-bash-completion-0.41.2-r1.198430a1.apk /var/cache/apk/readline-8.0.0-r0.32186548.apk /var/cache/apk/sudo-1.8.27-r2.fe94d45e.apk /var/cache/apk/virtualbox-guest-additions-6.0.8-r1.b025c3f0.apk /var/cache/apk/virtualbox-guest-additions-openrc-6.0.8-r1.b528fb1a.apk /var/cache/apk/virtualbox-guest-modules-virt-4.19.98-r0.f198f848.apk
==> virtualbox-iso: + dd 'if=/dev/zero' 'of=/EMPTY' 'bs=1M'
```
It will take some time to zero out the unused portion of the root drive, but Packer will continue:
```packer
==> virtualbox-iso: 59959+0 records in
==> virtualbox-iso: 59958+0 records out
==> virtualbox-iso: + rm -f /EMPTY
==> virtualbox-iso: + cat /dev/null
==> virtualbox-iso: + history -c
==> virtualbox-iso: + sync
==> virtualbox-iso: + sync
==> virtualbox-iso: + sync
==> virtualbox-iso: + exit 0
==> virtualbox-iso: Gracefully halting virtual machine...
==> virtualbox-iso: Preparing to export machine...
virtualbox-iso: Deleting forwarded port mapping for the communicator (SSH, WinRM, etc) (host port 3684)
==> virtualbox-iso: Exporting virtual machine...
virtualbox-iso: Executing: export alpine-3.10.0-x86_64 --output output-virtualbox-iso/alpine-3.10.0-x86_64.ova
==> virtualbox-iso: Deregistering and deleting VM...
==> virtualbox-iso: Running post-processor: vagrant
==> virtualbox-iso (vagrant): Creating Vagrant box for 'virtualbox' provider
virtualbox-iso (vagrant): Unpacking OVA: output-virtualbox-iso/alpine-3.10.0-x86_64.ova
virtualbox-iso (vagrant): Renaming the OVF to box.ovf...
virtualbox-iso (vagrant): Using custom Vagrantfile: configs/vagrantfile.tpl
virtualbox-iso (vagrant): Compressing: Vagrantfile
virtualbox-iso (vagrant): Compressing: alpine-3.10.0-x86_64-disk001.vmdk
virtualbox-iso (vagrant): Compressing: box.ovf
virtualbox-iso (vagrant): Compressing: metadata.json
Build 'virtualbox-iso' finished.
==> Builds finished. The artifacts of successful builds are:
--> virtualbox-iso: 'virtualbox' provider box: nemonik_alpine310.box
```
### 9.5.2. Vagrant
This class uses Vagrant, a command-line utility for managing the life cycle of virtual machines as a vagrant in that the VMs are not meant to hang around in the same place for very long.
Unless you want to pollute your machine with every imaginable programming language, framework and library version you'll find yourself often creating a virtual machine (VM) for each software project. Sometimes more than one. And if you're like me of the past you'll end up with a VirtualBox full of VMs. If you haven't gone about this the right way, you'll end up wondering what VM went with which project and now how did I create it? The anti-pattern around this problem is to write documentation. A better way that aligns with a DevOps repeatable practices is to create automation to provision and configure your development VMs. This is where Vagrant comes in as it is "a command-line utility for managing the life cycle of virtual machines."
#### 9.5.2.1. Vagrant documentation and source
Vagrant's documentation can be found at
It's canonical (i.e., authoritative) source can be found at
Vagrant is written in Ruby. In fact, a Vagrantfile is written in a Ruby DSL and I make full use of this to extend the functionality of the Vagrantfile..
#### 9.5.2.2. Installing Vagrant
1. If you are Windows or OS X download Vagrant from
The class has been verified to work with Version 2.2.10. Newer version may or may not work.
If you're using Linux, use your operating system's package manager to install `vagrant`. For exmple, to install on Arch Linux one would use
```bash
sudo pacman -Syu vagrant
```
2. Click on the installer once downloaded and follow along. On Windows, the installer may stall calculating for a bit and may bury modals you'll need to respond to in the Windows Task bar, so keep an eye out for that. The installer will automatically add the vagrant command to your system path so that it is available on the command line. If it is not found, the documentation advises you to try logging out and logging back into your system. This is particularly necessary sometimes for Windows. Windows will require a reboot, so remember to **come back and complete step-3, if you are on the MITRE corporate network**.
3. **If you're not on the MITRE corporate network please skip this step.**
- On Windows, use the File Explorer to replace the existing `C:\Hashicorp\vagrant\embedded\cacert.pem` file with the project's `vagrant_files/cacert.pem` by using the File Explorer.
- On Mac OS X, copy it to `/opt/vagrant/embedded` as *root* using
```bash
sudo cp vagrant_files/cacert.pem /opt/vagrant/embedded/.
```
**NOTE**
- Vagrant respects `SSL_CERT_FILE` and `CURL_CA_BUNDLE` environment variables used to point to cacert bundles. If you run into SSL errors, you may have `SSL_CERT_FILE` and/or `CURL_CA_BUNDLE` environment variable files set requiring you to add MITRE CA certificates to the file specified by these environment variables. If you use the `set_env.sh` at the root of the project it will unset these environment variables forcing vagrant to use its cacert.pem file you replace above.
- The same site has the Mac OS X download, whose installation is less involved.
#### 9.5.2.3. The Vagrantfile explained
The `Vagrantfile` found at the root of the project describes how to provision and configure one or more virtual machines.
Vagrant's own documentation puts it best:
> Vagrant is meant to run with one Vagrantfile per project, and the Vagrantfile is
> supposed to be committed to version control. This allows other developers involved
> in the project to check out the code, run vagrant up, and be on their way.
> Vagrantfiles are portable across every platform Vagrant supports.
If we were instead provisioning Amazon EC2 instances, we'd alternatively use [Terraform](https://www.terraform.io/), a tool for building, changing, and versioning infrastructure.
The following sub-sections enumerate the various sections of the `Vagrantfile` broken apart in order to discuss.
##### 9.5.2.3.1. Modelines
```ruby
# -*- mode: ruby -*-
# vi: set ft=ruby :
```
When authoring, tells your text editor (e.g. emacs or vim) to choose a specific editing mode for the Vagrantfile. Line one is a [modeline for emacs](http://www.gnu.org/software/emacs/manual/html_node/emacs/Choosing-Modes.html) and line two is a [modeline for vim](http://vim.wikia.com/wiki/Modeline_magic).
##### 9.5.2.3.2. Setting extra variables for Ansible roles
The following lines:
```ruby
# Vagrant will start at your current path and then move upward looking
# for a Vagrant file. The following will provide the path for the found
# Vagrantfile, so you can execute `vagrant` commands on the command-line
# anywhere in the project a Vagrantfile doesn't already exist.
vagrantfilePath = ""
if File.dirname(__FILE__).end_with?('Vagrantfile')
vagrantfilePath = File.dirname(File.dirname(__FILE__))
else
vagrantfilePath = File.dirname(__FILE__)
end
# Used to hold all the configuration variable and convienance methods for accessing
require File.join(vagrantfilePath, 'configuration_vars.rb')
```
in the Vagrantfile at the root of the project will include `configuration_vars.rb` Ruby module. This Ruby module defines the Ansible extra_vars used by Ansible roles found in `ansible/roles` and `box/ansible/role` as well as the bash scripting templates used by the `Vagrantfile` at the root of the project as well as the `box/Vagrantfile` as well as the bash scripts found in `box/`. You would update the version of GitLab used by the class by updating `:gitlab_version` here.
The contents of `configuration_vars.rb` Ruby module will resemble:
```ruby
#-*- mode: ruby -*-
# vi: set ft=ruby :
# Copyright (C) 2020 Michael Joseph Walsh - All Rights Reserved
# You may use, distribute and modify this code under the
# terms of the the license.
#
# You should have received a copy of the license with
# this file. If not, please email
module ConfigurationVars
# Define variablese
VARS = {
# The network block the cluster and apps will be in
network_prefix: "192.168.0",
# # Add OpenEBS drives ('yes'/'no')
# openebs_drives: 'yes',
openebs_drives: 'no',
# Sets the OpenEBS drive size in GB
openebs_drive_size_in_gb: 100,
# Provision and configure development vagrant ('yes'/'no')
# create_development: 'no',
create_development: 'yes',
# development_is_worker_node: 'yes',
# The number of nodes to provision the Kubernetes cluster. One will be a master.
nodes: 2,
# nodes: 1,
# The Vagrant box to base our DevOps box on. Pick just one.
base_box: 'centos/7',
base_box_version: '2004.01',
# base_box: 'ubuntu/bionic64',
# base_box_version: '20200304.0.0',
# base_box: 'nemonik/alpine310',
# base_box_version: '0',
vagrant_root_drive_size: '80GB',
ansible_version: '2.10.3',
default_retries: '60',
default_delay: '10',
docker_timeout: '300',
docker_retries: '60',
docker_delay: '10',
k3s_version: 'v1.19.4+k3s1',
k3s_cluster_secret: 'kluster_secret',
kubectl_version: 'v1.19.4',
kubectl_checksum: 'sha256:7df333f1fc1207d600139fe8196688303d05fbbc6836577808cda8fe1e3ea63f',
kubernetes_dashboard: 'yes',
kubernetes_dashboard_version: 'v2.0.0',
traefik: 'yes',
traefik_version: '1.7.26',
traefik_http_port: '80',
traefik_admin_port: '8080',
traefik_host: '192.168.0.206',
metallb: 'yes',
metallb_version: 'v0.9.5',
kompose_version: '1.18.0',
docker_compose_version: '1.27.4',
docker_compose_pip_version: '1.25.0rc2',
helm_cli_version: '3.2.1',
helm_cli_checksum: '018f9908cb950701a5d59e757653a790c66d8eda288625dbb185354ca6f41f6b',
registry_version: '2.7.1',
registry: 'yes',
registry_host: '192.168.0.10',
registry_port: '5000',
passthrough_registry: 'yes',
passthrough_registry_host: '192.168.0.10',
passthrough_registry_port: '5001',
registry_deploy_via: 'docker-compose',
gitlab: 'yes',
gitlab_version: '13.2.3',
gitlab_host: '192.168.0.202',
gitlab_port: '80',
gitlab_ssh_port: '10022',
gitlab_user: 'root',
drone: 'yes',
drone_version: '1.9.0',
drone_runner_docker_version: '1.4.0',
drone_host: '192.168.0.10',
drone_cli_version: 'v1.2.1',
plantuml_server: 'yes',
plantuml_server_version: 'latest',
plantuml_host: '192.168.0.203',
plantuml_port: '80',
taiga: 'yes',
taiga_version: 'latest',
taiga_host: '192.168.0.204',
taiga_port: '80',
sonarqube: 'yes',
sonarqube_version: '8.5.1-community',
sonarqube_host: '192.168.0.205',
sonarqube_port: '9000',
sonar_scanner_cli_version: '4.3.0.2102',
inspec_version: '4.18.39',
python_container_image: 'yes',
python_version: '2.7.18',
golang_container_image: 'yes',
golang_sonarqube_scanner_image: 'yes',
golang_version: '1.15',
selenium_standalone_chrome_version: '3.141',
standalone_firefox_container_image: 'yes',
selenium_standalone_firefox_version: '3.141',
owasp_zap2docker_stable_image: 'yes',
zap2docker_stable_version: '2.8.0',
openwhisk: 'yes',
openwhisk_host: '192.168.0.207',
cache_path: '/vagrant/cache',
images_cache_path: '/vagrant/cache/images',
create_cache: 'yes',
host_os: (
host_os = RbConfig::CONFIG['host_os']
case host_os
when /mswin|msys|mingw|cygwin|bccwin|wince|emc/
'windows'
when /darwin|mac os/
'macosx'
when /linux/
'linux'
when /bsd/
'unix'
else
raise Error, "unknown os: #{host_os.inspect}"
end
)
}
VARS[:ansible_python_version] = (
if VARS[:base_box].downcase.include? 'centos' and VARS[:base_box].to_s.include? '7'
'python2'
else
'python3'
end
)
def ConfigurationVars.as_string( http_proxy, https_proxy, ftp_proxy, no_proxy, certs)
vars = VARS
vars[:http_proxy] = (!http_proxy ? "" : http_proxy)
vars[:https_proxy] = (!https_proxy ? "" : https_proxy)
vars[:ftp_proxy] = (!ftp_proxy ? "" : ftp_proxy)
vars[:no_proxy] = (!no_proxy ? "" : no_proxy)
vars[:CA_CERTIFICATES] = ''
unless certs.nil? || certs == ''
vars[:CA_CERTIFICATES] = certs
end
vars_string = ''
vars.each do |key, value|
if ( ( key == :CA_CERTIFICATES ) && ( !value.nil? ) && value != '' )
vars_string = vars_string + "\\\"#{key}\\\":\\\["
value.each { |item|
vars_string = vars_string + "\\\"#{item}\\\","
}
vars_string = vars_string.chop + '\\],'
else
if (value.is_a? Integer)
value = value.to_s
end
vars_string = vars_string + "\\\"#{key}\\\":\\\"#{value}\\\","
end
end
return '\\{' + vars_string.chop + '\\}'
end
DETERMINE_OS_TEMPLATE = <<~SHELL
echo Determining OS...
os=""
if [[ $(command -v lsb_release | wc -l) == *"1"* ]]; then
os="$(lsb_release -is)-$(lsb_release -cs)"
elif [ -f "/etc/os-release" ]; then
if [[ $(cat /etc/os-release | grep -i alpine | wc -l) -gt "0" ]]; then
os="Alpine"
elif [[ $(cat /etc/os-release | grep -i "CentOS Linux 7" | wc -l) -gt "0" ]]; then
os="CentOS 7"
fi
else
echo -n "Cannot determine OS."
exit -1
fi
SHELL
OS_PACKAGES_FROM_CACHE_TEMPLATE = <<~SHELL
echo OS packages from cache...
mkdir -p /tmp/root-cache
box="#{VARS[:base_box]}"
case $os in
"Alpine")
package_manager="apk"
;;
"Ubuntu-bionic")
package_manager="apt"
;;
"CentOS 7")
package_manager="yum"
;;
*)
echo -n "${os} not supported."
exit -1
;;
esac
if [ -f "/vagrant/cache/TYPE/${box}/${package_manager}.tar.gz" ]; then
update=true
if [ -f "/tmp/root-cache/${package_manager}.tar.gz" ]; then
if ((`stat -c%s "/vagrant/cache/TYPE/${box}/${package_manager}.tar.gz"`!=`stat -c%s "/tmp/root-cache/${package_manager}.tar.gz"`)); then
update=false
fi
fi
if ($update == true); then
echo Installing ${box} ${package_manager} packages from cache...
cd /tmp/root-cache
cp /vagrant/cache/TYPE/${box}/${package_manager}.tar.gz ${package_manager}.tar.gz
tar zxf ${package_manager}.tar.gz
if [ -d "${package_manager}" ]; then
case $os in
"Alpine")
echo "installing apk packages from cache..."
apk add --repositories-file=/dev/null --allow-untrusted --no-network apk/*.apk
;;
"Ubuntu-bionic")
#echo "installing apt packages from cache..."
#cd apt/archives
#dpkg -i ./*.deb
#apt --fix-broken install
echo "not yet reliable..."
;;
"CentOS 7")
mv yum /var/cache/
;;
*)
echo "${os} not supported." 1>&2
exit -1
;;
esac
fi
rm -Rf ${package_manager}
else
echo No new ${box} packages in cache...
fi
else
echo No cached ${box} packages...
fi
SHELL
ROOT_INSTALL_ANSIBLE_DEPENDENCIES_TEMPLATE = <<~SHELL
echo Root installing ansible dependencies...
case $os in
"Alpine")
# install Alpine packages
apk add python3 python3-dev py3-pip musl-dev libffi-dev libc-dev py3-cryptography make gcc libressl-dev
;;
"Ubuntu-bionic")
apt update
apt upgrade -y
apt install -y python3 python3-dev python3-pip make gcc
;;
"CentOS 7")
yum update -y
yum install -y epel-release
yum install -y python-pip python-devel make gcc python-cffi
# pip uninstall -y bcrypt
# yum --enablerepo=epel install -y python2-bcrypt
;;
*)
echo "${os} not supported." 1>&2
exit -1
;;
esac
SHELL
USER_INSTALL_DEPENDENCIES_TEMPLATE = <<~SHELL
echo User installing ansible dependencies...
#{ConfigurationVars::VARS[:ansible_python_version]} -m pip install --user --upgrade pip
/home/vagrant/.local/bin/pip install --user --upgrade setuptools
/home/vagrant/.local/bin/pip install --user paramiko ansible==#{ConfigurationVars::VARS[:ansible_version]}
case $os in
"Alpine"|"Ubuntu-bionic")
;;
"CentOS 7")
# /home/vagrant/.local/bin/pip uninstall -y bcrypt
# /home/vagrant/.local/bin/pip install -y bcrypt==3.1.7
;;
*)
echo "${os} not supported." 1>&2
exit -1
;;
esac
SHELL
INSTALL_ANSIBLE_TEMPLATE = <<~SHELL
case $os in
"Alpine"|"Ubuntu-bionic"|"CentOS 7")
#{ConfigurationVars::VARS[:ansible_python_version]} -m pip install --user --upgrade pip setuptools
#{ConfigurationVars::VARS[:ansible_python_version]} -m pip install --user paramiko ansible==#{ConfigurationVars::VARS[:ansible_version]}
;;
*)
echo "${os} not supported." 1>&2
exit -1
;;
esac
SHELL
RUN_ANSIBLE_TEMPLATE = <<~SHELL
echo Running ansible-playbook PLAYBOOK_PATH...
case $os in
"Alpine"|"Ubuntu-bionic"|"CentOS 7")
n=0
until [ "$n" -ge #{ConfigurationVars::VARS[:default_retries]} ]; do
/home/vagrant/.local/bin/ansible-galaxy install --force --roles-path ansible/roles --role-file requirements.yml && break
n=$((n+1))
sleep #{ConfigurationVars::VARS[:default_delay]}
done
PYTHONUNBUFFERED=1 ANSIBLE_FORCE_COLOR=true /home/vagrant/.local/bin/ansible-playbook PLAYBOOK_PATH --limit="LIMIT" --extra-vars=ANSIBLE_EXTRA_VARS --extra-vars='ansible_python_interpreter="/usr/bin/env #{ConfigurationVars::VARS[:ansible_python_version]}"' --vault-password-file=VAULT_PASS_PATH -vvvv --connection=local --inventory=INVENTORY_PATH
;;
# "CentOS 7")
# n=0
# until [ "$n" -ge #{ConfigurationVars::VARS[:default_retries]} ]; do
# ansible-galaxy install --force --roles-path ansible/roles --role-file requirements.yml && break
# n=$((n+1))
# sleep #{ConfigurationVars::VARS[:default_delay]}
# done
# PYTHONUNBUFFERED=1 ANSIBLE_FORCE_COLOR=true ansible-playbook PLAYBOOK_PATH --limit="LIMIT" --extra-vars=ANSIBLE_EXTRA_VARS --vault-password-file=VAULT_PASS_PATH -vvvv --connection=local --inventory=INVENTORY_PATH
# ;;
*)
echo "${os} not supported." 1>&2
exit -1
;;
esac
SHELL
RESIZE_ROOT_TEMPLATE = <<~SHELL
echo Resizing root...
case $os in
"Alpine")
# install Alpine packages
echo "resize root not yet handled."
;;
"Ubuntu-bionic")
echo "resize root not yet handled."
;;
"CentOS 7")
echo "Resizing root volume..."
yum -y install cloud-utils-growpart
growpart /dev/sda 1
xfs_growfs /
;;
*)
echo "${os} not supported." 1>&2
exit -1
;;
esac
SHELL
SITE_PACKAGES_FROM_CACHE_TEMPLATE = <<~SHELL
echo Site packages from cache...
if [ -f "/vagrant/cache/TYPE/${box}/site-packages.tar.gz" ]; then
update=true
if [ -f "/tmp/root-cache/site-packages.tar.gz" ]; then
if ((`stat -c%s "/vagrant/cache/TYPE/${box}/site-packages"`!=`stat -c%s "/tmp/root-cache/site-packages"`)); then
update=false
fi
fi
if ($update == true); then
echo Unpacking #{ VARS[:ansible_python_version] } site-packages from cache...
cd /tmp/root-cache
cp /vagrant/cache/TYPE/${box}/site-packages.tar.gz site-packages.tar.gz
cd /usr/lib/#{ VARS[:ansible_python_version] }*
tar zxvf /tmp/root-cache/site-packages.tar.gz
else
echo No new updates to site-packages in cache...
fi
fi
SHELL
USER_CACHED_CONTENT_TEMPLATE = <<~SHELL
echo Use cached content...
mkdir -p /tmp/vagrant-cache
box="#{VARS[:base_box]}"
if [ "${os}" == "CentOS 7" ] && [ -f "/vagrant/cache/TYPE/${box}/rvm.tar.gz" ]; then
update=true
if [ -f "/tmp/vagrant-cache/rvm.tar.gz" ]; then
if ((`stat -c%s "/vagrant/cache/TYPE/${box}/rvm.tar.gz"`!=`stat -c%s "/tmp/vagrant-cache/rvm.tar.gz"`)); then
update=false
fi
fi
if ($update == true); then
echo Unpacking /home/vagrant/[.rvm .gnupg .bash_profile .bashrc .profile .mkshrc .zshrc .zlogin] from cache...
cp /vagrant/cache/TYPE/${box}/rvm.tar.gz /tmp/vagrant-cache/rvm.tar.gz
cd /home/vagrant/
tar zxvf /tmp/vagrant-cache/rvm.tar.gz
else
echo No new updates to /home/vagrant/[.rvm .gnupg .bash_profile .bashrc .profile .mkshrc .zshrc .zlogin] in cache...
fi
else
echo No /home/vagrant/[.rvm .gnupg .bash_profile .bashrc .profile .mkshrc .zshrc .zlogin] in cache...
fi
if [ -f "/vagrant/cache/TYPE/${box}/cache.tar.gz" ]; then
update=true
if [ -f "/tmp/vagrant-cache/cache.tar.gz" ]; then
if ((`stat -c%s "/vagrant/cache/TYPE/${box}/cache.tar.gz"`!=`stat -c%s "/tmp/vagrant-cache/cache.tar.gz"`)); then
update=false
fi
fi
if ($update == true); then
echo Unpacking /home/vagrant/.cache from cache...
cp /vagrant/cache/TYPE/${box}/cache.tar.gz /tmp/vagrant-cache/cache.tar.gz
cd /home/vagrant/
tar zxvf /tmp/vagrant-cache/cache.tar.gz
else
echo No new updates to /home/vagrant/.cache in cache...
fi
else
echo No /home/vagrant/.cache cache...
fi
if [ -f "/vagrant/cache/TYPE/${box}/local.tar.gz" ]; then
update=true
if [ -f "/tmp/vagrant-cache/local.tar.gz" ]; then
if ((`stat -c%s "/vagrant/cache/TYPE/${box}/local.tar.gz"`!=`stat -c%s "/tmp/vagrant-cache/local.tar.gz"`)); then
update=false
fi
fi
if ($update == true); then
echo Unpacking /home/vagrant/.local from cache...
cp /vagrant/cache/TYPE/${box}/local.tar.gz /tmp/vagrant-cache/local.tar.gz
cd /home/vagrant/
tar -zxvf /tmp/vagrant-cache/local.tar.gz
else
echo No new updates to /home/vagrant/.local in cache...
fi
else
echo No /home/vagrant/.local cache...
fi
if [ -f "/vagrant/cache/TYPE/${box}/gem.tar.gz" ]; then
update=true
if [ -f "/tmp/vagrant-cache/gem.tar.gz" ]; then
if ((`stat -c%s "/vagrant/cache/TYPE/${box}/gem.tar.gz"`!=`stat -c%s "/tmp/vagrant-cache/gem.tar.gz"`)); then
update=false
fi
fi
if ($update == true); then
echo Unpacking /home/vagrant/.gem from cache...
cp /vagrant/cache/TYPE/${box}/gem.tar.gz /tmp/vagrant-cache/gem.tar.gz
cd /home/vagrant
tar -zxvf /tmp/vagrant-cache/gem.tar.gz
else
echo No new updates to /home/vagrant/.gem in cache...
fi
else
echo No /home/vagrant/.gem cache...
fi
SHELL
end
```
##### 9.5.2.3.3. Automatically installing and removing the necessary Vagrant plugins
Vagrant can be extended by plugins and this class makes use of a number of them. One of which, the `vagrant-vbguest` plugin is not installed if the Vagrant's base operating system is Alpine, but is installed otherwise.
```ruby
uninstall_plugins = %w( vagrant-cachier vagrant-alpine )
required_plugins = %w( vagrant-timezone vagrant-proxyconf vagrant-certificates vagrant-disksize vagrant-reload ) # vagrant-disksize
if (not os.downcase.include? 'alpine')
required_plugins = required_plugins << "vagrant-vbguest"
else
# as alpine is currently not supported by vagrant-vbguest
uninstall_plugins = uninstall_plugins << "vagrant-vbguest"
end
# Uninstall the following plugins
plugin_uninstalled = false
uninstall_plugins.each do |plugin|
if Vagrant.has_plugin?(plugin)
system "vagrant plugin uninstall #{plugin}"
plugin_uninstalled = true
end
end
# Require the following plugins
plugin_installed = false
required_plugins.each do |plugin|
unless Vagrant.has_plugin?(plugin)
system "vagrant plugin install #{plugin}"
plugin_installed = true
end
end
#system "vagrant plugin update"
# if plugins were installed, restart
if plugin_installed || plugin_uninstalled
puts "restarting"
exec "vagrant #{ARGV.join' '}"
end
```
##### 9.5.2.3.4. Inserting Proxy setting via host environmental variables
Later in the Vagrantfile, a bit of code makes use of the `vagrant-proxyconf` plugin to configure the HTTP proxy settings for the vagrants (i.e., the transient VMs).
```ruby
# Set proxy settings for all vagrants
#
# Depends on install of vagrant-proxyconf plugin.
#
# To use:
#
# 1. Install `vagrant plugin install vagrant-proxyconf`
# 2. Set environmental variables for `http_proxy`, `https_proxy`, `ftp_proxy`, and `no_proxy`
#
# For example:
#
# ```
# export http_proxy=
# export https_proxy=
# export ftp_proxy=
# export no_proxy=
# ```
if (ENV['http_proxy'] || ENV['https_proxy'])
config.proxy.http = ENV['http_proxy']
config.proxy.https = ENV['https_proxy']
config.proxy.ftp = ENV['ftp_proxy']
config.proxy.no_proxy = ENV['no_proxy']
config.proxy.enabled = { docker: false }
if ( ARGV.include? 'up' ) || ( ARGV.include? 'provision' )
puts "INFO: HTTP Proxy variables set.".green
puts "INFO: http_proxy = #{ config.proxy.http }".green
puts "INFO: https_proxy = #{ config.proxy.https }".green
puts "INFO: ftp_proxy = #{ config.proxy.ftp }".green
puts "INFO: no_proxy = #{ config.proxy.no_proxy }".green
end
else
if ( ARGV.include? 'up' ) || ( ARGV.include? 'provision' )
puts "INFO: No http_proxy or https_proxy environment variables are set.".green
end
config.proxy.http = nil
config.proxy.https = nil
config.proxy.ftp = nil
config.proxy.no_proxy = nil
config.proxy.enabled = false
end
```
##### 9.5.2.3.5. Inserting enterprise CA certificates
Later still, a bit of code makes use of the `vagrant-certificates` plugin's to inject the specified certificates into the vagrants. This is useful, for example, if your enterprise network has a firewall (or appliance) which utilizes SSL interception. So, the existence of this plugin tell us more broadly others have to deal with the havoc SSL interception brings to development.
```ruby
# To add Enterprise CA Certificates to all vagrants
#
# Depends on the install of the vagrant-certificates plugin
#
# To use:
#
# 1. Install `vagrant plugin install vagrant-certificates`.
# 2. Set environement variable for `CA_CERTIFICATES` containing a comma separated list of certificate URLs.
#
# For example:
#
# ```
# export CA_CERTIFICATES=http://employeeshare.mitre.org/m/mjwalsh/transfer/MITRE%20BA%20ROOT.crt,http://employeeshare.mitre.org/m/mjwalsh/transfer/MITRE%20BA%20NPE%20CA-3%281%29.crt
# ```
#
# The Root certificate *must* be denotes as the root certificat like so:
#
# http://employeeshare.mitre.org/m/mjwalsh/transfer/MITRE%20BA%20ROOT.crt
#
if ENV['CA_CERTIFICATES']
# Because @williambailey's vagrant-ca-certificates has an issue https://github.com/williambailey/vagrant-ca-certificates/issues/34 I am using @Toilal fork, vagrant-certificates
if ( ARGV.include? 'up' ) || ( ARGV.include? 'provision' )
puts "INFO: CA Certificates set to #{ ENV['CA_CERTIFICATES'] }".green
end
config.certificates.enabled = true
config.certificates.certs = ENV['CA_CERTIFICATES'].split(',')
else
if ( ARGV.include? 'up' ) || ( ARGV.include? 'provision' )
puts "INFO: No CA_CERTIFICATES environment variable set.".green
end
config.certificates.certs = nil
config.certificates.enabled = false
end
```
##### 9.5.2.3.6. Auto-generate the Ansible inventory file
The Vagrantfile doesn't make use of the Vagrant's own [Ansible Local provisioner](https://www.vagrantup.com/docs/provisioning/ansible_local.html), but instead makes use of the `Shell` provisioner to install and use Ansible to configure the Vagrants.
To do this Ansible needs an inventory file and so the Vagrantfile dynamically creates the inventory file via the following code:
```ruby
# Write hosts file for Ansible
require 'erb'
@worker_nodes = [*1..ConfigurationVars::VARS[:nodes]-1]
template = <<-TEXT
#
# Do not change this file by hand as it is dynamically generated via the Vagrantfile.
#
development ansible_connection=local
master ansible_connection=local
<% for @node in @worker_nodes %>node<%= @node %> ansible_connection=local
<% end %>
[boxes]
box
[masters]
master
[workers]
<% for @node in @worker_nodes %>node<%= @node %>
<% end %>
[developments]
development
TEXT
open(File.join(vagrantfilePath, 'hosts'), 'w') do |f|
f.puts ERB.new(template).result
end
```
resulting in a `hosts` file being created in the root of the project upon `vagrant up` being called on the command-line.
The contents of the `hosts` file will resemeble:
```
#
# Do not change this file by hand as it is dynamically generated via the Vagrantfile.
#
development ansible_connection=local
master ansible_connection=local
[boxes]
box
[masters]
master
[workers]
[developments]
development
```
##### 9.5.2.3.7. Mounting the project folder into each vagrant
Each vagrant (e.g., master, worker nodes if provisioned, development) will have the class project fodler mounted at `vagrant`. If you or on OS X or Linux this will be done via the Network File Service requiring you to enter your password everytime prompted. If you are on Windows the the mounting will be handled by VirtualBox.
The code in the Vagrantfile to accomplish this is the following lines:
```ruby
# nfs does not appear to work reliably on OS X Catalina (See: https://github.com/hashicorp/vagrant/issues/11234)
if Vagrant::Util::Platform.windows?
config.vm.synced_folder '.', '/vagrant', owner: 'vagrant', group: 'vagrant', mount_options: ['dmode=775,fmode=664']
elsif Vagrant::Util::Platform.platform.include? 'darwin'
projectPath=File.join('/System/Volumes/Data/Users/', ENV['USER'], vagrantfilePath.split(ENV['USER']).last)
config.vm.synced_folder projectPath, '/vagrant', owner: 'vagrant', group: 'vagrant', mount_options: ['dmode=775,fmode=664']
else
config.vm.synced_folder '.', '/vagrant', type: 'nfs'
end
```
**NOTE**
- If you're host laptop or PC is using a version of Linux that Vagrant is unable configure `nfs` on, then you may have to comment out portions of this section in both `Vagrantfile` and `box/Vagrantfile`, so that the default `type` will be used like so
```ruby
# nfs does not appear to work reliably on OS X Catalina (See: https://github.com/hashicorp/vagrant/issues/11234)
# if Vagrant::Util::Platform.windows?
config.vm.synced_folder '.', '/vagrant', owner: 'vagrant', group: 'vagrant', mount_options: ['dmode=775,fmode=664']
# elsif Vagrant::Util::Platform.platform.include? 'darwin'
# projectPath=File.join('/System/Volumes/Data/Users/', ENV['USER'], vagrantfilePath.split(ENV['USER']).last)
#
# config.vm.synced_folder projectPath, '/vagrant', owner: 'vagrant', group: 'vagrant', mount_options: ['dmode=775,fmode=664']
# else
# config.vm.synced_folder '.', '/vagrant', type: 'nfs'
# end
```
##### 9.5.2.3.8. Build a Vagrant Box
The master, worker nodes (if created) and development vagrant will need a base box.
The following section of code will call the `build_box.sh` shell script found in the `box` directory to create this box.
```ruby
if ( ARGV.include? 'up' )
if (`vagrant box list | grep #{box}`.empty? )
puts "INFO: Creating #{box} box...".green
require 'open3'
Open3.popen2e('bash', '-c', 'cd box && ./build_box.sh') do |stdin, stdout, stderr|
puts stdout.each { |line| puts line }
end
else
puts "INFO: Using existing #{box} box...".green
end
end
```
Vagrant needs a image to start from in much the same reason one builds an Amazon EC2 off an Amazon Machine Image (AMI) or VMWare Image. Vagrant calls its images _Vagrant boxes_. One can create a Vagrant box via the vagrant command line interface (CLI) or alternatively one can use Hashicorp's Packer cli. The class use existing `centos/7` and `ubuntu/bionic` Vagrant boxes to create a box ontop of which the class vagrants (VMs) are built, but if you selected `nemonik/alpine310` as your `:base_box` in the `ConfigurationVars` and have istalled the `packer` cli packer will be used to create `nemonik/alpine310` base box and then take this box and use it to create a box via Vagrant ontop of which the vagrants will be created.
##### 9.5.2.3.9. Configuring the Kubernetes cluster vagrant(s)
By default the class is configured as a two node cluster, a single master node and one worker node by having `:nodes` set to `2` in the `ConfigurationVars` module.
The Vagrant file uses the value of `:nodes` and the shell scripting templates defined in the `ConfigurationVars` module to create the cluster starting with a master node and each subsequent node being created as a worker node.
```ruby
# shell scripting to install root user cached content
root_cached_template = ConfigurationVars::DETERMINE_OS_TEMPLATE + ConfigurationVars::OS_PACKAGES_FROM_CACHE_TEMPLATE + ConfigurationVars::SITE_PACKAGES_FROM_CACHE_TEMPLATE
# shell scripting to install user cached content
user_cached_template = ConfigurationVars::USER_CACHED_CONTENT_TEMPLATE
(0..ConfigurationVars::VARS[:nodes]-1).each do |node|
if (node == 0) then
hostname = 'master'
masters_root_cached = root_cached_template.gsub! /TYPE/, 'masters'
masters_user_cached = user_cached_template.gsub! /TYPE/, 'masters'
else
hostname = "node#{node}"
workers_root_cached = root_cached_template.gsub! /TYPE/, 'workers'
workers_user_cached = user_cached_template.gsub! /TYPE/, 'workers'
end
config.vm.define hostname do |vagrant|
vagrant.vm.network 'private_network', ip: "#{ ConfigurationVars::VARS[:network_prefix] }.#{10 + node}"
vagrant.vm.hostname = hostname
vagrant.vm.provider :virtualbox do |virtualbox|
virtualbox.name = "Hands-on DevOps class - #{os} - #{hostname}"
virtualbox.gui = false
# disable audio
virtualbox.customize ['modifyvm', :id, '--audio', 'none']
virtualbox.customize ['modifyvm', :id, '--nic1', 'nat']
virtualbox.customize ['modifyvm', :id, '--cableconnected1', 'on']
virtualbox.customize ["modifyvm", :id, "--natdnshostresolver1", "on"]
virtualbox.customize ["modifyvm", :id, "--natdnsproxy1", "on"]
virtualbox.customize [ "guestproperty", "set", :id, "/VirtualBox/GuestAdd/VBoxService/--timesync-interval", 10000 ]
virtualbox.customize [ "guestproperty", "set", :id, "/VirtualBox/GuestAdd/VBoxService/--timesync-min-adjust", 100 ]
virtualbox.customize [ "guestproperty", "set", :id, "/VirtualBox/GuestAdd/VBoxService/--timesync-set-on-restore", 1 ]
virtualbox.customize [ "guestproperty", "set", :id, "/VirtualBox/GuestAdd/VBoxService/--timesync-set-start", 1 ]
virtualbox.customize [ "guestproperty", "set", :id, "/VirtualBox/GuestAdd/VBoxService/--timesync-set-threshold", 1000 ]
if (node == 0) then
virtualbox.memory = 8192 #8192 #6144 #4096
virtualbox.cpus = 8 #8 #4
else
virtualbox.memory = 2048
virtualbox.cpus = 2
end
if (ConfigurationVars::VARS[:openebs_drives].downcase == 'yes') then # create OpenEBS drives on each node
openebs_disk = "./#{hostname}_openebs_disk.vdi"
# Add a second drive for OpenEBS
unless File.exist?(openebs_disk)
virtualbox.customize ['createmedium', '--filename', openebs_disk, '--size', ConfigurationVars::VARS[:openebs_drive_size_in_gb] * 1024]
end
# the value of storage_system_bus depends on your platform
storage_system_bus = "IDE"
# provisions the drive
virtualbox.customize ['storageattach', :id, '--storagectl', storage_system_bus, '--port', 1, '--device', 0, '--type', 'hdd', '--medium', openebs_disk]
end
end
# Configure via shell and Ansible
if (node == 0) then # the master node
# install root user cached content
vagrant.vm.provision 'root_cached_content', type: :shell, privileged: true, inline: "#{masters_root_cached}"
# install user cached content
vagrant.vm.provision 'user_cached_content', type: :shell, privileged: false, inline: "#{masters_user_cached}"
vagrant.vm.provision 'ansible', type: :shell, privileged: false, reset: true, inline: <<-SHELL
echo Configuring #{hostname} via Ansible...
cd /vagrant
n=0
until [ "$n" -ge #{ConfigurationVars::VARS[:default_retries]} ]; do
/home/vagrant/.local/bin/ansible-galaxy install --force --roles-path ansible/roles --role-file requirements.yml && break
n=$((n+1))
sleep #{ConfigurationVars::VARS[:default_delay]}
done
PYTHONUNBUFFERED=1 ANSIBLE_FORCE_COLOR=true /home/vagrant/.local/bin/ansible-playbook -vvvv --extra-vars=#{vars_string} --extra-vars='ansible_python_interpreter="/usr/bin/env #{ConfigurationVars::VARS[:ansible_python_version]}"' --vault-password-file=vau