https://github.com/dennyzhang/challenges-chef

:book: Challenges Your Chef Skills By Solving Real Questions.
https://github.com/dennyzhang/challenges-chef

chef configuration-management denny-challenges devops study-project

Last synced: 3 months ago
JSON representation

:book: Challenges Your Chef Skills By Solving Real Questions.

Host: GitHub
URL: https://github.com/dennyzhang/challenges-chef
Owner: dennyzhang
Created: 2017-11-15T19:04:24.000Z (over 8 years ago)
Default Branch: master
Last Pushed: 2018-11-05T23:38:07.000Z (over 7 years ago)
Last Synced: 2025-08-05T18:38:46.431Z (10 months ago)
Topics: chef, configuration-management, denny-challenges, devops, study-project
Language: Ruby
Homepage: https://www.dennyzhang.com/battle
Size: 2.33 MB
Stars: 26
Watchers: 11
Forks: 33
Open Issues: 2
Metadata Files:
- Readme: README.org

Awesome Lists containing this project

README

* Summary
#+BEGIN_HTML

#+END_HTML
* CheatSheet
File me [[https://github.com/DennyZhang/cheatsheet-kubernetes-A4/issues][Issues]] or star [[https://github.com/DennyZhang/cheatsheet-kubernetes-A4][this repo]].

See more challenges from Denny: [[https://github.com/topics/denny-challenges][#denny-challenges]]

** Scenario-101: Chef HelloWorld I
- Objective: Create a dummy cookbook, and test deployment in docker
- Requirements:
#+BEGIN_EXAMPLE
1. Use docker container to start a env with chef pre-installed
2. Create a dummy cookbook and apply it
#+END_EXAMPLE

- See more: [[Scenario-101][Scenario-101]]

** Scenario-102: Chef HelloWorld II
- Objective: Pure VM deployment
- Requirements:
#+BEGIN_EXAMPLE
1. Start a VM, install chef facility
2. Create a dummy cookbook to install jq package
3. Before install jq, run "apt-get update" by chef. So you need berkshelf.
4. Enforce rubocop and foodcritic for code static check
#+END_EXAMPLE

- See more: [[Scenario-102][Scenario-102]]

** Scenario-103: Chef HelloWorld III
- Objective: Setup chef server and chef client
- Requirements:
#+BEGIN_EXAMPLE
1. Start 3 containers to run chef server, knife workstation and chef client
2. Install and configure knife
3. From knife node run chef deployment in chef client node
#+END_EXAMPLE

#+BEGIN_HTML

#+END_HTML
- See more: [[Scenario-103][Scenario-103]]
- TODO

** Scenario-201: Enforce TDD For Your Chef Cookbooks I
- Objective: Test your cookbooks. Run kitchen docker in your laptop.
- Requirements:
#+BEGIN_EXAMPLE
1. Use kitchen to test your cookbook: start a container and test the logic
2. Enforce kitchen verify logic via serverspec
#+END_EXAMPLE

- See more: [[Scenario-201][Scenario-201]]

#+BEGIN_HTML

#+END_HTML
** Scenario-202: Enforce TDD For Your Chef Cookbooks II
- Objective: Deploy for 3 scenarios: docker, local VM and public cloud
- Requirements:
#+BEGIN_EXAMPLE
1. Use Kitchen to test local vm deployment
2. Use Kitchen to test docker deployment
3. Use Kitchen to test cloud VM deployment
#+END_EXAMPLE

- See more: [[Scenario-202][Scenario-202]]

#+BEGIN_HTML

#+END_HTML

** Scenario-301: Use Chef To Deploy Jenkins I
- Objective: Use chef to deploy one common service for real
- Requirements:
#+BEGIN_EXAMPLE
1. Deploy a standalone jenkins by chef
2. Chef shall add one jenkins user
3. Chef shall add a dummy job with slack notification enabled
4. Run deployment in docker, vagrant and EC2
5. Test both Ubuntu 14.04 and Centos 7
#+END_EXAMPLE

- See more: [[Scenario-301][Scenario-301]]

** Scenario-302: Use Chef To Deploy Jenkins II
- Objective: More Jenkins customization
- Requirements:
#+BEGIN_EXAMPLE
1. For better security, only registered user can login
2. For better security, Jenkins listen on port 18080, instead of 8080
3. When Jenkins is down, get alerts
#+END_EXAMPLE

- See more: [[Scenario-302][Scenario-302]]

** Scenario-303: Use Chef To Deploy Jenkins II
- Objective: More Jenkins customization
- Requirements:
#+BEGIN_EXAMPLE
1. Use Jenkinsfile to create a Jenkins pipeline
2. Enable and configure ThinBackup
#+END_EXAMPLE

- See more: [[Scenario-303][Scenario-303]]
- TODO

** Scenario-401: Use Chef To Deploy 2 Nodes Jenkins I
- Objective: Test and verify the deployment for both all-in-one and 2 nodes cluster
- Requirements:
#+BEGIN_EXAMPLE
1. Test 2 nodes jenkins cluster deployment in docker
2. Use Jenkinsfile to configure Jenkins
3. Define Jenkins pipeline
#+END_EXAMPLE

#+BEGIN_HTML

#+END_HTML
- See more: [[Scenario-303][Scenario-303]]
- TODO

#+BEGIN_HTML

#+END_HTML
* More Resources
License: Code is licensed under [[https://www.dennyzhang.com/wp-content/mit_license.txt][MIT License]].

- Useful links
#+BEGIN_EXAMPLE
https://github.com/chef-cookbooks/jenkins
https://github.com/jenkinsci/pipeline-examples
#+END_EXAMPLE

#+BEGIN_HTML

#+END_HTML
* org-mode configuration :noexport:
#+STARTUP: overview customtime noalign logdone showall
#+DESCRIPTION:
#+KEYWORDS:
#+AUTHOR: Denny Zhang
#+EMAIL: denny@dennyzhang.com
#+TAGS: noexport(n)
#+PRIORITIES: A D C
#+OPTIONS: H:3 num:t toc:nil \n:nil @:t ::t |:t ^:t -:t f:t *:t <:t
#+OPTIONS: TeX:t LaTeX:nil skip:nil d:nil todo:t pri:nil tags:not-in-toc
#+EXPORT_EXCLUDE_TAGS: exclude noexport
#+SEQ_TODO: TODO HALF ASSIGN | DONE BYPASS DELEGATE CANCELED DEFERRED
#+LINK_UP:
#+LINK_HOME:
* # --8<-------------------------- separator ------------------------>8-- :noexport:
* [#A] Routine job by chef :IMPORTANT:noexport:
# Delete old client
knife client delete mdmsandbox -c ~/.chef/knife_mdm.rb -y
knife node delete mdmsandbox -c ~/.chef/knife_mdm.rb -y
# Delete cookbook
knife cookbook delete nagios3 -c ~/.chef/knife_mdm.rb -y
** update osc envs
knife bootstrap www.oscgc.com --sudo -x ubuntu -N "dennytest" -c ~/.chef/knife_mdm.rb -V --node-ssl-verify-mode none
*** Internal Jenkins: 192.168.1.184:4022
knife bootstrap 192.168.1.184 --sudo -x root -P totvsJenkins -p 4022 -N "osc@cloud.com" -c ~/.chef/knife_mdm.rb -V --node-ssl-verify-mode none
** update mdm envs
*** MDM offical nagios: 104.236.159.226:22
knife bootstrap 104.236.159.226 --sudo -x root -N "mdmnagios" -c ~/.chef/knife_mdm.rb -V --node-ssl-verify-mode none
*** Internal sandbox: 10.165.4.67:7022
knife bootstrap 10.165.4.67 --sudo -x root -P sophia1 -p 7022 -N "dennysandbox" -c ~/.chef/knife_mdm.rb -V --node-ssl-verify-mode none
*** Internal Jenkins: 10.165.4.67:4022
knife bootstrap 10.165.4.67 --sudo -x root -P totvsJenkins -p 4022 -N "mdmnjenkins" -c ~/.chef/knife_mdm.rb -V --node-ssl-verify-mode none
*** MDM repo server, official Jenkins: 104.236.159.226:4022
knife bootstrap 104.236.159.226 --sudo -x root -P totvsRepo -p 4022 -N "mdmrepo" -c ~/.chef/knife_mdm.rb -V --node-ssl-verify-mode none

-r apt,jenkins-mdm -j "{\"jenkins_mdm\": {\"jobs\":\"BuildMDMRepo\", \"enable_email_alerting\":\"1\", \"enable_job_scheduled\":\"1\"}}"

http://104.236.159.226:18000
ssh -N -p 5022 -f root@104.236.159.226 -L 18080:localhost:18080 -n /bin/bash1
http://127.0.0.1:18080
*** mdm all-in-one docker test
docker pull denny/sshd:latest

docker run -d --privileged -t -p 2200:22 -i denny/sshd:latest /usr/sbin/sshd -D

knife bootstrap 104.236.180.184 --sudo -x root -P sophia1 -p 2200 -N "aiodocker" -c ~/.chef/knife_mdm.rb -V --node-ssl-verify-mode none -r apt,all-in-one -j "{\"all-in-one\": {\"mgmt_timeout\":\"900000\", \"max_timeout\":\"960000\"}, \"app_mdm\":{\"cb_bucket_retries\":\"10\", \"cb_bucket_retryinterval\":\"10000\", \"cb_bucket_timeout\": \"100000\"}}"
*** mdm local all-in-one test box
192.168.50.11

knife bootstrap 192.168.50.11 --sudo -x vagrant -P vagrant -p 22 -N "dennylocalbox" -c ~/.chef/knife_mdm.rb -V --node-ssl-verify-mode none -r apt,all-in-one -j "{\"all-in-one\": {\"enable_check\":\"0\",\"enable_nagios\":\"0\",\"mgmt_timeout\":\"900000\", \"max_timeout\":\"960000\"}, \"app_mdm\":{\"cb_bucket_retries\":\"10\", \"cb_bucket_retryinterval\":\"10000\", \"cb_bucket_timeout\": \"100000\"}}"
** upload cookbooks by berks
cd /Users/mac/Dropbox/private_data/project/chef/denny-chef-devops/cookbooks/devops-test
berks install
berks upload
berks upload nagios3

# upload cookbooks by knife
cd /Users/mac/Dropbox/private_data/work/totvs/code/mdmdevops/cookbooks/
ls -1 | xargs knife cookbook upload -c ~/.chef/knife_mdm.rb --force
knife cookbook upload -c ~/.chef/knife_mdm.rb backupdir

# delete cookbook
knife cookbook delete -c ~/.chef/knife_mdm.rb nagios3

# list cookbooks
knife cookbook list -c ~/.chef/knife_mdm.rb
** ~/.berkshelf/config.json
#+BEGIN_EXAMPLE
{
"chef": {
"chef_server_url": "https://104.236.159.226:443/organizations/digitalocean",
"node_name": "admin",
"client_key": "/Users/mac/.chef/chef_dennyzhang.pem",
"validation_client_name": "digitalocean-validator",
"validation_key_path": "/Users/mac/.chef/dennyzhang-validator.pem"
},
"ssl": {
"verify": false
}
}
#+END_EXAMPLE
** # --8<-------------------------- separator ------------------------>8--
** DONE apt-get update
CLOSED: [2015-04-26 Sun 17:21]
-r apt

Test:
apt-get update
** DONE [#A] enable email sending by gmail
CLOSED: [2015-04-24 Fri 11:09]
-r ssmtp -j "{\"ssmtp\": {\"credential_method\":\"plain\", \"auth_username\":\"navy.yang007@gmail.com\", \"auth_password\":\"file.navy1\", \"mailhub_name\":\"smtp.gmail.com\", \"mailhub_port\":587}}"

Test:

# ssmtp cookbook doesn't come with mailutils package installed
apt-get install mailutils
yum install mailx

echo "This is a test mail." | mail -s "test mail" 249950670@qq.com
** HALF enable nagios3 for monitoring and auto reporting
-r nagios3 -j "{\"nagios\": {\"server_ip\":\"127.0.0.1\", \"client_ip_list\":\"127.0.0.1\"}}"

Test:
http://127.0.0.1/nagios
nagiosadmin/password1234
** DONE enable backup script
CLOSED: [2015-04-24 Fri 11:22]
-r backupdir -j "{\"backupdir\": {\"dir_list\":\"/var/www/,/etc\"}}"

Test:
** DONE enable hostname
CLOSED: [2015-04-26 Sun 17:24]
-r hostname -j "{\"set_fqdn\": \"workstation.mdm.com\"}"

Test:
hostname -a
hostname -f
** setup hub registry
-r docker-registry2
** # --8<-------------------------- separator ------------------------>8--
** HALF enable mdm Jenkins
-r apt,jenkins-mdm -j "{\"jenkins_mdm\": {\"jobs\":\"BuildMDMRepo,UpdateSandboxMDM\"}}"
** TODO run mdm all-in-one
-r apt,all-in-one -j "{\"mdm\": {\"repo_server\":\"10.165.4.67:18000\"}}"
** # --8<-------------------------- separator ------------------------>8--
** DONE [#A] Use osc chef server
CLOSED: [2015-06-06 Sat 21:31]
cd /Users/mac/Dropbox/private_data/osc/chef/iamdevops/cookbooks
ls -1 | xargs knife cookbook upload -c ~/.chef/knife_osc.rb --force

knife bootstrap 192.168.1.185 --sudo -x root -P sophia1 -p 4022 -N "dennytest" -c ~/.chef/knife_osc.rb -V --node-ssl-verify-mode none -r apt,os-basic-auth -j "{\"os_basic\": {\"enable_firewall\":\"0\"}}"
* [#A] chef: a systems and cloud infrastructure automation framework :IMPORTANT:noexport:
:PROPERTIES:
:type: cloud
:END:

chef's cookbook: /usr/local/src/chef/cookbooks/mycookbook
| Num | Name | Summary |
|-----+--------------+------------------------------------------------------------|
| 1 | Resource | a statement of configuration policy |
| 2 | Knife | upload items from the chef-repo to the Chef server |
| 3 | workstation | |
| 4 | cookbooks | fundamental unit of configuration and policy distribution. |
|-----+--------------+------------------------------------------------------------|
| 5 | recipes | |
| 6 | Attribute | |
| 7 | Databags | a global variable that is stored as JSON data |
| 8 | Environments | |
#+TBLFM: $1=@-1+1;N

- knife
| Name | Summary |
|---------------------+-------------------------------------------------------------------------------------------------------|
| knife client list | |
| knife user list | |
| knife node list | |
| knife cookbook list | |
|---------------------+-------------------------------------------------------------------------------------------------------|
| Upload cookbooks | knife cookbook upload -a |
| Upload Data Bag | knife upload data_bags |
| Upload Roles | knife role from file base.rb starter.rb webserver.rb |
| Upload Environments | knife environment from file dev.rb production.rb |
| Boostrap VM | knife bootstrap --sudo -x root -P ChangeMe1 -N "node1" --bootstrap-version 11.12.8 |
| Configure Run_list | knife node run_list set node1 'role[webserver]' 'role[cron]' |

- Misc command
| Command | Summary |
|-----------------------------------------------------------------------------------+----------------------------------------------------------|
| chef-server-ctl test | |
| knife configure --initial | |
| sudo knife bootstrap 192.168.1.185 -x root -P ChangeMe -N centos --sudo | bootstrap a node |
| /root/.chef/knife.rb | knife configuration |
| /etc/chef/client.rb | chef client configuration |
| knife node edit client1 | |
|-----------------------------------------------------------------------------------+----------------------------------------------------------|
| curl https://127.0.0.1:443/clients | |
| curl https://centos-vm1.novalocal:443 | |
| open https://FQDN-OR-IP-OF-CHEF-SERVER | admin/p@ssw0rd1; Make sure iptables doesn't ban 443 port |
|-----------------------------------------------------------------------------------+----------------------------------------------------------|
| knife cookbook create apache-tutorial-1 -o ./ | |
| knife upload cookbooks cookbook-test | upload cookbook |
| knife node run_list add node1.example.com cookbook-test | add run_list of a cookbook to a given node |
| knife node run_list add centos187.osc.com 'recipe[don_cookbook1::testfile]' | add a recipe of a cookbook to a given node |
| knife node run_list remove ubuntu.dennyzhang.com 'recipe[fluig-os::conf_history]' | |
| knife node show dennyubuntu -r | show run_list |
|-----------------------------------------------------------------------------------+----------------------------------------------------------|
| chef-client | puppet agent |
| chef-client -l debug | |
| chef-client -i 3600 | poll every 3600 seconds for changes |
| chef-client -S https://XXX -K /etc/chef/chef-validator.pem | |

- The agents can be installed from the workstation using the knife tool that uses SSH for deployment, easing the installation burden.
** [#A] Linux install chef utility
https://docs.chef.io/install_omnibus.html

curl -L https://www.opscode.com/chef/install.sh | bash
*** DONE [#A] install chef facility with given version
CLOSED: [2016-05-05 Thu 16:45]
http://stackoverflow.com/questions/20205889/how-to-update-the-chef-client-version
(echo "version=12.4.1"; curl -L https://www.opscode.com/chef/install.sh) | sudo bash
** DONE [#A] Chef server migration/backup/restore from chef 11 to chef 12
CLOSED: [2015-04-22 Wed 16:53]
http://www.ameir.net/blog/archives/326-migrating-from-one-chef-server-to-another.html
http://docs.chef.io/server_backup_restore.html

- install new chef server

knife backup export -D ~/chef-backup/ -c ~/.chef/knife-orig.rb
knife backup restore -D ~/chef-backup -c ~/.chef/knife-new.rb
*** migration spchef
macs-MacBook-Air:.chef mac$ knife node list

macs-MacBook-Air:.chef mac$ knife backup restore -D ~/chef-backup
WARNING: This will overwrite existing data!
Do you want to restore backup, possibly overwriting exisitng data? (Y/N) y
=== Restoring clients ===
=== Restoring users ===
ERROR: Failed to create user[admin]: #; skipping
ERROR: Failed to create user[dennyadmin]: #; skipping
ERROR: Failed to create user[kungadmin]: #; skipping
=== Restoring nodes ===
Restoring nodes from /Users/mac/chef-backup/nodes/all-in-one-sp.json
Restoring nodes from /Users/mac/chef-backup/nodes/on-premise-deployment.json
Restoring nodes from /Users/mac/chef-backup/nodes/vapp_buxeo1mg2tlyoe1q1428409660944.json
Restoring nodes from /Users/mac/chef-backup/nodes/vapp_cg6b5hwpk56elgr31420578177412.json
Restoring nodes from /Users/mac/chef-backup/nodes/vapp_hjgyd06p8gc621vq1416950631623.json
Restoring nodes from /Users/mac/chef-backup/nodes/vapp_j920on7cdlpdb8m41417812176580.json
Restoring nodes from /Users/mac/chef-backup/nodes/vapp_n90igti0hp4za70j1420656472829.json
Restoring nodes from /Users/mac/chef-backup/nodes/vapp_yxd6ik7lv9xrapcd1418151775033.json
=== Restoring roles ===
=== Restoring data bags ===
=== Restoring environments ===
=== Restoring cookbooks ===
Restoring cookbook ["all-in-one"]
WARNING: * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
WARNING: The cookbooks: all-in-one exist in multiple places in your cookbook_path.
A composite version of these cookbooks has been compiled for uploading.

IMPORTANT: In a future version of Chef, this behavior will be removed and you will no longer
be able to have the same version of a cookbook in multiple places in your cookbook_path.
WARNING: The affected cookbooks are located:
vsftpd:
/Users/mac/chef-backup/cookbooks/vsftpd
/Users/mac/chef-backup/cookbooks/vsftpd-0.1.0
WARNING: * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
Uploading vsftpd [0.1.0]
Uploaded 1 cookbook.
macs-MacBook-Air:.chef mac$
*** TODO [#A] chef_server_url parameter need to be updated
#+BEGIN_EXAMPLE
root@kitchen-identity-sandbox:~# cat /etc/chef/client.rb
cat /etc/chef/client.rb
log_level :info
log_location STDOUT
node_name 'all-in-one-sp'
client_key '/etc/chef/client.pem'
chef_server_url 'https://104.131.157.119/organizations/digitalocean'
cache_type 'BasicFile'
no_lazy_load true
cache_options( :path => '/etc/chef/checksums' )
# TODO: improve later
ssl_verify_mode :verify_none
no_proxy 'no_proxy'root@kitchen-identity-sandbox:~#
#+END_EXAMPLE
*** web page: Migrating from one Chef server to another | ameir dot net
http://www.ameir.net/blog/archives/326-migrating-from-one-chef-server-to-another.html
**** webcontent :noexport:
#+begin_example
Location: http://www.ameir.net/blog/archives/326-migrating-from-one-chef-server-to-another.html
ameir dot net

Get your geek on

Home » Linux Luvin' » Migrating from one Chef server to another

Leave a comment Cancel reply

Your email address will not be published. Required fields are marked *

Name * [ ]

Email * [ ]

Website [ ]

[ ]
[ ]
[ ]
[ ]
[ ]
[ ]
[ ]
Comment [ ]

You may use these HTML tags and attributes:

ecosyste.ms

Data

Tools

Indexes

Applications

Experiments

Awesome

https://github.com/dennyzhang/challenges-chef

Awesome Lists containing this project

README