{"id":25668410,"url":"https://github.com/skalogs/skalogs-bundle","last_synced_at":"2025-10-15T19:15:35.269Z","repository":{"id":132728899,"uuid":"111741444","full_name":"skalogs/skalogs-bundle","owner":"skalogs","description":"Open Source data and event driven real time Monitoring and Analytics Platform","archived":false,"fork":false,"pushed_at":"2020-05-06T12:27:14.000Z","size":169,"stargazers_count":14,"open_issues_count":1,"forks_count":8,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-04-23T00:18:25.112Z","etag":null,"topics":["ansible","cattle","docker","elasticsearch","elk-stack","grafana","itoa","kafka","kibana","kubernetes","logmanagement","logstash","prometheus","rancher","siem","zookeeper"],"latest_commit_sha":null,"homepage":"https://skalogs.com","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/skalogs.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-11-22T23:15:03.000Z","updated_at":"2023-08-21T20:34:55.000Z","dependencies_parsed_at":null,"dependency_job_id":"4d3d56dc-5b57-4421-b8fa-a80ebf4dcde0","html_url":"https://github.com/skalogs/skalogs-bundle","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/skalogs/skalogs-bundle","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/skalogs%2Fskalogs-bundle","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/skalogs%2Fskalogs-bundle/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/skalogs%2Fskalogs-bundle/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/skalogs%2Fskalogs-bundle/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/skalogs","download_url":"https://codeload.github.com/skalogs/skalogs-bundle/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/skalogs%2Fskalogs-bundle/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279104921,"owners_count":26104633,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-15T02:00:07.814Z","response_time":56,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ansible","cattle","docker","elasticsearch","elk-stack","grafana","itoa","kafka","kibana","kubernetes","logmanagement","logstash","prometheus","rancher","siem","zookeeper"],"created_at":"2025-02-24T10:33:10.155Z","updated_at":"2025-10-15T19:15:35.234Z","avatar_url":"https://github.com/skalogs.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Project ansible to manage project on Rancher\n\n## Foreword\n\nIf you are not familiar with Rancher please read the official documentation http://docs.rancher.com/rancher/latest/en/.\n\nRancher is used to create segregated environments for each project instances.\n\nEach environment should exist as \"environments\" in rancher (example: my-uat-project my-production-project ).\nEach rancher environment is segregated at network level via Docker Overlay and IPSec tunnels (managed by rancher itself).\n\nOnly trusted ops can connect to the machines, unix accounts are managed by ansible.\n\nDeveloppers will not be allowed to access rancher by itself, rancher master will only be accessible to ops for the moment.\nEnd users will have tools in their project to see their application logs \u0026 metrics (elk, prometheus).\n\n### Machine requirements\n\nAt the moment this set of playbooks is designed to run on ubuntu. We recommend using latest LTS (16.04 at times of writing).\n\nEach host (including the master) must have at least the following requirements :\n* 2 vcpu\n* 4 Go ram\n* 2 disks with one for the system with 12Go (8Go for / + 4Go for swap) and one for /var/lib/docker with 8Go.\n\n### Example for test environments\n\n3 hosts per Rancher environment :\n* one for monitoring (ELK, Prometheus)\n* one for the cluster itself\n\n## Playbooks documentation\n\n### Requirements\n\n* Ansible \u003e= 2.3\n* Vagrant / Virtualbox (optional) if you want to run this on your own machine\n* Configure collectdata/inventory with the list of Host\n* Configure collectdata/all/Rancher with the ip of rancher master and docker_registries (if needed)\n\n### Roles dependencies\n\nFirst you need to fetch roles with ansible galaxy :\n```\nansible-galaxy install -r requirements.yml\n```\n\n### Rancher platforms\n\nEach Rancher platform will have its own configuration (inventory + group_vars + hosts_vars).\nBy default wrappers will use the production configuration.\n\nYou can use other configurations by changing inventory from the command line (everything is relative to the path of the inventory file).\n```\n./ansible-playbook_wrapper configure_host.yml -K -i path/to/inventory\n\n```\n\n### SSH configuration\n\nCopy the file `config_ssh.template` to `config_ssh` in this local folder,\nthen edit `config_ssh` to configure it. The user must match a user declared\nin ssh_users list in `\u003crancher_cluster_name\u003e/group_vars/all/vars`\n\nTo prevent typing sudo password when using ansible, create for each cluster\nthe file `\u003crancher_cluster_name\u003e/group_vars/all/private` with the following content:\n\n```\nansible_sudo_pass: your_sudo_password\n```\n\nWhere your_sudo_password is the password declared in ssh_users list in `\u003crancher_cluster_name\u003e/group_vars/all/vars`\n\nNote that if you are using this mechanism, it will always be used, even if you are using `-k` or `-K`\n\n### Vault\n\nAdd the vault password in `ansible/.ansible_vault_pass`\n\n### Workflow to setup rancher from scratch\n\n* run configure_host playbook\n* run create_master playbook\n* run create_project playbook (everytime you need to setup a new env)\n\n### Configure the Machine\nUse the configure_host.yml\n\nYou can bootstrap the machine for the first time with the following command\n\n``` ./ansible-playbook_wrapper configure_host.yml -u your_account -Kk ```\n\nThis ansible does the following :\n\n* Configures the kernel\n* Installs Docker\n* Sets machine hostname\n* Configures ssh (disable root login, enforce auth by key)\n* Removes the former ops account\n\n###  Create a Master\n\nCreate a tag for the new Master, for example\n```\n[rancher-master]\nhost-1\n```\n\nConfigure the create_master.yml for [rancher-master]\n\nLaunch the playbook\n```\n./ansible-playbook_wrapper create_master.yml -K\n```\n\nThis playbook will create a file apiKey in {{ inventory_dir }}/group_vars/all/apikey. This file contains the rancher api key and secrets.\nWe HIGHLY recommend to vault this file using ansible vault.\n\n### Create a project\n\nCreate a tag for your project into the inventory for example\n```\n[my-first-environment-project]\nhost-2\nhost-3\n```\n\nThe first host of the tag will become the \"tools\" host (running elk, etc...).\n\nLaunch the playbook\n```\n./ansible-playbook_wrapper create_project.yml -K -e \"NAME_PROJECT=my-first-environment-project\"\n```\n\nThis ansible does the following :\n\n* Creates a project into RANCHER\n* Creates \"API KEY ENVIRONMENT\" into rancher and write into group_vars/{{NAME_PROJECT}}\n* Adds Host into the project\n* Installs some stacks : Janitor\n\n### Create the project \"collect-data\"\n\nConfigure collectdata/inventory\n```\n[collect-data]\nhost-2\nhost-3\n```\nLaunch the playbook\n```\n./ansible-playbook_wrapper create_collect_data.yml -K -e \"NAME_PROJECT=collect-data\"\n```\n\nThis ansible does the following :\n\n* Creates a project into RANCHER\n* Creates \"API KEY ENVIRONMENT\" into rancher and write into group_vars/{{NAME_PROJECT}}\n* Adds Host into the project\n* Installs stacks :\n     * Janitor\n     * Elasticsearch\n     * Zookeeper\n     * Kafka\n     * Importer\n     * Kibana\n     * Packetbeat-dashboard\n     * Metricbeat-dashboard\n     * Elk-monitoring (to monitor your cluster)\n     * Prometheus\n     * Endpoints\n\n### Create the project \"demo\"\n\nThis project is a demo for collect data.\n\nLaunch the playbook\n```\n./ansible-playbook_wrapper create_project_demo.yml -K -e \"NAME_PROJECT=demo\"\n```\n\n* Installs stacks :\n    * demo-todo : application nodejs\n    * demo-petclinic : application springboot\n    * packetbeat : collect network informations from Host\n    * metricbeat : collect metric informations from Host\n\n## Disaster recovery\n\nIf for some reason the host running rancher-master dies, application will remain up and running, so there is no impact.\nAfter running troubleshooting steps, if you don't have any clue we recommend you to wipe the master machine and spawn a new master with \"create_master.yml\" playbook and run mysql restore script (see above).\n\n\n### Quick cleanup procedure (optional)\nThis step can be skipped if you spawn a new master.\n\nRun the following commands on your rancher-master\n```\nsudo service docker stop\nrm -rf /var/lib/docker/*\nrm -rf /var/lib/mysql/*\nsudo service docker start\n```\n\n### Make sure master is configured properly\n```\n./ansible-playbook_wrapper create_master.yml -K\n```\n\n### Restore mysql\n\nOn the master enter into the container itself and run the restore script\n```\nsudo docker exec -ti mysql-backup-s3 bash\n./restore.sh\n```\n\nThis will restore the latest backup find on s3.\n\nIf you want to restore a specific backup you can do\n```\nID_BUCKET_RESTORE=2016-07-21T133544Z.dump.sql.gz ./restore.sh\n```\n\n## Upgrading Rancher\n\nTo upgrade Rancher you just need to change the rancher_version version in group vars.\nExample: in production/group_vars/all/vars\n```\nrancher_version: \"v1.6.21\"\nrancher_agent_version: \"v1.2.11\"\n```\n\nWe highly recommend to stick to a specific version for production Rancher environment to make sure everything is repeatable.\n\nThen just run again the create_master.yml playbook.\nIt will upgrade rancher-master smoothly.\nWhen rancher-master is up again it will contact environments host agents and update it if necessary, this operation is done by Rancher itself.\n\nDocker version\n--------------\nDefault docker version is for Ubuntu 18.04 version.\nIf you want to get this role working on ubuntu 16.04 you may override docker_version variable to an older version :\n\n```\ndocker_version: \"18.02.0~ce-0~ubuntu\"\n```\nSuggested docker version and docker-compose version for Ubuntu 18.04.1 (bionic):\n\n```\ndocker_version: 5:18.09.0~3-0~ubuntu-bionic\ndocker_compose_version: 1.23.1\n```\nSee:\n\ndocker releases: https://github.com/docker/docker-ce/releases\n\ndocker-compose releases: https://github.com/docker/compose/releases/\n\nReference to localhost present in /etc/resolv.conf\n--------------------------------------------------\nSome Linux distributions will run a local DNS cache server like dnsmasq. If this is the case, the nameserver entry in /etc/resolv.conf will point to 127.0.0.1 (localhost). This configuration is re-used when running Docker containers, but inside a container you cannot reach dnsmasq on 127.0.0.1. Since rancher/agent:v1.2.7, the agent will fail to register and log:\n\n```\nERROR: DNS Checking loopback IP address 127.0.0.0/8, localhost or ::1 configured as the DNS server on the host file /etc/resolv.conf, can't accept it\n```\nTo fix this, you can either specify DNS servers for Docker or disable dnsmasq. Instructions on both options are provided in the [Docker documentation](https://docs.docker.com/install/linux/linux-postinstall/#dns-resolver-found-in-resolvconf-and-containers-cant-use-it)\n\nThis can be done by overriding docker_opts in group_vars/all/.\n```\ndocker_opts: \"--dns 8.8.8.8 --dns 8.8.4.4\"\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fskalogs%2Fskalogs-bundle","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fskalogs%2Fskalogs-bundle","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fskalogs%2Fskalogs-bundle/lists"}