{"id":13432120,"url":"https://github.com/jvandevelde/distributed-playground","last_synced_at":"2025-03-16T23:30:40.763Z","repository":{"id":75145494,"uuid":"54290366","full_name":"jvandevelde/distributed-playground","owner":"jvandevelde","description":"Distributed service playground with Vagrant, Consul, Docker \u0026 ASP.NET Core","archived":false,"fork":false,"pushed_at":"2016-04-16T22:48:34.000Z","size":164,"stargazers_count":42,"open_issues_count":0,"forks_count":4,"subscribers_count":6,"default_branch":"master","last_synced_at":"2024-10-27T11:48:40.063Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jvandevelde.png","metadata":{"files":{"readme":"README.adoc","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2016-03-19T22:41:19.000Z","updated_at":"2024-10-02T16:15:36.000Z","dependencies_parsed_at":"2023-06-05T14:00:39.379Z","dependency_job_id":null,"html_url":"https://github.com/jvandevelde/distributed-playground","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jvandevelde%2Fdistributed-playground","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jvandevelde%2Fdistributed-playground/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jvandevelde%2Fdistributed-playground/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jvandevelde%2Fdistributed-playground/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jvandevelde","download_url":"https://codeload.github.com/jvandevelde/distributed-playground/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243949777,"owners_count":20373651,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-31T02:01:08.745Z","updated_at":"2025-03-16T23:30:40.758Z","avatar_url":"https://github.com/jvandevelde.png","language":"Shell","funding_links":[],"categories":["Sample Projects","例子"],"sub_categories":["Workflow","工作流"],"readme":"= Getting Started\n:imagesdir: docs/images\nifdef::env-github[]\n:note-caption: :information_source:\n:tip-caption: :bulb:\n:warning-caption: :warning:\n:important-caption: :exclamation:\nendif::[]\n\n== Contents\nlink:README.adoc[Overview (This page)]\n\nlink:docs/logging.adoc[Logging System Details]\n\n== Overview\nThis repository contains utilizes the following stack to create a distributed\nservice environment that could be utilized to experiment with micro-services,\nASP.NET Core and any other tech stacks that require some part of the stack\nalready available.\n\n1. *https://www.vagrantup.com/[Vagrant]* - Virtual machine creation, provisioning \u0026 orchestration\n2. *https://www.consul.io/[Consul]* - Service registry \u0026 discovery platform\n3. *https://www.docker.com/[Docker]* - Service hosting environment allowing for easy deployment of multi-instance\nservices\n4. *https://get.asp.net/[ASP.NET 5/Core]* - A demo service is written using ASP.NET Core and built as a Docker\nimage automatically as part of the Vagrant provisioning of the `docker-1` \u0026 `docker-2`\nmachines. This image is then used to create several instances of the service on each\nhost machine.\n5. *https://github.com/eBay/fabio[Fabio HTTP Router]* - Front-end service router to make it easy to access services\nvia a specific address/port. Each request is automatically redirected\nand balanced across all available service instances, wherever they are actually located\n+\n.Fabio \u0026 Consul\n****\nFabio utilizes Consul as a backend to configure itself and immediately have new instances\nof a service available for request routing (vs needing to configure a load balancer each\ntime an new service instance is available or moves)\n****\n\n=== Environment Diagram\n\nimage::system_overview.png[]\n\n== Required Software\n\n1. https://www.vagrantup.com/downloads.html[Vagrant]\n2. https://www.virtualbox.org/wiki/Downloads[Virtual Box]\n+\nCurrently VirtualBox is the only provider supported in the repository's Vagrantfile.\nThis may be expanded in the future to work with Hyper-V/AWS/Azure/Digital Ocean\n+\nWARNING: If installing VirtualBox on a Windows 8+ machine, you will need to make sure\nto disable and/or un-install Hyper-V before being able to use VirtualBox. Un-installing\nthe Hyper-V Windows feature should not remove any existing Virtual Machine images and\nis fairly quick to do. This is one reason I would like to build Hyper-V support into\nthe Vagrantfile in the future.\n\n== Quickstart\n\n1. Add the `ubuntu/trusty64` box to your Vagrant installation by running\n+\n`vagrant box ubuntu/trusty64`\n2. Clone this repository\n3. From the repository location open a command prompt and run\n+\n`vagrant up`\n\nOnce the machines have been created and provisioned by Vagrant, you should be able to do\nthe following:\n\n- The demo Hello World service can be accessed via the Fabio Router address on port 9999.\nMaking multiple requests to this address will result in those requests being automatically\nbalanced across all running instances of the service container across the Docker hosts.\n*The service response includes which host \u0026 instance returned the result*\n+\nhttp://172.28.128.31:9999\n+\n- To view the Fabio HTTP Router Web UI to view all registered service instances:\n+\nhttp://172.28.128.31:9998\n+\n- Access the Consul Web UI to view all available services\n+\nhttp://172.28.128.30:8500/ui/\n- View all services and instances registered with Consul by using\nthe Consul REST API(s) on any of the `ext-n` machines\n+\n.Consul REST API examples\n|===\n|Task|URL\n\n|Query for all registered services\n(Can query all consul instances)\n|http://172.28.128.10:8500/v1/catalog/services\nhttp://172.28.128.11:8500/v1/catalog/services\nhttp://172.28.128.12:8500/v1/catalog/services\n\n|Query details of a specific service\n|http://172.28.128.11:8500/v1/catalog/service/demosvc.1\n|===\n+\n- Access the ElasticSearch REST APIs directly running on the `ext-1`, `ext-2`, `ext-3` nodes\n+\n.ElasticSearch REST API\n|===\n|Task|URL\n\n|Query details of a specific ES node\n|http://172.28.128.10:9200\n\n|Query cluster status\n(Can query all consul instances)\n|http://172.28.128.10:9200/_cluster/health\nhttp://172.28.128.11:9200/_cluster/health\nhttp://172.28.128.12:9200/_cluster/health\n|===\n+\n- Two ElasticSearch Management Plugins are also available that make viewing cluster information\nmuch simpler via their Web UIs\n+\n.Accessing ElasticSearch Plugins\n|===\n|Plugin|URL\n\n|ElasticSearch HQ\n|http://172.28.128.10:9200/_plugin/hq\n\n|ElasticSearch HEAD (much older)\n|http://172.28.128.10:9200/_plugin/head\n|===\n+\n- SSH into the Docker hosts to inspect Docker container status\n+\n`vagrant ssh docker-1`\n+\nOnce inside the Docker host, you can query Docker via\n+\n`docker ps` (view all running containers)\n+\n`docker ps -a` (view all containers)\n\n== Minimal startup\n\nA minimal startup of the environment requires at least one Consul server, a\nDocker host to run services on and the Fabio HTTP router.\n\nTo create and/or start only these Virtual machines, you can run:\n\n- run `vagrant up ext-1 docker-1 httprouter`\n\n== Machine Descriptions\n\n1. ext-1, ext-2, ext-3\n+\nThese machines host clusters outside of Docker. Currently only the Consul\ncluster is hosted on these virtual machines. Doing it this way may not be strictly\nnecessary but in the future *Docker Swarm* may be introduced which also requires a\nservice registry to be available and I'm not sure if it can be hosted on the Docker\nhosts being placed into a swarm.\n+\nElasticSearch master nodes may also be hosted on these machines in the future\n\n2. docker-1, docker-2\n+\nThese machines host Docker and automatically get provisioned with:\n+\n- *Registrator*\n   +\n  Used to automatically register any running Docker containers with the Consul cluster\n  making them available to be discovered\n  - *Consul Client*\n    +\n  Used to communicate with the main Consul cluster hosted on `ext-1, ext-2 \u0026 ext-3`\n\nNOTE: Registrator communicates with Consul via this container instead of the main Consul cluster\n  nodes. Because the cluster server nodes are responsible for managing cluster state/health, it is recommended\n  to use lighter weight Consul agent in client mode to perform service registration/discover requests\n  which are then forwarded to the server nodes.\n\n== Debugging\n\n=== Vagrant \u0026 VirtualBox Guest Additions Versioning\n\nThe `ubuntu\\trusty64` Vagrant boxes do not always have the most current VirtualBox Guest additions installed\non them which causes a warning message to be displayed when originally creating a Vagrant machine (ie: during 'vagrant up').\nThis message indicates that this condition could also cause the setup of Vagrant's synced folders to fail.\n\nThis has occasionally happened to me after upgrades of Docker or Vagrant. While the machines still work, Vagrant breaks out of\nprovisioning any remaining machines when it can't setup synced folders.\n\nYou can address this in a couple of ways:\n\n1. Manually install the correct version of the Guest Additions and update your\nlocal box definition with `vagrant package` See https://gist.github.com/fernandoaleman/5083680[this gist] for more details\n+\n*Note* This isn't a great solution because you would need to perform this for every box update you want might want to use\n2. Install the 'vagrant-vbguest' plugin that ensures the latest Virtual Box Guest Addtions package gets installed\non each Vagrant machine as it's being created. You can do this by running the following command:\n+\n`vagrant plugin install vagrant-vbguest`\n+\nYou can update this plugin with:\n+\n`vagrant plugin update vagrant-vbguest`\n+\nFor more information, visit the project repository on https://github.com/dotless-de/vagrant-vbguest/[Github]\n\n=== Windows 10\n\nWARNING: I've run into problems individually updating and running the latest Docker Toolbox, Vagrant and Virtual Box\non the latest Windows 10 Update (Version 1511 - released March 8th, 2016).\nDoing so may result in problems with Vagrant or the Docker Toolbox Quickstart Terminal not starting up properly due\nto \"Network interface issues\"\nThis appears to have something to do with the discussion found here:\nhttp://stackoverflow.com/questions/33725779/failed-to-open-create-the-internal-network-vagrant-on-windows10\n\nI was not able to get all 3 tools to work together until I did the following:\n\n1. Uninstall Docker Toolbox\n2. Uninstall Virtual Box\n3. Download and install an older version (10.2.2) of Docker Toolbox\n+\nhttps://github.com/docker/toolbox/releases/download/v1.10.2/DockerToolbox-1.10.2.exe\n+\nThis version of Docker Toolbox will install the following software/versions\n+\n.ToolBox 1.10.2 Software\n|===\n|Package|Version\n|Docker\n|v1.10.2\n|VirtualBox\n|v5.0.14\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjvandevelde%2Fdistributed-playground","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjvandevelde%2Fdistributed-playground","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjvandevelde%2Fdistributed-playground/lists"}