{"id":23122498,"url":"https://github.com/calgrimes/elasticsearch-aws-vm-ubuntu","last_synced_at":"2025-04-04T04:18:18.222Z","repository":{"id":237723741,"uuid":"795134299","full_name":"CalGrimes/ElasticSearch-AWS-VM-Ubuntu","owner":"CalGrimes","description":"Quick start setup for the ELK stack including ElasticSearch, Kibana, \u0026 Logstash on an Ubuntu VM hosted on AWS. Demo'd from a Widnows machine","archived":false,"fork":false,"pushed_at":"2024-10-14T11:45:45.000Z","size":1260,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-09T15:41:56.686Z","etag":null,"topics":["elasticsearch","elk","elk-stack","ubuntu"],"latest_commit_sha":null,"homepage":"","language":"HCL","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/CalGrimes.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-05-02T16:46:07.000Z","updated_at":"2024-10-14T11:45:49.000Z","dependencies_parsed_at":"2024-05-03T02:59:03.237Z","dependency_job_id":"c3dde618-7b67-47fa-bcad-1ac28796c9ca","html_url":"https://github.com/CalGrimes/ElasticSearch-AWS-VM-Ubuntu","commit_stats":null,"previous_names":["calgrimes/elasticsearch-aws-vm-ubuntu"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CalGrimes%2FElasticSearch-AWS-VM-Ubuntu","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CalGrimes%2FElasticSearch-AWS-VM-Ubuntu/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CalGrimes%2FElasticSearch-AWS-VM-Ubuntu/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CalGrimes%2FElasticSearch-AWS-VM-Ubuntu/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/CalGrimes","download_url":"https://codeload.github.com/CalGrimes/ElasticSearch-AWS-VM-Ubuntu/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247117762,"owners_count":20886440,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["elasticsearch","elk","elk-stack","ubuntu"],"created_at":"2024-12-17T07:27:45.205Z","updated_at":"2025-04-04T04:18:18.199Z","avatar_url":"https://github.com/CalGrimes.png","language":"HCL","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ELK Quick Start - AWS\n\n## Prerequisites\n\n- Permission to `manage` the following types of resources in your AWS Cloud Infrastructure tenancy: `vpcs`, `internet-gateways`, `route-tables`, `security-groups`, `subnets`, and `instances`.\n\n- Quota to create the following resources: 1 VPC, 1 subnet, 1 Internet Gateway, 1 route rule, and 1 compute instance.\n\n## Deploy Using the Terraform CLI\n\n### Clone the repository\nStart with a local copy of this repo.  You can make that with the commands:\n\n    git clone https://github.com/CalGrimes/ElasticSearch-AWS-VM-Ubuntu.git\n    cd ElasticSearch-AWS-VM-Ubuntu\n\n### Initialize the deployment\n\nWe now need to initialize the directory.  This makes the module aware of the AWS provider.  You can do this by running:\n\n    terraform init\n\nThis gives the following output:\n\n![](./images/terraform-init.png)\n\n### Modify the configuration\nCreate a `terraform.tfvars` file and add the following configuration to it:\n\n```hcl\naws_access_key=\"your_aws_access_key\"\naws_secret_key=\"your_aws_secret_key\"\nregion=\"us-west-2\"\nmy_ip=\"your_public_ip\"\n```\n\n### Deploy the module\nNow for the main attraction.  Let's make sure the plan looks good:\n\n    terraform plan  -var-file=\"terraform.tfvars\"\n\nThat gives:\n\n![](./images/terraform-plan.png)\n\nIf that's good, we can go ahead and apply the deployment with:\n\n    terraform apply -var-file=\"terraform.tfvars\"\n\nYou'll need to enter `yes` when prompted.  Once complete, you'll see something like this:\n\n![](./images/terraform-apply.png)\n\nWhen the apply is complete, the infrastructure will be deployed, but cloud-init scripts will still be running.  Those will wrap up asynchronously. So, it'll be a few more minutes before your cluster is accessible.  Now is a good time to get a coffee.\n\n### Configuring Elasticsearch and Kibana\nOnce the module is deployed and you have waited for a few minutes, you can connect to the ELK VM and configure Elasticsearch and Kibana.\n\nThe output will show the ELK VM public IP and the generated ssh private key. The private key will be saved locally as ELK_private_key.pem. You can use this key to SSH into the ELK VM.\n\nNow let's build SSH tunnels for each product of ELK:\n\n`ELK_VM_public_IP = 13.40.13.172`\n\nCreate an SSH tunnel for ports `9200` and `5601` with the following command:\n\n```\nssh -i ELK_private_key.pem -L 9200:localhost:9200 -L 5601:localhost:5601 ubuntu@\u003cELK_VM_public_IP\u003e\n```\n\nOnce you are connected to the ELK VM. Ensure the cloud-init scripts have completed by checking status of elastic and kibana services with the following command:\n```\nsystemctl status elasticsearch kibana\n```\n![](./images/elastic-kibana-status.png)\n\nNow you need to configure the network host in the Elasticsearch and Kibana configuration files. Run the following command to open the elasticsearch configuration file:\n\n```\nsudo vi /etc/elasticsearch/elasticsearch.yml\n```\n![](./images/elastic-config.png)\n\nAdd the following line to the configuration file:\n\n`network.host: 0.0.0.0`\n\nSave and exit the file.\n\nNow open the kibana configuration file:\n```\nsudo vi /etc/kibana/kibana.yml\n```\n![](./images/kibana-config.png)\n\nAdd the following line to the configuration file:\n`server.host: \"0.0.0.0\"`\n\nSave and exit the file.\n\nNow restart the Elasticsearch and Kibana services with the following command:\n\n```\nsudo systemctl restart elasticsearch kiabana\n```\n\n\n\n\n### Connect to Elasticsearch and Kibana\nNow you can browse to (http://localhost:9200) for Elasticsearch, and (http://localhost:5601) for Kibana.\n\n![](./images/elasticsearch.png)\n\nTo fetch your enrollment token, you can run the following command:\n\n```\nsudo /usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token --scope kibana\n```\n\nPaste the enrollment token and click 'Configure Elastic'\n\n![](./images/kibana-verification.png)\n\nNext, collect the Kibana verfication code by running the following command:\n\n```\nsudo /usr/share/kibana/bin/kibana-verification-code\n```\n\nNow wait for Elastic to set everything up. \n![](./images/starting-elastic.png)\n\nYou'll then be presented with a login screen. The superuser is elastic. \n![](./images/elastic-login.png)\nTo retrieve the password, you can run the following command:\n\n```\nsudo /usr/share/elasticsearch/bin/elasticsearch-reset-password -u elastic\n```\n\nAt last you can login to Kibana with the superuser credentials and start exploring the dashboard.\n![](./images/dashboard.png)\n\n## View the Cluster in the Console\nYou can also login to the web console [here](https://eu-west-2.console.aws.amazon.com/ec2/home?region=eu-west-2#Instances:) to view the IaaS that is running the cluster.\n\n![](./images/console.png)\n\n### Destroy the Deployment\nWhen you no longer need the deployment, you can run this command to destroy it:\n\n    terraform destroy\n\nYou'll need to enter `yes` when prompted.\n\n![](../images/simple/terraform-destroy.png)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcalgrimes%2Felasticsearch-aws-vm-ubuntu","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcalgrimes%2Felasticsearch-aws-vm-ubuntu","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcalgrimes%2Felasticsearch-aws-vm-ubuntu/lists"}