{"id":20184842,"url":"https://github.com/gerhalt/mining-camp","last_synced_at":"2025-04-10T06:06:06.293Z","repository":{"id":46815278,"uuid":"100408184","full_name":"gerhalt/mining-camp","owner":"gerhalt","description":"Easy automated configuration and deployment of Minecraft servers on AWS spot instances, featuring automatic backups and restoration using S3.","archived":false,"fork":false,"pushed_at":"2021-09-23T23:21:48.000Z","size":90,"stargazers_count":45,"open_issues_count":4,"forks_count":6,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-04-10T06:05:52.240Z","etag":null,"topics":["ansible","aws","ec2","ec2-spot","minecraft","minecraft-server","python","route53","s3","server","spot-instances","terraform"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/gerhalt.png","metadata":{"files":{"readme":"README.adoc","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-08-15T18:44:23.000Z","updated_at":"2024-05-27T19:16:59.000Z","dependencies_parsed_at":"2022-07-22T19:02:44.902Z","dependency_job_id":null,"html_url":"https://github.com/gerhalt/mining-camp","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gerhalt%2Fmining-camp","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gerhalt%2Fmining-camp/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gerhalt%2Fmining-camp/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gerhalt%2Fmining-camp/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/gerhalt","download_url":"https://codeload.github.com/gerhalt/mining-camp/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248166925,"owners_count":21058481,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ansible","aws","ec2","ec2-spot","minecraft","minecraft-server","python","route53","s3","server","spot-instances","terraform"],"created_at":"2024-11-14T03:09:26.793Z","updated_at":"2025-04-10T06:06:06.251Z","avatar_url":"https://github.com/gerhalt.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"Mining Camp\n===========\n:toc:\n:toc-placement: preamble\n:toclevels: 3\n\nEasy automated configuration and deployment of Minecraft servers on AWS spot\ninstances, with features such as instance shutdown monitoring and automatic\nbackups and restorations using S3.\n\nimage::https://i.imgur.com/jvJzU6v.png[Mining Camp]\n\n== Introduction\n\nAmazon sells spare EC2 instances that aren't currently reserved, called \"spot\ninstances\", for a fraction of the cost of a reserving them. If you can mitigate\ntheir volatility risk, they're an excellent deal and powerful enough to host\ndemanding game servers.\n\n_Advantages:_\n\n* More control over everything. SSH into the box, troubleshoot, adjust anything\nyou like on the fly.\n* Better bang for your buck. More powerful systems than most hosting services\noffer.\n* Pay for exactly what you use. Spot instances are billed by the second, so if\nyour server isn't going to be used for a few days, or even overnight, just turn\nit off.\n* Linux-based. This might be a disadvantage, depending on your comfort level.\nWindows instances are not only more of a pain, they're also far more expensive.\n* Works with any AWS region, so you can minimize latency for your player base.\n\n_Disadvantages:_\n\n* Spot instances can be terminated at any time. This is mitigated by both the\nemergency shutdown monitoring script, but in practice reasonable bids for spot\ninstances result in excellent uptime.\n* Requires initial work to setup. This isn't a push-button solution, but once\nyou're done with the setup day-to-day interactions are extremely easy.\n\n=== Sample Cost Breakdowns\n\nI've provided a couple samples of pricing based on different instance types\nbelow. When deciding what to use, cross reference your requirements against\nthe https://aws.amazon.com/ec2/spot/pricing/[spot instance pricing] and the\nhttps://aws.amazon.com/ec2/instance-types/[EC2 instance types]. Be aware that\nspot instance prices can differ not only between AWS regions, but between\navailability zones as well.\n\nAll calculations below are based on an instance in `us-east-1`, and assume a\n31-day month.\n\nNOTE: AWS charges for absolutely everything. It's critical that you monitor\nyour bill and be aware of where you're incurring charges.\n\n==== i3.large\n\nAn _i3.large_ is a quite powerful instance with ~16GB of RAM and a 500GB NVMe\ndrive, meaning an EBS (beyond the 8GB root volume) isn't required. Price\nestimates for the various moving pieces:\n\n* Current spot instance price for an _i3.large_ is $0.0371 per hour at present.\n* An 8GB EBS is used as the root volume for the instance, at $0.10 per\n    GB-month, or roughly $0.00013 an hour. This is deleted on instance shutdown,\n    so no charges are incurred when the server is off.\n* Elastic IPs are free when assigned to a running instance, and $0.005 per hour\n    when unassigned.\n* AWS data transfer is pretty expensive, at $0.09 per GB. I use about 1.4GB per\n    day, but my server has a lot of time when it's completely idle. For the\n    sake of round numbers, 2GB of bandwidth a day winds up being $0.0075 an\n    hour. This cost is very subjective.\n* S3 is actually quite cheap, especially with lifecycle management rules\n    cleaning up old backups. 20GB of combined server archives and rotating\n    backups is $0.023 per GB, $0.46 a month, or $0.0006 an hour.\n\nFrom these numbers, when my server is running, I'm paying roughly *$0.045* an\nhour or *$33.73* a month to run it full-time. When my server is off, but I want\nto preserved my backups and elastic IP address, I'm paying *$4.17* a month. Just\npreserving backups is only *$0.46*, which is very reasonable.\n\n==== t2.medium\n\nOptions with less RAM more be suited to smaller loads. One example is a\n_t2.medium_, which has only 4GB of RAM and no SSD, which means it requires an\nEBS volume. The upside is that it only costs $0.0134 an hour. If we use a 30GB\nephemeral EBS volume that's deleted on instance termination, that adds roughly\n$0.0004 an hour.\n\nUsing this less powerful, less expensive instance brings considerable savings:\n*$0.0255* an hour, or *$18.97* when run for an entire month.\n\n== Setup\n\nStart by checking out this repository, and navigating to your checkout-out\ndirectory. Sample bash commands provided are from the root directory unless\notherwise noted.\n\nYou'll need to download and install the following requirements to get started\n(versions I've tested with are in parentheses):\n\n* `pip` (20.1)\n* `terraform` (0.12.24)\n\nYou'll also need:\n\n* An appropriate version of the JDK/JVM to run the server locally at least once,\n  allowing it to generate the necessary configuration files.\n* A fully configured AWS developer account; you'll be connecting to it to set\n  up the required infrastructure.\n\n=== Configuration\n\n==== Credentials\n\nYou'll need your AWS credentials available for most of these operations, under\nthe `minecraft` profile. `~/.aws/credentials` will look like:\n\n```\n[minecraft]\naws_access_key_id = \u003cyour_access_key_here\u003e\naws_secret_access_key = \u003cyour_secret_key_here\u003e\n```\n\nTo tell Ansible to use this profile, you can do one of the following:\n\n* `export AWS_PROFILE=minecraft` in the shell you'll be running the Ansible\n    commands in.\n* Prefix each command with `AWS_PROFILE=minecraft`, which sets the environment\n    for that particular invocation.\n* Change the `minecraft` section title to `default` (or duplicate it). This\n    will cause those credentials to be tried by default.\n\n==== SSH Key Pair\n\nYou'll need a key pair for accessing your instance. Generate a public-private\nkey pair. As an example, you can do this with `ssh-keygen`:\n\n```\nssh-keygen -t rsa -b 4096 -C \"AWS\"\n```\n\nIn the EC2 console, select _Import Key Pair_ on the\n_NETWORK \u0026 SECURITY -\u003e Key Pairs_ page. Upload your public key, and name it\n\"aws-public\". The launch configuration Terraform creates includes this key,\nallowing SSH access to Ansible (and for troubleshooting!)\n\n==== Instance Access\n\nYou'll need to _choose one of the following methods_ to access your instance:\n\n* Use the public IP automatically assigned to your spot instance. This\n  requires no setup on your part, but the address will be different each time\n  the instance is started. The IP of your instance can be found under\n  _EC2 → Instances_ in the \"IPv4 Public IP\" column.\n* Use an elastic IP. This associates a known IP with your\n  instance each time it's started. Setup for this must be done by hand.\n  Elastic IPs also incur a cost while they're not in use, as well as\n  additional charges if you make too many assignments per month.\n  **(DEPRECATED)**\n* Use Route 53 to host an owned domain or subdomain, pointed at your server.\n  Requires a domain and minor manual setup with your registrar.\n\n===== via Elastic IP (DEPRECATED)\n\nYou'll need to create an elastic IP for association with your instance,\nproviding a convenient public-facing IP. In the AWS console, do the following:\n\n1. Enter the EC2 service.\n2. Click on _Elastic IPs_, under the _NETWORK \u0026 SECURITY_ menu on the left-hand\nside of the screen.\n3. Click _Allocate new address_.\n4. Leave the scope as \"VPC\", and click close.\n5. You should see your new elastic IP in the list. Save the _Allocation ID_ for\nlater use during the setup.\n\nOnce a server has been spun up, this elastic IP will be attached to it.\nAn allocated elastic IP is included in the price of a running instance, but\nyou will be billed for any unassigned EIPs by the hour. For this reason, if\nyou plan to stop your Minecraft server for long periods of time, be sure to\ndelete your EIPs and create new ones when you're ready to begin hosting again.\n\n===== via Domain\n\nProceed through the rest of the setup, specifying a server hostname. Once\nyou've applied your Terraform config, return to do this section and do the\nfollowing:\n\n1. Enter the AWS console, and navigate to the _Route 53_ service.\n2. Select the hosted zone for the hostname you chose.\n3. You should see an `NS` record with four hosts.\n4. Add a corresponding `NS` record to your domain for each host. I leave this\n  as an exercise for the reader.\n5. Now, when your server is booted, it will automatically add an `A` record\n  with a short TTL pointed at your server's public IP.\n\n==== Virtual Environment \u0026 Python Requirements\n\nUsing pip, install the necessary Python 2.7 requirements. I recommend using\nhttps://virtualenv.pypa.io/en/stable/[virtualenv] and\nhttps://pypi.python.org/pypi/virtualenvwrapper/[virtualenvwrapper]. Running the\nfollowing installs Ansible, the AWS command-line interface, and libraries\nrequired for interacting with AWS programmatically.\n\n```\n$ mkvirtualenv minecraft\n(minecraft) $ pip install -r requirements.txt\n```\n\n==== Minecraft Server Archive\n\nYou'll need to create a Minecraft server archive to be pulled onto your\ninstance each time the box is spun up. In this example, I'll be creating an\narchive for my Feed the Beast server named `daftcyborg`.\n\n```\n$ # Create a base directory named after your server name\n$ mkdir daftcyborg\n$ cd daftcyborg\n\n$ # Get your base server pack. In my case, I've already downloaded the FTB server\n$ ls\nFTBRevelationServer_1.0.0.zip\n$ unzip FTBRevelationServer_1.0.0.zip\n\n$ # Install the server requirements, if the pack requires it\n$ sh ./FTBInstall.sh\n\n$ # Create and populate the bootstrap script - mining camp will use this to\n$ # launch your server. Copy from _utitilies/_, and update with memory limits\n$ # and the `.jar` you're using.\n$ cp \u003crepo-checkout-directory\u003e/utilities/server-start.sh .\n\n$ # Launch the server. You'll need to do this twice, once to create the\n$ # eula.txt and once to generate the base\n$ sh ./server-start.sh\nMissing eula.txt. Startup will fail and eula.txt will be created\nMake sure to read eula.txt before playing!\nTo continue press \u003center\u003e\n```\n\nOpen `eula.txt`, and agree (or don't) to the terms and conditions.\n\nLaunch the server again, and wait for it to complete. This will generate the\nworld base, and any settings and properties files necessary. Quit the server,\nand do the following as desired:\n\n* Remove the `world` directory, which is the world directory name used by\ndefault and which will (assuming you update the `server.properties` file) be\nnamed differently when your server is run.\n* Edit `server.properties` as desired. It is important that the _server-port_\nbe left as _25565_, otherwise you'll need to adjust the Terraform\nconfiguration. Fields I recommend changing are _level-name_, _level-seed_, and\n_motd_.\n* Add yourself and any other players desired to `ops.json`.\n* Update `server-icon.png` to a custom icon.\n\nCopy server.properties to `ansible/files/server.properties`, which Ansible will\ninstall every time over the top of the properties file in the archive, allowing\neasy configuration changes.\n\nNow, clean up your leftover base archive, since you don't need it anymore:\n\n```\n$ rm FTBRevelationServer_1.0.0.zip\n```\n\nNavigate up a level, and create a gzipped tarball:\n\n```\n$ cd ..\n$ tar -cvzf daftcyborg-server-12-20-2017.tgz daftcyborg/\n```\n\nLastly, push the archive to S3:\n\n```\n$ # The parameterized command is 'aws s3 cp \u003cserver_file\u003e s3://\u003cbucket_name\u003e/\u003cserver_name\u003e/'\n$ # My version looks like:\n$ aws s3 cp daftcyborg-server-12-20-2017.tgz s3://josh-minecraft/daftcyborg/\n```\n\nLastly, save the full name, including file extension, of the archive you\ngenerated; it will be required when you run the setup wizard.\n\n==== Settings\n\nThe recommended way to configure the system is to run the setup wizard\nfrom the root repo directory, like so:\n\n```\n$ ./utilities/setup.py\n```\n\nThis guides you through each required setting, offering default values if\navailable. It then takes your input and renders out `terraform/variables.tf`\nand `ansible/group_vars/all` from corresponding `*.j2` templates. If you like,\nyou can populate those templates by hand as well.\n\n* It's important you choose the right _aws_availability_zone_, since spot\n  prices can vary substantially from zone to zone.\n* Maximum spot price determines the maximum price you're willing to pay per\n  hour. Setting this wisely will prevent you from being surprised by a large\n  bill at the end of the month.\n\n==== Terraform\n\nTerraform allows you to easily setup EC2 and S3 to match your needs. To apply\nthe terraform configuration, run:\n\n```\nterraform apply terraform/\n```\n\nOnce this has successfully completed, your AWS configuration is done. Unless\nyou change your configuration, you won't need to run this again.\n\n==== Ansible\n\nThe first time you do the configurations, you'll need to bundle your local\nAnsible configuration and push it to your S3 bucket, under the same directory\nas your server is stored. A provided playbook takes care of this for you:\n\n```\ncd ansible/\nansible bundle.yml\n```\n\nNOTE: You'll need to repeat this step each time you adjust any server settings.\n\n\n== Instance Interactions\n\n=== Launch\n\nJump to the `ansible` directory, and run the `start.yml` playbook to configure\nthe instance and launch the minecraft server:\n\n```\ncd ansible\nansible-playbook start.yml\n```\n\nThis merely increases the size of the auto-scaling group from 0 → 1; the server\nwon't be available for a few more minutes as it provisions.\n\n=== Shutdown\n\nShutting down your server is very similar, with a few extra command options\nthat allow Ansible to SSH into the host and save the server state before\nkilling the box itself.\n\n```\ncd ansible\nansible-playbook -i ec2.py --private-key=~/.ssh/aws -u ubuntu -c ssh stop.yml\n```\n\nWhen this playbook finishes, your instance will be gone, but the state of the\nserver will have been preserved and pushed to S3, ready for the next time you\nlaunch it.\n\nNOTE: If the auto-inventory script is taking too long, you can update\n`ansible/ec2.ini`'s `regions` entry with the particular AWS region you're using.\n\nNOTE: If using an older version of Ansible, the Paramiko library used by\ndefault may run into errors when gathering facts from the remote host. If this\nhappens, add `-c ssh` to the `ansible-playbook` command above.\n\n== Troubleshooting\n\nLogs from the initial provision are available via the console as well as in\n`/var/log/user-data`.\n\n== Tests\n\nTests are currently available for the Prospector tool. You'll need to install\nthe requirements in the test directory in order to run them. From the root,\nwith your virtual environment active:\n\n```\n(minecraft) $ pip install -r utilities/tests/requirements.txt\n```\n\nNow you can launch the test suite:\n\n```\n(minecraft) $ python -m unittest -v utilities.tests.test_prospector\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgerhalt%2Fmining-camp","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgerhalt%2Fmining-camp","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgerhalt%2Fmining-camp/lists"}