{"id":19952957,"url":"https://github.com/ethersphere/ethereum-environments","last_synced_at":"2026-03-08T16:31:51.233Z","repository":{"id":15081923,"uuid":"17808353","full_name":"ethersphere/ethereum-environments","owner":"ethersphere","description":"ethereum environments provisioning and automated vm builds","archived":false,"fork":false,"pushed_at":"2014-06-15T17:40:59.000Z","size":300,"stargazers_count":15,"open_issues_count":0,"forks_count":3,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-04-07T20:21:24.591Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Puppet","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ethersphere.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2014-03-16T20:23:14.000Z","updated_at":"2025-01-19T11:27:57.000Z","dependencies_parsed_at":"2022-08-30T05:12:07.011Z","dependency_job_id":null,"html_url":"https://github.com/ethersphere/ethereum-environments","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ethersphere%2Fethereum-environments","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ethersphere%2Fethereum-environments/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ethersphere%2Fethereum-environments/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ethersphere%2Fethereum-environments/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ethersphere","download_url":"https://codeload.github.com/ethersphere/ethereum-environments/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252241967,"owners_count":21717081,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-13T01:14:53.968Z","updated_at":"2026-03-08T16:31:51.195Z","avatar_url":"https://github.com/ethersphere.png","language":"Puppet","funding_links":[],"categories":[],"sub_categories":[],"readme":"ethereum-environments\n=====================\n\nThis projects provides the environments that allows you to create virtual machines (both remote and local) that run the various kinds of ethereum clients. Basically it allows you to build a full ethereum node with one command. \n\nThe two node types currently supported are the cpp and go implementations of the ethereum client. The full-node client is set up to run as a system service on the VM.\n\n## Features\n- apt management with unattended security updates \n- time server \n- users (ubuntu admin and ethereum)\n- ethereum full node cli built from head\n- ethereum launched as system service running as ethereum user \n- ssh setup only public key connection (authorized keys setup)\n- ufw firewall only open ethereum port and ssh\n- fail2ban against ssh ddos\n\n## Disclaimer\n\nThe ethereum project is in inception phase. All software and tools being developed are alpha. Adjust your expectations.\n\n## TL;DR\n\n### remote vm:\n\ncpp:\n\n    packer build -var 'node=cpp-ethereum' -var 'name=cpp-ethereum' packer/aws.json\n    vagrant box add aws-cpp-ethereum boxes/aws-cpp-ethereum.aws.box\n    ETH_NODE=cpp-ethereum vagrant up aws-cpp-ethereum --provider=aws  --no-provision   \n\ngo:\n\n    packer build -var 'node=go-ethereum' -var 'name=go-ethereum' packer/aws.json\n    vagrant box add aws-go-ethereum boxes/aws-go-ethereum.aws.box\n    ETH_NODE=go-ethereum vagrant up aws-go-ethereum --provider=aws  --no-provision   \n\n### local vm:\n\ncpp:\n\n    ETH_NODE=cpp-ethereum vagrant up virtualbox-cpp-ethereum\n\ngo:\n\n    ETH_NODE=go-ethereum vagrant up virtualbox-go-ethereum\n\n## What is installed on the VM\n\n### Client version \n\nBy default the clients are installed from their github repo using the latest code on the master branch. The actual branch is controlled by a puppet facter variable, which you can set on the command line:\n\n    ETH_NODE=go-ethereum ETH_BRANCH=develop vagrant up virtualbox-go-ethereum\n\nYou can also set up multiple vm instances of the same node-type running different client versions. In this case you need to set an alternative name via ETH_NAME variable (which defaults to the nodename). The vm name argument you pass to vagrant should match this name. \n\n    ETH_NODE=go-ethereum ETH_BRANCH=master vagrant up virtualbox-go-ethereum\n    ETH_NODE=go-ethereum ETH_BRANCH=develop ETH_NAME=go-ethereum-dev vagrant up virtualbox-go-ethereum-dev\n\nthe same for remote aws instances: \n\n    packer build -var 'node=go-ethereum' -var 'name=go-ethereum' packer/aws.json\n    packer build -var 'node=go-ethereum' -var 'name=go-ethereum-deb' --branch=develop' packer/aws.json\n    ETH_NODE=go-ethereum ETH_BRANCH=master vagrant up aws-go-ethereum --provider=aws  --no-provision   \n    ETH_NODE=go-ethereum ETH_BRANCH=develop ETH_NAME=go-ethereum-dev vagrant up aws-go-ethereum-dev --provider=aws  --no-provision   \n\n\n### System service\n\nAfter the VM is provisioned with puppet, the ethereum client will be running as a system service using upstart. The clients use data directory in `/usr/local/share/cpp-ethereum/` or `/usr/local/share/go-ethereum/` and logging in `/var/log/go-ethereum/cpp-ethereum.log` or `/var/log/go-ethereum/go-ethereum.log`. These locations can be reset in `puppet/hiera/common.yaml`. The ethereum client service logs can be viewed on the vm as ethereum user:\n\n    sudo su ethereum tail -f /var/log/go-ethereum/go-ethereum.log\n\n### GUI client  \n\nBy default the GUI client is not installed. You can choose to install the GUI client by setting the ETH_GUI variable (passed to puppet as the gui facter variable). \n\n    ETH_NODE=go-ethereum ETH_GUI=true vagrant up virtualbox-go-ethereum\n\nThis only makes sense for local VM really. You need to use virtualbox app to start your vm with a GUI and start the ethereum gui clients from your vm screen.\n\nIf the GUI is installed, the system service full node will not be running by default. You can still start it by ssh-ing to your vm and \n\n    sudo start go-ethereum\n\nor\n\n    sudo start cpp-ethereum\n\nsee upstart documentation on how to control upstart system services. \n\n### \n\n## Prerequisites\n\n* packer - http://www.packer.io for remote \n* virtualbox - for local \n* vagrant - http://www.vagrantup.com/ for local and remote \n* vagrant plugins recommended: aws, vbguest\n\nTested on OSX with \n* Packer v0.5.2, Vagrant 1.5.1, vagrant-aws (0.4.1), virtualbox 4.3.8 (4.3.10 buggy on OSX), vagrant-vbguest (0.10.0)\n* Packer v0.6.0, Vagrant 1.6.3, vagrant-aws (0.4.1), virtualbox 4.3.12 vagrant-vbguest (0.10.0)\n\n### Linux\n\nPacker is distributed as a binary package and i know of no way to install it with a package manager. \n\nOn deb style systems, vagrant installs simply with:\n\n    sudo apt-get -y install vagrant\n\n### on Mac OS X\n\nThere are various ways to install packer and vagrant. Here is a pure command line version using homebrew and cask.\n\n    brew tap homebrew/binary\n    brew install packer\n    brew tap phinze/cask\n    brew install brew-cask\n    brew cask install vagrant\n\n### vagrant plugins \n\nThe `vbguest` plugin is useful to keep your guest editions uptodate (with virtualbox version). Version mismatch can often result in nasty errors.\n\n    vagrant plugin install vagrant-vbguest\n\nif you use this plugin, `vagrant up` will not be able to download the basebox, so you need to add it manually\n\n    vagrant box add --name ubuntu14.04 --provider virtualbox http://cloud-images.ubuntu.com/vagrant/trusty/current/trusty-server-cloudimg-amd64-vagrant-disk1.box\n\nvagrant-aws is used to manage remote aws vm-s with vagrant.\n    \n    vagrant plugin install vagrant-aws\n\n## base OS for VMs\n\nThe base OS used for VMs here is cutting edge Ubuntu 14.04 (trusty):\n\n- AWS EC2 eu-west-1 region ami: ami-335da344\n- vagrant box: http://cloud-images.ubuntu.com/vagrant/trusty/current/trusty-server-cloudimg-amd64-vagrant-disk1.box\n\nNote if you change to an older base OS, you need to make sure puppet 3.x is installed on the VM (not locally). For instance, ubuntu precise has puppet 2.7.x which is too old; to upgrade to puppet 3.x follow http://docs.puppetlabs.com/guides/puppetlabs_package_repositories.html#for-debian-and-ubuntu:\n\n    wget https://apt.puppetlabs.com/puppetlabs-release-precise.deb\n    sudo dpkg -i puppetlabs-release-precise.deb\n    sudo apt-get update\n    sudo apt-get install puppet\n\nto automate this step you can add the lines to the shell provisioning section in `packer/aws-template.json`:\n\n## setting ssh access via authorized_keys\n\nBefore either local or remote vm provisioning, you need to set up your ssh keys.\n\nCreate a file `puppet/modules/users/files/.ssh/authorized_keys` within your working copy. Put your favourite public keys in there to grant access to the VM for both VM users: `ethereum` and `ubuntu` (admin). This file should *not* be under source control to avoid leaking email address etc. (as a precaution it is added to `.gitignore`). The format of the file is your usual `~/.ssh/authorized_keys`, simply one public key per line.\n\nNote that we do not allow unsafe access to the remote vm by vagrant instead force it to connect as `ubuntu` user and your aws private key. This means you must add at least your aws public key to this file, otherwise `vagrant ssh` will be denied access after provisioning. \n\nNote that ssh access to your remote VM is also controlled by your instance's security group. If you explicitly whitelisted IP addresses, access will be limited to connections coming from those.\n\n## local VM\n\nThe multi-machine `Vagrantfile` includes a section for local vms using virtualbox as provider. So you can use it to boot up temporary local instances for any node locally. This requires virtualbox to be installed on your host. \n\n    ETH_NODE=go-ethereum vagrant up virtualbox-go-ethereum\n\nPerform https://github.com/ethersphere/ethereum-environments#setting-ssh-access-via-authorized_keys\n\nThen:\n\n    ETH_NODE=go-ethereum vagrant ssh virtualbox-go-ethereum\n\n## remote VM\n\nYou can create remote vms on amazon ec2 (called ami-s). This requires packer installed as well as having an amazon aws account. The setup is detailed below.\n\n### aws ec2 setup\n\nAssuming you are set up on amazon, go to console \u003e \naccount \u003e security credentials https://console.aws.amazon.com/iam/home?#security_credential\nand export your credentials (AWSAccessKeyId, AWSSecretKey) in `rootkey.cvs`\n\n    AWSAccessKeyId=XXXXXXXXXXXXXXX\n    AWSSecretKey=XXXXXXXXXXXXXX\n\nTo enable network access to your instances, you must allow inbound traffic to your instance. Create a security group within ec2 and add an inbound rule allowing all TCP traffic from anywhere. This sounds dangerous but we provision the server with firewall.\nhttps://console.aws.amazon.com/ec2/home?region=us-east-1#s=SecurityGroups\nRemember your security group name and id (this will be environment vars `AWS_SECURITY_GROUP` and `AWS_SECURITY_GROUP_ID`).\n\nYou should also create a named keypair and export its key into a `.pem` file. The path to this file should be `AWS_PRIVATE_KEY_FILE` and the name is `AWS_KEYPAIR_NAME`.\n\nAll credentials and other ec2 related variables are set via user variables http://www.packer.io/docs/templates/user-variables.html reading environment variables. E.g., `packer/aws-template.json`\n\n    \"variables\": {\n        \"aws_access_key\": \"{{env `AWSAccessKeyId`}}\",\n        \"aws_secret_key\": \"{{env `AWSSecretKey`}}\"\n    },\n\nSo create a file (say `aws.env`) setting environment variables (never share or commit this, as a precaution I added this to .gitignore): \n\n    export AWS_ACCESS_KEY_ID=\n    export AWS_SECRET_KEY=\n    export AWS_PRIVATE_KEY_FILE=\n    export AWS_KEYPAIR_NAME=\n    export AWS_SECURITY_GROUP=\n    export AWS_SECURITY_GROUP_ID=\n    export AWS_REGION=\n\nYou need to source this file in your shell terminal.\n\n    source ./aws.env\n\n### building remote VMs on aws ec2\n\n`packer/aws-template.json` is the template to create amazon machine instances (ami-s). For each node, there is a var file in `packer/nodes`. So to build an ec2 ami for say `go-ethereum` node: \n\n    source ./aws.env\n    packer build -var 'node=go-ethereum' -var 'name=go-ethereum' packer/aws.json\n\nVMs are available for the following nodes:\n\n* cpp-ethereum (ethereum full node client cpp implementation built from head of master branch)\n* go-ethereum (ethereum full node client go implementation built from head of master branch)\n\nuser variables in `packer/aws.json` can be overwritten on the command line.\n- `node`: should match a top-level manifest basename with node def\n- `name`: used in the ami name and passed as facter variable to puppet to set client id \n- `source_ami`: base ami (by default it is a eu-west1 region ubuntu trusty)\n- `instance_type`: aws instance type (e.g., m1.small)\n\nOnce the ami is created, you can make it public, see http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/sharingamis-intro.html\nIf you play around and create ami-s you no longer want, make sure that you deregister them on the ec2 console or amazon will charge you (a minuscule fee) for storing them.\n\n### bringing up VMs with vagrant \n\nAfter packer builds the ami, it also exports a corresponding aws vagrant box (using the vagrant postprocessor http://www.packer.io/intro/getting-started/vagrant.html. The box is saved under `boxes/aws-\u003cNAME\u003e.box`. This box contains the actual ami number so you do not need to set it.\n\nTo use this box, you need to install the `vagrant-aws` plugin for vagrant https://github.com/mitchellh/vagrant-aws, simply with\n\n    vagrant plugin install vagrant-aws\n\nOnce you got the ami built with packer, you add the box:\n\n    vagrant box add aws-go-ethereum boxes/aws-go-ethereum.box\n\nNow you can use the provided multi-machine `Vagrantfile` to boot up temporary instances for any node. (Note the extra `--provider=aws`):\n\n    ETH_NODE=go-ethereum vagrant up aws-go-ethereum --provider=aws --no-provision\n\nIf your packer build was successful, you can safely use the ` --no-provision` option.\nYou can ssh into your remote instance (if you added your aws public key to the ssh authorized key file):\n\n    ETH_NODE=go-ethereum vagrant ssh aws-go-ethereum\n\nor you can reprovision your instance using:\n\n    ETH_NODE=go-ethereum vagrant provision aws-go-ethereum \n\nThis is set up to use the exact same puppet masterless process as packer. Remote provisioning with vagrant is only useful if you develop this project and want to test modifications in provisioning without recreating an instance with packer. It is also useful if packer provisioning fails. In this case, just delete the puppet section from the aws packer template, create the instance and then try provisioning with vagrant which you can debug properly by ssh-ing into the vm.\n\nIf you do `vagrant destroy`, the instance will indeed be terminated (in aws lingo):\n\n    ETH_NODE=go-ethereum vagrant destroy aws-go-ethereum\n\nIf you recreate an instance with packer, you need to remove and add the box again to vagrant.\n\n## Hiera\n\nI use hiera as parameter abstraction layer. A bit overkill at this stage but nice to document options.\nHiera calls in puppet should not use defaults, better style documenting all hiera variables by giving the default in `puppet/hiera/common.yaml`\n\n## Developer notes\n\nCompiling on the VM is a bit of a hack since it merges two distinct steps.\nThe ideal scenario is that we have a continuous release setup that creates unstable or head binary packages using development/compiler baseboxes. Node VMs on the other hand would then be created using these packages, ie., the relevant puppet modules would just install from a repo using a node basebox. \nThis setup cuts across these two problems and implements it in one step until a binary repo with automated dev builds is available.\nAn additional benefit is that now developers can use the exact same environment to compile and test using vagrant on their private or remote aws instances. \n\n### Vision of a third layer for network testing \n\nOnce the node VMs are created, their clones can be launched with automated scripts resulting in ethereum testnets composed of nodes with uniform and mixed implementations. \nThese isolated testnets could then be used for integration testing and benchmarking: in one test round consisting of X blocks a suite of precanned transactions and contracts would be fired at the testnet and checked for correctness of operation as well as for expected measures on various network and mining statistics.\n\n## Troubleshooting\n\n### memory\ncpp-ethereum compilation needs a lot of memory. If you get a mysterious `c++: internal compiler error: Killed (program cc1plus)` error, try increase your VM-s memory. In Vagrant, a generous 2GB is requested since the default 512MB is not enough. For aws m1.small instance type was chosen since m1.micro don't cut it.\n\n### \npacker fails with `Build 'amazon-ebs' errored: extra data in buffer` or `Build 'amazon-ebs' errored: gob: decoding array or slice: length exceeds input size`,just run it again. \n\n## Credits\n* https://github.com/zelig\n* https://github.com/valzav\n* https://github.com/caktux\n\n## TODO\n* support builds from other branches (needs more trix for go client)\n* nodes running multiple clients\n* add peer server nodes or components\n* sort out miners address/key export and import \n* add packer template to support other cloud providers\n\n##Contribute\n\nPlease contribute with pull requests.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fethersphere%2Fethereum-environments","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fethersphere%2Fethereum-environments","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fethersphere%2Fethereum-environments/lists"}