{"id":16931966,"url":"https://github.com/thenickdude/proxmox-on-ec2","last_synced_at":"2025-03-23T17:30:51.493Z","repository":{"id":63591557,"uuid":"547714079","full_name":"thenickdude/proxmox-on-ec2","owner":"thenickdude","description":"A guide to installing Proxmox VE 7 on AWS EC2","archived":false,"fork":false,"pushed_at":"2023-04-17T21:58:39.000Z","size":31,"stargazers_count":38,"open_issues_count":2,"forks_count":12,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-03-18T22:11:31.487Z","etag":null,"topics":["aws","cloud-computing","ec2","proxmox","proxmox-ve"],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"wtfpl","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/thenickdude.png","metadata":{"files":{"readme":"Readme.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-10-08T06:31:27.000Z","updated_at":"2025-01-22T16:09:58.000Z","dependencies_parsed_at":"2024-10-28T21:49:44.886Z","dependency_job_id":null,"html_url":"https://github.com/thenickdude/proxmox-on-ec2","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thenickdude%2Fproxmox-on-ec2","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thenickdude%2Fproxmox-on-ec2/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thenickdude%2Fproxmox-on-ec2/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thenickdude%2Fproxmox-on-ec2/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/thenickdude","download_url":"https://codeload.github.com/thenickdude/proxmox-on-ec2/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245140774,"owners_count":20567447,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws","cloud-computing","ec2","proxmox","proxmox-ve"],"created_at":"2024-10-13T20:45:08.252Z","updated_at":"2025-03-23T17:30:51.191Z","avatar_url":"https://github.com/thenickdude.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# Installing Proxmox VE 7 on AWS EC2\n\nGetting Proxmox to run on EC2 is a little tricky, but by using my cloud-init config you can automate the install process\nand get an instance running in minutes.\n\n## Note for hosting VM guests\n\nPlease note that if you want to be able to host VM guests in EC2 (rather than containers), Proxmox needs to run on an\n\"metal\" instance type (I only tested Intel, AMD might work too) to have access to VT-x/AMD-V and hence KVM hardware \nacceleration (e.g. `c5n.metal`, `m5zn.metal`). \n\nThese instances are **very expensive** (starting at around $4/hour), because rather than getting a small slice of a \nmachine to run your VM on, you're getting an entire EC2 server, with hundreds of gigs of RAM and many cores. \nSignificant cost reductions (75%) are possible if you are able to take advantage of Spot Instances.\n\nSee the list of available metal instance types here, and be sure to filter by Physical Processor type to exclude \nGraviton:\n\nhttps://instances.vantage.sh/?filter=metal\n\nIt's possible to run VMs without KVM support by turning it off on the \"Options\" tab of the VM, but this is \n*excruciatingly* slow and only useful for experimentation.\n\n## Launching your Proxmox VM\n\nLaunch a new VM with the following settings:\n\n- AMI: Use an official Debian 11 AMD64 AMI, e.g. from this list: https://wiki.debian.org/Cloud/AmazonEC2Image/Bullseye\n  \n  You can fetch the latest Debian 11 AMI ID programmatically with the AWS CLI like so:\n  ```bash\n  aws ssm get-parameter --name /aws/service/debian/daily/bullseye/latest/amd64\n  ```\n- Instance type: Something with at least 2GiB of RAM, e.g. `t3.small` for hosting containers, or a `metal` instance type\n  for hosting VMs. \n \n  `t3.micro` (with 1GiB of RAM) rapidly runs out of memory and freezes, even with no workloads running.\n- Network settings: To get started you can set up a security group which allows access to all TCP ports to your IP \n  address only.\n\n  **Avoid exposing Proxmox to the Web at large!** If you need access from multiple IP addresses then set up a VPN to \n  protect access to Proxmox.\n- Storage: You can set up the initial size of your root disk here. I set mine to 64GB since I'm planning to store some\n  ISOs and containers on the root disk.\n- Advanced details: In this section, scroll all the way down to **User Data**. Paste in \n  [this user-data script](https://raw.githubusercontent.com/thenickdude/proxmox-on-ec2/master/cloud-init.yaml). \n \n  At the start of that script you can edit the `hostname` and `fqdn` fields to set the hostname of your server.\n  e.g. you might set `hostname` to `pve` and `fqdn` to `pve.example.com`. The hostname becomes the name of your Proxmox\n  node and is not easily changed later.\n\nNow launch the instance. In the background, the user-data script will automatically download and install Proxmox and \nconfigure the network for you.\n\nYou can check the installation progress in the EC2 console by clicking Actions \u003e Monitor and Troubleshoot \u003e \nEC2 Serial Console, this is equivalent to Proxmox's console monitor. Or you can SSH in and run `tail -f /var/log/syslog` \ninstead. \n\nNote that the serial console is not supported on `metal` instance types.\n\nYou need to SSH in to set a password for the `root` account so that you'll be able to use this\npassword to log in to Proxmox's Web console, you can do this while the installation is still running.\n\nRun `ssh root@YOUR_PUBLIC_IP` (using your private key), then run `passwd` to set the password for\nroot.\n\nOnce setup completes (the last line'll be `[  OK  ] Reached target Cloud-init target.`, which I reached in about 7 \nminutes on `t3.small`), you can access Proxmox's console by visiting `https://\u003cyour instance's public IP\u003e:8006/`,\nand log in using `root` as your username and the password you set. \n\n**You need to reboot Proxmox now** to complete the installation of system updates and switch to the Proxmox kernel (containers \nwill fail to launch otherwise, with fatal AppArmor errors!)\n\n## Note on host network config\n\nThe system supports ENIs (network adapters) being attached dynamically at runtime, so the config for these is \nautomatically generated using `/etc/network/cloud-interfaces-template` and stored into `/run/network/interfaces.d`. \nDHCP is used for these interfaces to pick up the correct private IPv4 addresses from EC2.\n\nBecause Proxmox doesn't know about these config files, using Proxmox to edit the network config causes it to generate\nnon-functional stub configs for those network adapters, which breaks networking.\n\n**So always edit `/etc/network/interfaces` by hand** instead of using Proxmox's \"System \u003e Network\" panel in the WebUI.\n\nIf you end up locking yourself out due to bad network config, you can log in using the Serial Console to fix up\n`/etc/network/interfaces`, and then run `systemctl restart networking.service` afterwards to apply your changes.\n\n**No IPv6 support is enabled.** This is possible, but as I don't have a usecase for it myself I haven't tested it \nout.\n\n## Guest networking\n\n### NAT guests\n\nOn EC2 your Proxmox server only has one IP address by default, so VMs/containers will need to use NAT to share the host's\nIP address to talk to the rest of the network. \n\nMy install script automatically sets up `vmbr0` for you at `10.10.10.1/24`, using NAT, with a `dnsmasq` DHCP server \navailable on that interface. \n\nSo if you add guests to `vmbr0`, and enable DHCP in your guest's network settings, they should get an IP address \nautomatically and be able to connect to the Internet.\n\nThese guests don't have an IP address which is visible to the EC2 network, they can only be connected to from Proxmox\nand from other guests. If you need this support, read on to the next section!\n\n### Routed guests\n\nIf you want to assign a proper IP address to your guests, you can add multiple local IP addresses to your instance's\nENI. Check this article to see what the limit is for your instance type:\n\nhttps://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-eni.html\n\ne.g. for `t3.small`, this limit is 4 local IP addresses (1 for Proxmox and 3 for guests).\n\nMy instance's subnet's CIDR is `172.31.0.0/20`, so I decide on a subnet somewhere within that range to allocate my \nguest IP addresses in, `172.31.14.8/29`. Of that subnet, the first address will be used by Proxmox's bridge \n(`172.31.14.8`), so I add the subsequent addresses `172.31.14.9`, `172.31.14.10`, and `172.31.14.11` to my ENI's IP \nprivate address pool:\n\n![Adding more private IP addresses](assigning-ip-addresses.png)\n\nI edit Proxmox's `/etc/network/interfaces` to add a definition for a new bridge, `vmbr1`, with its address set to the \nfirst IP address of that subnet:\n\n```\nauto vmbr1\niface vmbr1 inet static\n# Routed network for guests with manually-allocated secondary private ENI IPs\n    address 172.31.14.8/29\n    bridge-ports none\n    bridge-stp off\n    bridge-fd 0\n\n    post-up echo 1 \u003e /proc/sys/net/ipv4/ip_forward\n    post-up echo 1 \u003e /proc/sys/net/ipv4/conf/MAIN_NETWORK_ADAPTER_NAME_HERE/proxy_arp\n```\n\nLoad the new configuration by running `systemctl restart networking.service`.\n\nThen when launching guests, attach them to `vmbr1`, set their IP address manually from\n`vmbr1`'s subnet, e.g. `172.31.14.9/29` (`/29` is equivalent to a subnet mask of `255.255.255.248`), and set their \ngateway to the IP address for `vmbr1`, e.g. `172.31.14.8`. You can set their DNS server to `169.254.169.253`,\nwhich is the AWS VPC DNS server address, or otherwise just point it at your favourite public DNS service, like `8.8.8.8`.\n\n**The security group for the ENI applies to both the host and to all guests**, so if you want to restrict \naccess to specific guests you'll need to use Proxmox's firewall rules to achieve that.\n\nNow your guests can access and be accessed by hosts on the local VPC network. But right now they don't have a \npublic IP assigned, so they can't connect to the Internet or be connected to from it.\n\n**To give a guest a public IP,** you need to allocate an Elastic IP address, associate it with the ENI, and bind it to the\ncorresponding Private IP address. \n\nNote that if you do this, Proxmox itself will no longer be allocated a temporary public IP address automatically on \ninstance startup, so you need to attach an Elastic IP address to that too if you need to retain public-IP access to it.\n\n## Turning Proxmox into an AMI\n\nYou can create an AMI from Proxmox and use that to launch a new EC2 instance, although note that the caveats described\nhere still apply if you want to change the hostname (can't have any containers or VMs created beforehand):\n\nhttps://pve.proxmox.com/wiki/Renaming_a_PVE_node\n\nWhen you launch a new instance from the AMI, in the launch wizard use this User Data script to set the hostname (in \nplace of the IP-address based hostname it would otherwise generate automatically):\n\n```yaml\n#cloud-config\nhostname: proxmox\nfqdn: proxmox.local\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthenickdude%2Fproxmox-on-ec2","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fthenickdude%2Fproxmox-on-ec2","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthenickdude%2Fproxmox-on-ec2/lists"}