{"id":27387766,"url":"https://github.com/metomi/metomi-vms","last_synced_at":"2026-03-08T15:36:25.510Z","repository":{"id":27791491,"uuid":"31280410","full_name":"metomi/metomi-vms","owner":"metomi","description":":computer: Vagrant virtual machines with FCM + Rose + Cylc installed","archived":false,"fork":false,"pushed_at":"2025-02-07T13:13:06.000Z","size":266,"stargazers_count":17,"open_issues_count":0,"forks_count":14,"subscribers_count":14,"default_branch":"master","last_synced_at":"2025-02-07T14:23:41.254Z","etag":null,"topics":["shell"],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/metomi.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2015-02-24T20:30:56.000Z","updated_at":"2025-02-07T13:13:12.000Z","dependencies_parsed_at":"2024-06-12T21:06:47.698Z","dependency_job_id":"bdebba2c-1c9e-4573-8082-590a1b248f0e","html_url":"https://github.com/metomi/metomi-vms","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/metomi%2Fmetomi-vms","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/metomi%2Fmetomi-vms/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/metomi%2Fmetomi-vms/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/metomi%2Fmetomi-vms/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/metomi","download_url":"https://codeload.github.com/metomi/metomi-vms/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248758461,"owners_count":21156964,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["shell"],"created_at":"2025-04-13T18:19:17.982Z","updated_at":"2026-03-08T15:36:25.501Z","avatar_url":"https://github.com/metomi.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# metomi-vms\n\n[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.1341042.svg)](https://doi.org/10.5281/zenodo.1341042)\n\nVagrant virtual machines with [FCM](http://metomi.github.io/fcm/doc/) + [Rose](http://metomi.github.io/rose/) + [Cylc](http://cylc.github.io/cylc/) installed.\n\nTable of contents:\n* [Software Requirements](#software-requirements)\n* [Setting up the Default Virtual Machine](#setting-up-the-default-virtual-machine)\n* [Using the Default Virtual Machine](#using-the-default-virtual-machine)\n* [Disabling the Desktop Environment](#disabling-the-desktop-environment)\n* [Using other Virtual Machines](#using-other-virtual-machines)\n* [Optional Windows Software](#optional-windows-software)\n  * [Git BASH](#git-bash)\n  * [Cygwin](#cygwin)\n* [Ubuntu Pro](#ubuntu-pro)\n* [VMware](#vmware)\n* [libvirt](#libvirt)\n* [Troubleshooting](#troubleshooting)\n* [Amazon AWS](#amazon-aws)\n\n## Software Requirements\n\nIn order to use a virtual machine (VM), you must first install:\n * [VirtualBox](https://www.virtualbox.org/), software that enables running of virtual machines (version 5.1.x or later required).\n   * As an alternative, [VMware Workstation Player](https://www.vmware.com/uk/products/workstation-player.html) (version 16 or later) or [VMware Fusion Player](https://www.vmware.com/uk/products/fusion.html) (version 12 or later) can also be used to run the virtual machine. See the [VMware section](#vmware) for further information.\n * [Vagrant](https://www.vagrantup.com/), software that allows easy configuration of virtual machines (version 2.0.x or later required).\n\nThese applications provide point-and-click installers for Windows and can usually be installed via the package manager on Linux systems.\n\n## Setting up the Default Virtual Machine\n\nAfter you have installed VirtualBox and Vagrant, download the metomi VM setup files from github:\n * https://github.com/metomi/metomi-vms/archive/master.zip.\n\nThen extract the files which will be put into a directory called `metomi-vms-master`.\n\nThe default VM uses Ubuntu 18.04.\nIf necessary you can customise the VM by editing the file `Vagrantfile.ubuntu-1804` as follows:\n* By default the VM will be built with support for accessing the Met Office Science Repository Service.\n  If you don't want this (or don't have access) then remove `mosrs` from the `args` in the `config.vm.provision` line.\n* As described below, you may prefer not to install the desktop environment.\n  To do this remove `desktop` from the `args` in the `config.vm.provision` line and comment out the line `v.gui = true`.\n* By default the VM is configured with 1 GB memory and 2 CPUs.\n  You may want to increase these if your host machine is powerful enough.\n\nSee the [Vagrant documentation](https://docs.vagrantup.com/v2/virtualbox/configuration.html) for more details on configuration options.\n\nBefore proceeding you need to be running a terminal with your current directory set to `metomi-vms-master`.\n* Windows users can navigate to the directory using Windows File Explorer and then use `Shift-\u003e Right Mouse Click -\u003e Open command window here`.\n\nNow run the command `vagrant up` to build and launch the VM.\nThis involves downloading a base VM and then installing lots of additional software so it can take a long time (depending on the speed of your internet connection).\nNote that, although a login screen will appear in a separate window, you will not be able to login at this stage.\nOnce the installation is complete the VM will shutdown.\n\n## Using the Default Virtual Machine\n\nRun the command `vagrant up` to launch the VM.\nA separate window should open containing a lightweight Linux desktop environment ([LXDE](http://lxde.org/)) with a terminal already opened.\n\nBoth Cylc 7 and Cylc 8 are installed ([see the migration guide](https://cylc.github.io/cylc-doc/stable/html/7-to-8/index.html)).\nCylc 8 is the default.\nTo use Cylc 7, run the following command before running any Cylc or Rose commands:\n```\nexport CYLC_VERSION=7\n```\n\nIf your VM includes support for the Met Office Science Repository Service then you will be prompted for your password (and also your user name the first time you use the VM).\nIf you get your username or password wrong and Subversion fails to connect, just run `mosrs-cache-password` to try again.\n\nThe VM is configured with a local [Rose suite repository](http://metomi.github.io/rose/doc/html/tutorial/rose/rosie.html) and with the suite log viewer running under apache.\nIf you want to learn more about Rose and Cylc you can follow the tutorials contained in the [Rose User Guide](http://metomi.github.io/rose/).\n\nTo shutdown the VM you can either use the menu item available in the bottom right hand corner of the Linux desktop or you can issue the command `vagrant halt` from the command window where you launched the VM.\n\nNote that the desktop environment is configured to use a UK keyboard.\nIf you need to change this, take a look at how this is configured in the file `install-desktop.sh`.\n\n## Disabling the Desktop Environment\n\nIf you are using the VM on a Mac or Linux system where you already have a X server running then you may find it easier to not install the desktop environment.\nIn order to do this, edit the file `Vagrantfile.ubuntu-1804` as described above.\nThen run the command `vagrant up` to launch the VM in the normal way.\nNote that, unlike when installing the desktop environment, it will not shutdown after the initial installation.\n\nOnce the VM is running, run the command `vagrant ssh` to connect to it.\n\nTo shutdown the VM you can either run the command `sudo shutdown -h now` from within your ssh session or you can exit your ssh session and then issue the command `vagrant halt`.\n\n## Using other Virtual Machines\n\nIn addition to the default VM, additional VMs are supported in separate files named `Vagrantfile.\u003cdistribution\u003e`, e.g. `Vagrantfile.centos-7`.\nThese other VMs are provided primarily for the purpose of testing FCM, Rose \u0026 Cylc on other Linux distributions and providing a reference install on these platforms.\nNote that they are not as well tested as the default VM and may not include a desktop environment.\n\nTo use a different VM, modify the file which is loaded in the default `Vagrantfile` before running `vagrant up`.\nAlternatively you can set the environment variable `VAGRANT_VAGRANTFILE`, for example:\n```\nexport VAGRANT_VAGRANTFILE=Vagrantfile.ubuntu-2204\n```\n(use `set` in place of `export` when using the command window on Windows).\n\n## Optional Windows Software\n\n### Git BASH\n\nFor an alternative to the normal command window, Windows users can install Git BASH (which comes with [Git for Windows](https://git-for-windows.github.io/)).\nAs well as providing a nicer interface (more familiar for Linux users) this also means that you can use git to clone the metomi-vms repository (instead of downloading the zip file):\n```\ngit clone https://github.com/metomi/metomi-vms.git\n```\nIt is then easy to track any local changes, pull down updates, etc.\n\n### Cygwin\n\nIf you want to run a VM without a desktop environment on Windows then, in order to enable GUI programs to work, you will need to install [Cygwin](https://www.cygwin.com/), making sure to select the `xinit` and `xorg-server` packages from the `X11` section and the `openssh` and `openssl` packages from the `Net` section.\n\nThen, instead of using a normal command window for launching the VM, you should use a Cygwin-X terminal, which you can find in the Start Menu as `Cygwin-X \u003e XWin Server`.\nIn Cygwin-X terminals, you can use many common Unix commands (e.g. cd, ls).\nFirstly run the command  `cd /cygdrive` followed by `ls` and you should see your Windows drives.\nThen use the `cd` command to navigate to the directory where you have extracted the setup files (e.g. `c/Users/User/metomi-vms-master`).\n\n## Ubuntu Pro\n\nWhile Ubuntu 18.04 LTS went end-of-life in May 2023, an [Ubuntu Pro](https://ubuntu.com/pro) subscription can be used to get security updates for a further 5 years. This is free for personal use for up to 5 machines and the process is documented in a [Tutorial](https://ubuntu.com/pro/tutorial).\n\nWhen you reboot your VM you may get the error \"Vagrant was unable to mount VirtualBox shared folders\". This can be fixed by [re-installing the VirtualBox guest additions](https://www.virtualbox.org/manual/ch04.html#additions-linux), which can be done via the command-line by\n```\nsudo apt install -y virtualbox-guest-additions-iso\n```\nYou may find that this is sufficient to fix the error after rebooting. If it is not, you can manually install them by\n```\nsudo mount -o loop /usr/share/virtualbox/VBoxGuestAdditions.iso /media/cdrom\nsudo /media/cdrom/VBoxLinuxAdditions.run\nsudo umount /media/cdrom\n```\nand then rebooting the VM.\n\n## VMware\n\nAs an alternative to VirtualBox, [VMware Workstation Player](https://www.vmware.com/uk/products/workstation-player.html) (Windows, Linux) or ([VMware Fusion Player](https://www.vmware.com/uk/products/fusion.html) (macOS) can be used to host the virtual machine. VMware Workstation Player is free for non-commercial use and VMware Fusion Player is free with a Personal Use License.\n\nYou will need to download and install\n\n* [VMware Workstation Player](https://www.vmware.com/uk/products/workstation-player.html) (version 16 or later) **or** [VMware Fusion Player](https://www.vmware.com/uk/products/fusion.html) (version 12 or later)\n* The [Vagrant VMware utility](https://www.vagrantup.com/vmware/downloads)\n* The Vagrant VMware plugin by running the command `vagrant plugin install vagrant-vmware-desktop`\n\nThe configuration settings can be found in the [Vagrantfile.vmware_ubuntu-1804](Vagrantfile.vmware_ubuntu-1804) file. To bring the box up using VMware, you should [set your VAGRANT_VAGRANTFILE](#using-other-virtual-machines) to `Vagrantfile.vmware_ubuntu-1804` before running the command\n```\nvagrant up --provider=vmware_desktop\n```\nYou may have issues if both VMware and VirtualBox are installed, or if the Hyper-V hypervisor is also running. If you are using an Apple Silicon device with VMware Fusion you will need to set the following\n```\nconfig.vm.box = \"uwbbi/bionic-arm64\"\n```\nin the [Vagrantfile.vmware_ubuntu-1804](Vagrantfile.vmware_ubuntu-1804) file as you cannot use an amd64-based installation on Apple Silicon (ARM-based) hardware.\n\n## libvirt\n\nAnother alternative to VirtualBox and VMware is to use the [libvirt virtualisation API](https://libvirt.org/index.html), which also has a [Vagrant plugin](https://vagrant-libvirt.github.io/vagrant-libvirt/). You will need to install libvirt on your host system. You should [set your VAGRANT_VAGRANTFILE](#using-other-virtual-machines) to [Vagrantfile.libvirt_ubuntu-1804](Vagrantfile.libvirt_ubuntu-1804) before running the command\n```\nvagrant up --provider=libvirt\n```\nto provision the VM. The current [Vagrantfile](Vagrantfile.libvirt_ubuntu-1804) has been used on a GNU/Linux host without a graphical login.\n\nThe advantage of this method is that it allows for PCI passthrough, allowing the guest OS to directly access hardware on the host, for instance allowing the guest to access a GPU on the host machine. On a GNU/Linux system you can use the `lspci -v` command to determine the device information, and then include a block such as this within the `config.vm.provider` section of the [Vagrantfile.libvirt_ubuntu-1804](Vagrantfile.libvirt_ubuntu-1804) file:\n```\n    # VGA controller on 65:00.0\n    v.pci :domain =\u003e '0x0000', :bus =\u003e '0x65', :slot =\u003e '0x00', :function =\u003e '0x0'\n    # Audio controller on 65:00.1\n    v.pci :domain =\u003e '0x0000', :bus =\u003e '0x65', :slot =\u003e '0x00', :function =\u003e '0x1'\n    # USB controller on 65:00.2\n    v.pci :domain =\u003e '0x0000', :bus =\u003e '0x65', :slot =\u003e '0x00', :function =\u003e '0x2'\n    # Serial bus controller on 65:00.3\n    v.pci :domain =\u003e '0x0000', :bus =\u003e '0x65', :slot =\u003e '0x00', :function =\u003e '0x3'\n```\nThis step needs to be done on the initial creation of the VM. The GPU would also need to be isolated from the host system.\n\n## Troubleshooting\n\n### Guest display resolution\n\nWhen you resize the VirtualBox window (e.g. to mazimise it) the display resolution of your Linux VM should adjust to match.\nIf this doesn't work it may be due to the Guest Additions installed in your VM not matching the version of VirtualBox you have installed.\nThe easiest way to fix this is to install the [vagrant-vbguest Vagrant plugin](https://github.com/dotless-de/vagrant-vbguest).\nNote that, if the plugin does update your Guest Additions then you will need to shutdown and restart your VM (using `vagrant up`) in order for them to take effect.\n\n## Amazon AWS\n\nIt is possible to run using Vagrant on an Amazon AWS EC2 virtual machine. To do this you will need to install the [`vagrant-aws`](https://github.com/geckoboard/vagrant-aws) plugin via\n```\nvagrant plugin install vagrant-gecko-aws --entry-point vagrant-aws\n```\nYou do not need VirtualBox. You should ensure that you are using a recent version of Vagrant to enable the AWS plugin to work, and you will first need to run the command\n```\nvagrant box add dummy https://github.com/mitchellh/vagrant-aws/raw/master/dummy.box\n```\nin a different directory to your `metomi-vms` directory.\n\nSome set-up is required within the AWS console. You will first need to:\n\n1. Generate a key to allow you to connect to the VM\n2. Create a security group and restrict incoming access to the IP address of your computer \n3. Create a user and make a note of the required information \n\nThe information in points 1 \u0026 2 will need to be saved to a file called **_aws-credentials_** - an example one is provided which looks like\n```\nexport VAGRANT_VAGRANTFILE=Vagrantfile.aws_ubuntu-1804\nexport AWS_KEY='AAAAAAAAAAAAAAAAAAAA'\nexport AWS_SECRET='BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB'\nexport AWS_KEYNAME='CCCCCCCCC'\nexport AWS_KEYPATH='/full/path/to/CCCCCCCCC'\n```\nIf you are using Windows you may want to replace the options in the Vagrantfile directly, but be careful not to commit this information back to a public repository.\n\n**Note** that because the default username for EC2 Ubuntu VMs is **_ubuntu_** the `/home/ubuntu` directory has also been symbolically linked to `/home/vagrant`. You should continue to work under the `ubuntu` user as normal.\n\n### VM size\n\nThere are many different sizes of VM to choose from (known as [instance types](https://aws.amazon.com/ec2/instance-types/)), some of which will be eligible for the free tier, e.g. `t2.micro` that has 1 CPU and 1GB of memory. To be able to run the UM you will need to select a larger type, such as `t2.medium`(2 CPUs and 4GB of memory) or `t2.large`(2 CPUs and 8GB of memory). This is changed in the `aws.instance_type` setting in the Vagrantfile. You can also select faster hardware, e.g. the `m5` hardware uses faster Intel Xeon processors with a greater network and storage bandwidth, and may give better performance. Larger and faster options are available, but these will all come with an associated cost.\n\nThe hard disk size of the VM is set to 30GB in the Vagrantfile in the `aws.block_device_mapping` setting. This can be changed as required. The default `t2.micro` size is 8GB if nothing is set.\n\nIt is possible to resize your VM by changing the instance type once it has been created. To do this you need to first stop it using\n```\nvagrant halt\n```\nand then [follow the instructions](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-resize.html) for changing the instance using the AWS console. \n\n### Chose your region\n\nOn the [AWS console](https://aws.amazon.com/) you should change your region to the one where you want the VM to be provisioned by using the drop-down menu on the top right of the page. The default settings may put you in `us-east-2` (US East (Ohio)), but you may want to change this to, e.g., London (or `eu-west-2`). \n\nFrom here you should click the **All services** drop-down menu, and then click **EC2** to enter the EC2 Dashboard.\n\nThere are many different types of EC2 VMs (e.g. Ubuntu, Amazon Linux etc.), which are identified by their unique **ami-** identifier. This identifier is also unique to a particular region. The setting for Ubuntu 18.04 LTS or Ubuntu 22.04 LTS in the London (eu-west-2) region has already been set in the `aws.ami` setting in the provided Vagrantfiles, but you may need to first subscribe to use these images. If you wish to use a different region you will need to search for the correct _ami-_ identifier from the **Launch instance** option within the EC2 Dashboard and then set this in the Vagrantfile accordingly.\n\n### Create your key pair\n\nIn the EC2 Dashboard scroll down the left-hand menu until you find **Network \u0026 Security** and click **Key Pairs**, and then click **Create key pair** on the top right. \n\nHere you should give your key a name, e.g. \"vagrant\" or \"metomi-vms\" etc. You should keep the **pem** file format, and then click **Create key pair**. Save this file to your local machine, and ensure it has the correct permissions so that it is only readable by you.\n\nYou should add the name of and full path to your key to your **_aws-credentials_** file. Note that the name should not include any extension (e.g. `.pem`), but the full path should.\n\n### Create a security group to limit IP access to your VM\n\nIn the EC2 Dashboard scroll down the left-hand menu until you find **Network \u0026 Security** and click **Security Groups**, and then click **Create security group** on the top right. \n\nYou should give it a name, e.g. **MyIP** as is used in the Vagrantfile, and a description (e.g. \"limit access to my IP\"). Scroll down to the **Inbound rule** section and click **Add rule**.\n\nHere, use the drop-down menus to change the _Type_ to **All traffic** and the _Source_ to **My IP** (your current IP address will be automatically added). Scroll down to the bottom of the page and click **Create security group**.\n\nIf you used a name other than \"MyIP\" for the name of the group you will need to update the `aws.security_groups` setting in the AWS Vagrantfile.\n\n### Create a user\n\nYou will need to create a user with the correct permissions to access your EC2 VM, which again is done via the console. This is not done within the EC2 Dashboard, but is instead done within the **IAM Dashboard** (Identity and Access Management). To get to this from the EC2 Dashboard first click the AWS logo on the top left of the page to bring you back to the console front page, and then click the **All services** drop-down menu, and then click **IAM** under the \"Security, Identity, \u0026 Compliance\" section.\n\nUnder **Access Management** click **Users** and then click **Add user**. You should give them a name, e.g. _vagrant_ or _metomi-vms_ etc.. Tick the box for **Programmatic access** and then click the **Next: Permissions** button.\n\nHere you should click the tab labelled **Attach existing policies directly** and search for **AmazonEC2FullAccess** and then tick the check-box next to this option. Now click the **Next: Tags** button. You can then click the **Next: Review** button. \n\nNow click **Create user**. This will bring you to a page listing the username, the _Access key ID_ and the _Secret access key_. **THE SECRET ACCESS KEY INFORMATION WILL BE DISPLAYED ONLY ONCE**. \n\nYou should copy this information into your **_aws-credentials_** file and download and save the `.csv` file containing this information. Again, do not upload this information (either the aws-credentials file or the csv file) to a public repository.\n\n### Provision your AWS VM\n\nOnce you have all the information for your aws-credentials file, you should first _source_ this file\n```\nsource aws-credentials\n```\nOnce you have created (\u0026 potentially added) the security group to your AWS Vagrantfile, you should then provision the VM by\n```\nvagrant up --provider=aws\n```\nIf this hangs on the line\n```\nWaiting for SSH to become available...\n```\nthen you should check the security group settings above. You may also recieve an email saying _\"You recently requested an AWS Service that required additional validation\"_, which may have also caused a delay. It will take several minutes to provision the VM for the first time.\n\nOnce the required packages have been installed you will need to run\n```\nvagrant up\n```\nagain, before being able to connect via\n```\nvagrant ssh\n```\n\nIf the VM becomes unresponsive you many need to force-stop it via the EC2 Dashboard and run\n```\nvagrant up\n```\nagain. In the instance list it will be called **_metomi-vms_**.\n\n### Connecting via X2Go\n\nTo be able to connect to a full desktop, you can use X2Go. You should install the [X2Go client](https://wiki.x2go.org/doku.php/download:start).\n\n1. Fill out the settings details on the new session popup as in the table below, then press _OK_.\n\n2. Click on your new session on the right hand side of the x2goclient window.\n\n3. It should automatically login after asking you if you wish to allow the connection.\n\nIf you stop the instance and then later restart it, the IP address may change. You can find the IP address of your EC2 instance on the EC2 Dashboard. You will need to change this in your X2Go settings and allow the connection when prompted.\n\n**X2Go settings for AWS EC2 instance**\n\n| Option | Setting |\n| :--- | :--- |\n| Session name | *e.g.* AWS |\n| Host | *IP address for the VM, e.g.* 3.8.24.92 |\n| Login | ubuntu |\n| Use RSA/DSA key for ssh connection | *The full path to the key file you created earlier (navigate via button)* |\n| Session type | *Select* LXDE *from drop-down menu* |\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmetomi%2Fmetomi-vms","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmetomi%2Fmetomi-vms","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmetomi%2Fmetomi-vms/lists"}