Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/aws/amazon-ec2-metadata-mock

A tool to simulate Amazon EC2 instance metadata
https://github.com/aws/amazon-ec2-metadata-mock

aws-ec2 ec2-instance-metadata ec2-instance-metadata-mock hacktoberfest scheduled-events-mock spot-interruption-mock

Last synced: 27 days ago
JSON representation

A tool to simulate Amazon EC2 instance metadata

Awesome Lists containing this project

README 

        
# Amazon EC2 Metadata Mock



**Amazon EC2 Metadata Mock (AEMM)** is a tool to simulate [Amazon EC2 instance metadata service](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html) for local testing.








   

   latest release

   

   

   go-version

   

   

   license

   

   

   docker-pulls

   




![EC2 Metadata Mock CI and Release](https://github.com/aws/amazon-ec2-metadata-mock/workflows/EC2%20Metadata%20Mock%20CI%20and%20Release/badge.svg)



# Table of Contents



   * [Project Summary](#project-summary)

   * [Major Features](#major-features)

   * [Supported Metadata Categories](#supported-metadata-categories)

   * [Getting Started](#getting-started)

      * [Installation](#installation)

      * [Starting AEMM](#starting-aemm)

      * [Making a Request](#making-a-request)

   * [Configuration](#configuration)

      * [Defaults](#defaults)

      * [Overrides](#overrides)

   * [Usage](#usage)

      * [Spot Interruption](#spot-interruption)

      * [Scheduled Events](#events)

      * [Instance Metadata Service Versions](#instance-metadata-service-versions)

      * [Static Metadata](#static-metadata)

   * [Troubleshooting](#troubleshooting)

      * [Warnings and Expected Outcome](#warnings-and-expected-outcome)

   * [Integrations](#integrations)

   * [Building](#building)

   * [Communication](#communication)

   * [Contributing](#contributing)

   * [License](#license)



# Summary

AWS EC2 Instance metadata is data about your instance that you can use to configure or manage the running instance. Instance metadata is divided into categories like hostname, instance id, maintenance events, spot instance action. See the complete list of metadata categories [here](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-categories.html).



The instance metadata can be accessed from within the instance. Some instance metadata is available only when an instance is affected by the event. E.g. A Spot instance's metadata item `spot/instance-action` is available only when AWS decides to interrupt the Spot instance.

These bring forth some challenges like not being able to test one's application in the event of Spot interruption or other such events and requiring an EC2 instance for testing.

This project attempts to bridge these gaps by providing mocks for **most** of these metadata categories. The mock responses are designed to replicate those from the actual instance metadata service for accurate, local testing.



# Major Features

- Simulate Spot Instance Interruption (ITN) & EC2 Rebalance Recommendation events for Spot instances

- Delay mock response from the mock serve start time

- Configure metadata in mock responses via CLI flags, config file, env variables

- IMDSv1 and v2 support (configurable for IMDSv2 support only)

- Save processed configuration to a local file



# Supported Metadata Categories

AEMM supports most [metadata categories](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-categories.html) **except for:**

* ancestor-ami-ids

* events/maintenance/history



PRs for any of the above paths are always welcome! Please see our [Contributing](#contributing) section for details.



## Windows Metadata

Paths specific to Windows instances such as `elastic-gpus/associations/elastic-gpu-id` will **not** be supported at this time. Feel free to open

an issue to discuss if you would like AEMM to support these paths in the future.



# Getting Started

AEMM is simple to get up and running.



## Installation



### Install w/ Homebrew



```

brew tap aws/tap

brew install ec2-metadata-mock

```

### Install w/ Curl



#### MacOS/Linux

```

curl -Lo ec2-metadata-mock https://github.com/aws/amazon-ec2-metadata-mock/releases/download/v1.11.2/ec2-metadata-mock-`uname | tr '[:upper:]' '[:lower:]'`-amd64

chmod +x ec2-metadata-mock

```



#### ARM Linux

```

curl -Lo ec2-metadata-mock https://github.com/aws/amazon-ec2-metadata-mock/releases/download/v1.11.2/ec2-metadata-mock-linux-arm

```



```

curl -Lo ec2-metadata-mock https://github.com/aws/amazon-ec2-metadata-mock/releases/download/v1.11.2/ec2-metadata-mock-linux-arm64

```



#### Windows

```

curl -Lo ec2-metadata-mock https://github.com/aws/amazon-ec2-metadata-mock/releases/download/v1.11.2/ec2-metadata-mock-windows-amd64.exe

```



### Install w/ Docker

```

docker pull public.ecr.aws/aws-ec2/amazon-ec2-metadata-mock:v1.11.2

docker run -it --rm -p 1338:1338 public.ecr.aws/aws-ec2/amazon-ec2-metadata-mock:v1.11.2

```



### On Kubernetes

#### Supported versions

* Kubernetes >= 1.14



#### Helm

We are hosting helm-charts for Amazon EC2 Metadata Mock in ecr-public. The chart for this project is hosted in [helm/amazon-ec2-metadata-mock](https://gallery.ecr.aws/aws-ec2/helm/amazon-ec2-metadata-mock).

 

Detailed instructions on installing Amazon EC2 Metadata Mock using Helm can be found here [Helm README](https://github.com/aws/amazon-ec2-metadata-mock/blob/main/helm/amazon-ec2-metadata-mock/README.md)



#### kubectl

kubectl apply -f https://github.com/aws/amazon-ec2-metadata-mock/releases/download/v1.11.2/all-resources.yaml



## Starting AEMM

Use `ec2-metadata-mock --help` to view examples and explanations of supported flags and commands:



```

$ ec2-metadata-mock --help



ec2-metadata-mock is a tool to mock Amazon EC2 instance metadata.



Usage:

  ec2-metadata-mock  [arguments] [flags]

  ec2-metadata-mock [command]



Examples:

  ec2-metadata-mock --mock-delay-sec 10	mocks all metadata paths

  ec2-metadata-mock spot --action terminate	mocks spot ITN only



Available Commands:

  events      Mock EC2 maintenance events

  help        Help about any command

  spot        Mock EC2 Spot interruption notice



Flags:

  -c, --config-file string              config file for cli input parameters in json format (default: $HOME/aemm-config.json)

  -h, --help                            help for ec2-metadata-mock

  -n, --hostname string                 the HTTP hostname for the mock url (default: 0.0.0.0)

  -I, --imdsv2                          whether to enable IMDSv2 only, requiring a session token when submitting requests (default: false, meaning both IMDS v1 and v2 are enabled)

  -d, --mock-delay-sec int              spot itn delay in seconds, relative to the application start time (default: 0 seconds)

  -x, --mock-ip-count int               number of IPs in a cluster that can receive a Spot Interrupt Notice and/or Scheduled Event (default 2)

      --mock-trigger-time string        spot itn trigger time in RFC3339 format. This takes priority over mock-delay-sec (default: none)

  -p, --port string                     the HTTP port where the mock runs (default: 1338)

      --rebalance-delay-sec int         rebalance rec delay in seconds, relative to the application start time (default: 0 seconds)

      --rebalance-trigger-time string   rebalance rec trigger time in RFC3339 format. This takes priority over rebalance-delay-sec (default: none)

  -s, --save-config-to-file             whether to save processed config from all input sources in .ec2-metadata-mock/.aemm-config-used.json in $HOME or working dir, if homedir is not found (default: false)

      --version                         version for ec2-metadata-mock



Use "ec2-metadata-mock [command] --help" for more information about a command.

```



Starting AEMM with default configurations using `ec2-metadata-mock` will start the server on the default host and port:



```

$ ec2-metadata-mock



Initiating ec2-metadata-mock for all mocks on port 1338

Serving the following routes: /latest/meta-data/product-codes, /latest/meta-data/iam/info, /latest/meta-data/instance-type, ...(truncated for readability)

```



## Making a Request

With the server running, send a request to one of the supported routes above using `curl localhost:1338/`. Example request to display all supported routes:



```

$ curl localhost:1338/latest/meta-data



ami-id

ami-launch-index

ami-manifest-path

block-device-mapping/

elastic-inference/

events/

hostname

iam/

instance-action

instance-id

instance-life-cycle

instance-type

kernel-id

local-hostname

local-ipv4

mac

network/

placement/

product-codes

public-hostname

public-ipv4

public-keys/

ramdisk-id

reservation-id

security-groups

services/

spot/

tags/

```



# Configuration

AEMM's wide-range of configurability ranges from overriding port numbers to enabling IMDSv2-only to updating specific metadata values and paths.

These configurations can be loaded from various sources with a deterministic precedence.



## Defaults

Defaults for AEMM configuration are sourced throughout code. Examples below:

* **CLI flags**

  * [server config defaults](https://github.com/aws/amazon-ec2-metadata-mock/blob/master/pkg/config/server.go#L22)

* **Metadata mock responses**

  * [aemm-metadata-default-values.json](https://github.com/aws/amazon-ec2-metadata-mock/blob/master/pkg/config/defaults/aemm-metadata-default-values.json)

* **Commands**

  * [events](https://github.com/aws/amazon-ec2-metadata-mock/blob/master/pkg/cmd/events/events.go#L72)



## Overrides

AEMM supports configuration from various sources including: cli flags, env variables, and config files. Details regarding

configuration steps, behavior, and precedence are outlined [here](https://github.com/aws/amazon-ec2-metadata-mock/blob/master/docs/configuration.md).



# Usage

AEMM is primarily used as a developer tool to help test behavior related to Metadata Service. Popular use cases include: emulating spot instance interrupts after a designated delay, mocking scheduled maintenance events, IMDSv2 migrations,

and requesting static metadata. This section outlines the common use cases of AEMM; advanced usage and behavior are documented [here](https://github.com/aws/amazon-ec2-metadata-mock/blob/master/docs/usage.md).



## Spot Interruption

To view the available flags for the Spot Interruption command use `spot --help`:

```

$ ec2-metadata-mock spot --help

Mock EC2 Spot interruption notice



Usage:

  ec2-metadata-mock spot [--action ACTION] [flags]



Aliases:

  spot



Examples:

  ec2-metadata-mock spot -h 	spot help

  ec2-metadata-mock spot -d 5 --action terminate		mocks spot interruption only



Flags:

  -a, --action string                  action in the spot interruption notice (default: terminate)

                                       action can be one of the following: terminate,hibernate,stop

  -h, --help                           help for spot

  -r, --rebalance-rec-time string      rebalance rec time specifies the approximate time when the rebalance recommendation notification will be emitted in RFC3339 format

  -t, --time string                    termination time specifies the approximate time when the spot instance will receive the shutdown signal in RFC3339 format to execute instance action E.g. 2020-01-07T01:03:47Z (default: request time + 2 minutes in UTC)



Global Flags:

  -c, --config-file string              config file for cli input parameters in json format (default: $HOME/aemm-config.json)

  -n, --hostname string                 the HTTP hostname for the mock url (default: 0.0.0.0)

  -I, --imdsv2                          whether to enable IMDSv2 only, requiring a session token when submitting requests (default: false, meaning both IMDS v1 and v2 are enabled)

  -d, --mock-delay-sec int              spot itn delay in seconds, relative to the application start time (default: 0 seconds)

  -x, --mock-ip-count int               number of IPs in a cluster that can receive a Spot Interrupt Notice and/or Scheduled Event (default 2)

      --mock-trigger-time string        spot itn trigger time in RFC3339 format. This takes priority over mock-delay-sec (default: none)

  -p, --port string                     the HTTP port where the mock runs (default: 1338)

      --rebalance-delay-sec int         rebalance rec delay in seconds, relative to the application start time (default: 0 seconds)

      --rebalance-trigger-time string   rebalance rec trigger time in RFC3339 format. This takes priority over rebalance-delay-sec (default: none)

  -s, --save-config-to-file             whether to save processed config from all input sources in .ec2-metadata-mock/.aemm-config-used.json in $HOME or working dir, if homedir is not found (default: false)

```



1.) **Starting AEMM with `spot`**:  `spot` routes available immediately:

```

$ ec2-metadata-mock spot

Initiating ec2-metadata-mock for EC2 Spot interruption notice on port 1338

Serving the following routes: ... (truncated for readability)

```

Send the request:

```

$ curl localhost:1338/latest/meta-data/spot/instance-action

{

	"action": "terminate",

	"time": "2020-04-24T17:11:44Z"

}

```



2.) **Starting AEMM with `spot` after Delay**: Users can apply a *delay* duration in seconds for when the `spot` metadata will become available:



```

$ ec2-metadata-mock spot -d 10

Initiating ec2-metadata-mock for EC2 Spot interruption notice on port 1338



Flags:

mock-delay-sec: 10



Serving the following routes: ... (truncated for readability)

```



Sending a request to `spot` paths before the delay has passed will return **404 - Not Found:**

```

$ curl localhost:1338/latest/meta-data/spot/instance-action



 

  404 - Not Found

 

 

  
404 - Not Found


 



// Server log

Delaying the response by 10s as requested. The mock response will be available in 2s. Returning `notFoundResponse` for now

```



Once the delay is complete, querying `spot` paths return expected results:

```

$ curl localhost:1338/latest/meta-data/spot/instance-action



{

	"action": "terminate",

	"time": "2020-04-24T17:19:32Z"

}



```



Alternatively a trigger time can be configured using `--mock-trigger-time` which can be useful to synchronize spot interruption simulation over multiple instances.



### EC2 Instance Rebalance Recommendation

The Rebalance Recommendation notification is also available under the **spot** command as it is sent when EC2 emits this signal when Spot Instances are at an elevated risk of interruption:

```

$ ec2-metadata-mock spot

Initiating ec2-metadata-mock for EC2 Spot interruption notice on port 1338

Serving the following routes: ... (truncated for readability)

```

Send the request:

```

$ curl localhost:1338/latest/meta-data/events/recommendations/rebalance

{

        "noticeTime": "2020-10-16T19:18:24Z"

}

```

*Note: although Rebalance Recommendation path contains `events` it will **not be available** when starting AEMM with the `events` command*



## Events

Similar to spot, the `events` command, view the local flags using `events --help`:



```

$ ec2-metadata-mock events --help

Mock EC2 Scheduled Events



Usage:

  ec2-metadata-mock events [--code CODE] [--state STATE] [--not-after] [--not-before-deadline] [flags]



Aliases:

  events, se, scheduledevents



Examples:

  ec2-metadata-mock events -h 	events help

  ec2-metadata-mock events -o instance-stop --state active -d		mocks an active and upcoming scheduled event for instance stop with a deadline for the event start time



Flags:

  -o, --code string                  event code in the scheduled event (default: system-reboot)

                                     event-code can be one of the following: instance-reboot,system-reboot,system-maintenance,instance-retirement,instance-stop

  -h, --help                         help for events

  -a, --not-after string             the latest end time for the scheduled event in RFC3339 format E.g. 2020-01-07T01:03:47Z default: application start time + 7 days in UTC))

  -b, --not-before string            the earliest start time for the scheduled event in RFC3339 format E.g. 2020-01-07T01:03:47Z (default: application start time in UTC)

  -l, --not-before-deadline string   the deadline for starting the event in RFC3339 format E.g. 2020-01-07T01:03:47Z (default: application start time + 9 days in UTC)

  -t, --state string                 state of the scheduled event (default: active)

                                     state can be one of the following: active,completed,canceled



(Truncated Global Flags for readability)

```



1.) **Starting AEMM with `events`**: `events` route available immediately and `spot` routes will no longer be available due to the implementation of Commands [detailed here](https://github.com/aws/amazon-ec2-metadata-mock/blob/master/docs/usage.md):



```

$ ec2-metadata-mock events --code instance-reboot -a 2020-01-07T01:03:47Z  -b 2020-01-01T01:03:47Z -l 2020-01-10T01:03:47Z --state completed

Initiating ec2-metadata-mock for EC2 Events on port 1338

Serving the following routes: ... (truncated for readability)



```



Send the request:

```

$ curl localhost:1338/latest/meta-data/events/maintenance/scheduled

{

	"Code": "instance-reboot",

	"Description": "The instance is scheduled for instance-reboot",

	"State": "completed",

	"EventId": "instance-event-1234567890abcdef0",

	"NotBefore": "1 Jan 2020 01:03:47 GMT",

	"NotAfter": "7 Jan 2020 01:03:47 GMT",

	"NotBeforeDeadline": "10 Jan 2020 01:03:47 GMT"

}

```



## Instance Metadata Service Versions

AEMM supports [both versions](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/configuring-instance-metadata-service.html) of Instance Metadata service. By default, AEMM starts with supporting v1 and v2; however, it is possible to enable **IMDSv2 only** via overrides.



1.) **Starting AEMM with IMDSv2 only:** session tokens are required for all requests; v1 requests will return **401 - Unauthorized:**



```

$ ec2-metadata-mock --imdsv2

```

Send a v1 request:

```

$ curl localhost:1338/latest/meta-data/mac



   

      401 - Unauthorized

   

   

      
401 - Unauthorized


   



```



Send a v2 request:

```

TOKEN=`curl -X PUT "localhost:1338/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600"` \

&& curl -H "X-aws-ec2-metadata-token: $TOKEN" localhost:1338/latest/meta-data/mac

0e:49:61:0f:c3:11

```



Requesting a token outside the TTL bounds (between 1-2600 seconds) will return **400 - Bad Request:**

```

$ curl -X PUT "localhost:1338/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 0"



   

      400 - Bad Request

   

   

      
400 - Bad Request


   



```



Providing an expired token is synonymous to no token at all resulting in **401 - Unauthorized**.



## Static Metadata

Static metadata is classified as instance-specific metadata that is **always** available regardless of which command is used to start the tool.



Examples of static metadata include *all* [metadata categories](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-categories.html) from the non-dynamic category table (ami-id, instance-id, mac) **except for events and spot categories (classified as commands in AEMM).**



*Note that 'static' naming is used within the context of this tool ONLY*



1.) **Requesting static metadata `instance-id`**:

```

$ ec2-metadata-mock

```



Send the request:

```

$ curl localhost:1338/latest/meta-data/instance-id

i-1234567890abcdef0

```



Details on overriding static metadata values and behavior can be found [here](https://github.com/aws/amazon-ec2-metadata-mock/blob/master/docs/usage.md#static-metadata)



# Troubleshooting



## Warnings and Expected Outcome

|Warning Displayed|Cause|Effect|

|---|---|---|

|Warning: Config File _file name_ Not Found in _locations_ |input configuration file not found|other input sources are used (See above for details)|

|Warning: Failed to save the final configuration to local file - Failed to create directory for final configuration at _path/to/dir_: _error string_|Failure to create the hidden directory `.amazon-ec2-metadata-mock` to store the final configuration file| configuration used by the tool is NOT saved to a local file. The tool continues with its primary job of mocking metadata paths |

|Warning: Failed to save the final configuration to local file - The destination '_path/to/dir_' for saving the configuration already exists, but is not a directory |Failure to create the hidden directory `.amazon-ec2-metadata-mock` to store the final configuration file, because a resource by that name already exists| configuration used by the tool is NOT saved to a local file. The tool continues with its primary job of mocking metadata paths |

|Warning: Failed to save the final configuration to local file _path/to/local/file_: _error string_  |Failure to save final configuration to a file |configuration used by the tool is NOT saved to a local file. The tool continues with its primary job of mocking metadata paths |

|Warning: Failed to find home directory due to error: _error string_|Failure to get home directory| working directory is used instead|



# Integrations

[aws-node-termination-handler](https://github.com/aws/aws-node-termination-handler) uses AEMM in its [e2e test suite](https://github.com/aws/aws-node-termination-handler/tree/master/test/e2e)

to mock the metadata service and interrupt events.



In the [NTH imds-v2-test](https://github.com/aws/aws-node-termination-handler/blob/master/test/e2e/imds-v2-test), Helm is used

to download the latest AEMM release, install it onto the cluster, and start it with imdsv2-only access. NTH then acquires the v2 token from AEMM, consumes the interrupt event, then

cordons the worker node and evicts the test pod, thus validating NTH functionality. For more details on NTH e2e tests refer to the documentation [here](https://github.com/aws/aws-node-termination-handler/blob/master/test/README.md).



# Building

For build instructions, please consult [BUILD.md](https://github.com/aws/amazon-ec2-metadata-mock/blob/master/BUILD.md)



# Communication

If you've run into a bug or have a new feature request, please open an [issue](https://github.com/aws/amazon-ec2-metadata-mock/issues/new).



Check out the open source [Amazon EC2 Spot Instances Integrations Roadmap](https://github.com/aws/ec2-spot-instances-integrations-roadmap) to see what we're working on and give us feedback!



# Contributing

Contributions are welcome! Please read our [guidelines](https://github.com/aws/amazon-ec2-metadata-mock/blob/master/CONTRIBUTING.md) and our [Code of Conduct](https://github.com/aws/amazon-ec2-metadata-mock/blob/master/CODE_OF_CONDUCT.md)



# License

This project is licensed under the Apache-2.0 License.