https://github.com/localstack/localstack-extension-platform-observability
Experimental extension to log various metrics of the locastack platform
https://github.com/localstack/localstack-extension-platform-observability
Last synced: 9 months ago
JSON representation
Experimental extension to log various metrics of the locastack platform
- Host: GitHub
- URL: https://github.com/localstack/localstack-extension-platform-observability
- Owner: localstack
- Created: 2024-01-09T21:41:56.000Z (almost 2 years ago)
- Default Branch: main
- Last Pushed: 2024-07-16T23:39:10.000Z (over 1 year ago)
- Last Synced: 2025-02-20T00:31:45.427Z (10 months ago)
- Language: Python
- Size: 24.4 KB
- Stars: 1
- Watchers: 19
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
LocalStack Extension: Platform observability
===============================
LocalStack extension for providing insights into the LocalStack platform in the form of service-specific traces and metrics.
## Install
```bash
localstack extensions install localstack-extension-platform-observability
```
## Install local development version
To install the extension into localstack in developer mode, you will need Python 3.10, and create a virtual environment in the extensions project.
In the newly generated project, simply run
```bash
make install
```
Then, to enable the extension for LocalStack, run
```bash
localstack extensions dev enable .
```
You can then start LocalStack with `EXTENSION_DEV_MODE=1` to load all enabled extensions:
```bash
EXTENSION_DEV_MODE=1 localstack start
```
## Install from GitHub repository
To distribute your extension, simply upload it to your github account. Your extension can then be installed via:
```bash
localstack extensions install "git+https://github.com/localstack/localstack-extension-platform-observability/#egg=localstack-extension-platform-observability"
```
## Usage
### Metrics
Fetch all metrics
```bash
curl localhost:4566/_extension/observability/metrics
```
Fetch a specific instrument
```bash
curl localhost:4566/_extension/observability/metrics/
```
The following instruments exist
* `system`: system metrics like number of threads
* `sns`: sns topic statistics
* `sqs`: sqs queue statistics
* `gateway`: HTTP gateway statistics on number of requests
Example:
```bash
curl -s "localhost:4566/_extension/observability/metrics" | jq .
```
```json
{
"system": [
{
"active_thread_count": 15,
"max_rss": 15
}
],
"gateway": [
{
"total": 14,
"sqs.SendMessage": 2,
"sqs.ReceiveMessage": 1,
"sns.Publish": 1,
"dynamodb.PutItem": 0,
"dynamodb.GetItem": 0,
"dynamodb.BatchWriteItem": 0,
"dynamodb.BatchGetItem": 0,
"lambda.Invoke": 0
}
],
"sqs": [
{
"queue": "arn:aws:sqs:us-east-1:000000000000:input-dead-letter-queue",
"visible": 0,
"invisible": 0,
"delayed": 0
},
{
"queue": "arn:aws:sqs:us-east-1:000000000000:input-queue",
"visible": 2,
"invisible": 0,
"delayed": 0
},
{
"queue": "arn:aws:sqs:us-east-1:000000000000:recovery-queue",
"visible": 0,
"invisible": 0,
"delayed": 0
}
],
"sns": [
{
"topic_arn": "arn:aws:sns:us-east-1:000000000000:localstack-topic",
"published": 1,
"delivered": 0,
"failed": 0
}
],
"timestamp": 1704986115.3762584
}
```
### Trace logs
#### Lambda
Find lambda traces in
```bash
/var/lib/localstack/cache/observability/traces-lambda-events/
```
Here is an example:
```json
{"timestamp": 1704984270.1660516, "event": "enqueued", "request_id": "ad2df0ed-c952-4f48-881c-8b944dad44c6", "lambda_arn": "arn:aws:lambda:us-east-1:000000000000:function:test-lambda-perf-e0c504b2", "failure_cause": null}
{"timestamp": 1704984270.184178, "event": "enqueued", "request_id": "d5d2efb3-e781-411a-b718-e2345c118c39", "lambda_arn": "arn:aws:lambda:us-east-1:000000000000:function:test-lambda-perf-e0c504b2", "failure_cause": null}
{"timestamp": 1704984270.3365452, "event": "submitted", "request_id": "ad2df0ed-c952-4f48-881c-8b944dad44c6", "lambda_arn": "arn:aws:lambda:us-east-1:000000000000:function:test-lambda-perf-e0c504b2", "failure_cause": null}
{"timestamp": 1704984270.3368104, "event": "invoking", "request_id": "ad2df0ed-c952-4f48-881c-8b944dad44c6", "lambda_arn": "arn:aws:lambda:us-east-1:000000000000:function:test-lambda-perf-e0c504b2", "failure_cause": null}
{"timestamp": 1704984270.4253993, "event": "submitted", "request_id": "d5d2efb3-e781-411a-b718-e2345c118c39", "lambda_arn": "arn:aws:lambda:us-east-1:000000000000:function:test-lambda-perf-e0c504b2", "failure_cause": null}
```
#### Lambda SQS Event source listeners
Find traces that relate to lambda sqs event source listeners in
```bash
/var/lib/localstack/cache/observability/traces-lambda-sqs/
```
Here is an example of queueing two message to trigger a lambda.
```json
{"timestamp": 1705009138.683765, "event": "message_queued", "message_id": "94c3e579-dd40-48a6-bfaa-5d1d04c79044", "event_source_arn": "arn:aws:sqs:us-east-1:000000000000:test-queue-a5d98750", "lambda_arn": null, "request_id": null, "failure_cause": null}
{"timestamp": 1705009138.6840491, "event": "message_queued", "message_id": "00787a9f-1d70-452d-9fec-f25bf7064e32", "event_source_arn": "arn:aws:sqs:us-east-1:000000000000:test-queue-a5d98750", "lambda_arn": null, "request_id": null, "failure_cause": null}
{"timestamp": 1705009139.6799114, "event": "message_dequeued", "message_id": "94c3e579-dd40-48a6-bfaa-5d1d04c79044", "event_source_arn": "arn:aws:sqs:us-east-1:000000000000:test-queue-a5d98750", "lambda_arn": "arn:aws:lambda:us-east-1:000000000000:function:test-lambda-perf-33b02082", "request_id": null, "failure_cause": null}
{"timestamp": 1705009139.6799738, "event": "invoke_queued", "message_id": "94c3e579-dd40-48a6-bfaa-5d1d04c79044", "event_source_arn": "arn:aws:sqs:us-east-1:000000000000:test-queue-a5d98750", "lambda_arn": "arn:aws:lambda:us-east-1:000000000000:function:test-lambda-perf-33b02082", "request_id": "0d616a5e-2511-4c88-a7b2-de0f0a7161ed", "failure_cause": null}
{"timestamp": 1705009139.6801724, "event": "invoke", "message_id": "94c3e579-dd40-48a6-bfaa-5d1d04c79044", "event_source_arn": "arn:aws:sqs:us-east-1:000000000000:test-queue-a5d98750", "lambda_arn": "arn:aws:lambda:us-east-1:000000000000:function:test-lambda-perf-33b02082", "request_id": "0d616a5e-2511-4c88-a7b2-de0f0a7161ed", "failure_cause": null}
{"timestamp": 1705009140.0882578, "event": "invoke_success", "message_id": "94c3e579-dd40-48a6-bfaa-5d1d04c79044", "event_source_arn": "arn:aws:sqs:us-east-1:000000000000:test-queue-a5d98750", "lambda_arn": "arn:aws:lambda:us-east-1:000000000000:function:test-lambda-perf-33b02082", "request_id": "0d616a5e-2511-4c88-a7b2-de0f0a7161ed", "failure_cause": null}
{"timestamp": 1705009140.6862357, "event": "message_dequeued", "message_id": "00787a9f-1d70-452d-9fec-f25bf7064e32", "event_source_arn": "arn:aws:sqs:us-east-1:000000000000:test-queue-a5d98750", "lambda_arn": "arn:aws:lambda:us-east-1:000000000000:function:test-lambda-perf-33b02082", "request_id": null, "failure_cause": null}
{"timestamp": 1705009140.6865497, "event": "invoke_queued", "message_id": "00787a9f-1d70-452d-9fec-f25bf7064e32", "event_source_arn": "arn:aws:sqs:us-east-1:000000000000:test-queue-a5d98750", "lambda_arn": "arn:aws:lambda:us-east-1:000000000000:function:test-lambda-perf-33b02082", "request_id": "101a0017-172b-401b-9381-34aa1a4d3e7c", "failure_cause": null}
{"timestamp": 1705009140.6868262, "event": "invoke", "message_id": "00787a9f-1d70-452d-9fec-f25bf7064e32", "event_source_arn": "arn:aws:sqs:us-east-1:000000000000:test-queue-a5d98750", "lambda_arn": "arn:aws:lambda:us-east-1:000000000000:function:test-lambda-perf-33b02082", "request_id": "101a0017-172b-401b-9381-34aa1a4d3e7c", "failure_cause": null}
{"timestamp": 1705009140.6951976, "event": "invoke_success", "message_id": "00787a9f-1d70-452d-9fec-f25bf7064e32", "event_source_arn": "arn:aws:sqs:us-east-1:000000000000:test-queue-a5d98750", "lambda_arn": "arn:aws:lambda:us-east-1:000000000000:function:test-lambda-perf-33b02082", "request_id": "101a0017-172b-401b-9381-34aa1a4d3e7c", "failure_cause": null}
```