https://github.com/daisaru11/beats-aws-ecs
Beats processor plugin for AWS ECS Task
https://github.com/daisaru11/beats-aws-ecs
beats ecs go
Last synced: 3 months ago
JSON representation
Beats processor plugin for AWS ECS Task
- Host: GitHub
- URL: https://github.com/daisaru11/beats-aws-ecs
- Owner: daisaru11
- Created: 2019-03-17T08:31:45.000Z (over 6 years ago)
- Default Branch: master
- Last Pushed: 2019-05-04T10:44:54.000Z (over 6 years ago)
- Last Synced: 2024-04-22T15:25:49.172Z (over 1 year ago)
- Topics: beats, ecs, go
- Language: Go
- Homepage:
- Size: 26.4 KB
- Stars: 1
- Watchers: 2
- Forks: 2
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
## Beats plugin for AWS ECS
[Beats](https://github.com/elastic/beats) processor plugin for AWS ECS Task.
This plugin appends AWS ECS Task and container metadata to beats events.
```
{
"@timestamp": "2019-04-14T16:17:49.374Z",
...
"aws_ecs_task": {
"task": {
"cluster": "ecs-local-cluster",
"arn": "arn:aws:ecs:us-west-2:111111111111:task/ecs-local-cluster/37e873f6-37b4-42a7-af47-eac7275c6152",
"family": "esc-local-task-definition",
"revision": "1"
},
"container": {
"image_id": "sha256:031c45582fce6e8234175ed01cfea828a8f096e5b1ed3cdd41142d2a40244d27",
"name": "test-nginx",
"docker_name": "test-nginx",
"image": "nginx:alpine"
}
},
"message": "192.168.176.1 - - [14/Apr/2019:15:26:07 +0000] \"GET / HTTP/1.1\" 200 612 \"-\" \"curl/7.54.0\" \"-\"",
...
}
```
- Supported beats
- Filebeat
- Supported endpoint versions of AWS ECS Task Metadata
- v2
- v3
## Usage
### Filebeat
Add `add_aws_ecs_task_metadata` processor to your filebeat config.
```yaml:filebeat.yml
processors:
- add_aws_ecs_task_metadata:
indexers:
- container_name:
matchers:
- container_name:
name: nginx
```
Run a filebeat container as a sidecar with the container which outputs logs you want to collect. The logs is shared through volume.
```json:taskDefinition.json
{
"family": "nginx",
"taskRoleArn": "xxxx",
"executionRoleArn": "xxxx",
"networkMode": "awsvpc",
"cpu": 256,
"memory": 512,
"requiresCompatibilities": ["FARGATE"],
"volumes": [
{
"name": "nginx_logs"
}
],
"containerDefinitions": [
{
"essential": true,
"image": "nginx:alpine",
"name": "nginx",
"portMappings": [
{
"containerPort": 80
}
],
"command": [
"/bin/sh",
"-c",
"rm /var/log/nginx/*.log && exec nginx -g 'daemon off;'"
],
"mountPoints": [
{
"sourceVolume": "nginx_logs",
"containerPath": "/var/log/nginx"
}
]
},
{
"essential": true,
"image": "daisaru11/beats-aws-ecs:filebeat",
"name": "filebeat",
"secrets": [
{
"name": "FILEBEAT_CONFIG",
"valueFrom": "xxxx"
}
],
"command": [
"/bin/sh",
"-c",
"echo \"$FILEBEAT_CONFIG\" > /tmp/filebeat.yml && filebeat --plugin beats-aws-ecs.so -e -c /tmp/filebeat.yml"
],
"volumesFrom": [
{
"sourceContainer": "nginx"
}
]
}
]
}
```