https://github.com/digitalkaoz/aws-log-shipper
Ship AWS Logs (CF, ALB, ELB, S3) stored in S3 to Cloudwatch (or others)
https://github.com/digitalkaoz/aws-log-shipper
alb aws aws-lambda cloudfront-logs cloudwatch elb logs
Last synced: 4 months ago
JSON representation
Ship AWS Logs (CF, ALB, ELB, S3) stored in S3 to Cloudwatch (or others)
- Host: GitHub
- URL: https://github.com/digitalkaoz/aws-log-shipper
- Owner: digitalkaoz
- Created: 2018-12-03T08:26:01.000Z (over 7 years ago)
- Default Branch: master
- Last Pushed: 2022-12-08T21:39:48.000Z (over 3 years ago)
- Last Synced: 2025-02-01T18:46:10.318Z (over 1 year ago)
- Topics: alb, aws, aws-lambda, cloudfront-logs, cloudwatch, elb, logs
- Language: TypeScript
- Size: 1010 KB
- Stars: 3
- Watchers: 3
- Forks: 0
- Open Issues: 18
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# aws-log-shipper
This module lets you ship AWS Logs from services that are only able to store them in an S3 Bucket (pretty useless) to other Services (e.g. Cloudwatch).
It ships the Data as JSON encoded String (so easy for every System afterwards)
## Installation
```
yarn add aws-log-shipper
```
## Usage
```js
const logShipper = require("aws-log-shipper");
module.exports.handler = logShipper.default;
```
Then configure your Lambda to use `index.handler` as handler endpoint.
You should invoke this Lambda with `s3:PutEvent` Bucket Notification of those Logfiles.
## Implementations
currently we can ship logs from the following services:
* Cloudfront
* ALB
* ELB (not yet, its hard to differntiate from ALB logs just by looking at file itself)
* S3
## Configuration
we configure the Lambda with `ENV` vars completely
* `SHIPPER` : where to ship the logs (atm the only option is `cloudwatch`)
* `CF_LOG_GROUP` : which Cloudwatch Loggroup to use for Cloudfront logs
* `ALB_LOG_GROUP` : which Cloudwatch Loggroup to use for ALB logs
* `ELB_LOG_GROUP` : which Cloudwatch Loggroup to use for ELB logs
* `S3_LOG_GROUP` : which Cloudwatch Loggroup to use for S3 logs
## TODO
this was just a quick evening hack, so most things are a bit rough
* Tests
* Docs
* decouple the ENV vars a bit (to tied to Cloudwatch Shipper)
* differentiate between ELB and ALB logs