https://github.com/signiant/paperwatch
Automatically forward CloudWatch logs posted by Amazon services to PaperTrail
https://github.com/signiant/paperwatch
Last synced: 9 months ago
JSON representation
Automatically forward CloudWatch logs posted by Amazon services to PaperTrail
- Host: GitHub
- URL: https://github.com/signiant/paperwatch
- Owner: Signiant
- License: mit
- Created: 2016-08-04T14:16:27.000Z (over 9 years ago)
- Default Branch: master
- Last Pushed: 2024-03-05T12:24:31.000Z (about 2 years ago)
- Last Synced: 2025-04-20T11:43:59.816Z (about 1 year ago)
- Language: JavaScript
- Homepage:
- Size: 29.3 KB
- Stars: 19
- Watchers: 4
- Forks: 14
- Open Issues: 4
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# PaperWatch
A tool for automatically forwarding CloudWatch logs from your lambda functions to Papertrail.
## Setup
Follow these steps to set up and deploy PaperWatch to AWS:
#### 1. Configure
The two lambda functions reads the following configuration information from `config/papertrail.json`
| Key | Required | Type | Description |
| ----- | ----- | ---------- | ----------|
| `host` | Yes | _String_ | The paper trail endpoint's address |
| `port` | Yes | _Number_ | The paper trail endpoint's port |
| `consumer` | Yes | _String_ | The name of the consumer function. Used by the Subscribe function to avoid subscribing the consumer to itself. Do not change unless you are modifying the CFN template.|
| `retentionPeriod` | No | _Number_ | If included, updates the CloudWatch log group's retention period (in days) |
| `sources` | Yes | _Array_ of _Objects_ | List of log sources. A log source is identified by the _prefix_ property. Log's generated by a source are passed to it's _consumer_ function. Default configuration includes log sources for lambda functions and api gateway |
| `exclude` | No | _Array_ of _Strings_ | List of lambda function names that the Subscribe function should ignore |
| `logLevelExtractor` | Yes | _RegExp_ | Regular expression that matches the incoming CloudWatch logs and extracts the log level from them. The extracted log level will be used in the PaperTrail log. Wrap the log level part in a regex group and you are good to go
| `defaultLogLevel` | Yes | _String_ | If logLevelExtractor is unable to match, use the level provided here.
*Example config:*
```
{
"host": "samplehost.papertrailapp.com",
"port": 12345,
"retentionPeriod": 3
"sources": [
{
"prefix": "/aws/lambda/",
"consumer": "LambdaLogConsumer"
},
{
"prefix": "API-Gateway-Execution-Logs_",
"consumer": "APIGatewayLogConsumer"
}
]
"exclude": [
"TestFunction",
"LoudFunction"
],
"logLevelExtractor": ".* - (error|warn|info|verbose|debug|emerg|alert|crit|notice): ",
"defaultLogLevel": "info"
}
```
#### 2. Build
Running the _build_ script will install dependencies and create the .zip deployment package (as lambda.zip)
```
paperwatch $ npm run build
```
#### 3. Deploy
First, upload the deployment package to an s3 bucket. Take note of the bucket name, you will need this later.
This can be done with the aws-cli as follows:
```
paperwatch $ aws s3 cp ./lambda.zip s3://
```
Next, create a new cloudformation stack using from the template at ```cfn/deploy.cfn.json```.
Enter a stack name. You will also be prompted to enter the following parameters:
- SourceBucket - the name of the s3 bucket you uploaded the deployment package to
- SourceKey - name of the deployment package in the s3 bucket (lambda.zip), including any prefixes if applicable
Once deployed, logs will be forwarded to papertrail for all new lambda functions.
Existing functions must be subscribed manually.
### Manual Subscription
To manually subscribe a function, navigate to the CloudWatch console and follow these steps:
1. Locate the log group for the function you wish to subscribe, selecting it by checking the box on the left
2. Open the _Actions_ dropdown and choose _Subscription filters_ -> _Create lambda subscription filter_
3. From the Lambda Function dropdown, select the consumer function (LambdaLogConsumer or APIGatewayLogConsumer by default)
4. From the Log Format dropdown, select _Other_ and click next
5. For the _Subscription filter name_ enter Papertrail
6. Click _Start Streaming_
PaperWatch will now begin forwarding the functions logs to Papertrail