{"id":17241786,"url":"https://github.com/j256/cloudwatch-logback-appender","last_synced_at":"2025-03-17T11:30:32.245Z","repository":{"id":18693014,"uuid":"84955557","full_name":"j256/cloudwatch-logback-appender","owner":"j256","description":"Appender that publishes logback log entries to AWS CloudWatch","archived":false,"fork":false,"pushed_at":"2023-11-29T21:47:42.000Z","size":140,"stargazers_count":58,"open_issues_count":12,"forks_count":39,"subscribers_count":7,"default_branch":"master","last_synced_at":"2025-02-27T23:05:18.553Z","etag":null,"topics":["aws","aws-cloudwatch","cloudwatch","java","logback-appender"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"isc","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/j256.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":"j256","patreon":"j256"}},"created_at":"2017-03-14T14:00:27.000Z","updated_at":"2025-01-22T11:07:30.000Z","dependencies_parsed_at":"2024-10-27T12:52:08.064Z","dependency_job_id":"9a46b229-7e9b-4306-a71c-7bf5d6d3639d","html_url":"https://github.com/j256/cloudwatch-logback-appender","commit_stats":null,"previous_names":[],"tags_count":19,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/j256%2Fcloudwatch-logback-appender","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/j256%2Fcloudwatch-logback-appender/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/j256%2Fcloudwatch-logback-appender/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/j256%2Fcloudwatch-logback-appender/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/j256","download_url":"https://codeload.github.com/j256/cloudwatch-logback-appender/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243858056,"owners_count":20359271,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws","aws-cloudwatch","cloudwatch","java","logback-appender"],"created_at":"2024-10-15T06:11:21.067Z","updated_at":"2025-03-17T11:30:31.648Z","avatar_url":"https://github.com/j256.png","language":"Java","funding_links":["https://github.com/sponsors/j256","https://patreon.com/j256"],"categories":[],"sub_categories":[],"readme":"Logback log appender for AWS CloudWatch\n=======================================\n\n# Background\n\nThis package provides a logback appender that writes its log events to Cloudwatch.  Before you say it,\nthere seem to be many projects like this out there but I could find none of them that were\nself-contained and that were published to the central Maven repo.\n\n* Code available from the [git repository](https://github.com/j256/cloudwatch-logback-appender).  [![CircleCI](https://circleci.com/gh/j256/cloudwatch-logback-appender.svg?style=svg)](https://circleci.com/gh/j256/cloudwatch-logback-appender) [![CodeCov](https://img.shields.io/codecov/c/github/j256/cloudwatch-logback-appender.svg)](https://codecov.io/github/j256/cloudwatch-logback-appender/)\n* Maven packages are published via [![Maven Central](https://maven-badges.herokuapp.com/maven-central/com.j256.cloudwatchlogbackappender/cloudwatchlogbackappender/badge.svg?style=flat-square)](https://maven-badges.herokuapp.com/maven-central/com.j256.cloudwatchlogbackappender/cloudwatchlogbackappender/) [![javadoc](https://javadoc.io/badge2/com.j256.cloudwatchlogbackappender/cloudwatchlogbackappender/javadoc.svg)](https://javadoc.io/doc/com.j256.cloudwatchlogbackappender/cloudwatchlogbackappender)\n\nEnjoy. Gray Watson\n\n# Maven Configuration\n\n``` xml\n\u003cdependencies\u003e\n\t\u003cdependency\u003e\n\t\t\u003cgroupId\u003ecom.j256.cloudwatchlogbackappender\u003c/groupId\u003e\n\t\t\u003cartifactId\u003ecloudwatchlogbackappender\u003c/artifactId\u003e\n\t\t\u003c!-- NOTE: change the version to the most recent release version from the repo --\u003e\n\t\t\u003cversion\u003e2.0\u003c/version\u003e\n\t\u003c/dependency\u003e\n\u003c/dependencies\u003e\n```\n\n## Dependencies\n\nBy default the appender has dependencies on logback (duh) but also the log (cloudwatch) and ec2 AWS SDK\npackages.  You can add a exclusion for these packages if you want to depend on different versions.\n\n``` xml\n\u003cdependency\u003e\n\t\u003cgroupId\u003ecom.amazonaws\u003c/groupId\u003e\n\t\u003cartifactId\u003eaws-java-sdk-logs\u003c/artifactId\u003e\n\t\u003cversion\u003e1.11.914\u003c/version\u003e\n\u003c/dependency\u003e\n\u003cdependency\u003e\n\t\u003cgroupId\u003ecom.amazonaws\u003c/groupId\u003e\n\t\u003cartifactId\u003eaws-java-sdk-ec2\u003c/artifactId\u003e\n\t\u003cversion\u003e1.11.914\u003c/version\u003e\n\u003c/dependency\u003e\n```\n\n# logback.xml Configuration\n\nMinimal logback appender configuration:\n\n``` xml\n\u003cappender name=\"CLOUDWATCH\" class=\"com.j256.cloudwatchlogbackappender.CloudWatchAppender\"\u003e\n\t\u003cregion\u003eus-east-1\u003c/region\u003e\n\t\u003clogGroup\u003eyour-log-group-name-here\u003c/logGroup\u003e\n\t\u003clogStream\u003eyour-log-stream-name-here\u003c/logStream\u003e\n\t\u003clayout\u003e\n\t\t\u003c!-- possible layout pattern --\u003e\n\t\t\u003cpattern\u003e[%thread] %level %logger{20} - %msg%n%xThrowable\u003c/pattern\u003e\n\t\u003c/layout\u003e\n\u003c/appender\u003e\n```\n\nCloudwatch unfortunately does not allow multiple hosts to write to the same log-stream.  If multiple servers are writing\nlogs, you should configure the log-stream name with an instance-name suffix or something.  The `logStream` name setting\nuses the `Ec2PatternLayout` to generate the name, which can also be used to format your log lines.  This allows you to\nuse the standard `%token` such as `%date` in the name of the log-stream – see the\n[logback documentation](http://logback.qos.ch/manual/layouts.html#conversionWord).  The `Ec2PatternLayout` class also\nadds support for additional tokens:\n\n| Property | Description |\n| -------- | ----------- |\n| `instanceName` | Name of the EC2 instance or ID if the name is not available. |\n| `instance` | Same as instanceName. |\n| `in` | Same as instanceName. |\n| `instanceId` | ID of the EC2 instance. |\n| `iid` | Same as instanceId. |\n| `uuid` | Random UUID as a string |\n| `hostName` | Name of the host from `InetAddress.getLocalHost()`. |\n| `host` | Same as hostName. |\n| `hostAddress` | IP address of the host from `InetAddress.getLocalHost()`. |\n| `address` | Same as hostAddress. |\n| `addr` | Same as hostAddress. |\n| `systemProperty` | Value of a system-property whose name is set as an {option}.  Ex: %systemProperty{os.version}`. |\n| `property` | Same as systemProperty. |\n| `prop` | Same as systemProperty. |\n| `systemEnviron` | Value of a environmental variable whose name is set as an {option}.  Ex: %systemEnviron{SHELL}`. |\n| `environ` | Same as systemEnviron. |\n| `env` | Same as systemEnviron. |\n\nFor example:\n\n``` xml\n\t\u003clogGroup\u003eyour-log-group-name-here\u003c/logGroup\u003e\n\t\u003clogStream\u003egeneral-%instance-%date{yyyyMMdd,UTC}-%uuid\u003c/logStream\u003e\n```\n\nThis will generate a log-stream name with the prefix \"general-\" and then with the instance-name,\ndate in UTC timezone, and a random UUID.\n\n**NOTE:** The instance-name and instance-id tokens will only work when running on an EC2 instance that\nsupports the EC2MetadataUtils methods for looking up the information.  You can call\n`Ec2InstanceNameConverter.setInstanceName(...)` or `Ec2InstanceIdConverter.setInstanceId(...)` early in your\nprogram if you want to set them yourself. \n\n**NOTE:** `logGroup` must match the regex pattern `[\\.\\-_/#A-Za-z0-9]+`.  `logStream` cannot contain the ':' character\nwhich will be replaced by '_'.\n\nThe appender also adds the support for the previous list of % tokens to be expanded on each log line:\n\n``` xml\n\u003cappender name=\"CLOUDWATCH\" class=\"com.j256.cloudwatchlogbackappender.CloudWatchAppender\"\u003e\n\t...\n\t\u003clayout class=\"com.j256.cloudwatchlogbackappender.Ec2PatternLayout\"\u003e\n\t\t\u003cpattern\u003e\\[%instance\\] \\[%thread\\] %level %logger{20} - %msg%n%xThrowable\u003c/pattern\u003e\n\t\u003c/layout\u003e\n```\n\nHere is the complete list of the appender properties.\n\n| Property | Type | Default | Description |\n| -------- | ---- | ------- | ----------- |\n| `region` | *string* | none | AWS region needed by CloudWatch API |\n| `logGroup` | *string* | none | Log group name |\n| `logStream` | *string* | none | Log stream name |\n| `accessKeyId` | *string* | none | AWS API access key ID, see AWS Permissions below.\u003cbr /\u003e  Code will use ```DefaultAWSCredentialsProviderChain``` by default. |\n| `secretKey` | *string* | none | AWS API secret key, see AWS Permissions below.\u003cbr /\u003e  Code will use ```DefaultAWSCredentialsProviderChain``` by default. |\n| `maxBatchSize` | *int* | 128 | Maximum number of log events put into CloudWatch in single request. |\n| `maxBatchTimeMillis` | *long* | 5000 | Maximum time in milliseconds to collect log events to submit batch. |\n| `maxQueueWaitTimeMillis` | *long* | 100 | Maximum time in milliseconds to wait if internal queue is full before using the emergency appender (see below). |\n| `initialWaitTimeMillis` | *long* | 0 | Initial wait time before logging messages.  Helps if server needs to configure itself initially. |\n| `internalQueueSize` | *int* | 8192 | Size of the internal log event queue. |\n| `createLogDests` | *boolean* | true | Create the CloudWatch log and stream if they don't exist. |\n| `maxEventMessageSize` | *int* | 256k | Maximum size of event message before it is truncated or sent to emergency appender. |\n| `truncateEventMessages` | *boolean* | true | If an event it too large, should the message be truncated.  If false then it will be sent to emergency appender. |\n| `copyEvents` | *boolean* | true | Copies the event for logging by the background thread. |\n| `printRejectedEvents` | *boolean* | false | Print any rejected events to stderr if the emergency appender doesn't work. |\n\n## Emergency Appender\n\nSince this appender is queuing up log events and then writing them remotely, there are a number of situations which\nmight result in log events not getting remoted correctly.  To protect against this, you can add in an \"emergency\"\nappender to write events to the console or a file by adding the following to your CLOUDWATCH appender stanza:\n\n``` xml\n\u003cappender name=\"CLOUDWATCH\" class=\"com.j256.cloudwatchlogbackappender.CloudWatchAppender\"\u003e\n\t...\n\t\u003cappender-ref ref=\"EMERGENCY_FILE\" /\u003e\n```\n\nThis appender will be used if:\n\n* there was some problem configuring the CloudWatch or other AWS APIs\n* the internal queue fills up and messages can't be written remotely fast enough\n* there was some problem with the actual put events CloudWatch call – maybe a transient network failure\n\nIf no emergency appender is configured and a problem does happen then the log messages will be not be persisted.\n\n# AWS Permissions\n\nYou can specify the AWS CloudWatch permissions in a number of ways.  If you use the `accessKeyId` and `secretKey`\nsettings in the `logback.xml` file then the appender will use those credentials directly.  You can also set the\n`cloudwatchappender.aws.accessKeyId` and `cloudwatchappender.aws.secretKey` Java System properties which will be\nused.  If neither of those are specified then the appender will use the `DefaultAWSCredentialsProviderChain` which\nlooks for the access and secret keys in:\n\n* Environment Variables: `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` or `AWS_ACCESS_KEY` and `AWS_SECRET_KEY`\n* Java System Properties: `aws.accessKeyId` and `aws.secretKey`\n* Credential file at the default location (`~/.aws/credentials`) shared by all AWS SDKs and the AWS CLI\n* Instance profile credentials delivered through the Amazon EC2 metadata service\n\n## IAM Permissions\n\nWhen making any AWS API calls, we typically create a IAM user with specific permissions so if any API keys are stolen,\nthe hacker only have limited access to our AWS services.  To get the appender to be able to publish to CloudWatch,\nthe following IAM policy is required to create the log group and put log events to CloudWatch.\n\nThe `logs:CreateLogGroup` and `logs:CreateLogStream` actions are only required if the appender is creating the\nlog-group and stream itself (see `createLogDests` option above).  The `ec2:DescribeTags` action is only required\nif you want the appender to query for the ec2 instance name it is on – see `Ec2PatternLayout` above.\n\n```json\n{\n    \"Version\": \"2012-10-17\",\n    \"Statement\": [\n        {\n            \"Effect\": \"Allow\",\n            \"Action\": [\n                \"logs:CreateLogGroup\",\n                \"logs:CreateLogStream\",\n                \"logs:DescribeLogGroups\",\n                \"logs:DescribeLogStreams\",\n                \"logs:PutLogEvents\",\n                \"ec2:DescribeTags\"\n            ],\n            \"Resource\": [\n                \"*\"\n            ]\n        }\n    ]\n}\n```\n\nI couldn't figure out how to restrict to all ec2 instances.  If you are only doing log requests then\nyou should be able to limit it to the resource `arn:aws:logs:*:*:*`.\n\n# ChangeLog Release Notes\n\nSee the [ChangeLog.txt file](src/main/javadoc/doc-files/changelog.txt).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fj256%2Fcloudwatch-logback-appender","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fj256%2Fcloudwatch-logback-appender","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fj256%2Fcloudwatch-logback-appender/lists"}