{"id":16645129,"url":"https://github.com/cosmo0920/embulk-input-cloudwatch_logs","last_synced_at":"2026-03-10T01:31:02.128Z","repository":{"id":62557908,"uuid":"250209342","full_name":"cosmo0920/embulk-input-cloudwatch_logs","owner":"cosmo0920","description":"Cloudwatch Logs input plugin for Embulk","archived":false,"fork":false,"pushed_at":"2023-08-02T01:37:55.000Z","size":89,"stargazers_count":1,"open_issues_count":0,"forks_count":2,"subscribers_count":1,"default_branch":"master","last_synced_at":"2026-03-03T13:53:36.366Z","etag":null,"topics":["aws","cloudwatch-logs","embulk","java"],"latest_commit_sha":null,"homepage":null,"language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cosmo0920.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-03-26T09:05:05.000Z","updated_at":"2020-04-15T06:10:43.000Z","dependencies_parsed_at":"2024-11-17T23:42:56.262Z","dependency_job_id":null,"html_url":"https://github.com/cosmo0920/embulk-input-cloudwatch_logs","commit_stats":{"total_commits":14,"total_committers":1,"mean_commits":14.0,"dds":0.0,"last_synced_commit":"a328953945449b0928a6cddf4ae27cd82981f93e"},"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/cosmo0920/embulk-input-cloudwatch_logs","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cosmo0920%2Fembulk-input-cloudwatch_logs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cosmo0920%2Fembulk-input-cloudwatch_logs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cosmo0920%2Fembulk-input-cloudwatch_logs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cosmo0920%2Fembulk-input-cloudwatch_logs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cosmo0920","download_url":"https://codeload.github.com/cosmo0920/embulk-input-cloudwatch_logs/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cosmo0920%2Fembulk-input-cloudwatch_logs/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30320885,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-09T20:05:46.299Z","status":"ssl_error","status_checked_at":"2026-03-09T19:57:04.425Z","response_time":61,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws","cloudwatch-logs","embulk","java"],"created_at":"2024-10-12T08:13:22.150Z","updated_at":"2026-03-10T01:31:02.099Z","avatar_url":"https://github.com/cosmo0920.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Cloudwatch Logs input plugin for Embulk\n\n## Overview\n\n* **Plugin type**: input\n* **Resume supported**: no\n* **Cleanup supported**: yes\n* **Guess supported**: no\n\n## Configuration\n\n- **log_group_name**: CloudWathcLogs log group name (string, required)\n\n- **log_stream_name**: CloudWathcLogs log stream name (string, default: `null`)\n\n- **use_log_stream_name_prefix**: Whether using **log_stream_name** as stream name prefix (boolean, default: `false`)\n\n- **column_name**: Column name for CloudWatchLogs' message column (string, default: `\"message\"`)\n\n- **start_time**: CloudWatchLogs start-time (string, optional, default: `null`)\n\n- **end_time**: CloudWatchLogs end-time (string, optional, default: `null`)\n\n- **time_range_format**: Time range format. Internally, it will be used as `yyyy-MM-dd HH:mm:ss` by default. (string, optional, default: `null`)\n\n- **region** CloudWatchLogs region. Currently this should be required. (string, optional)\n\n- **authentication_method**: name of mechanism to authenticate requests (basic, env, instance, profile, properties, anonymous, or session. default: basic)\n\n  - \"basic\": uses access_key_id and secret_access_key to authenticate.\n\n    - **aws_access_key_id**: AWS access key ID (string, required)\n\n    - **aws_secret_access_key**: AWS secret access key (string, required)\n\n  - \"env\": uses `AWS_ACCESS_KEY_ID` (or `AWS_ACCESS_KEY`) and `AWS_SECRET_KEY` (or `AWS_SECRET_ACCESS_KEY`) environment variables.\n\n  - \"instance\": uses EC2 instance profile.\n\n  - \"profile\": uses credentials written in a file. Format of the file is as following, where `[...]` is a name of profile.\n\n    - **aws_profile_file**: path to a profiles file. (string, default: given by `AWS_CREDENTIAL_PROFILES_FILE` environment varialbe, or ~/.aws/credentials).\n\n    - **aws_profile_name**: name of a profile. (string, default: `\"default\"`)\n\n    ```\n    [default]\n    aws_access_key_id=YOUR_ACCESS_KEY_ID\n    aws_secret_access_key=YOUR_SECRET_ACCESS_KEY\n\n    [profile2]\n    ...\n    ```\n\n  - \"properties\": uses aws.accessKeyId and aws.secretKey Java system properties.\n\n  - \"anonymous\": uses anonymous access. This auth method can access only public files.\n\n  - \"session\": uses temporary-generated access_key_id, secret_access_key and session_token.\n\n    - **aws_access_key_id**: AWS access key ID (string, required)\n\n    - **aws_secret_access_key**: AWS secret access key (string, required)\n\n    - **aws_session_token**: session token (string, required)\n\n  - \"default\": uses AWS SDK's default strategy to look up available credentials from runtime environment. This method behaves like the combination of the following methods.\n\n    1. \"env\"\n    1. \"properties\"\n    1. \"profile\"\n    1. \"instance\"\n\n## Example\n\n```yaml\nin:\n  type: cloudwatch_logs\n  log_group_name: fluentd\n  log_stream_name: cloudwatchlogs_test\n  use_log_stream_name_prefix: true\n  region: ap-northeast-1\n  aws_access_key_id: ABCXYZ123ABCXYZ123\n  aws_secret_access_key: AbCxYz123aBcXyZ123\n```\n\nSpecifying time range is also supported.\n`start_time` and `end_time` are corresponds to `startTime` and `endTime` in GetLogEvents request.\nThey are handled as number of milliseconds after Jan 1, 1970 00:00:00 UTC.\n\nref: https://docs.aws.amazon.com/AmazonCloudWatchLogs/latest/APIReference/API_GetLogEvents.html#API_GetLogEvents_RequestSyntax\n\n```yaml\n  start_time: 2020-01-18 00:00:00Z\n  end_time: 2020-03-20 00:00:00Z\n  time_range_format: \"YY-MM-dd HH:mm:ss'Z'\"\n```\n\nTo use `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` environment variables:\n\n```yaml\nin:\n  type: cloudwatch_logs\n  log_group_name: fluentd\n  log_stream_name: cloudwatchlogs_test\n  use_log_stream_name_prefix: true\n  region: ap-northeast-1\n  authentication_method: env\n```\n\n## Build\n\n```\n$ ./gradlew gem  # -t to watch change of files and rebuild continuously\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcosmo0920%2Fembulk-input-cloudwatch_logs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcosmo0920%2Fembulk-input-cloudwatch_logs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcosmo0920%2Fembulk-input-cloudwatch_logs/lists"}