{"id":16746552,"url":"https://github.com/fluent/fluent-plugin-s3","last_synced_at":"2025-05-13T19:11:42.391Z","repository":{"id":37622720,"uuid":"2472984","full_name":"fluent/fluent-plugin-s3","owner":"fluent","description":"Amazon S3 input and output plugin for Fluentd","archived":false,"fork":false,"pushed_at":"2025-02-18T05:32:54.000Z","size":659,"stargazers_count":319,"open_issues_count":29,"forks_count":219,"subscribers_count":25,"default_branch":"master","last_synced_at":"2025-05-05T10:06:56.275Z","etag":null,"topics":["aws","fluentd","fluentd-plugin","s3"],"latest_commit_sha":null,"homepage":"https://docs.fluentd.org/output/s3","language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/fluent.png","metadata":{"files":{"readme":"README.md","changelog":"ChangeLog","contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":"AUTHORS","dei":null,"publiccode":null,"codemeta":null}},"created_at":"2011-09-28T05:24:03.000Z","updated_at":"2025-04-14T05:13:05.000Z","dependencies_parsed_at":"2024-12-18T02:31:04.193Z","dependency_job_id":"c8c2b001-6818-4c5f-9f37-9f8ac6ca4d47","html_url":"https://github.com/fluent/fluent-plugin-s3","commit_stats":{"total_commits":535,"total_committers":94,"mean_commits":5.691489361702128,"dds":0.6261682242990654,"last_synced_commit":"95fffb870b8231e4825e198bb9384a79fd182569"},"previous_names":[],"tags_count":92,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fluent%2Ffluent-plugin-s3","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fluent%2Ffluent-plugin-s3/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fluent%2Ffluent-plugin-s3/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fluent%2Ffluent-plugin-s3/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/fluent","download_url":"https://codeload.github.com/fluent/fluent-plugin-s3/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253417880,"owners_count":21905309,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws","fluentd","fluentd-plugin","s3"],"created_at":"2024-10-13T02:07:05.089Z","updated_at":"2025-05-13T19:11:42.371Z","avatar_url":"https://github.com/fluent.png","language":"Ruby","readme":"# Amazon S3 plugin for [Fluentd](http://github.com/fluent/fluentd)\n\n[\u003cimg src=\"https://travis-ci.org/fluent/fluent-plugin-s3.svg?branch=master\"\nalt=\"Build Status\" /\u003e](https://travis-ci.org/fluent/fluent-plugin-s3) [\u003cimg\nsrc=\"https://codeclimate.com/github/fluent/fluent-plugin-s3/badges/gpa.svg\"\n/\u003e](https://codeclimate.com/github/fluent/fluent-plugin-s3)\n\n## Overview\n\n**s3** output plugin buffers event logs in local file and upload it to S3\nperiodically.\n\nThis plugin splits files exactly by using the time of event logs (not the time\nwhen the logs are received). For example, a log '2011-01-02 message B' is\nreached, and then another log '2011-01-03 message B' is reached in this order,\nthe former one is stored in \"20110102.gz\" file, and latter one in\n\"20110103.gz\" file.\n\n**s3** input plugin reads data from S3 periodically. This plugin uses\nSQS queue on the region same as S3 bucket.\nWe must setup SQS queue and S3 event notification before use this plugin.\n\n:warning: Be sure to keep a close eye on S3 costs, as a few user have reported [unexpectedly high costs](https://github.com/fluent/fluent-plugin-s3/issues/160).\n\n## Requirements\n\n| fluent-plugin-s3  | fluentd | ruby |\n|-------------------|---------|------|\n| \u003e= 1.0.0 | \u003e= v0.14.0 | \u003e= 2.1 |\n|  \u003c 1.0.0 | \u003e= v0.12.0 | \u003e= 1.9 |\n\n## Installation\n\nSimply use RubyGems:\n\n    # install latest version\n    $ gem install fluent-plugin-s3 --no-document # for fluentd v1.0 or later\n    # If you need to install specifiv version, use -v option\n    $ gem install fluent-plugin-s3 -v 1.3.0 --no-document\n    # For v0.12. This is for old v0.12 users. Don't use v0.12 for new deployment\n    $ gem install fluent-plugin-s3 -v \"~\u003e 0.8\" --no-document # for fluentd v0.12\n\n\n## Configuration: credentials\n\nBoth S3 input/output plugin provide several credential methods for authentication/authorization.\n\nSee [Configuration: credentials](docs/credentials.md) about details.\n\n## Output Plugin\n\nSee [Configuration: Output](docs/output.md) about details.\n\n## Input Plugin\n\nSee [Configuration: Input](docs/input.md) about details.\n\n## Tips and How to\n\n* [Object Metadata Added To Records](docs/howto.md#object-metadata-added-to-records)\n* [IAM Policy](docs/howto.md#iam-policy)\n* [Use your (de)compression algorithm](docs/howto.md#use-your-decompression-algorithm)\n\n## Migration guide\n\nSee [Migration guide from v0.12](docs/v0.12.md) about details.\n\n## Website, license, et. al.\n\n| Web site          | http://fluentd.org/                       |\n|-------------------|-------------------------------------------|\n| Documents         | http://docs.fluentd.org/                  |\n| Source repository | http://github.com/fluent/fluent-plugin-s3 |\n| Discussion        | http://groups.google.com/group/fluentd    |\n| Author            | Sadayuki Furuhashi                        |\n| Copyright         | (c) 2011 FURUHASHI Sadayuki               |\n| License           | Apache License, Version 2.0               |\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffluent%2Ffluent-plugin-s3","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffluent%2Ffluent-plugin-s3","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffluent%2Ffluent-plugin-s3/lists"}