{"id":17496130,"url":"https://github.com/brint/fluent-plugin-rackspace-cloud-files","last_synced_at":"2025-04-02T14:13:25.109Z","repository":{"id":36524931,"uuid":"40830689","full_name":"brint/fluent-plugin-rackspace-cloud-files","owner":"brint","description":"Rackspace Cloud Files output plugin for Fluent event collector, based on fluent-plugin-swift","archived":false,"fork":false,"pushed_at":"2015-09-07T16:27:33.000Z","size":260,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"master","last_synced_at":"2025-03-10T21:04:45.010Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://rubygems.org/gems/fluent-plugin-rackspace-cloud-files","language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/brint.png","metadata":{"files":{"readme":"README.rdoc","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2015-08-16T17:38:09.000Z","updated_at":"2015-08-17T01:24:22.000Z","dependencies_parsed_at":"2022-09-09T01:01:00.371Z","dependency_job_id":null,"html_url":"https://github.com/brint/fluent-plugin-rackspace-cloud-files","commit_stats":null,"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/brint%2Ffluent-plugin-rackspace-cloud-files","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/brint%2Ffluent-plugin-rackspace-cloud-files/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/brint%2Ffluent-plugin-rackspace-cloud-files/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/brint%2Ffluent-plugin-rackspace-cloud-files/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/brint","download_url":"https://codeload.github.com/brint/fluent-plugin-rackspace-cloud-files/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246828503,"owners_count":20840474,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-19T14:59:18.739Z","updated_at":"2025-04-02T14:13:25.083Z","avatar_url":"https://github.com/brint.png","language":"Ruby","funding_links":[],"categories":[],"sub_categories":[],"readme":"= Rackspace Cloud Files output plugin for Fluent event collector\n\n== Overview\n\n{\u003cimg src=\"https://travis-ci.org/brint/fluent-plugin-rackspace-cloud-files.svg\" alt=\"Build Status\" /\u003e}[https://travis-ci.org/brint/fluent-plugin-rackspace-cloud-files]\n{\u003cimg src=\"https://badge.fury.io/rb/fluent-plugin-rackspace-cloud-files.svg\" alt=\"Gem Version\" /\u003e}[http://badge.fury.io/rb/fluent-plugin-rackspace-cloud-files]\n{\u003cimg src=\"https://gemnasium.com/brint/fluent-plugin-rackspace-cloud-files.svg\" alt=\"Dependency Status\" /\u003e}[https://gemnasium.com/brint/fluent-plugin-rackspace-cloud-files]\n\nThis has only been tested on fluentd version 0.12.12.\n\n*rackspace-cloud-files* output plugin buffers event logs in local file and upload it to Rackspace Cloud Files periodically.\n\nThis plugin splits files exactly by using the time of event logs (not the time when the logs are received). For example, a log '2011-01-02 message B' is reached, and then another log '2011-01-03 message B' is reached in this order, the former one is stored in \"20110102.gz\" file, and latter one in \"20110103.gz\" file.\n\n\n== Installation\n\nSimply use RubyGems:\n\n     gem install fluent-plugin-rackspace-cloud-files\n\nIf you have fluentd installed, you will need to use it's embedded ruby installation to complete the install:\n\n     /usr/sbin/td-agent-gem install fluent-plugin-rackspace-cloud-files\n\n\n== Configuration\n\n    \u003cmatch pattern\u003e\n      type rackspace_cloud_files\n\n      rackspace_auth_url https://identity.api.rackspacecloud.com/v2.0\n      rackspace_username rax-user\n      rackspace_api_key apikey\n      rackspace_container container_name\n      rackspace_region dfw\n\n      object_key_format %{path}%{time_slice}_%{index}.%{file_extension}\n      ssl_verify false\n\n      path logs/\n      buffer_path /var/log/td-agent/rackspace-cloud-files\n\n      time_slice_format %Y%m%d-%H\n      time_slice_wait 10m\n      utc\n    \u003c/match\u003e\n\n[rackspace_auth_url] Authentication URL. Defaults to the Rackspace US endpoint.\n\n[rackspace_username] Authentication User Name.\n\n[rackspace_api_key] Authentication API Key.\n\n[rackspace_container] Container name.\n\n[object_key_format] The format of Rackspace Cloud Files object keys. You can use several built-in variables:\n\n- %{path}\n- %{time_slice}\n- %{index}\n- %{file_extension}\n\nto decide keys dynamically.\n\n%{path} is exactly the value of *path* configured in the configuration file. E.g., \"logs/\" in the example configuration above.\n%{time_slice} is the time-slice in text that are formatted with *time_slice_format*.\n%{index} is the sequential number starts from 0, increments when multiple files are uploaded to S3 in the same time slice.\n%{file_extention} is always \"gz\" for now.\n\nThe default format is \"%{path}%{time_slice}_%{index}.%{file_extension}\".\n\nFor instance, using the example configuration above, actual object keys on Rackspace Cloud Files will be something like:\n\n    \"logs/20130111-22_0.gz\"\n    \"logs/20130111-23_0.gz\"\n    \"logs/20130111-23_1.gz\"\n    \"logs/20130112-00_0.gz\"\n\nWith the configuration:\n\n    object_key_format %{path}/events/ts=%{time_slice}/events_%{index}.%{file_extension}\n    path log\n    time_slice_format %Y%m%d-%H\n\nYou get:\n\n    \"log/events/ts=20130111-22/events_0.gz\"\n    \"log/events/ts=20130111-23/events_0.gz\"\n    \"log/events/ts=20130111-23/events_1.gz\"\n    \"log/events/ts=20130112-00/events_0.gz\"\n\nThe {fluent-mixin-config-placeholders}[https://github.com/tagomoris/fluent-mixin-config-placeholders] mixin is also incorporated, so additional variables such as %{hostname}, %{uuid}, etc. can be used in the s3_object_key_format. This could prove useful in preventing filename conflicts when writing from multiple servers.\n\n    object_key_format %{path}/events/ts=%{time_slice}/events_%{index}-%{hostname}.%{file_extension}\n\n[store_as] archive format on Rackspace Cloud Files. You can use serveral format:\n\n- gzip (default)\n- json\n- text\n\n[auto_create_container] Create Cloud Files container if it does not exists. Default is true.\n\n[path] path prefix of the files on Cloud Files. Default is \"\" (no prefix).\n\n[buffer_path (required)] path prefix of the files to buffer logs.\n\n[time_slice_format] Format of the time used as the file name. Default is '%Y%m%d'. Use '%Y%m%d%H' to split files hourly.\n\n[time_slice_wait] The time to wait old logs. Default is 10 minutes. Specify larger value if old logs may reache.\n\n[utc] Use UTC instead of local time.\n\n\n== Copyright\n\nCopyright:: Copyright (c) 2015 Brint O'Hearn.\n\nThis software is based on fluent-plugin-swift ( https://github.com/yuuzi41/fluent-plugin-swift ), written by Yuji Hagiwara, licensed by Apache License, Version 2.0.\n\nLicense::   Apache License, Version 2.0\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbrint%2Ffluent-plugin-rackspace-cloud-files","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbrint%2Ffluent-plugin-rackspace-cloud-files","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbrint%2Ffluent-plugin-rackspace-cloud-files/lists"}