{"id":26999906,"url":"https://github.com/chrishein/s3_uploader","last_synced_at":"2025-04-04T03:18:21.536Z","repository":{"id":4658034,"uuid":"5803818","full_name":"chrishein/s3_uploader","owner":"chrishein","description":"Multithreaded recursive directory upload to S3 using FOG","archived":false,"fork":false,"pushed_at":"2016-08-26T20:43:59.000Z","size":75,"stargazers_count":36,"open_issues_count":5,"forks_count":26,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-04-02T16:03:08.459Z","etag":null,"topics":["aws-s3","fog","ruby"],"latest_commit_sha":null,"homepage":"","language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/chrishein.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2012-09-14T02:26:40.000Z","updated_at":"2021-03-20T16:19:55.000Z","dependencies_parsed_at":"2022-08-06T17:16:36.840Z","dependency_job_id":null,"html_url":"https://github.com/chrishein/s3_uploader","commit_stats":null,"previous_names":[],"tags_count":17,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chrishein%2Fs3_uploader","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chrishein%2Fs3_uploader/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chrishein%2Fs3_uploader/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chrishein%2Fs3_uploader/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/chrishein","download_url":"https://codeload.github.com/chrishein/s3_uploader/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247112756,"owners_count":20885606,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws-s3","fog","ruby"],"created_at":"2025-04-04T03:18:20.206Z","updated_at":"2025-04-04T03:18:21.524Z","avatar_url":"https://github.com/chrishein.png","language":"Ruby","funding_links":[],"categories":[],"sub_categories":[],"readme":"# S3Uploader\n\n[![Gem Version](https://badge.fury.io/rb/s3_uploader.svg)](https://badge.fury.io/rb/s3_uploader)\n[![Build Status](https://travis-ci.org/chrishein/s3_uploader.svg?branch=master)](https://travis-ci.org/chrishein/s3_uploader)\n[![Coverage Status](https://coveralls.io/repos/github/chrishein/s3_uploader/badge.svg?branch=master)](https://coveralls.io/github/chrishein/s3_uploader?branch=master)\n\nMultithreaded recursive directory uploader to S3 using [fog](https://github.com/fog/fog).\n\nIt recursively transverses all contents of the directory provided as source parameter, uploading all files to the destination bucket.\nA destination folder where to put the uploaded files tree inside the bucket can be specified too.\n\nBy default, it uses 5 threads to upload files in parallel, but the number can be configured as well.\n\nFiles are stored as non public if not otherwise specified.\n\nA CLI binary is included.\n\n## Installation\n\nAdd this line to your application's Gemfile:\n\n    gem 's3_uploader'\n\nAnd then execute:\n\n    $ bundle\n\nOr install it yourself as:\n\n    $ gem install s3_uploader\n\n## Usage\n\n```ruby\n\tuploader = S3Uploader::Uploader.new({\n         :s3_key =\u003e YOUR_KEY,\n         :s3_secret =\u003e YOUR_SECRET_KEY,\n         :destination_dir =\u003e 'test/',\n         :region =\u003e 'eu-west-1',\n         :threads =\u003e 10\n\t\t})\n\n  uploader.upload('/tmp/test', 'mybucket')\n```\n\nor\n\n```ruby\n\tS3Uploader.upload('/tmp/test', 'mybucket',\n\t\t{ \t :s3_key =\u003e YOUR_KEY,\n\t\t\t   :s3_secret =\u003e YOUR_SECRET_KEY,\n\t\t\t   :destination_dir =\u003e 'test/',\n\t\t\t   :region =\u003e 'eu-west-1',\n\t\t\t   :threads =\u003e 4,\n\t\t\t   :metadata =\u003e { 'Cache-Control' =\u003e 'max-age=315576000' }\n\t\t})\n```\n\nFormer static method upload_directory is still supported for backwards compatibility.\n\n```ruby\n\tS3Uploader.upload_directory('/tmp/test', 'mybucket', { :destination_dir =\u003e 'test/', :threads =\u003e 4 })\n```\n\nIf no keys are provided, it uses S3_KEY and S3_SECRET environment variables. us-east-1 is the default region.\n\nMetadata headers are documented [here](http://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectPUT.html)\n\nOr as a command line binary\n\n\ts3uploader -r eu-west-1 -k YOUR_KEY -s YOUR_SECRET_KEY -d test/ -t 4 /tmp/test mybucket\n\nAgain, it uses S3_KEY and S3_SECRET environment variables if non provided in parameters.\n\n\ts3uploader -d test/ -t 4 /tmp/test mybucket\n\n## Compress files\n\nIf the `:gzip` options is used, files not already compressed are packed using GZip before upload. A GZip working\ndirectory is required in this case.\n\n```ruby\n  S3Uploader.upload_directory('/tmp/test', 'mybucket',\n    {    :s3_key =\u003e YOUR_KEY,\n         :s3_secret =\u003e YOUR_SECRET_KEY,\n         :destination_dir =\u003e 'test/',\n         :region =\u003e 'eu-west-1',\n         :gzip =\u003e true,\n         :gzip_working_dir =\u003e '/tmp/gzip_working_dir'\n    })\n```\n\n## Contributing\n\n1. Fork it\n2. Create your feature branch (`git checkout -b my-new-feature`)\n3. Commit your changes (`git commit -am 'Added some feature'`)\n4. Push to the branch (`git push origin my-new-feature`)\n5. Create new Pull Request\n\n## Contributors\n\n* [Mark Wagner](https://github.com/theSociableme)\n* [Brandon Hilkert](https://github.com/brandonhilkert)\n* [Philip Cunningham](https://github.com/unsymbol)\n* [Ludwig Bratke](https://github.com/bratke)\n* [John Pignata](https://github.com/jpignata)\n* [eperezks](https://github.com/eperezks)\n\n## License\n\nDistributed under the MIT License. See LICENSE file for further details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchrishein%2Fs3_uploader","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fchrishein%2Fs3_uploader","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchrishein%2Fs3_uploader/lists"}