{"id":15616435,"url":"https://github.com/texpert/shrine-aws-lambda","last_synced_at":"2025-08-30T09:16:46.752Z","repository":{"id":58032508,"uuid":"356377873","full_name":"texpert/shrine-aws-lambda","owner":"texpert","description":"AWS Lambda integration for Shrine File Attachment toolkit for Ruby applications","archived":false,"fork":false,"pushed_at":"2024-07-14T09:08:16.000Z","size":44,"stargazers_count":6,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-08-11T05:27:06.723Z","etag":null,"topics":["asynchronous","aws","aws-lambda","processing","ruby","s3","shrine","webhooks"],"latest_commit_sha":null,"homepage":"","language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/texpert.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-04-09T19:26:05.000Z","updated_at":"2025-04-25T07:58:21.000Z","dependencies_parsed_at":"2024-07-14T10:25:18.319Z","dependency_job_id":"13806327-2101-4c75-a5d7-e02e809a6db7","html_url":"https://github.com/texpert/shrine-aws-lambda","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"purl":"pkg:github/texpert/shrine-aws-lambda","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/texpert%2Fshrine-aws-lambda","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/texpert%2Fshrine-aws-lambda/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/texpert%2Fshrine-aws-lambda/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/texpert%2Fshrine-aws-lambda/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/texpert","download_url":"https://codeload.github.com/texpert/shrine-aws-lambda/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/texpert%2Fshrine-aws-lambda/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":272829021,"owners_count":25000140,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-30T02:00:09.474Z","response_time":77,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["asynchronous","aws","aws-lambda","processing","ruby","s3","shrine","webhooks"],"created_at":"2024-10-03T07:08:31.429Z","updated_at":"2025-08-30T09:16:46.722Z","avatar_url":"https://github.com/texpert.png","language":"Ruby","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Shrine::Plugins::AwsLambda\nProvides [AWS Lambda] integration for the [Shrine] File Attachment toolkit for Ruby applications\n\nThis is a gem, renamed from the initial [shrine-lambda](https://github.com/texpert/shrine-lambda) to [shrine-aws-lambda](https://github.com/texpert/shrine-aws-lambda) for clarity\n\n## Description\n\n### AWS Lambda\n\n[AWS Lambda] is a serverless computing platform provided by [Amazon] as a part of the [Amazon Web Services]. It is a \ncompute service that could run someone's uploaded code in response to events and/or requests, and automatically manages \nand scales the compute resources required by that code.\n\n### Shrine\n\n[Shrine] is the best and most versatile file attachment toolkit for Ruby applications, developed by [Janko \nMarohnić][Janko]. It has a vast collection of plugins with support for direct uploads, background processing and \ndeleting, processing on upload or on-the-fly, and ability to use with other ORMs\n\n### Shrine-AWS-Lambda\n\nShrine-AWS-Lambda is a plugin for invoking [AWS Lambda] functions for processing files already stored in some [AWS S3 \nbucket][AWS S3]. Specifically, it was designed for invoking an image resizing [AWS Lambda] function like [this \none][lambda-image-resize], but it could be used to invoke any other function, due to [Shrine]'s modular plugin \narchitecture design.\n\nThe function is invoked to run asynchronously. Function's result will be sent by [AWS Lambda] back to the \ninvoking application in a HTTP request's payload. The HTTP request would target a callback URL specified in the \nShrine-AWS-Lambda's setup. So, the invoking application must provide a HTTP endpoint (a webhook) to catch the results.\n\n#### Setup\n\nAdd the Shrine-AWS-Lambda gem to the application's Gemfile:\n\n```ruby\ngem 'shrine-aws-lambda'\n```\n\nRun `$ bundle install` command in the application's root folder to install the gem.\n\n\nNote, that for working with AWS, the AWS credentials (the `access_key_id` and the `secret_access_key`) should be set \neither in the [Shrine] initializer, or in [default profile][AWS profiles] in the `~/.aws` folder.\n\n```ruby\n# config/initializers/shrine.rb:\n\n# ...\n\n  s3_options = { access_key_id:     'your_aws_access_key_id',\n                 secret_access_key: 'your_aws_secret_access_key',\n                 region:            'your AWS bucket region' }\n```\n\nAlso, for Lambda functions to work, various [AWS Lambda permissions] should be managed on the [Amazon Web Services] side.\n\nAdd to the [Shrine]'s initializer file the Shrine-AWS-Lambda plugin registration with the `:callback_url` parameter, \nand the [AWS Lambda] functions list retrieval call (which will retrieve the functions list on application initialization \nand will store the list into the `Shrine.opts[:lambda_function_list]` for further checking):\n\n```ruby\n# config/initializers/shrine.rb:\n\n# ...\n\n  shrine.plugin :aws_lambda, s3_options.merge(callback_url: \"https://#{ENV.fetch('APP_HOST')}/lambda\")\n  Shrine.lambda_function_list\n```\n\nBy default, Shrine-AWS-Lambda is using the S3 bucket named `:cache` for retrieving the original file, and the `:store` \nnamed S3 bucket for storing the resulting files.\n\nShrine-AWS-Lambda uses the [Shrine backgrounding plugin] for asynchronous operation, so this plugin should be also \nincluded into the Shrine's initializer.\n\nHere is a full example of a Shrine initializer of a [Rails] application using a [Roda] endpoint for presigned_url \n(used for direct file uploads to [AWS S3]) and [AWS Lambda] callbacks):\n\n```ruby\n# config/initializers/shrine.rb:\n\n# frozen_string_literal: true\n\nrequire 'shrine'\n\nif Rails.env.test?\n  require 'shrine/storage/file_system'\n\n  Shrine.storages = {\n    cache: Shrine::Storage::FileSystem.new('public', prefix: 'uploads/cache'), \n    store: Shrine::Storage::FileSystem.new('public', prefix: 'uploads/store'),\n  }\nelse\n  require 'shrine/storage/s3'\n\n  aws_credentials = Rails.application.credentials.aws\n\n  s3_options = { access_key_id:     aws_credentials[:access_key_id],\n                 secret_access_key: aws_credentials[:secret_access_key],\n                 region:            'us-east-2' }\n\n  if Rails.env.production?\n    cache_bucket = store_bucket = aws_credentials.aws_s3_bucket\n  else\n    cache_bucket = 'texpert-test-cache'\n    store_bucket = 'texpert-test-store'\n  end\n\n  Shrine.storages = { cache: Shrine::Storage::S3.new(prefix: 'cache', **s3_options.merge!(bucket: cache_bucket)), \n                      store: Shrine::Storage::S3.new(prefix: 'store', **s3_options.merge!(bucket: store_bucket)) }\n\n  ActiveSupport::Reloader.to_prepare do\n   lambda_callback_url = if Rails.env.development? \u0026\u0026 NGROK_ENABLED\n                          \"#{NGROK_URL}/rapi/lambda\"\n                         else\n                          \"https://#{ENV['APP_HOST'] || 'localhost'}/rapi/lambda\"\n                         end\n\n   Shrine.plugin :aws_lambda, s3_options.merge!(callback_url: lambda_callback_url)\n   Shrine.lambda_function_list\n  end\nend\n\nShrine.plugin :activerecord\nShrine.plugin :backgrounding\nShrine.plugin :cached_attachment_data # for forms\n\nShrine.logger = Rails.logger\nShrine.plugin :instrumentation\n\nShrine.plugin :presign_endpoint, presign_options: lambda { |request|\n filename     = request.params['filename']\n extension    = File.extname(filename)\n content_type = Rack::Mime.mime_type(extension)\n\n { content_length_range: 0..1.gigabyte,                         # limit filesize to 1 GB\n   content_disposition: \"attachment; filename=\\\"#{filename}\\\"\", # download with original filename\n   content_type:        content_type }                          # set correct content type\n}\n\nShrine.plugin :rack_file # for non-Rails apps\nShrine.plugin :remote_url, max_size: 1.gigabyte\n\nShrine::Attacher.promote_block { PromoteJob.enqueue(self.class.name, record.class.name, record.id, name, file_data) }\nShrine::Attacher.destroy_block { DeleteJob.enqueue(self.class.name, data) }\n```\n\nTake notice that the promote job is a default, not AWS Lambda job: \n\n```\nShrine::Attacher.promote_block { PromoteJob.enqueue(self.class.name, record.class.name, record.id, name, file_data) }\n```\n\nThis is made to be able to use other than AWS storages in the test environment (like Shrine's `FileSystem` storage) \nand, also, other uploaders which are not using [AWS Lambda]. To use an AWS Lambda job, this job must be overridden \nto a `LambdaPromoteJob` directly in the uploaders' classes which will use [AWS Lambda] - see below in the **Usage** \nchapter. \n \nAnother thing used in this initializer is the [ngrok-wrapper] gem for exposing the localhost to the world for \ncatching the Lambda callback requests.\n\n#### How it works\n\nShrine-AWS-Lambda works in such a way that an \"assembly\" should be created in the `LambdaUploader`, which contains all \nthe information about how the file should be processed. A random generated string is appended to the assembly, stored \ninto the cached file metadata, and used by the Lambda function to sign the requests to the `:lambda_callback_url`, \nalong with the AWS `:access_key_id` from the AWS credentials Lambda function is running with.\n\nProcessing itself happens asynchronously - the invoked Lambda function will issue a PUT HTTP request to the  \n`:lambda_callback_url`, specified in the Shrine's initializer, with the request's payload containing the processing  \nresults.\n\nThe request should be intercepted by a endpoint at the `:lambda_callback_url`, and its payload transferred to the \n`lambda_save` method on successful request authorization. \n\nThe authorization is calculating the HTTP request signature using the random string stored in the cached file and \nthe AWS Lambda function's `:access_key_id` received in the request authorization header. Then, the calculated \nsignature is compared to the received in the same authorization header AWS Lambda signature.\n\n#### Usage\n\nShrine-AWS-Lambda assemblies are built inside the `#lambda_process_versions` method in the `LambdaUploader` class:\n\n```ruby\n# app/uploaders/lambda_uploader.rb:\n\n# frozen_string_literal: true\n\nclass LambdaUploader \u003c Uploader\n  unless Rails.env.test?\n    Attacher.promote_block do\n      LambdaPromoteJob.enqueue(self.class.name, record.class.name, record.id, name, file_data)\n    end\n  end\n\n  plugin :upload_options, store: -\u003e(_io, context) do\n    if %i[avatar logo].include?(context[:name])\n      {acl: \"public-read\"}\n    else\n      {acl: \"private\"}\n    end\n  end\n\n  plugin :versions\n\n  def lambda_process_versions(io, context)\n    assembly = { function: 'ImageResizeOnDemand' } # Here the AWS Lambda function name is specified\n\n    # Check if the original file format is a image format supported by the Sharp.js library\n    if %w[image/gif image/jpeg image/pjpeg image/png image/svg+xml image/tiff image/x-tiff image/webm]\n      .include?(io\u0026.data\u0026.dig('metadata', 'mime_type'))\n      case context[:name]\n        when :avatar\n          assembly[:versions] =\n            [{ name: :size40, storage: :store, width: 40, height: 40, format: :jpg }]\n        when :logo\n          assembly[:versions] =\n            [{ name: :size270_180, storage: :store, width: 270, height: 180, format: :jpg }]\n        when :doc\n          assembly[:versions] =\n            [\n              { name: :size40, storage: :store, width: 40, height: 40, format: :png },\n              { name: :size80, storage: :store, width: 80, height: 80, format: :jpg },\n              { name: :size120, storage: :store, width: 120, height: 120, format: :jpg }\n            ]\n      end\n    end\n    assembly\n  end\nend\n```\n\nThe above example is built to interact with the [lambda-image-resize] function, which is using the [Sharp] Javascript\nlibrary for image processing. It is not yet implemented in this function to use the `:target_storage` as default \nbucket for all the processed files, that's why the `:storage` key is specified on every file version. If the file's \nmime type is not supported by the [Sharp] library, no `:versions` will be inserted into the `:assembly` so the \noriginal file will just be copied to the `:store` S3 bucket.\n\nThe [Shrine upload_options plugin] is used to specify the S3 bucket ACL and the [Shrine versions plugin] is used to \nenable the uploader to deal with different processed versions of the original file.\n\nThe default options used by Shrine-AWS-Lambda plugin are the following:\n\n```ruby\n  { callbackURL:    Shrine.opts[:callback_url],\n    copy_original:  true,\n    storages:       Shrine.buckets_to_use(%i[cache store]),\n    target_storage: :store }\n```\n\nThese options could be overridden in the `LambdaUploader` specifying them as the `assembly` keys:\n\n```ruby\n  assembly[:callbackURL]    = some_callback_url\n  assembly[:copy_original]  = false               # If this is `false`, only the processed file versions will be stored     \n  assembly[:storages]       = Shrine.buckets_to_use(%i[cache store other_store])\n  assembly[:target_storage] = :other_store\n```\n\nAny S3 buckets could be specified, as long as the buckets are defined in the Shrine's initializer file.\n\n\n#### Webhook\n\nA `:callbackUrl` endpoint should be implemented to catch the [AWS Lambda] processing results, authorize, and save them. \nHere is an example of a [Roda] endpoint:\n\n```ruby\n# lib/rapi/base.rb:\n\n# frozen_string_literal: true\n\n# On Rails autoload is done by ActiveSupport from the `autoload_paths` - no need to require files\n# require 'roda'\n# require 'roda/plugins/json'\n# require 'roda/plugins/static_routing'\n\nmodule RAPI\n  class Base \u003c Roda\n    plugin :json\n    plugin :request_headers\n    plugin :static_routing\n\n    static_put '/lambda' do\n      auth_result = Shrine::Attacher.lambda_authorize(request.headers, request.body.read)\n      if !auth_result\n        response.status = 403\n        { 'Error' =\u003e 'Signature mismatch' }\n      elsif auth_result.is_a?(Array)\n        attacher = auth_result[0]\n        if attacher.lambda_save(auth_result[1])\n          { 'Result' =\u003e 'OK' }\n        else\n          response.status = 500\n          { 'Error' =\u003e 'Backend record update error' }\n        end\n      else\n        response.status = 500\n        { 'Error' =\u003e 'Backend Lambda authorization error' }\n      end\n    end\n  end\nend\n```\n\n#### Backgrounding\n\nEven though submitting a Lambda assembly doesn't require any uploading, it still does a HTTP request, so it is better \nto put it into a background job. This is configured in the `LambdaUploader` class:\n\n```ruby\nunless Rails.env.test?\n  Attacher.promote_block do\n    LambdaPromoteJob.enqueue(self.class.name, record.class.name, record.id, name, file_data)\n  end\nend\n```\n\nThen the job file should be implemented:\n\n```ruby\n# app/jobs/lambda_promote_job.rb:\n\n# frozen_string_literal: true\n\nclass LambdaPromoteJob \u003c Que::Job\n  def run(...)\n    ActiveRecord::Base.transaction do\n     Shrine::Attacher.lambda_process(...)\n    end\n  end\nend\n```\n\n### Gem Maintenance\n\n#### Preparing a release\n\nMerge all the pull requests that should make it into the new release into the `main` branch, then checkout and pull the\nbranch and run the `github_changelog_generator`, specifying the new version as a `--future-release` command line \nparameter:\n\n```bash\n$ git checkout main\n$ git pull\n\n$ github_changelog_generator -u texpert -p shrine-aws-lambda --future-release v0.1.1\n```\n\nThen add the changes to `git`, commit and push the `Preparing the new release` commit directly into the `main` branch:\n\n```bash\n$ git add .\n$ git commit -m 'Preparing the new v0.1.1 release'\n$ git push\n```\n\n#### RubyGems credentials\n\nEnsure you have the RubyGems credentials located in the `~/.gem/credentials` file.\n\n#### Adding a gem owner\n\n```bash\n$ gem owner shrine-aws-lambda -a friend@example.com\n```\n\n#### Building a new gem version\n\nAdjust the new gem version number in the `lib/shrine/plugins/lambda/version.rb` file. It is used when building the gem\nby the following command:\n\n```bash\n$ gem build shrine-aws-lambda.gemspec\n```\n\nAssuming the version was set to `0.2.0`, a `shrine-aws-lambda-0.2.0.gem` binary file will be generated at the root of \nthe app (repo).\n\n- The binary file shouldn't be added into the `git` tree, it will be pushed into the RubyGems and to the GitHub releases\n\n#### Pushing a new gem release to RubyGems\n\n```bash\n$ gem push shrine-aws-lambda-0.2.0.gem # don't forget to specify the correct version number\n```\n\n#### Crafting the new release on GitHub\n\nOn the [Releases page](https://github.com/texpert/shrine-aws-lambda/releases) push the `Draft a new release` button.\n\nThe new release editing page opens, on which the following actions could be taken:\n\n- Choose the repo branch (default is `main`)\n- Insert a tag version (usually, the tag should correspond to the gem's new version, v0.0.1, for example)\n - the tag will be created by GitHub on the last commit into the chosen branch\n- Fill the release Title and Description\n- Attach the binary file with the generated gem version\n- If the release is not yet ready for production, mark the `This is a pre-release` checkbox\n- Press either the `Publish release`, or the `Save draft button` if you want to publish it later\n - After publishing the release, the the binary gem file will be available on GitHub and could be removed locally\n\n\n## Inspiration\n\nI want to thank [Janko Marohnić][Janko] for the awesome [Shrine] gem and, also, for guiding me to look at his  \nimplementation of a similar plugin - [Shrine-Transloadit].\n\nAlso thanks goes to [Tim Uckun] for providing a link to the [article about resizing images on the fly][AWS blog \narticle], which pointed me to use the [Sharp] library for image resizing.\n\n## License\n\n[MIT](/LICENSE.txt)\n\n[Amazon]: https://www.amazon.com\n[Amazon Web Services]: https://aws.amazon.com\n[AWS blog article]: https://aws.amazon.com/blogs/compute/resize-images-on-the-fly-with-amazon-s3-aws-lambda-and-amazon-api-gateway/\n[AWS Lambda]: https://aws.amazon.com/lambda\n[AWS Lambda permissions]: https://docs.aws.amazon.com/lambda/latest/dg/intro-permission-model.html\n[AWS profiles]: https://docs.aws.amazon.com/cli/latest/userguide/cli-multiple-profiles.html\n[AWS S3]: https://aws.amazon.com/s3/\n[Janko]: https://github.com/janko-m\n[lambda-image-resize]: https://github.com/texpert/lambda-image-resize.js\n[ngrok-wrapper]: https://github.com/texpert/ngrok-wrapper\n[Rails]: http://rubyonrails.org\n[Roda]: http://roda.jeremyevans.net\n[Sharp]: https://github.com/lovell/sharp\n[Shrine]: https://github.com/janko-m/shrine\n[Shrine backgrounding plugin]: http://shrinerb.com/rdoc/classes/Shrine/Plugins/Backgrounding.html\n[Shrine-Transloadit]: https://github.com/janko-m/shrine-transloadit\n[Shrine upload_options plugin]: http://shrinerb.com/rdoc/classes/Shrine/Plugins/UploadOptions.html\n[Shrine versions plugin]: http://shrinerb.com/rdoc/classes/Shrine/Plugins/Versions.html\n[Tim Uckun]: https://github.com/timuckun\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftexpert%2Fshrine-aws-lambda","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftexpert%2Fshrine-aws-lambda","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftexpert%2Fshrine-aws-lambda/lists"}