{"id":17141625,"url":"https://github.com/martinklepsch/s3-beam","last_synced_at":"2025-04-07T11:07:53.152Z","repository":{"id":22129536,"uuid":"25460377","full_name":"martinklepsch/s3-beam","owner":"martinklepsch","description":"🚀 direct-to-S3 uploading using ClojureScript","archived":false,"fork":false,"pushed_at":"2020-05-23T16:55:46.000Z","size":72,"stargazers_count":93,"open_issues_count":8,"forks_count":17,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-03-31T09:08:52.227Z","etag":null,"topics":["aws","cljs","clojurescript","digitalocean","direct-upload","s3","signing-server"],"latest_commit_sha":null,"homepage":"","language":"Clojure","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"epl-1.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/martinklepsch.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2014-10-20T09:53:44.000Z","updated_at":"2025-01-14T10:53:05.000Z","dependencies_parsed_at":"2022-08-19T15:10:16.909Z","dependency_job_id":null,"html_url":"https://github.com/martinklepsch/s3-beam","commit_stats":null,"previous_names":[],"tags_count":9,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/martinklepsch%2Fs3-beam","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/martinklepsch%2Fs3-beam/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/martinklepsch%2Fs3-beam/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/martinklepsch%2Fs3-beam/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/martinklepsch","download_url":"https://codeload.github.com/martinklepsch/s3-beam/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247640463,"owners_count":20971557,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws","cljs","clojurescript","digitalocean","direct-upload","s3","signing-server"],"created_at":"2024-10-14T20:26:02.478Z","updated_at":"2025-04-07T11:07:53.134Z","avatar_url":"https://github.com/martinklepsch.png","language":"Clojure","funding_links":[],"categories":[],"sub_categories":[],"readme":"# s3-beam [![Dependencies Status](https://versions.deps.co/martinklepsch/s3-beam/status.svg)](https://versions.deps.co/martinklepsch/s3-beam)\n\n[Usage](#usage) | [Changes](#changes)\n\n`s3-beam` is a Clojure/Clojurescript library designed to help you upload files\nfrom the browser to S3 (CORS upload). `s3-beam` can also upload files from the browser\nto [DigitalOcean Spaces](https://www.digitalocean.com/products/object-storage/).\n\n[](dependency)\n```clojure\n[org.martinklepsch/s3-beam \"0.6.0-alpha5\"] ;; latest release\n```\n[](/dependency)\n\n## Usage\n\nTo upload files directly to S3 you need to send special request\nparameters that are based on your AWS credentials, the file name, mime\ntype, date etc.\nSince we **don't want to store our credentials in the client** these\nparameters need to be generated on the server side.\nFor this reason this library consists of two parts:\n\n1. A pluggable route that will send back the required parameters for a\n   given file-name \u0026 mime-type\n2. A client-side core.async pipeline setup that will retrieve the\n   special parameters for a given File object, upload it to S3 and\n   report back to you\n\n### 1. Enable CORS on your S3 bucket\n\nPlease follow [Amazon's official documentation](http://docs.aws.amazon.com/AmazonS3/latest/dev/cors.html).\n\nFor DigitalOcean Spaces, please follow [DigitalOceans official documentation](https://developers.digitalocean.com/documentation/spaces/#set-bucket-cors).\n\n### 2. Plug-in the route to sign uploads\n\n```clj\n(ns your.server\n  (:require [s3-beam.handler :as s3b]\n            [compojure.core :refer [GET defroutes]]\n            [compojure.route :refer [resources]]))\n\n(def bucket \"your-bucket\")\n(def aws-zone \"eu-west-1\")\n(def access-key \"your-aws-access-key\")\n(def secret-key \"your-aws-secret-key\")\n\n(defroutes routes\n  (resources \"/\")\n  (GET \"/sign\" {params :params} (s3b/s3-sign bucket aws-zone access-key secret-key)))\n```\n\nIf you want to use a route different than `/sign`, define it in the\nhandler, `(GET \"/my-cool-route\" ...)`, and then pass it in the options\nmap to `s3-pipe` in the frontend.\n\nIf you are serving your S3 bucket from DigitalOcean Spaces, with CloudFront, or another CDN/proxy, you can pass\n`upload-url` as a fifth parameter to `s3-sign`, so that the ClojureScript client is directed\nto upload through this bucket. You still need to pass the bucket name, as the policy that is\ncreated and signed is based on the bucket name.\n\n### 3. Integrate the upload pipeline into your frontend\n\nIn your frontend code you can now use `s3-beam.client/s3-pipe`.\n`s3-pipe`'s argument is a channel where completed uploads will be\nreported. The function returns a channel where you can put File\nobjects of a file map that should get uploaded. It can also take an\nextra options map with the previously mentioned `:server-url` like so:\n\n    (s3/s3-pipe uploaded {:server-url \"/my-cool-route\"}) ; assuming s3-beam.client is NS aliased as s3\n\nThe full options map spec is:\n\n- `:server-url` the signing server url, defaults to \"/sign\"\n- `:response-parser` a function to process the signing response from the signing server into EDN\n                     defaults to read-string.\n- `:key-fn` a function used to generate the object key for the uploaded file on S3\n                   defaults to nil, which means it will use the passed filename as the object key.\n- `:headers-fn` a function used to create the headers for the GET request to the signing server.\n                   The returned headers should be a Clojure map of header name Strings to corresponding\n                   header value Strings.\n- `:progress-events?` If set to true, it will push progress events to the channel during the transfer, false per default.\n\nIf you choose to place a file map instead of a `File` object, you file map should follow:\n\n- `:file`                  A `File` object\n- `:identifier` (optional) A variable used to uniquely identify this file upload.\n                           This will be included in the response channel.\n- `:key` (optional)        The file-name parameter that is sent to the signing server. If a `:key` key\n                           exists in the input-map it will be used instead of the key-fn as an object-key.\n- `:metadata` (optional)   Metadata for the object. See [Amazon's API docs](http://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectPOST.html)\n                           for full details on which keys are supported. Keys and values can be strings or keywords.\n                           **N.B.** Keys not on that list will not be accepted. If you want to set arbitrary metadata,\n                           it needs to be prefixed with `x-amz-meta-*`.\n\nAn example using it within an Om component:\n\n```clj\n(ns your.client\n  (:require [s3-beam.client :as s3]\n  ...))\n\n(defcomponent upload-form [app-state owner]\n  (init-state [_]\n    (let [uploaded (chan 20)]\n      {:dropped-queue (chan 20)\n       :upload-queue (s3/s3-pipe uploaded)\n       :uploaded uploaded\n       :uploads []}))\n  (did-mount [_]\n    (listen-file-drop js/document (om/get-state owner :dropped-queue))\n    (go (while true\n          (let [{:keys [dropped-queue upload-queue uploaded uploads]} (om/get-state owner)]\n            (let [[v ch] (alts! [dropped-queue uploaded])]\n              (cond\n               (= ch dropped-queue) (put! upload-queue v)\n               (= ch uploaded) (om/set-state! owner :uploads (conj uploads v))))))))\n  (render-state [this state]\n    ; ....\n    )\n```\n\n#### Return values\n\nThe spec for the returned map (in the example above the returned map is `v`):\n\n- `:type` `:success`\n- `:file` The `File` object from the uploaded file\n- `:response` The upload response from S3 as a map with:\n - `:location` The S3 URL of the uploaded file\n - `:bucket` The S3 bucket where the file is located\n - `:key` The S3 key for the file\n - `:etag` The etag for the file\n- `:xhr` The `XhrIo` object used to POST to S3\n- `:identifier` A value used to uniquely identify the uploaded file\n\nOr, if an error occurs during upload processing, an error-map will be placed on the response channel:\n\n- `:type` `:error`\n- `:identifier` A variable used to uniquely identify this file upload. This will be included in the response channel.\n- `:error-code` The error code from the XHR\n- `:error-message` The debug message from the error code\n- `:http-error-code` The HTTP error code\n\nIf `:progress-events?` are set to `true`, it will also forward those events from XhrIo:\n\n- `:type` `:progress`\n- `:file` The `File` object from the uploaded file\n- `:bytes-sent` Bytes uploaded\n- `:bytes-total` Total file size in bytes\n- `:xhr` The `XhrIo` object used to POST to S3\n- `:identifier` A value used to uniquely identify the uploaded file\n\n## Changes\n\n#### 0.6.0-alpha5\n\n- Fix compilation issues with shadow-cljs ([#47](https://github.com/martinklepsch/s3-beam/pull/47))\n- Upgrade dependencies ([#48](https://github.com/martinklepsch/s3-beam/pull/48))\n\n#### 0.6.0-alpha4\n\n- Add support for DigitalOcean Spaces ([#44](https://github.com/martinklepsch/s3-beam/issues/44))\n\n#### 0.6.0-alpha3\n\n- Add support for progress events ([#40](https://github.com/martinklepsch/s3-beam/pull/40))\n\n#### 0.6.0-alpha1\n\n- Add support for assigning metadata to files when uploading them. See the file-map spec above for more details. [#37](https://github.com/martinklepsch/s3-beam/pull/37)\n- Tweak keys and parameters for communication between the client and server parts of the library. This is backwards and\n  forwards compatible between clients and servers running 0.5.2 and 0.6.0-alpha1.\n\n#### 0.5.2\n\n- Allow the user to upload to S3 through a custom URL as an extra parameter to `sign-upload`\n- Support bucket names with a '.' in them\n- Add asserts that arguments are provided\n\n#### 0.5.1\n\n- Allow the upload-queue to be passed an input-map instead of a file. This\n  input-map follows the spec:\n\n    - `:file`                  A `File` object\n    - `:identifier` (optional) A variable used to uniquely identify this file upload.\n                               This will be included in the response channel.\n    - `:key` (optional)        The file-name parameter that is sent to the signing server. If a `:key` key\n                               exists in the input-map it will be used instead of the key-fn as an object-key.\n- Introduce error handling. When an error has been thrown while uploading a file to S3\n  an error-map will be put onto the channel. The error-map follows the spec:\n\n    - `:identifier`      A variable used to uniquely identify this file upload. This will be\n                         included in the response channel.\n    - `:error-code`      The error code from the XHR\n    - `:error-message`   The debug message from the error code\n    - `:http-error-code` The HTTP error code\n- New options are available in the options map:\n\n    - `:response-parser` a function to process the signing response from the signing server into EDN\n                         defaults to read-string.\n    - `:key-fn`          a function used to generate the object key for the uploaded file on S3\n                         defaults to nil, which means it will use the passed filename as the object key.\n    - `:headers-fn`      a function used to create the headers for the GET request to the signing server.\n- Places a map into the upload-channel with:\n    - `:file`       The `File` object from the uploaded file\n    - `:response`   The upload response from S3 as a map with:\n     - `:location` The S3 URL of the uploaded file\n     - `:bucket`   The S3 bucket where the file is located\n     - `:key`      The S3 key for the file\n     - `:etag`     The etag for the file\n    - `:xhr`        The `XhrIo` object used to POST to S3\n    - `:identifier` A value used to uniquely identify the uploaded file\n\n#### 0.4.0\n\n- Support custom ACLs. The `sign-upload` function that can be used to\n  implement custom signing routes now supports an additional `:acl` key\n  to upload assets with a different ACL than `public-read`.\n\n        (sign-upload {:file-name \"xyz.html\" :mime-type \"text/html\"}\n                     {:bucket bucket\n                      :aws-zone aws-zone\n                      :aws-access-key access-key\n                      :aws-secret-key secret-key\n                      :acl \"authenticated-read\"})\n- Changes the arity of `s3-beam.handler/policy` function.\n\n#### 0.3.1\n\n- Correctly look up endpoints given a zone parameter ([#10](https://github.com/martinklepsch/s3-beam/pull/10/files))\n\n#### 0.3.0\n\n-  Allow customization of server-side endpoint ([1cb9b27](https://github.com/martinklepsch/s3-beam/commit/1cb9b2703691e172e275a95490b3fc8209dfa409))\n\n        (s3/s3-pipe uploaded {:server-url \"/my-cool-route\"})\n\n#### 0.2.0\n\n- Allow passing of `aws-zone` parameter to `s3-sign` handler function ([b880736](https://github.com/martinklepsch/s3-beam/commit/b88073646b7c92b5493a168ce25d27feaa130c9e))\n\n## Contributing\n\nPull requests and issues are welcome. There are a few things I'd like to improve:\n\n* **Testing:** currently there are no tests\n* **Error handling:** what happens when the request fails?\n\n## Maintainers\n\nMartin Klepsch\nDaniel Compton\n\n## License\n\nCopyright © 2014 Martin Klepsch\n\nDistributed under the Eclipse Public License either version 1.0 or (at\nyour option) any later version.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmartinklepsch%2Fs3-beam","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmartinklepsch%2Fs3-beam","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmartinklepsch%2Fs3-beam/lists"}