{"id":17136891,"url":"https://github.com/jessestuart/serverless-image-organizer","last_synced_at":"2026-05-10T09:38:49.007Z","repository":{"id":72494028,"uuid":"119822052","full_name":"jessestuart/serverless-image-organizer","owner":"jessestuart","description":"A simple Lambda-based stack to process and organize image files.","archived":false,"fork":false,"pushed_at":"2019-04-21T05:10:21.000Z","size":24,"stargazers_count":4,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-12T23:33:53.170Z","etag":null,"topics":["apex","aws","aws-lambda","aws-s3","go","serverless","terraform"],"latest_commit_sha":null,"homepage":"https://jessestuart.com/posts/20180201-image-managment-s3/","language":"HCL","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jessestuart.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-02-01T10:43:05.000Z","updated_at":"2020-01-18T00:01:50.000Z","dependencies_parsed_at":null,"dependency_job_id":"2b2c0998-3ed7-47ff-816c-008dd77e8cb1","html_url":"https://github.com/jessestuart/serverless-image-organizer","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jessestuart%2Fserverless-image-organizer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jessestuart%2Fserverless-image-organizer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jessestuart%2Fserverless-image-organizer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jessestuart%2Fserverless-image-organizer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jessestuart","download_url":"https://codeload.github.com/jessestuart/serverless-image-organizer/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245223993,"owners_count":20580362,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["apex","aws","aws-lambda","aws-s3","go","serverless","terraform"],"created_at":"2024-10-14T20:05:40.343Z","updated_at":"2025-10-14T12:10:43.883Z","avatar_url":"https://github.com/jessestuart.png","language":"HCL","funding_links":[],"categories":[],"sub_categories":[],"readme":"## `serverless-image-organizer`\n\nI like to keep things organized -- especially when it comes to things like my\nphotography.\n\nI also like things being done automatically for me. And playing around with\nshiny technologies.\n\nSo [serverless-image-organizer][gh-link] was born! This was an afternoon project\nthat does one simple thing: listens for uploads to a source S3 bucket, then\nsorts all incoming photos into a destination bucket, grouped into \"folders\" by\ndate (in ISO 8601 date format, e.g., `2018-02-01/foobar.jpg`).\n\nIn order for that simple task to be executed \"serverlessly\", however, requires\nthe integration of a non-trivial network of components. There are, of course,\nplenty of services to abstract much of the complexity out of this (for a price),\nbut I'm more the type to roll up my sleeves and get to know how things work\nunder the hood. IAM Users, Roles, custom Policies, and more? Sounds fun! Let's\ndig in.\n\n### Deploying functions \u0026 infrastructure\n\nThis project uses [Apex][apex] to manage deployments of Lambda functions,\nproviding a thin but convenient layer of abstraction over the\n\"refactor-zip-upload\" workflow previously required. Deploying a new version of\nthe Lambda is as simple as:\n\n```\nλ apex deploy image-upload-handler\n   • config unchanged          env= function=image-upload-handler\n   • updating function         env= function=image-upload-handler\n   • updated alias current     env= function=image-upload-handler version=12\n   • function updated          env= function=image-upload-handler name=serverless-image-organizer_image-upload-handler version=12\n```\n\nHowever, the Lambda function isn't of very much use on its own -- several other\nAWS infrastructure components are required to wire everything together (IAM, S3,\nCloudWatch, etc). And for declaring and maintaining these resources, nothing\nbeats Hashicorp's [Terraform][terraform] -- it has its shortcomings, but it's\nwell worth checking out if you're interested in the declarative\ninfrastructure-as-code.\n\nLuckily, Apex integrates well with Terraform -- the `apex infra` command\nessentially proxies all commands directly while exposing a few Apex-provided\nvariables, like the ARN of any deployed functions.\n\nTL;DR, here's what goes down to put this all together:\n\n1. Create an IAM role with a few required permissions: e.g., S3 R/W access, the\n   ability to invoke Lambda functions, and (optional but recommended) write\n   access to CloudWatch logs (because you know, [telemetry][telemetry]).\n1. Attach a custom IAM policy granting access for that role to respond to S3\n   events.\n1. Create a Lambda function, and hook it up to the aforementioned IAM role.\n1. When the Lambda is triggered, a simple `Go` function is invoked that queries\n   S3, downloads the JPEG asset, parses out the EXIF data (including the date\n   created), and executes a final `PutObject` on the destination bucket with the\n   updated data.\n\n### What's next?\n\n* I started working on a system to persist EXIF metadata properties in a\n  lightweight DB like DynamoDB -- the motivation here would be limiting the\n  overhead of EXIF processing to a one-time serverless function, as opposed to\n  requiring that computation to be performed client-side on every page load for\n  e.g., a photo gallery.\n\n- The system is currently tightly coupled to AWS -- it probably wouldn't be too\n  difficult to implement support for any S3-compliant Object Storage provider\n  (e.g., [Minio][minio], [DigitalOcean Spaces][do-spaces], [Openstack\n  Swift][os-swift]).\n\n### FAQ\n\n* **Q**: Can I use the same bucket for the source and destination?\n\n- **A**: Not without introducing more opinionated logic / filtering to the\n  Lambda function -- I wasn't able to determine a trivial way to distinguish\n  between images uploaded by a user versus images (re)-uploaded by the Lambda\n  (both are treated as a `PutObject` request), resulting in... yeah, an infinite\n  loop.\n\n### Takeaways\n\nThe tooling around this stack may still be a little rough around the edges --\nafter all, native Go support was only announced in January of 2018 -- but thanks\nto the efforts of organizations like Hashicorp, Apex, and the rest of the OSS\ncommunity, it's now possible to rapidly iterate on an idea from prototype to\nproduction-ready in a matter of just a few hours.\n\nAnd of course, I'd be remiss not to mention the other benefit of working in this\nstack: it costs almost nothing to operate, with AWS's free tier extending up to\n1 million invocations/month at time of writing.\n\n[apex]: https://github.com/apex/apex\n[do-spaces]: https://www.digitalocean.com/products/spaces/\n[gh-link]: https://github.com/jessestuart/serverless-image-organizer\n[minio]: https://minio.io\n[os-swift]: https://www.openstack.org/software/releases/ocata/components/swift\n[telemetry]: https://www.safaribooksonline.com/library/view/the-devops-handbook/9781457191381/DOHB-ch_14.xhtml\n[terraform]: https://github.com/hashicorp/terraform\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjessestuart%2Fserverless-image-organizer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjessestuart%2Fserverless-image-organizer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjessestuart%2Fserverless-image-organizer/lists"}