{"id":19810997,"url":"https://github.com/accenture/serverless-ephemeral","last_synced_at":"2025-05-01T08:32:19.764Z","repository":{"id":26141553,"uuid":"103312496","full_name":"Accenture/serverless-ephemeral","owner":"Accenture","description":"This is a Serverless Framework plugin that helps bundling any stateless zipped library to AWS Lambda.","archived":false,"fork":false,"pushed_at":"2023-01-03T18:48:22.000Z","size":908,"stargazers_count":67,"open_issues_count":23,"forks_count":15,"subscribers_count":31,"default_branch":"master","last_synced_at":"2025-04-19T14:35:55.622Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Accenture.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-09-12T19:33:08.000Z","updated_at":"2023-02-01T18:15:48.000Z","dependencies_parsed_at":"2023-01-14T04:06:31.020Z","dependency_job_id":null,"html_url":"https://github.com/Accenture/serverless-ephemeral","commit_stats":null,"previous_names":[],"tags_count":6,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Accenture%2Fserverless-ephemeral","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Accenture%2Fserverless-ephemeral/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Accenture%2Fserverless-ephemeral/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Accenture%2Fserverless-ephemeral/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Accenture","download_url":"https://codeload.github.com/Accenture/serverless-ephemeral/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251847828,"owners_count":21653582,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-12T09:24:18.528Z","updated_at":"2025-05-01T08:32:19.457Z","avatar_url":"https://github.com/Accenture.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Serverless Ephemeral\n\n[![NPM Version][npm-image]][npm-url]\n[![Build Status][travis-image]][travis-url]\n\n![Serephem](img/logo192.png)\n\nServerless Ephemeral (or Serephem) is a [Serverless Framework plugin](https://serverless.com/framework/docs/providers/aws/guide/plugins/) that helps bundle any stateless library into a Lambda deployment artifact.\n\n## Pre-requirements\n\n- Node \u003e= 6.9\n- Serverless Framework \u003e= 1.12.0\n- Docker (with docker compose)\n\n## Examples\n\n- [TensorFlow Lambda (CPU only, Python 2.7)](examples/tensorflow-lambda): Pulls in or builds (via Docker) a TensorFlow package. For reference on how to manually build a TensorFlow package for Lambdas, see [docs/build-tensorflow-package.md](docs/build-tensorflow-package.md).\n\n- [Image Processor](examples/image-processor): Builds and adds a Python [Pillow](https://python-pillow.org/) package in order to resize an image uploaded to an S3 bucket.\n\n## Add the plugin\n\n1. Install it\n\n   ```bash\n   npm i --save-dev serverless-ephemeral\n   ```\n\n1. Add it to your `serverless.yml` file and exclude the `.ephemeral` directory\n\n   ```yaml\n   plugins:\n     - serverless-ephemeral\n\n   package:\n     exclude:\n       - package.json\n       - package-lock.json\n       - node_modules/**\n       - .ephemeral/**\n   ```\n\n1. Add the `.ephemeral` directory to `.gitignore`\n\n```text\n# Serverless Framework\n.serverless\n.ephemeral\n```\n\n## Configuration\n\nThe configuration for the Ephemeral plugin is set inside the `custom` section of the `serverless.yml` file. In it, you can define the list of stateless libraries you wish to pull into the final Lambda artifact.\n\nThere are two types of configuration:\n\n- [Build a library during runtime](#build-a-library)\n- [Download a library](#download-a-library)\n\nBoth can be enhanced with [global configuration options](#global-options).\n\n### Build a library\n\nYou can build a specific library during runtime. This is achieved via a Docker container that outputs a zip library.\n\nThe Serepehm plugin provides some useful packagers out of the box. However, you can create your own packager via Docker files.\n\n#### Serephem packagers\n\nYou can use one of the Docker packagers provided with the Serephem plugin.\n\n##### TensorFlow\n\n```yaml\ncustom:\n  ephemeral:\n    libraries:\n      - packager:\n          name: tensorflow\n          version: 1.8.0\n```\n\n- **packager.name** is required. This is the packager name identifier for TensorFlow: **tensorflow**\n- **packager.version** is required. This will determine which TensorFlow version you want to build.\n\n#### Build your own packager\n\nYou can create your own packager via Docker. To do so:\n\n1. Create a directory where you will store all your Docker files:\n\n   ```bash\n   mkdir my-packager\n   cd my-packager\n   ```\n\n1. Create a `docker-compose.yml` file. For example:\n\n   ```yaml\n   version: \"3\"\n   services:\n     packager:\n       build: .\n   ```\n\n   Keep note of the name of your packager service, in this case `packager`.\n\n1. Create a `Dockerfile` and any other support files. For example:\n\n   **`Dockerfile`**\n\n   ```yaml\n   FROM amazonlinux\n\n   COPY scripts/build.sh scripts/build.sh\n   RUN yum -y install zip \u0026\u0026 \\\n   chmod +x scripts/build.sh\n\n   CMD [ \"scripts/build.sh\" ]\n   ```\n\n   **`scripts/build.sh`**\n\n   ```bash\n   # create zip destination directory\n   mkdir -p /tmp/lambda-libraries\n\n   # download library files\n   mkdir /tmp/files\n   cd /tmp/files\n   curl http://example.com/file-1.py --output file-1.py\n   curl http://example.com/file-2.py --output file-2.py\n   zip -9rq /tmp/lambda-libraries/library-a.zip *\n   ```\n\n   **IMPORTANT**: the container must generate a zip file containing the stateless library files. Thus:\n\n   - Your container must zip the stateless library files.\n\n   - You must create a directory where the final zip(s) will be stored. This directory will be mounted to the Serephem's libraries directory, so add only the necessary zip files.\n\n   - It is recommended that your Docker container extends from `amazonlinux` image to maximize compatibility with the Lambda environment.\n\n1. Add this configuration to your `serverless.yml`:\n\n   ```yaml\n   custom:\n     ephemeral:\n       libraries:\n         - packager:\n             compose: my-packager/docker-compose.yml\n             service: packager\n             output: /tmp/lambda-libraries/library-a.zip\n   ```\n\n   Notice how each of the values correspond to a setting previously created:\n\n   - `compose`: points to your Docker compose file, inside the directory you created\n\n   - `service`: the name of the service inside the `docker-compose.yml` file\n\n   - `output`: the output path for the zip file in the Docker container\n\n### Download a library\n\nYou can download a previously zipped library that contains all the necessary files and automatically add it to your Lambda deployment.\n\n```yaml\ncustom:\n  ephemeral:\n    libraries:\n      - url: https://xxxxx.s3.amazonaws.com/my-library.zip\n```\n\n- **url** is required. This is the packaged library you want to include. The library must be a zip file.\n\n\u003e Documentation explaining how to create a deployable TensorFlow zipped package can be found here: [docs/build-tensorflow-package.md](docs/build-tensorflow-package.md). This approach can be used as a base to create other stateless libraries.\n\n### Global options\n\n```yaml\ncustom:\n  ephemeral:\n    libraries:\n      - packager:\n          name: tensorflow\n          version: 1.8.0\n        directory: tfpackage\n      - url: https://xxxxx.s3.amazonaws.com/my-library.zip\n        nocache: true\n```\n\n- **directory** is optional. When ommitted, the package contents will be unzipped at service root level. If entered, a new folder will be created at that level using the specified name and everything will be unzipped there. The folder can only be named using alphanumeric characters and the symbols `. _ -`\n\n- **nocache** is optional. When ommitted or set to _false_, it will use the locally cached copy of the library. Otherwise, if set to _true_, it will re-fetch (download or build) the library every time the service is packaged.\n\n  \u003e Note: the **forceDownload** option has been deprecated in favor of **nocache**\n\n## Deploy\n\nDeploy your service normally with the `serverless deploy` (or `sls deploy`) command. If you use the `-v` option, Ephemeral will show more information about the process.\n\n   ```bash\n   sls deploy -v\n   ```\n\n   \u003e If the Serverless deployment is timing out, use the `AWS_CLIENT_TIMEOUT` environment variable: \u003chttps://github.com/serverless/serverless/issues/490#issuecomment-204976134\u003e\n\n### The .ephemeral directory\n\nDuring the deployment process, the `.ephemeral` directory will be created. The purpose of this directory is:\n\n- Saving the libraries' zip files inside the `.ephemeral/lib` folder\n- Bundling the libraries and the Serverless Lambda function file(s) inside the `.ephemeral/pkg` folder\n\n---\n\n## Development\n\nThis plugin is created with Node and uses the Serverless Framework hooks to execute the necessary actions.\n\n### Getting started\n\n1. Clone this repo\n2. Install dependencies via `npm i`\n\n### Running Lint\n\nThe plugin code uses the AirBnB ESLint rule set with some enhancements (see `.eslintrc` file). To run the linter:\n\n```bash\nnpm run lint\n```\n\n### Tests\n\nThe unit tests are coded with [Ava](https://github.com/avajs/ava) and [SinonJS](http://sinonjs.org/docs/). They can be found inside the `spec` folder.\n\nTo run the tests:\n\n```bash\nnpm test\n```\n\nTo run tests on \"watch\" mode and add verbosity:\n\n```bash\nnpm test -- --watch -v\n```\n\n[npm-image]: https://img.shields.io/npm/v/serverless-ephemeral.svg\n[npm-url]: https://npmjs.org/package/serverless-ephemeral\n[travis-image]: https://img.shields.io/travis/Accenture/serverless-ephemeral/master.svg\n[travis-url]: https://travis-ci.org/Accenture/serverless-ephemeral\n\n### Test via examples\n\nRefer to the [`examples`](examples) directory, for instance the [`TensorFlow example`](examples/tensorflow-lambda/README.md).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faccenture%2Fserverless-ephemeral","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Faccenture%2Fserverless-ephemeral","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faccenture%2Fserverless-ephemeral/lists"}