{"id":19458667,"url":"https://github.com/luontola/native-clojure-lambda","last_synced_at":"2025-04-25T06:30:32.897Z","repository":{"id":66604265,"uuid":"320678422","full_name":"luontola/native-clojure-lambda","owner":"luontola","description":"Example project of Clojure + GraalVM Native Image + AWS Lambda container images","archived":false,"fork":false,"pushed_at":"2020-12-21T16:07:50.000Z","size":39,"stargazers_count":29,"open_issues_count":2,"forks_count":2,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-04-03T17:22:59.293Z","etag":null,"topics":["aws-lambda","clojure","graalvm-native-image"],"latest_commit_sha":null,"homepage":"","language":"HCL","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/luontola.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-12-11T20:35:23.000Z","updated_at":"2023-07-16T13:38:09.000Z","dependencies_parsed_at":"2023-03-01T10:16:04.739Z","dependency_job_id":null,"html_url":"https://github.com/luontola/native-clojure-lambda","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/luontola%2Fnative-clojure-lambda","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/luontola%2Fnative-clojure-lambda/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/luontola%2Fnative-clojure-lambda/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/luontola%2Fnative-clojure-lambda/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/luontola","download_url":"https://codeload.github.com/luontola/native-clojure-lambda/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250766963,"owners_count":21483895,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws-lambda","clojure","graalvm-native-image"],"created_at":"2024-11-10T17:28:03.678Z","updated_at":"2025-04-25T06:30:32.881Z","avatar_url":"https://github.com/luontola.png","language":"HCL","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Native Clojure Lambda\n\nExample project of [Clojure](https://clojure.org/) +\n[GraalVM Native Image](https://www.graalvm.org/reference-manual/native-image/) +\n[AWS Lambda container images](https://aws.amazon.com/blogs/aws/new-for-aws-lambda-container-image-support/).\n\n## Code walkthrough\n\nThe application entrypoint is [src/hello_world/main.clj](src/hello_world/main.clj)\nwhere [lambada](https://github.com/uswitch/lambada) is used to generate a lambda handler class `hello_world.Handler`\nwhich implements `com.amazonaws.services.lambda.runtime.RequestStreamHandler`.\n\nTo run our lambda handler inside a container image, we need\na [runtime interface client](https://docs.aws.amazon.com/lambda/latest/dg/runtimes-images.html#runtimes-api-client). For\nJava, it's the `com.amazonaws.services.lambda.runtime.api.client.AWSLambda` class from\nthe [aws-lambda-java-runtime-interface-client](https://github.com/aws/aws-lambda-java-libs) library. We could call the\nmain method in `AWSLambda` from the command line, but to avoid having to specify the name of our lambda handler class on\nthe command line, we wrap it in our own main method in [src/hello_world/main.clj](src/hello_world/main.clj).\n\nThe runtime interface client looks for\nthe [AWS_LAMBDA_RUNTIME_API](https://docs.aws.amazon.com/lambda/latest/dg/runtimes-api.html) environment variable, which\ncontains the host and port of the AWS Lambda runtime endpoint. When running the code outside Lambda, we need to use\nthe [Lambda Runtime Interface Emulator](https://github.com/aws/aws-lambda-runtime-interface-emulator/) which will set\nthe AWS_LAMBDA_RUNTIME_API and start our application process. To run our container image conveniently both in and\noutside Lambda, we use the [lambda-bootstrap.sh](lambda-bootstrap.sh) script to detect whether AWS_LAMBDA_RUNTIME_API is\ndefined, and use the runtime emulator if the container is running locally.\n\nThere are three Dockerfiles for packaging the application:\n\n- [Dockerfile-jvm](Dockerfile-jvm) packages it with a normal OpenJDK JVM.\n- [Dockerfile-graalvm](Dockerfile-graalvm) packages it with GraalVM and also enables\n  the [native-image-agent](https://www.graalvm.org/reference-manual/native-image/BuildConfiguration/#assisted-configuration-of-native-image-builds)\n  Java agent to help in writing Native Image configuration files. Normally `native-image-agent` generates the\n  configuration when the process exits, but the application won't receive a shutdown signal through `aws-lambda-rie`, so\n  the `config-write-period-secs` parameter is needed.\n- [Dockerfile-native](Dockerfile-native) AOT compiles the application\n  with [Native Image](https://www.graalvm.org/reference-manual/native-image/) and packages the resulting native binary\n  (about 24 MB) into a minimal container image. The `native-image` command requires lots of time and memory. Even this\n  hello world application takes 55 seconds to compile on a 2020 Intel Macbook Pro 13\" while utilizing all the CPU cores\n  and about 4 GB memory.\n\nDue to [Native Image's limitations](https://www.graalvm.org/reference-manual/native-image/Limitations/), the AOT\ncompilation requires configuration files which among other things list all classes that are accessed using reflection\nand all resources that the application loads at run time. These configuration files must be included in the application\nJAR file under [META-INF/native-image](resources/META-INF/native-image) or its subdirectories. You can also\ncreate [native-image.properties](https://www.graalvm.org/reference-manual/native-image/BuildConfiguration/#embedding-a-configuration-file)\nfiles to specify the command line arguments to the `native-image` command.\n\nWhen you run `./scripts/build.sh \u0026\u0026 docker-compose up -d --build graalvm` to start the application under GraalVM\nwith `native-image-agent` enabled, and invoke the application with `./smoke-test.sh`, the `native-image-agent` will\ngenerate Native Image configuration to the `target/native-image` directory. The generated configuration can be\nsimplified manually, and it could be missing some entries if the tests did not execute all code paths, so it's\nrecommended to inspect it before including it under `META-INF/native-image`.\n\nThis project includes a bunch of\n[aws-lambda-java-runtime-interface-client specific configuration](resources/META-INF/native-image/com.amazonaws/aws-lambda-java-runtime-interface-client),\nbut I'm working on a PR to embed that configuration inside the library, so that in the future it would work out of the box.\n\nThe only application specific Native Image configuration is\nin [resources/META-INF/native-image/reflect-config.json](resources/META-INF/native-image/reflect-config.json) - the\nruntime interface client uses reflection to instantiate our lambda handler. Normal Clojure code uses very little\nreflection, provided your [Leiningen configuration](project.clj) has `:global-vars {*warn-on-reflection* true}` and you\nadd type hints where necessary.\n\nWith the Native Image configuration bundled inside the JAR, `native-image` can be called\nin [Dockerfile-native](Dockerfile-native). For Clojure applications, the mandatory parameters\nare `--report-unsupported-elements-at-runtime` and `--initialize-at-build-time`. The former suppresses warnings about\nthe Clojure compiler's code generator, which is normally called when a namespace is loaded, but Native Image doesn't\nsupport runtime code generation. The latter loads all Clojure namespaces at build time.\n\n### Performance\n\nHere are some informal measurements (in December 2020) that how long it takes to invoke this hello world lambda\napplication, as reported by the lambda's billed duration:\n\n|                        | JIT compiled (OpenJDK) | AOT compiled (Substrate VM) |\n|------------------------|------------------------|-----------------------------|\n| Local, cold start      | 1500-1600 ms           | 8-11 ms                     |\n| Local, warm start      | 3-7 ms                 | 2-3 ms                      |\n| AWS Lambda, cold start | 3500-4500 ms           | 350-1850 ms                 |\n| AWS Lambda, warm start | 5-80 ms                | 1-2 ms                      |\n\nThe AOT compiled application itself starts in just a few milliseconds, but there is quite much overhead in the AWS\nLambda infrastructure. Normally a cold start is 300-400 ms billed duration, of which 99% is just waiting for the AWS\nLambda infrastructure to initialize. Occasionally it takes up to 2 or 3 seconds.\n\nBy the time the lambda handler is called inside AWS Lambda, the AOT compiled application process has an uptime of about\n100 ms, which means the first call to `http://${AWS_LAMBDA_RUNTIME_API}/2018-06-01/runtime/invocation/next` takes 100\nms. The remaining billed 200 ms then happens either before our application process is even started, or in the call\nto `http://${AWS_LAMBDA_RUNTIME_API}/2018-06-01/runtime/invocation/${REQUEST_ID}/response` after the lambda handler is\nfinished.\n\nHopefully Amazon will optimize the cold start of container images and/or only bill the time taken by our application.\n\n## Developing\n\n### Prerequisites\n\nBuild tools:\n\n- [Java 11 JDK](https://www.oracle.com/java/technologies/javase-downloads.html)\n- [Leiningen](https://leiningen.org/)\n- [Docker](https://www.docker.com/)\n- [Docker Compose](https://docs.docker.com/compose/)\n\nDeployment tools:\n\n- [tfenv](https://github.com/tfutils/tfenv)\n- [AWS CLI](https://aws.amazon.com/cli/)\n\n### Building\n\nBuild the app:\n\n    ./scripts/build.sh\n\nRun the app with OpenJDK:\n\n    docker-compose up -d --build jvm\n\nRun the app with GraalVM. This also writes native-image configuration to `target/native-image` directory, from where you\ncan copy them to `resources/META-INF/native-image` or its subdirectories:\n\n    docker-compose up -d --build graalvm\n\nRun the app with GraalVM Native Image/Substrate VM:\n\n    docker-compose up -d --build native\n\nTry calling the app. This is useful for exercising all code paths to generate native-image configuration:\n\n    ./smoke-test.sh\n\nView logs:\n\n    docker-compose logs --follow\n\nShutdown the app:\n\n    docker-compose down\n\n### Deploying\n\nPrepare the deployment environment in AWS:\n\n    # \u003cchange AWS_PROFILE in scripts/env-setup.sh\u003e \n    # \u003cchange terraform backend in deployment/main.tf\u003e\n    . ./scripts/env-setup.sh \n    cd deployment\n    terraform init\n    terraform apply -target=aws_ecr_repository.releases -target=data.aws_region.current\n\nDeploy the app:\n\n    ./scripts/build.sh\n    docker-compose build native         # or \"jvm\"\n    . ./scripts/env-setup.sh \n    ./scripts/deploy.sh\n\n### Terraform commands\n\n*All Terraform commands need to be run in the `deployment` directory.*\n\nInitialize the working directory. This creates a local cache in the `.terraform` directory:\n\n    terraform init\n\nPreview pending changes:\n\n    terraform plan\n\nApply pending changes:\n\n    terraform apply\n\nUpgrade Terraform providers:\n\n    terraform init -upgrade\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fluontola%2Fnative-clojure-lambda","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fluontola%2Fnative-clojure-lambda","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fluontola%2Fnative-clojure-lambda/lists"}