{"id":15591495,"url":"https://github.com/metaskills/llamafile-on-lambda","last_synced_at":"2025-06-23T21:40:02.693Z","repository":{"id":247137002,"uuid":"824565144","full_name":"metaskills/llamafile-on-lambda","owner":"metaskills","description":"Serverless AI Inference with Gemma 2 using Mozilla's llamafile on AWS Lambda","archived":false,"fork":false,"pushed_at":"2024-07-30T01:54:43.000Z","size":885,"stargazers_count":11,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-06-20T13:09:16.430Z","etag":null,"topics":["ai","aws-lambda","gemma","gemma2","lambda","llamafile"],"latest_commit_sha":null,"homepage":"https://www.unremarkable.ai/serverless-ai-inference-with-gemma-2-using-mozillas-llamafile-on-aws-lambda","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/metaskills.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-07-05T12:03:09.000Z","updated_at":"2024-08-03T18:32:40.000Z","dependencies_parsed_at":"2025-04-28T10:38:14.597Z","dependency_job_id":"ee5efdb3-c8c6-4e00-b5ca-2def278f2c8c","html_url":"https://github.com/metaskills/llamafile-on-lambda","commit_stats":null,"previous_names":["metaskills/llamafile-on-lambda"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/metaskills/llamafile-on-lambda","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/metaskills%2Fllamafile-on-lambda","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/metaskills%2Fllamafile-on-lambda/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/metaskills%2Fllamafile-on-lambda/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/metaskills%2Fllamafile-on-lambda/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/metaskills","download_url":"https://codeload.github.com/metaskills/llamafile-on-lambda/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/metaskills%2Fllamafile-on-lambda/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":261561175,"owners_count":23177542,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","aws-lambda","gemma","gemma2","lambda","llamafile"],"created_at":"2024-10-02T23:41:31.053Z","updated_at":"2025-06-23T21:40:02.662Z","avatar_url":"https://github.com/metaskills.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Serverless AI Inference with Gemma 2 using Mozilla's llamafile on AWS Lambda\n\nhttps://www.unremarkable.ai/serverless-ai-inference-with-gemma-2-using-mozillas-llamafile-on-aws-lambda\n\n![Llamafile on AWS Lambda](public/llamafile-on-lambda.jpg)\n\n## Setup\n\nWe assume you have the following configured or installed.\n\n1. An AWS account with credentials configured.\n2. The [AWS SAM CLI](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/install-sam-cli.html) installed for fast and easy serverless deployments.\n3. Docker [installed](https://www.docker.com) for easy container builds and deployments.\n\nAfter you clone the repo, setup your dependencies with the following command:\n\n```shell\nnpm install\n```\n\n## Usage\n\nNow you can run the following commands from the root directory.\n\n* `./bin/build` - To download  and build a llamafile container for deployment.\n* `./bin/server` - To run the download (above) llamafile server locally.\n* `./bin/deploy` - Deploy to AWS Lambda. Also does a build if needed.\n\n## Chat\n\nThis project uses [Inquirer.js](https://www.npmjs.com/package/inquirer) to chat with the model using OpenAI's API. The model can be running locally using `bin/server` or deployed to Lambda using `bin/deploy`. Inquirer will ask for your local or function URL at the beginning of the chat session.\n\n![Llamafile on AWS Lambda](public/llamafile-on-lambda.gif)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmetaskills%2Fllamafile-on-lambda","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmetaskills%2Fllamafile-on-lambda","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmetaskills%2Fllamafile-on-lambda/lists"}