{"id":16129479,"url":"https://github.com/klutchell/balena-serge","last_synced_at":"2025-04-06T13:43:19.909Z","repository":{"id":160025382,"uuid":"635010147","full_name":"klutchell/balena-serge","owner":"klutchell","description":"Run an LLM on your edge device with balena.io","archived":false,"fork":false,"pushed_at":"2024-07-29T13:19:39.000Z","size":145,"stargazers_count":3,"open_issues_count":1,"forks_count":2,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-04-04T14:03:43.279Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/klutchell.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2023-05-01T19:04:05.000Z","updated_at":"2024-07-29T13:19:42.000Z","dependencies_parsed_at":"2023-05-05T13:46:26.906Z","dependency_job_id":"280517a9-7a58-47ac-be9b-d0da2f393a20","html_url":"https://github.com/klutchell/balena-serge","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/klutchell%2Fbalena-serge","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/klutchell%2Fbalena-serge/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/klutchell%2Fbalena-serge/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/klutchell%2Fbalena-serge/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/klutchell","download_url":"https://codeload.github.com/klutchell/balena-serge/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247492464,"owners_count":20947541,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-09T22:10:58.449Z","updated_at":"2025-04-06T13:43:19.869Z","avatar_url":"https://github.com/klutchell.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# balena-serge\n\nA chat interface based on llama.cpp for running Alpaca models.\n\nEntirely self-hosted, no API keys needed. Fits on 4GB of RAM and runs on the CPU.\n\nYou can read more on the [official project README](https://github.com/serge-chat/serge).\n\n## Hardware required\n\nLLaMA will just crash if you don't have enough available memory for your model.\n\n- 7B requires about 4.5GB of free RAM\n- 13B requires about 12GB free\n- 30B requires about 20GB free\n\nI've tested on Intel NUC, but any `amd64` or `aarch64` device with at least 5GB of memory should work!\n\nIn theory Raspberry Pi 4 8GB model should work but I haven't tried it myself!\n\n## Getting Started\n\nYou can one-click-deploy this project to balena using the button below:\n\n[![deploy button](https://balena.io/deploy.svg)](https://dashboard.balena-cloud.com/deploy?repoUrl=https://github.com/klutchell/balena-serge)\n\n## Manual Deployment\n\nAlternatively, deployment can be carried out by manually creating a [balenaCloud account](https://dashboard.balena-cloud.com) and application, flashing a device,\ndownloading the project and pushing it via the [balena CLI](https://github.com/balena-io/balena-cli).\n\n### Environment Variables\n\n| Name | Default | Purpose                                                                                                                                 |\n| ---- | ------- | --------------------------------------------------------------------------------------------------------------------------------------- |\n| `TZ` | `UTC`   | The timezone in your location. Find a [list of all timezone values here](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones). |\n\n## Usage\n\nOnce your device joins the fleet you'll need to allow some time for it to download the application containers.\n\nWhen it's done you should be able to access the app on port 80 of the device.\n\nYou can read more on the [official project README](https://github.com/serge-chat/serge).\n\n## Contributing\n\nPlease open an issue or submit a pull request with any features, fixes, or changes.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fklutchell%2Fbalena-serge","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fklutchell%2Fbalena-serge","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fklutchell%2Fbalena-serge/lists"}