{"id":14057014,"url":"https://github.com/seanmor5/honeycomb","last_synced_at":"2025-08-21T00:31:47.108Z","repository":{"id":251964064,"uuid":"836821148","full_name":"seanmor5/honeycomb","owner":"seanmor5","description":"Fast LLM inference with Elixir and Bumblebee","archived":false,"fork":false,"pushed_at":"2024-08-06T18:24:05.000Z","size":37,"stargazers_count":55,"open_issues_count":11,"forks_count":1,"subscribers_count":3,"default_branch":"main","last_synced_at":"2024-12-18T18:58:46.134Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Elixir","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/seanmor5.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-08-01T16:17:29.000Z","updated_at":"2024-12-11T06:07:02.000Z","dependencies_parsed_at":"2024-08-06T22:07:02.321Z","dependency_job_id":"c8f577a1-d6cd-4a02-b529-211467b854a3","html_url":"https://github.com/seanmor5/honeycomb","commit_stats":null,"previous_names":["seanmor5/honeycomb"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/seanmor5%2Fhoneycomb","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/seanmor5%2Fhoneycomb/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/seanmor5%2Fhoneycomb/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/seanmor5%2Fhoneycomb/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/seanmor5","download_url":"https://codeload.github.com/seanmor5/honeycomb/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":230471175,"owners_count":18231193,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-13T02:00:34.618Z","updated_at":"2024-12-19T17:09:54.817Z","avatar_url":"https://github.com/seanmor5.png","language":"Elixir","funding_links":[],"categories":["LLM Clients \u0026 SDKs","Generative AI"],"sub_categories":["How to Join","LLM Tools"],"readme":"# Honeycomb\n\nFast LLM inference built on Elixir, [Bumblebee](https://github.com/elixir-nx/bumblebee), and [EXLA](https://github.com/elixir-nx/nx/tree/main/exla).\n\n## Usage\n\nHoneycomb can be used as a standalone inference service or as a dependency in an existing Elixir project.\n\n### As a separate service\n\nTo use Honeycomb as a separate service, you just need to clone the project and run:\n\n```shell\nmix honeycomb.serve \u003cconfig\u003e\n```\n\nThe following arguments are required:\n\n  * `--model` - HuggingFace model repo to use\n\n  * `--chat-template` - Chat template to use\n\nThe following arguments are optional:\n\n  * `--max-sequence-length` - Text generation max sequence length. Total sequence\n    length accounts for both input and output tokens.\n\n  * `--hf-auth-token` - HuggingFace auth token for accessing private or gated repos.\n\nThe Honeycomb server is compatible with the OpenAI API, so you can use it as a drop-in replacement by changing the `api_url` in the OpenAI client.\n\n### As a dependency\n\nTo use Honeycomb as a dependency, first add it to your `deps`:\n  \n```elixir\ndefp deps do\n  [{:honeycomb, github: \"seanmor5/honeycomb\"}]\nend\n```\n\nNext, you'll need to configure the serving options:\n\n```elixir\nconfig :honeycomb, Honeycomb.Serving,\n  model: \"microsoft/Phi-3-mini-4k-instruct\",\n  chat_template: \"phi3\",\n  auth_token: System.fetch_env!(\"HF_TOKEN\")\n```\n\nThen you can call Honeycomb directly:\n\n```elixir\nmessages = [%{role: \"user\", content: \"Hello!\"}]\nHoneycomb.chat_completion(messages: messages)\n```\n\n## Benchmarks\n\nHoneycomb ships with some basic benchmarks and profiling utilities. You can benchmark and/or profile your inference configuration by running:\n\n```shell\nmix honeycomb.benchmark \u003cconfig\u003e\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fseanmor5%2Fhoneycomb","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fseanmor5%2Fhoneycomb","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fseanmor5%2Fhoneycomb/lists"}