{"id":15715569,"url":"https://github.com/elithrar/llm-cloudflare","last_synced_at":"2025-05-06T20:45:31.272Z","repository":{"id":256028776,"uuid":"854145384","full_name":"elithrar/llm-cloudflare","owner":"elithrar","description":"An LLM CLI plugin for Cloudflare Workers AI models.","archived":false,"fork":false,"pushed_at":"2025-03-05T21:56:34.000Z","size":103,"stargazers_count":18,"open_issues_count":5,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-05-06T03:57:16.437Z","etag":null,"topics":["ai","cli","cloudflare","llm","workers-ai"],"latest_commit_sha":null,"homepage":"https://developers.cloudflare.com/workers-ai/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/elithrar.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-09-08T13:53:36.000Z","updated_at":"2025-02-28T23:03:55.000Z","dependencies_parsed_at":"2024-10-24T12:57:45.927Z","dependency_job_id":"dcf645ca-8a9d-475a-9740-43506a89166e","html_url":"https://github.com/elithrar/llm-cloudflare","commit_stats":null,"previous_names":["elithrar/llm-cloudflare"],"tags_count":15,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elithrar%2Fllm-cloudflare","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elithrar%2Fllm-cloudflare/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elithrar%2Fllm-cloudflare/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elithrar%2Fllm-cloudflare/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/elithrar","download_url":"https://codeload.github.com/elithrar/llm-cloudflare/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252769128,"owners_count":21801373,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","cli","cloudflare","llm","workers-ai"],"created_at":"2024-10-03T21:42:01.136Z","updated_at":"2025-05-06T20:45:31.219Z","avatar_url":"https://github.com/elithrar.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"## llm-cloudflare\n\nA plugin for the [`llm`](https://llm.datasette.io/en/stable/) CLI that allows you to use the text generation models (LLMs) running on globally on Cloudflare [Workers AI](https://developers.cloudflare.com/workers-ai/models/#text-generation), including models like Llama 3.1, Mistral 7B, Gemma and a number of task-specific fine tunes.\n\n`llm-cloudflare` is useful for:\n\n* Using and building with LLMs that may not efficiently run on your local machine (limited GPU, memory, etc) vs. having Workers AI run it on a GPU near you.\n* Validating the performance of and/or comparing multiple models.\n* Experimenting without needing to download models ahead-of-time.\n\n## Usage\n\n**Prerequisite**: You'll need the `llm` CLI [installed first](https://llm.datasette.io/en/stable/setup.html).\n\nInstall and setup the plugin:\n\n```sh\n# Install the plugin from pip\nllm install llm-cloudflare\n\n# Provide a valid Workers AI token\n# Docs: https://developers.cloudflare.com/workers-ai/get-started/rest-api/#1-get-api-token-and-account-id\nllm keys set cloudflare\n\n# Set your Cloudflare account ID\n# Docs: https://developers.cloudflare.com/workers-ai/get-started/rest-api/#1-get-api-token-and-account-id\nexport CLOUDFLARE_ACCOUNT_ID=\"33charlonghexstringhere\"\n```\n\nUse it by specifying a Workers AI model:\n\n```sh\nllm -m \"@cf/meta/llama-3.1-8b-instruct\" \"Write a Cloudflare Worker in ESM format that returns an empty JSON object as a response. Show only the code.\"\n```\n\nYou can set a Workers AI model as [the default model](https://llm.datasette.io/en/stable/setup.html#setting-a-custom-default-model) in `llm`:\n\n```sh\n# Set Llama 3.1 8B as the default\nllm models default \"@cf/meta/llama-3.1-8b-instruct\" \n# See what model is set as the default\nllm models default\n# @cf/meta/llama-3.1-8b-instruct\n```\n\n## Available models\n\nThis plugin provides access to the [text generation models](https://developers.cloudflare.com/workers-ai/models/#text-generation) (LLMs) provided by Workers AI.\n\nTo see what models are available, invoke `llm models`. Models prefixed with `Cloudflare Workers AI` are provided by this plugin.\n\nThe supported models are generated by scripts. New models thus rely on this plugin being updated periodically.\n\nIn the future, this plugin may also add support for Workers AI's [embedding models](https://developers.cloudflare.com/workers-ai/models/#text-embeddings) for use with [`llm embed`](https://llm.datasette.io/en/stable/embeddings/index.html).\n\n## Credits\n\nCredit to [@hex](https://github.com/hex) for https://github.com/hex/llm-perplexity, which heavily inspired the design of this plugin.\n\n## License\n\nCopyright Cloudflare, Inc (2024). Apache-2.0 licensed. See the LICENSE file for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Felithrar%2Fllm-cloudflare","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Felithrar%2Fllm-cloudflare","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Felithrar%2Fllm-cloudflare/lists"}