{"id":24911255,"url":"https://github.com/mutablelogic/go-llm","last_synced_at":"2026-02-15T20:05:58.725Z","repository":{"id":275312140,"uuid":"923644405","full_name":"mutablelogic/go-llm","owner":"mutablelogic","description":"Large Language Model API interface","archived":false,"fork":false,"pushed_at":"2026-02-08T16:28:33.000Z","size":37273,"stargazers_count":7,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-02-08T20:48:12.456Z","etag":null,"topics":["anthropic","anthropic-api","anthropic-claude","golang","llm","mcp","mcp-server","mistral","mistral-api","ollama","ollama-api"],"latest_commit_sha":null,"homepage":"https://pkg.go.dev/github.com/mutablelogic/go-llm","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mutablelogic.png","metadata":{"files":{"readme":null,"changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-01-28T16:06:44.000Z","updated_at":"2026-02-08T16:28:37.000Z","dependencies_parsed_at":"2025-05-05T16:33:41.165Z","dependency_job_id":"97c13ecd-e489-43db-bf16-52bbf8e69db4","html_url":"https://github.com/mutablelogic/go-llm","commit_stats":null,"previous_names":["mutablelogic/go-llm"],"tags_count":10,"template":false,"template_full_name":null,"purl":"pkg:github/mutablelogic/go-llm","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mutablelogic%2Fgo-llm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mutablelogic%2Fgo-llm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mutablelogic%2Fgo-llm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mutablelogic%2Fgo-llm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mutablelogic","download_url":"https://codeload.github.com/mutablelogic/go-llm/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mutablelogic%2Fgo-llm/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29487813,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-15T19:29:10.908Z","status":"ssl_error","status_checked_at":"2026-02-15T19:29:10.419Z","response_time":118,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["anthropic","anthropic-api","anthropic-claude","golang","llm","mcp","mcp-server","mistral","mistral-api","ollama","ollama-api"],"created_at":"2025-02-02T04:18:53.180Z","updated_at":"2026-02-15T20:05:58.706Z","avatar_url":"https://github.com/mutablelogic.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# go-llm\n\nThe module implements a simple API interface for large language models\nwhich run on [Ollama](https://github.com/ollama/ollama/blob/main/docs/api.md),\n[Anthopic](https://docs.anthropic.com/en/api/getting-started), [Mistral](https://docs.mistral.ai/)\nand [OpenAI](https://platform.openai.com/docs/api-reference). The module implements the ability to:\n\n* Maintain a session of messages;\n* Tool calling support, including using your own tools (aka Tool plugins);\n* Create embedding vectors from text;\n* Stream responses;\n* Multi-modal support (aka, Images, Audio and Attachments);\n* Text-to-speech (OpenAI only) for completions\n\nThere is a command-line tool included in the module which can be used to interact with the API.\nIf you have docker installed, you can use the following command to run the tool, without\ninstallation:\n\n```bash\n# Display version, help\ndocker run ghcr.io/mutablelogic/go-llm version\ndocker run ghcr.io/mutablelogic/go-llm --help\n\n# Interact with Claude to retrieve news headlines, assuming\n# you have an API key for both Anthropic and NewsAPI\ndocker run -e ANTHROPIC_API_KEY -e NEWSAPI_KEY \\\n  ghcr.io/mutablelogic/go-llm \\\n  chat mistral-small-latest --prompt \"What is the latest news?\"\n```\n\nSee below for more information on how to use the command-line tool (or how to\ninstall it if you have a `go` compiler).\n\n## Programmatic Usage\n\nSee the documentation [here](https://pkg.go.dev/github.com/mutablelogic/go-llm)\nfor integration into your own code.\n\n### Agent Instantiation\n\nFor each LLM provider, you create an agent which can be used to interact with the API.\nTo create an\n[Ollama](https://pkg.go.dev/github.com/mutablelogic/go-llm/pkg/anthropic)\nagent,\n\n```go\nimport (\n  \"github.com/mutablelogic/go-llm/pkg/ollama\"\n)\n\nfunc main() {\n  // Create a new agent - replace the URL with the one to your Ollama instance\n  agent, err := ollama.New(\"https://ollama.com/api/v1/\")\n  if err != nil {\n    panic(err)\n  }\n  // ...\n}\n```\n\nTo create an\n[Anthropic](https://pkg.go.dev/github.com/mutablelogic/go-llm/pkg/anthropic)\nagent with an API key stored as an environment variable,\n\n```go\nimport (\n  \"github.com/mutablelogic/go-llm/pkg/anthropic\"\n)\n\nfunc main() {\n  // Create a new agent\n  agent, err := anthropic.New(os.Getenv(\"ANTHROPIC_API_KEY\"))\n  if err != nil {\n    panic(err)\n  }\n  // ...\n}\n```\n\nFor [Mistral](https://pkg.go.dev/github.com/mutablelogic/go-llm/pkg/mistral) models, you can use:\n\n```go\nimport (\n  \"github.com/mutablelogic/go-llm/pkg/mistral\"\n)\n\nfunc main() {\n  // Create a new agent\n  agent, err := mistral.New(os.Getenv(\"MISTRAL_API_KEY\"))\n  if err != nil {\n    panic(err)\n  }\n  // ...\n}\n```\n\nSimilarly for [OpenAI](https://pkg.go.dev/github.com/mutablelogic/go-llm/pkg/openai)\nmodels, you can use:\n\n```go\nimport (\n  \"github.com/mutablelogic/go-llm/pkg/openai\"\n)\n\nfunc main() {\n  // Create a new agent\n  agent, err := openai.New(os.Getenv(\"OPENAI_API_KEY\"))\n  if err != nil {\n    panic(err)\n  }\n  // ...\n}\n```\n\nYou can append options to the agent creation to set the client/server communication options,\nsuch as user agent strings, timeouts, debugging, rate limiting, adding custom headers, etc. See [here](https://pkg.go.dev/github.com/mutablelogic/go-client#readme-basic-usage) for more information.\n\nThere is also an _aggregated_ agent which can be used to interact with multiple providers at once. This is useful if you want\nto use models from different providers simultaneously.\n\n```go\nimport (\n  \"github.com/mutablelogic/go-llm/pkg/agent\"\n)\n\nfunc main() {\n  // Create a new agent which aggregates multiple providers\n  agent, err := agent.New(\n    agent.WithAnthropic(os.Getenv(\"ANTHROPIC_API_KEY\")), \n    agent.WithMistral(os.Getenv(\"MISTRAL_API_KEY\")),\n    agent.WithOpenAI(os.Getenv(\"OPENAI_API_KEY\")),\n    agent.WithOllama(os.Getenv(\"OLLAMA_URL\")),\n  )\n  if err != nil {\n    panic(err)\n  }\n  // ...\n}\n```\n\n### Completion\n\nYou can generate a **completion** as follows,\n\n```go\nimport (\n  \"github.com/mutablelogic/go-llm\"\n)\n\nfunc completion(ctx context.Context, agent llm.Agent) (string, error) {\n  completion, err := agent.\n    Model(ctx, \"claude-3-5-haiku-20241022\").\n    Completion((ctx, \"Why is the sky blue?\")\n  if err != nil {\n    return \"\", err\n  } else {\n    return completion.Text(0), nil\n  }\n}\n```\n\nThe zero index argument on `completion.Text(int)` indicates you want the text from the zero'th completion choice, for providers who can generate serveral different choices simultaneously.\n\nUse one of the following options as an argument to the `Completion` method to customize the output format of the completion, which needs to be paired with the right model:\n\n* `llm.WithFormat(\"text\")` - Generate text output (default).\n* `llm.WithFormat(\"json\")` - Generate JSON output.\n* `llm.WithFormat(\"image\", \"jpeg\")` - Generate JPEG image output (for models which support it).\n* `llm.WithFormat(\"audio\", \"mp3\")` - Generate audio output (for models which support it). Possible values are `mp3`, `opus`, `aac`, `flac`, `wav`, and `pcm`.\n\n### Chat Sessions\n\nYou create a **chat session** with a model as follows,\n\n```go\nimport (\n  \"github.com/mutablelogic/go-llm\"\n)\n\nfunc session(ctx context.Context, agent llm.Agent) error {\n  // Create a new chat session\n  session := agent.\n    Model(ctx, \"claude-3-5-haiku-20241022\").\n    Context()\n\n  // Repeat forever\n  for {\n    err := session.FromUser(ctx, \"hello\")\n    if err != nil {\n      return err\n    }\n\n    // Print the response for the zero'th completion\n    fmt.Println(session.Text(0))\n  }\n}\n```\n\nThe `Context` object will continue to store the current session and options, and will\nensure the session is maintained across multiple completion calls.\n\n### Embedding Generation\n\nYou can generate embedding vectors using an appropriate model with Ollama, OpenAI and Mistral models:\n\n```go\nimport (\n  \"github.com/mutablelogic/go-llm\"\n)\n\nfunc embedding(ctx context.Context, agent llm.Agent) error {\n  vector, err := agent.\n    Model(ctx, \"mistral-embed\").\n    Embedding(ctx, \"hello\")\n  // ...\n}\n```\n\n### Attachments \u0026 Image Caption Generation\n\nSome models have `vision` capability and others can also summarize text. For example, to\ngenerate captions for an image,\n\n```go\nimport (\n  \"github.com/mutablelogic/go-llm\"\n)\n\nfunc generate_image_caption(ctx context.Context, agent llm.Agent, path string) (string, error) {\n  f, err := os.Open(path)\n  if err != nil {\n    return \"\", err\n  }\n  defer f.Close()\n\n  completion, err := agent.\n    Model(ctx, \"claude-3-5-sonnet-20241022\").\n    Completion((ctx, \"Provide a short caption for this image\", llm.WithAttachment(f))\n  if err != nil {\n    return \"\", err  \n  }    \n\n  return completion.Text(0), nil\n}\n```\n\nTo summarize a text or PDF document is exactly the same using an Anthropic model, but maybe\nwith a different prompt.\n\n### Streaming\n\nStreaming is supported with all providers, but Ollama cannot be used with streaming and tools\nsimultaneously. You provide a callback function of signature `func(llm.Completion)` which will\nbe called as a completion is received.\n\n```go\nimport (\n  \"github.com/mutablelogic/go-llm\"\n)\n\nfunc generate_completion(ctx context.Context, agent llm.Agent, prompt string) (string, error) {\n   completion, err := agent.\n    Model(ctx, \"claude-3-5-haiku-20241022\").\n    Completion((ctx, \"Why is the sky blue?\", llm.WithStream(stream_callback))\n  if err != nil {\n    return \"\", err\n  } else {\n    return completion.Text(0), nil\n  }\n}\n\nfunc stream_callback(completion llm.Completion) {\n  // Print out the completion text on each call\n  fmt.Println(completion.Text(0))\n}\n\n```\n\n### Tool Support\n\nAll providers support tools, but not all models. Your own tools should implement the\nfollowing interface:\n\n```go\npackage llm\n\n// Definition of a tool\ntype Tool interface {\n  Name() string                     // The name of the tool\n  Description() string              // The description of the tool\n  Run(context.Context) (any, error) // Run the tool with a deadline and \n                                    // return the result\n}\n```\n\nFor example, if you want to implement a tool which adds two numbers,\n\n```go\npackage addition\n\ntype Adder struct {\n  A float64 `name:\"a\" help:\"The first number\" required:\"true\"`\n  B float64 `name:\"b\" help:\"The second number\" required:\"true\"`\n}\n\nfunc (Adder) Name() string {\n  return \"add_two_numbers\"\n}\n\nfunc (Adder) Description() string {\n  return \"Add two numbers together and return the result\"\n}\n\nfunc (a Adder) Run(context.Context) (any, error) {\n  return a.A + a.B, nil\n}\n```\n\nThen you can include your tool as part of the completion. It's possible that a\ncompletion will continue to call additional tools, in which case you should\nactually loop through completions until no tool calls are made.\n\n```go\nimport (\n  \"github.com/mutablelogic/go-llm\"\n  \"github.com/mutablelogic/go-llm/pkg/tool\"\n)\n\nfunc add_two_numbers(ctx context.Context, agent llm.Agent) (string, error) {\n  context := agent.Model(ctx, \"claude-3-5-haiku-20241022\").Context()\n  toolkit := tool.NewToolKit()\n  toolkit.Register(\u0026Adder{})\n\n  // Get the tool call\n  if err := context.FromUser(ctx, \"What is five plus seven?\", llm.WithToolKit(toolkit)); err != nil {\n    return \"\", err\n  }\n\n  // Call tools\n  for {\n    calls := context.ToolCalls(0)\n    if len(calls) == 0 {\n      break\n    }\n\n    // Print out any intermediate messages\n    if context.Text(0) != \"\" {\n      fmt.Println(context.Text(0))      \n    }\n\n    // Get the results from the toolkit\n    results, err := toolkit.Run(ctx, calls...)\n    if err != nil {\n      return \"\", err\n    }\n\n    // Get another tool call or a user response\n    if err := context.FromTool(ctx, results...); err != nil {\n      return \"\", err\n    }\n  }\n\n  // Return the result\n  return context.Text(0)\n}\n```\n\nParameters are implemented as struct fields, with tags. The tags you can include are:\n\n* `name:\"\"` - Set the name for the parameter\n* `json:\"\"` - If `name` is not used, then the name is set from the `json` tag\n* `help:\":` - Set the description for the parameter\n* `required:\"\"` - The parameter is required as part of the tool call\n* `enum:\"a,b,c\"` - The parameter value should be one of these comma-separated options\n\nThe transation of field types is as follows:\n\n* `string` - Translates as JSON `string`\n* `uint`, `int` - Translates to JSON `integer`\n* `float32`, `float64` - Translates to JSON `number`\n\n## The Command Line Tool\n\nYou can use the command-line tool to interact with the API. To build the tool, you can use the following command:\n\n```bash\ngo install github.com/mutablelogic/go-llm/cmd/llm@latest\nllm --help\n```\n\nThe output is something like:\n\n```text\nUsage: llm \u003ccommand\u003e [flags]\n\nLLM agent command line interface\n\nFlags:\n  -h, --help                      Show context-sensitive help.\n      --debug                     Enable debug output\n  -v, --verbose                   Enable verbose output\n      --timeout=DURATION          Agent connection timeout\n      --ollama-endpoint=STRING    Ollama endpoint ($OLLAMA_URL)\n      --anthropic-key=STRING      Anthropic API Key ($ANTHROPIC_API_KEY)\n      --mistral-key=STRING        Mistral API Key ($MISTRAL_API_KEY)\n      --open-ai-key=STRING        OpenAI API Key ($OPENAI_API_KEY)\n      --gemini-key=STRING         Gemini API Key ($GEMINI_API_KEY)\n      --news-key=STRING           News API Key ($NEWSAPI_KEY)\n\nCommands:\n  agents       Return a list of agents\n  models       Return a list of models\n  tools        Return a list of tools\n  download     Download a model (for Ollama)\n  chat         Start a chat session\n  complete     Complete a prompt, generate image or speech from text\n  embedding    Generate an embedding\n  version      Print the version of this tool\n\nRun \"llm \u003ccommand\u003e --help\" for more information on a command.\n```\n\n### Prompt Completion\n\nTo have the model respond to a prompt, you can use the `complete` command. For example, to\nhave the model respond to the prompt \"What is the capital of France?\" using the `claude-3-5-haiku-20241022`\nmodel, you can use the following command:\n\n```bash\nllm complete \"What is the capital of France?\"\n```\n\nThe first time you use the command use the ``--model`` flag to specify the model you want to use. Your\nchoice of model will be remembered for subsequent completions.\n\n### Explain computer code\n\nTo have the model explain a piece of computer code, you can pipe the code into the `complete` command.\nFor example, to have the model explain the code in the file `example.go`, you can use the following command:\n\n```bash\ncat example.go | llm complete\n```\n\n### Caption an image\n\nTo have the model generate a caption for an image, you can use the `complete` command with the `--file`\nflag. For example, to have the model generate a caption for the image in the file `example.jpg`, you can use\nthe following command:\n\n```bash\nllm complete --file picture.png \"Explain this image\"\n```\n\n### Generate an image\n\nTo have the model generate an image from a prompt, you can use the `complete` command with the `--format image`\noption. For example, to have the model generate an image from the prompt \"A picture of a cat\", you can use\nthe following command:\n\n```bash\nllm complete --model dall-e-3 --format image \"A picture of a cat\"\n```\n\nFlags `--size`, `--quality` and `--style` can be used to specify the image parameters. It will write the image\nfile in the current working directory.\n\n### Convert text to speech\n\nTo have a model generate text from speech:\n\n```bash\necho book.txt | llm complete --model tts-1 --format mp3 --voice coral\n```\n\nIt will write the audio file in the current working directory. You can currently write\nthe following audio formats and voices:\n\n* Formats: `--format mp3`, `--format opus`, `--format aac`, `--format flac`, `--format wav`, `--format pcm`\n* Voices: `--voice alloy`, `--voice ash`, `--voice coral`, `--voice echo`, `--voice fable`, `--voice onyx`, `--voice nova`, `--voice sage`, `--voice shimmer`\n\n## Contributing \u0026 Distribution\n\n_This module is currently in development and subject to change_. Please do file\nfeature requests and bugs [here](https://github.com/mutablelogic/go-llm/issues).\nThe [license is Apache 2](LICENSE) so feel free to redistribute. Redistributions in either source\ncode or binary form must reproduce the copyright notice, and please link back to this\nrepository for more information:\n\n\u003e **go-llm**\\\n\u003e [https://github.com/mutablelogic/go-llm/](https://github.com/mutablelogic/go-llm/)\\\n\u003e Copyright (c) 2025 David Thorpe, All rights reserved.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmutablelogic%2Fgo-llm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmutablelogic%2Fgo-llm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmutablelogic%2Fgo-llm/lists"}