{"id":22475400,"url":"https://github.com/modfin/bellman","last_synced_at":"2025-06-19T06:34:01.862Z","repository":{"id":265448187,"uuid":"896018927","full_name":"modfin/bellman","owner":"modfin","description":"Golang lib for LLM APIs, ChatGPT, Gemini and Anthropic","archived":false,"fork":false,"pushed_at":"2025-06-11T15:52:33.000Z","size":155,"stargazers_count":56,"open_issues_count":0,"forks_count":2,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-06-11T17:13:16.234Z","etag":null,"topics":["anthropic","chatgpt","claude","gemini","llm","llm-api","ollama","ollama-client","openai"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/modfin.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-11-29T11:35:29.000Z","updated_at":"2025-06-11T15:51:53.000Z","dependencies_parsed_at":"2025-02-03T09:21:28.510Z","dependency_job_id":"b48320e6-273b-4ce7-9fb5-ad1e5c6e386c","html_url":"https://github.com/modfin/bellman","commit_stats":null,"previous_names":["modfin/bellman"],"tags_count":28,"template":false,"template_full_name":null,"purl":"pkg:github/modfin/bellman","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/modfin%2Fbellman","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/modfin%2Fbellman/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/modfin%2Fbellman/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/modfin%2Fbellman/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/modfin","download_url":"https://codeload.github.com/modfin/bellman/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/modfin%2Fbellman/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":260702446,"owners_count":23049251,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["anthropic","chatgpt","claude","gemini","llm","llm-api","ollama","ollama-client","openai"],"created_at":"2024-12-06T13:16:06.538Z","updated_at":"2025-06-19T06:33:56.840Z","avatar_url":"https://github.com/modfin.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Bellman\n\nIt tries to be unified interface to interact with LLMs and embedding models.\nIn particular, it seeks to make it easier to switch between models and vendors \nalong woth lowering tha barrier to get started.\nBellman supports  `VertexAI/Gemini`, `OpenAI`, `Anthropic`, `VoyageAI` and `Ollama`   \n\nBellman consists of two parts. The library and the service.\nThe go library enables you to interact with the different LLM vendors directly while the service, \n`bellmand` creates a proxy service that lets you connect to all providers with one api key.\n\nBellman supports the common things that we expect in modern llm models.\nChat, Structured, Tools and binary input.\n\n## Why\nThis project was built to the lack of official sdk/clients for the major players along with the\nslight differences in API.\nIt also became clear when we started to play around with different LLMs in our projects that the \ndifferences, while slight, had implications and for each new model introduced it became an overhead.\nThere are other projects out there, like go version of langchain, that deals with some of it.\nBut having one proxy to hadle all types of models made things alot easier for us to iterate over \nproblems, models and solutions.\n\n## The Service\n\n`bellmand` is a simple web service that implements the bellman library and exposes it through a http api\n\nThe easiest way to get started is to simply run it as a docker service.\n\n### Prerequisite\n\n- Docker being installed\n- API Keys to Anthropic, OpenAI, VertexAI(Google Gemini) and/or VoyageAI \n- Installing Ollama, https://ollama.com/ (very cool project imho)\n\n### Run\n\n```sh\n## Help / Man\ndocker run --rm -it modfin/bellman --help  \n\n## Example \ndocker run --rm -d modfin/bellman \\\n  --prometheus-metrics-basic-auth=\"user:pass\"\n  --ollama-url=http://localhost:11434 \\\n  --openai-key=\"$(cat ./credentials/openai-api-key.txt)\" \\\n  --anthropic-key=\"$(cat ./credentials/anthropic-api-key.txt)\" \\\n  --voyageai-key=\"$(cat ./credentials/voyageai-api-key.txt)\" \\\n  --google-credential=\"$(cat ./credentials/google-service-account.json)\" \\\n  --google-project=your-google-project \\\n  --google-region=europe-north1 \\\n  --api-key=qwerty \n```\n\nThis will start the bellmand service that proxies requests to the model you define in the request.\n\n## The Library \n\n### Installation\n\n```bash\ngo get github.com/modfin/bellman\n```\n\n## Usage\n\nThe library provides clients for Anthropic, Ollama, OpenAI, VertexAI, VoyageAI and Bellmand itself.\n\nAll the clients implement the same interfaces, `gen.Generator` and `embed.Embeder`, \nand can there for be used interchangeably. \n\n```go \nclient, err := anthropic.New(...)\nclient, err := ollama.New(...)\nclient, err := openai.New(...)\nclient, err := vertexai.New(...)\nclient, err := voyageai.New(...)\nclient, err := bellman.New(...)\n```\n\n\n## bellman.New() \n\nThe benefit of using the bellman client,\nwhen you are running `bellmand`,\nis that we can interchangeably use any model that we wish to interact with.\n\n\n```go \n\nclient, err := bellman.New(...)\nllm := client.Generator() \nres, err := llm.Model(openai.GenModel_gpt4o_mini).\n   Prompt(\n       prompt.AsUser(\"What company made you?\"),\n    )\nfmt.Println(res, err)\n// OpenAI\n\n\nres, err := llm.Model(vertexai.GenModel_gemini_1_5_flash).\n    Prompt(\n        prompt.AsUser(\"What company made you?\"),\n    )\nfmt.Println(res, err)\n// Google\n\n// or even a custom model that you created yourself (trained) \n// or a new model that is not in the library yet\nmodel := gen.Model{\n    Provider: vertexai.Provider,\n    Name:     \"gemini-2.0-flash-exp\",\n    Config:   map[string]interface{}{\"region\": \"us-central1\"},\n}\nres, err := llm.Model(model).\n    Prompt(\n        prompt.AsUser(\"What company made you?\"),\n    )\nfmt.Println(res, err)\n// Google\n\n\n```\n\n\n## Prompting \n\nJust normal conversation mode\n\n```go \n\nres, err  := openai.New(apiKey).Generator().\n    Model(openai.GenModel_gpt4o_mini).\n    Prompt(\n        prompt.AsUser(\"What is the distance to the moon?\"),\n    )\nif err != nil {\n    log.Fatalf(\"Prompt() error = %v\", err)\n}\n\nawnser, err := res.AsText()\n\nfmt.Println(awnser, err)\n// The average distance from Earth to the Moon is approximately 384,400 kilometers \n// (about 238,855 miles). This distance can vary slightly because the Moon's orbit\n// is elliptical, ranging from about 363,300 km (225,623 miles) at its closest \n// (perigee) to 405,500 km (251,966 miles) at its farthest (apogee). \u003cnil\u003e\n```\n\n## System Prompting\n\nJust normal conversation mode\n\n```go \n\nres, err := openai.New(apiKey).Generator().\n    Model(openai.GenModel_gpt4o_mini).\n    System(\"You are a expert movie quoter and lite fo finish peoples sentences with a movie reference\").\n    Prompt(\n        prompt.AsUser(\"Who are you going to call?\"),\n    )\nif err != nil {\n    log.Fatalf(\"Prompt() error = %v\", err)\n}\n\nawnser, err := res.AsText()\n\nfmt.Println(awnser, err)\n// Ghostbusters! \u003cnil\u003e\n```\n\n\n\n## General Configuration\n\nSetting things like temperature, max tokens, top p, and stop secuences\n\n```go \n\nres, err := openai.New(apiKey).Generator().\n    Model(openai.GenModel_gpt4o_mini).\n\t    Temperature(0.5).\n\t    MaxTokens(100).\n\t    TopP(0.9). // should really not be used with temperature\n        StopAt(\".\", \"!\", \"?\").\n    Prompt(\n        prompt.AsUser(\"Write me a 2 paragraph text about gophers\"),\n    )\nif err != nil {\n    log.Fatalf(\"Prompt() error = %v\", err)\n}\n\nawnser, err := res.AsText()\n\nfmt.Println(awnser, err)\n// Gophers are small, \n// burrowing rodents belonging to the family Geomyidae, \n// primarily found in North America\n```\n\n\n## Structured Output\nFrom many models, you can now specify a schema that you want the models to output. \n\nA supporting library that can transforming your go struct to json schema is provided. `github.com/modfin/bellman/schema`\n\n```go\n\ntype Quote struct {\n    Character string `json:\"character\"`\n    Quote     string `json:\"quote\"`\n}\ntype Responese struct {\n    Quotes []Quote `json:\"quotes\"`\n}\n\n\nllm := vertexai.New(googleConfig).Generator()\nres, err := llm.\n    Model(vertexai.GenModel_gemini_1_5_pro).\n    Output(schema.From(Responese{})).\n    Prompt(\n        prompt.AsUser(\"give me 3 quotes from different characters in Hamlet\"),\n    )\nif err != nil {\n    log.Fatalf(\"Prompt() error = %v\", err)\n}\n\nawnser, err := res.AsText() // will return the json of the struct\nfmt.Println(awnser, err)\n//{\n//  \"quotes\": [\n//    {\n//      \"character\": \"Hamlet\",\n//      \"quote\": \"To be or not to be, that is the question.\"\n//    },\n//    {\n//      \"character\": \"Polonius\",\n//      \"quote\": \"This above all: to thine own self be true.\"\n//    },\n//    {\n//      \"character\": \"Queen Gertrude\",\n//      \"quote\": \"The lady doth protest too much, methinks.\"\n//    }\n//  ]\n//}  \u003cnil\u003e\n\nvar result Result\nerr := res.Unmarshal(\u0026result) // Just a shorthand to marshal it into your struct\nfmt.Println(result, err)\n// {[\n//      {Hamlet To be or not to be, that is the question.} \n//      {Polonius This above all: to thine own self be true.} \n//      {Queen Gertrude The lady doth protest too much, methinks.}\n// ]} \u003cnil\u003e\n\n```\n\n\n\n## Tools\nThe Bellman library allows you to define and use tools in your prompts. \nHere is an example of how to define and use a tool:\n\n1. Define a tool:\n   ```go\n\n    type Args struct {\n         Name string `json:\"name\"`\n    }\n\n    getQuote := tools.NewTool(\"get_quote\",\n       tools.WithDescription(\n            \"a function to get a quote from a person or character in Hamlet\",\n       ),\n       tools.WithArgSchema(Args{}),\n       tools.WithCallback(func(jsondata string) (string, error) {\n           var arg Args\n           err := json.Unmarshal([]byte(jsondata), \u0026arg)\n           if err != nil {\n               return \"\",err\n           }\n\t\t   return dao.GetQuoateFrom(arg.Name)\n       }),\n   )\n   ```\n\n2. Use the tool in a prompt:\n   ```go\n      res, err := anthopic.New(apiKey).Generator().\n          Model(anthropic.GenModel_3_5_haiku_latest)).\n          System(\"You are a Shakespeare quote generator\").\n          Tools(getQuote).\n          // Configure a specific too to be used, or the setting for it\n          Tool(tools.RequiredTool). \n          Prompt(\n              prompt.AsUser(\"Give me 3 quotes from different characters\"),\n          )\n   \n      if err != nil {\n          log.Fatalf(\"Prompt() error = %v\", err)\n      }\n   \n      // Evaluate with callback function\n      err = res.Eval()\n      if err != nil {\n          log.Fatalf(\"Eval() error = %v\", err)\n      }\n      \n      \n      // or Evaluate your self\n      \n      tools, err := res.Tools()\n      if err != nil {\n            log.Fatalf(\"Tools() error = %v\", err)\n      }\n      \n      for _, tool := range tools {\n          log.Printf(\"Tool: %s\", tool.Name)\n          switch tool.Name {\n             // ....\n          }\n      }\n      \n   ```\n\n\n## Binary Data\nImages is supported by Gemini, OpenAI and Anthropic.\\\nPDFs is only supported by Gemini and Anthropic\n\n#### Image\n```go \n\n   image := \"/9j/4AAQSkZJRgABAQEBLAEsAAD//g......gM4OToWbsBg5mGu0veCcRZO6f0EjK5Jv5X/AP/Z\"\n   data, err := base64.StdEncoding.DecodeString(image)\n   if err != nil {\n      t.Fatalf(\"could not decode image %v\", err)\n   }\n   res, err := llm.\n      Prompt(\n          prompt.AsUserWithData(prompt.MimeImageJPEG, data),\n          prompt.AsUser(\"Describe the image to me\"),\n      )\n   \n   if err != nil {\n      t.Fatalf(\"Prompt() error = %v\", err)\n   }\n   fmt.Println(res.AsText())\n   // The image contains the word \"Hot!\" in red text. The text is centered on a white background. \n   // The exclamation point is after the word.  The image is a simple and straightforward \n   // depiction of the word \"hot.\" \u003cnil\u003e\n\n```\n\n\n#### PDF\n```go \n\n   pdf, err := os.ReadFile(\"path/to/pdf\")\n   if err != nil {\n      t.Fatalf(\"could open file, %v\", err)\n   }\n   \n   res, err := anthopic.New(apiKey).Generator().\n      Prompt(\n          prompt.AsUserWithData(prompt.MimeApplicationPDF, pdf),\n          prompt.AsUser(\"Describe to me what is in the PDF\"),\n      )\n   \n   if err != nil {\n      t.Fatalf(\"Prompt() error = %v\", err)\n   }\n   fmt.Println(res.AsText())\n   // The image contains the word \"Hot!\" in red text. The text is centered on a white background. \n   // The exclamation point is after the word.  The image is a simple and straightforward \n   // depiction of the word \"hot.\" \u003cnil\u003e\n\n```\n\n## RAG Example\n\nSupporter lib for \"automated\" RAG (Retrieval-Augmented Generation) is supported by Gemini, OpenAI and Anthropic.\n\n```go \n\ntype GetQuoteArg struct {\n   StockId int `json:\"stock_id\" json-description:\"the id of a stock for which  quote to get\"`\n}\ntype Search struct {\n   Name string `json:\"name\" json-description:\"the name of a stock being looked for\"`\n}\n\ngetQuote := tools.NewTool(\"get_quote\",\n   tools.WithDescription(\"a function get a stock quote based on stock id\"),\n   tools.WithArgSchema(GetQuoteArg{}),\n   tools.WithCallback(func(jsondata string) (string, error) {\n       var arg GetQuoteArg\n       err := json.Unmarshal([]byte(jsondata), \u0026arg)\n       if err != nil {\n           return \"\", err\n       }\n       return `{\"stock_id\": ` + strconv.Itoa(arg.StockId) + `,\"price\": 123.45}`, nil\n   }),\n)\n\ngetStock := tools.NewTool(\"get_stock\",\n   tools.WithDescription(\"a function a stock based on name\"),\n   tools.WithArgSchema(Search{}),\n   tools.WithCallback(func(jsondata string) (string, error) {\n       var arg GetQuoteArg\n       err := json.Unmarshal([]byte(jsondata), \u0026arg)\n       if err != nil {\n           return \"\", err\n       }\n       return `{\"stock_id\": 98765}`, nil\n   }),\n)\n\n\ntype Result struct {\n   StockId int     `json:\"stock_id\"`\n   Price   float64 `json:\"price\"`\n}\n\nllm := anthopic.New(apiKey).Generator()\nllm = llm.SetTools(getQuote, getStock)\n\nres, err := rag.Run[Result](5, llm, prompt.AsUser(\"Get me the price of Volvo B\"))\nif err != nil {\n   t.Fatalf(\"Prompt() error = %v\", err)\n}\n\nfmt.Printf(\"==== Result after %d calls ====\\n\", res.Depth)\nfmt.Printf(\"%+v\\n\", res.Result)\nfmt.Printf(\"==== Conversation ====\\n\")\n\nfor _, p := range res.Promps {\n   fmt.Printf(\"%s: %s\\n\", p.Role, p.Text)\n}\n\n// ==== Result after 2 calls ====\n// {StockId:98765 Price:123.45}\n// ==== Conversation ====\n// user:       Get me the price of Volvo B\n// assistant:  tool function call: get_stock with argument: {\"name\":\"Volvo B\"}\n// user:       result: get_stock =\u003e {\"stock_id\": 98765}\n// assistant:  tool function call: get_quote with argument: {\"stock_id\":98765}\n// user:       result: get_quote =\u003e {\"stock_id\": 98765,\"price\": 123.45}\n// assistant:  tool function call: __bellman__rag_result_callback with argument: {\"price\":123.45,\"stock_id\":98765}\n\n\n\n```\n\n\n## License\n\nThis project is licensed under the MIT License. See the `LICENSE` file for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmodfin%2Fbellman","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmodfin%2Fbellman","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmodfin%2Fbellman/lists"}