{"id":33574256,"url":"https://github.com/voocel/litellm","last_synced_at":"2026-04-01T23:43:54.444Z","repository":{"id":305275396,"uuid":"1002344635","full_name":"voocel/litellm","owner":"voocel","description":"LiteLLM for Go, the easiest way to write LLM-based programs in Go","archived":false,"fork":false,"pushed_at":"2026-02-07T15:10:18.000Z","size":271,"stargazers_count":30,"open_issues_count":0,"forks_count":6,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-02-07T23:54:52.445Z","etag":null,"topics":["ai","claude","gateway","gemini","go","litellm","llm","openai"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/voocel.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-06-15T09:09:23.000Z","updated_at":"2026-02-07T15:10:21.000Z","dependencies_parsed_at":"2025-08-29T16:14:01.371Z","dependency_job_id":"231a707d-066d-422e-8763-770f7fbc4283","html_url":"https://github.com/voocel/litellm","commit_stats":null,"previous_names":["voocel/litellm"],"tags_count":17,"template":false,"template_full_name":null,"purl":"pkg:github/voocel/litellm","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/voocel%2Flitellm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/voocel%2Flitellm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/voocel%2Flitellm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/voocel%2Flitellm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/voocel","download_url":"https://codeload.github.com/voocel/litellm/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/voocel%2Flitellm/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29407845,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-13T06:24:03.484Z","status":"ssl_error","status_checked_at":"2026-02-13T06:23:12.830Z","response_time":78,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","claude","gateway","gemini","go","litellm","llm","openai"],"created_at":"2025-11-28T13:01:38.571Z","updated_at":"2026-04-01T23:43:54.423Z","avatar_url":"https://github.com/voocel.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# LiteLLM (Go) — Multi‑Provider LLM Client\n\n[中文](README_CN.md) | English\n\nLiteLLM is a small, typed Go client that lets you call multiple LLM providers through one API.\n\n## Get Started\n\n### Install\n\n```bash\ngo get github.com/voocel/litellm\n```\n\n### 1) Prepare an API key\n\n```bash\nexport OPENAI_API_KEY=\"your-key\"\n```\n\n### 2) Quick examples (minimal runnable)\n\n#### Text (chat)\n\n```go\npackage main\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"log\"\n\t\"os\"\n\n\t\"github.com/voocel/litellm\"\n)\n\nfunc main() {\n\tclient, err := litellm.NewWithProvider(\"openai\", litellm.ProviderConfig{\n\t\tAPIKey: os.Getenv(\"OPENAI_API_KEY\"),\n\t})\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\n\tresp, err := client.Chat(context.Background(), \u0026litellm.Request{\n\t\tModel:    \"gpt-4o-mini\",\n\t\tMessages: []litellm.Message{litellm.UserMessage(\"Explain AI in one sentence.\")},\n\t})\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tfmt.Println(resp.Content)\n}\n```\n\n#### Tool calling\n\n```go\npackage main\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"log\"\n\t\"os\"\n\n\t\"github.com/voocel/litellm\"\n)\n\nfunc main() {\n\tclient, err := litellm.NewWithProvider(\"openai\", litellm.ProviderConfig{\n\t\tAPIKey: os.Getenv(\"OPENAI_API_KEY\"),\n\t})\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\n\ttools := []litellm.Tool{\n\t\tlitellm.NewTool(\"get_weather\", \"Get weather for a city.\", map[string]any{\n\t\t\t\"type\": \"object\",\n\t\t\t\"properties\": map[string]any{\n\t\t\t\t\"city\": map[string]any{\"type\": \"string\"},\n\t\t\t},\n\t\t\t\"required\": []string{\"city\"},\n\t\t}),\n\t}\n\n\tresp, err := client.Chat(context.Background(), \u0026litellm.Request{\n\t\tModel:      \"gpt-4o-mini\",\n\t\tMessages:   []litellm.Message{litellm.UserMessage(\"Weather in Tokyo?\")},\n\t\tTools:      tools,\n\t\tToolChoice: \"auto\",\n\t})\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tfmt.Println(resp.Content)\n}\n```\n\n#### Streaming (collect)\n\n```go\npackage main\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"log\"\n\t\"os\"\n\n\t\"github.com/voocel/litellm\"\n)\n\nfunc main() {\n\tclient, err := litellm.NewWithProvider(\"openai\", litellm.ProviderConfig{\n\t\tAPIKey: os.Getenv(\"OPENAI_API_KEY\"),\n\t})\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\n\tstream, err := client.Stream(context.Background(), \u0026litellm.Request{\n\t\tModel:    \"gpt-4o-mini\",\n\t\tMessages: []litellm.Message{litellm.UserMessage(\"Tell me a joke.\")},\n\t})\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tdefer stream.Close()\n\nresp, err := litellm.CollectStream(stream)\nif err != nil {\n\tlog.Fatal(err)\n}\nfmt.Println(resp.Content)\n}\n```\n\nIf you need real-time streaming and a final aggregated response:\n\n```go\nresp, err := litellm.CollectStreamWithHandler(stream, func(chunk *litellm.StreamChunk) {\n\tif chunk.Type == litellm.ChunkTypeContent \u0026\u0026 chunk.Content != \"\" {\n\t\tfmt.Print(chunk.Content)\n\t}\n\tif chunk.ReasoningDone {\n\t\tfmt.Print(\"\\n[reasoning done]\")\n\t}\n})\nif err != nil {\n\tlog.Fatal(err)\n}\nfmt.Println(\"\\n---\")\nfmt.Println(resp.Content)\n```\n\n\u003e Notes\n\u003e - The `providers` package is an internal implementation detail. End users should only import `github.com/voocel/litellm`.\n\u003e - LiteLLM does not auto-discover providers or auto-route models. You must configure providers explicitly.\n\n## Core API\n\n- `New(provider, opts...)` builds a client with an explicit provider.\n- `NewWithProvider(name, config, opts...)` builds a client from a provider name and config.\n- `Request` is provider‑agnostic: set `Model` and `Messages`, then optional controls like `MaxTokens`, `Temperature`, `TopP`, `Stop`, etc.\n- `Chat(ctx, req)` returns a unified `Response`.\n- `Stream(ctx, req)` returns a `StreamReader` (not goroutine‑safe). Always `defer stream.Close()`.\n- `CollectStream(stream)` collects a stream into a unified `Response`.\n- `CollectStreamWithHandler(stream, onChunk)` collects and also handles each chunk.\n- `CollectStreamWithCallbacks(stream, callbacks)` adds content/reasoning/tool callbacks.\n- `Request.Thinking` controls thinking output when explicitly set; otherwise behavior is provider-specific.\n- `ListModels(ctx)` lists available models for the current provider (only some providers; fields are best‑effort).\n\n### Streaming (minimal)\n\n```go\nstream, err := client.Stream(ctx, \u0026litellm.Request{\n\tModel: \"gpt-4o-mini\",\n\tMessages: []litellm.Message{\n\t\t{Role: \"user\", Content: \"Tell me a joke.\"},\n\t},\n})\nif err != nil {\n\tlog.Fatal(err)\n}\ndefer stream.Close()\n\nresp, err := litellm.CollectStream(stream)\nif err != nil {\n\tlog.Fatal(err)\n}\nfmt.Print(resp.Content)\n```\n\n## Advanced Features (optional)\n\nEach feature below works across providers. Longer runnable examples live in `examples/`.\n\n### Model listing (some providers)\n\n\u003e Notes\n\u003e - Supported today: OpenAI / Anthropic / Gemini / OpenRouter / DeepSeek / Bedrock\n\u003e - Returned fields vary by provider; `ModelInfo` is best‑effort\n\u003e - Gemini model IDs are normalized (the `models/` prefix is removed)\n\u003e - Bedrock control plane can be overridden via `ProviderConfig.Extra[\"control_plane_base_url\"]`\n\n```go\nmodels, err := client.ListModels(ctx)\nif err != nil {\n\tlog.Fatal(err)\n}\nfor _, m := range models {\n\tfmt.Println(m.ID, m.Name)\n}\n```\n\n### Structured outputs\n\n```go\nschema := map[string]any{\n\t\"type\": \"object\",\n\t\"properties\": map[string]any{\n\t\t\"name\": map[string]any{\"type\": \"string\"},\n\t\t\"age\":  map[string]any{\"type\": \"integer\"},\n\t},\n\t\"required\": []string{\"name\", \"age\"},\n}\n\nresp, err := client.Chat(ctx, \u0026litellm.Request{\n\tModel: \"gpt-4o-mini\",\n\tMessages: []litellm.Message{{Role: \"user\", Content: \"Generate a person.\"}},\n\tResponseFormat: litellm.NewResponseFormatJSONSchema(\"person\", \"\", schema, true),\n})\n_ = resp\n```\n\n### Function calling\n\n```go\ntools := []litellm.Tool{\n\t{\n\t\tType: \"function\",\n\t\tFunction: litellm.FunctionDef{\n\t\t\tName: \"get_weather\",\n\t\t\tParameters: map[string]any{\n\t\t\t\t\"type\": \"object\",\n\t\t\t\t\"properties\": map[string]any{\n\t\t\t\t\t\"city\": map[string]any{\"type\": \"string\"},\n\t\t\t\t},\n\t\t\t\t\"required\": []string{\"city\"},\n\t\t\t},\n\t\t},\n\t},\n}\n\nresp, err := client.Chat(ctx, \u0026litellm.Request{\n\tModel: \"gpt-4o-mini\",\n\tMessages: []litellm.Message{{Role: \"user\", Content: \"Weather in Tokyo?\"}},\n\tTools: tools,\n\tToolChoice: \"auto\",\n})\n_ = resp\n```\n\n### Thinking output\n\n```go\nresp, err := client.Chat(ctx, \u0026litellm.Request{\n\tModel:    \"claude-haiku-4-5-20251001\",\n\tMessages: []litellm.Message{litellm.UserMessage(\"Explain the tradeoffs.\")},\n\tThinking: litellm.NewThinkingEnabled(1024),\n})\n_ = resp\n```\n\nTo disable:\n\n```go\nreq := \u0026litellm.Request{\n\tModel:    \"claude-haiku-4-5-20251001\",\n\tMessages: []litellm.Message{litellm.UserMessage(\"Explain the tradeoffs.\")},\n\tThinking: litellm.NewThinkingDisabled(),\n}\n_ = req\n```\n\n### OpenAI Responses API\n\n```go\nresp, err := client.Responses(ctx, \u0026litellm.OpenAIResponsesRequest{\n\tModel: \"o3-mini\",\n\tMessages: []litellm.Message{{Role: \"user\", Content: \"Solve 15*8 step by step.\"}},\n\tReasoningEffort:  \"medium\",\n\tReasoningSummary: \"auto\",\n\tThinking:         litellm.NewThinkingEnabled(0),\n\tMaxOutputTokens:  litellm.IntPtr(800),\n})\n_ = resp\n```\n\n### Retries \u0026 timeouts\n\n```go\nres := litellm.DefaultResilienceConfig()\nres.MaxRetries = 3\nres.InitialDelay = 1 * time.Second\nres.RequestTimeout = 60 * time.Second\n\nclient, _ := litellm.NewWithProvider(\"openai\", litellm.ProviderConfig{\n\tAPIKey:     os.Getenv(\"OPENAI_API_KEY\"),\n\tResilience: res,\n})\n_ = client\n```\n\n### Provider‑specific knobs\n\n`Request.Extra` is validated per provider. Unsupported providers will return an error.\n\nSupported keys:\n- Gemini: `tool_name` (string) for tool response naming\n\n### Cost calculation\n\nCalculate request costs based on token usage. Pricing data is fetched from [BerriAI/litellm](https://github.com/BerriAI/litellm) and loaded automatically on first use.\n\n```go\nresp, err := client.Chat(ctx, req)\nif err != nil {\n\tlog.Fatal(err)\n}\n\n// Calculate cost (pricing data loads automatically)\nif cost, err := litellm.CalculateCostForResponse(resp); err == nil {\n\tfmt.Printf(\"Cost: $%.6f (input: $%.6f, output: $%.6f)\\n\",\n\t\tcost.TotalCost, cost.InputCost, cost.OutputCost)\n}\n\n// Or use the standalone function\ncost, err := litellm.CalculateCost(resp.Model, resp.Usage)\n\n// Set custom pricing for unlisted models\nlitellm.SetModelPricing(\"my-model\", litellm.ModelPricing{\n\tInputCostPerToken:  0.000001,\n\tOutputCostPerToken: 0.000002,\n})\n```\n\n## Custom Providers\n\nImplement `litellm.Provider` and register it:\n\n```go\ntype MyProvider struct {\n\tname   string\n\tconfig litellm.ProviderConfig\n}\n\nfunc (p *MyProvider) Name() string                     { return p.name }\nfunc (p *MyProvider) Validate() error                 { return nil }\n\nfunc (p *MyProvider) Chat(ctx context.Context, req *litellm.Request) (*litellm.Response, error) {\n\treturn \u0026litellm.Response{Content: \"hello\", Model: req.Model, Provider: p.name}, nil\n}\nfunc (p *MyProvider) Stream(ctx context.Context, req *litellm.Request) (litellm.StreamReader, error) {\n\treturn nil, fmt.Errorf(\"streaming not implemented\")\n}\n\nfunc init() {\n\t_ = litellm.RegisterProviderWithDescriptor(litellm.ProviderDescriptor{\n\t\tName:       \"myprovider\",\n\t\tDefaultURL: \"https://api.example.com/v1\",\n\t\tFactory: func(cfg litellm.ProviderConfig) litellm.Provider {\n\t\t\treturn \u0026MyProvider{name: \"myprovider\", config: cfg}\n\t\t},\n\t})\n}\n```\n\nCustom provider contract:\n\n- `Chat` should validate basic request fields and return canonical fields when possible: `Response.Provider`, `Response.Model`, `Response.FinishReason`, and `Response.Usage`.\n- `Stream` should emit text deltas as `ChunkTypeContent`, reasoning deltas via `ReasoningContent`, tool deltas via `ToolCallDelta`, and finish with a chunk where `Done=true`.\n- `ToolCallDelta.Index` must stay stable for the same streamed tool call so the accumulator can rebuild complete tool calls.\n- Prefer returning `LiteLLMError` values, or errors wrapped by `WrapError`, so upper layers can classify auth, rate limit, timeout, model, and validation failures consistently.\n\n## Supported Providers\n\nBuiltin providers: OpenAI, Anthropic, Google Gemini, DeepSeek, Qwen (DashScope), GLM, AWS Bedrock, OpenRouter.\n\nLiteLLM does not rewrite model IDs. Always use official model IDs.\n\n## Configuration\n\nConfigure providers explicitly:\n\n```go\nclient, err := litellm.NewWithProvider(\"openai\", litellm.ProviderConfig{\n\tAPIKey:  os.Getenv(\"OPENAI_API_KEY\"),\n\tBaseURL: os.Getenv(\"OPENAI_BASE_URL\"), // optional\n})\n_ = client\n```\n\n## License\n\nApache License\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvoocel%2Flitellm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvoocel%2Flitellm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvoocel%2Flitellm/lists"}