{"id":24199247,"url":"https://github.com/simp-lee/llmconnector","last_synced_at":"2025-07-26T13:39:14.132Z","repository":{"id":247057426,"uuid":"822153411","full_name":"simp-lee/llmconnector","owner":"simp-lee","description":"llmconnector is a Go package designed to simplify communication with diverse LLM APIs, featuring robust configurations for retries, rate limits, and proxy support.","archived":false,"fork":false,"pushed_at":"2024-12-11T04:43:17.000Z","size":16,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-03T09:26:22.431Z","etag":null,"topics":["go","golang","llm","llm-api"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/simp-lee.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-06-30T12:29:52.000Z","updated_at":"2024-07-21T14:11:28.000Z","dependencies_parsed_at":"2024-07-21T15:33:09.357Z","dependency_job_id":null,"html_url":"https://github.com/simp-lee/llmconnector","commit_stats":null,"previous_names":["simp-lee/llmconnector","simp-lee/llm"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/simp-lee/llmconnector","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/simp-lee%2Fllmconnector","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/simp-lee%2Fllmconnector/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/simp-lee%2Fllmconnector/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/simp-lee%2Fllmconnector/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/simp-lee","download_url":"https://codeload.github.com/simp-lee/llmconnector/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/simp-lee%2Fllmconnector/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":267175742,"owners_count":24047921,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-26T02:00:08.937Z","response_time":62,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["go","golang","llm","llm-api"],"created_at":"2025-01-13T20:35:09.763Z","updated_at":"2025-07-26T13:39:14.040Z","avatar_url":"https://github.com/simp-lee.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# LLMConnector\n\n`llmconnector` is a Go package designed to facilitate communication with various Large Language Model (LLM) APIs, such as OpenAI and Alibaba. It provides a unified interface for chat and embedding functionalities, making it easier to integrate and switch between different LLM providers.\n\n## Features\n\n- **Chat Operations**: Interact with language models to generate text based on input messages.\n- **Embedding Operations**: Generate embeddings for text inputs, useful for various NLP tasks.\n- **Strategy Pattern**: Supports multiple strategies for chat and embedding operations, allowing easy integration of new providers.\n- **Customizable Options**: Provides a variety of options to customize chat and embedding requests, such as model type, temperature, max tokens, and more.\n- **Advanced HTTP Client**: Uses `github.com/simp-lee/gohttpclient` for robust HTTP operations with features like retries, rate limiting, and proxy support.\n- **Extensible**: Easy to add support for new language model providers.\n- **Concurrent-Safe**: Designed to be used in concurrent environments.\n- **Common Configuration**: Shared configuration options across different providers for consistency and ease of use.\n\n## Installation\n\nTo install LLMConnector, use the following command:\n\n```shell\ngo get github.com/simp-lee/llmconnector\n```\n\n## Usage\n\n### Setting Up the Model Context\n\n```go\npackage main\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"github.com/simp-lee/llmconnector\"\n)\n\nfunc main() {\n\tctx := context.Background()\n\n\t// Create a new model context\n\tmodelContext := llmconnector.NewModelContext()\n\n\t// Set up OpenAI strategy for chat\n\topenAIConfig := llmconnector.Config{\n\t\tAPIKey: \"your-openai-api-key\",\n\t}\n\topenAIStrategy, err := llmconnector.NewOpenAIStrategy(openAIConfig)\n\tif err != nil {\n\t\tfmt.Println(\"Error setting up OpenAI strategy:\", err)\n\t\treturn\n\t}\n\tmodelContext.SetChatStrategy(openAIStrategy)\n\t\n\t// Set up Alibaba strategy for embedding\n\talibabaConfig := llmconnector.Config{\n\t\tAPIKey: \"your-alibaba-api-key\",\n\t}\n\talibabaStrategy, err := llmconnector.NewAlibabaStrategy(alibabaConfig)\n\tif err != nil {\n\t\tfmt.Println(\"Error setting up Alibaba strategy:\", err)\n\t\treturn\n\t}\n\tmodelContext.SetEmbedStrategy(alibabaStrategy)\n\n\t// Use the strategies as needed... \n}\n```\n\n### Performing Chat Operations\n\n```go\nchatMessages := []llmconnector.ChatMessage{\n\t{Role: \"user\", Content: \"Hello, how are you?\"},\n}\nchatResponse, err := modelContext.Chat(ctx, chatMessages,\n\tllmconnector.WithChatModel(\"gpt-3.5-turbo\"),\n\tllmconnector.WithTemperature(0.7),\n)\nif err != nil {\n\tfmt.Println(\"Error performing chat operation:\", err)\n\treturn\n}\nfmt.Println(\"Chat Response:\", chatResponse.GetContent())\n```\n\n### Performing Embedding Operations\n\n```go\ntexts := []string{\"Hello world\", \"How are you?\"}\nembedResponse, err := modelContext.Embed(ctx, texts,\n\tllmconnector.WithEmbedModel(\"text-embedding-ada-002\"),\n)\nif err != nil {\n\tfmt.Println(\"Error performing embedding operation:\", err)\n\treturn\n}\nfmt.Println(\"Embeddings:\", embedResponse.GetEmbeddings())\n```\n\n### Customizing Options\n\nYou can customize the chat and embedding requests using various options:\n\n```go\nchatResponse, err := modelContext.Chat(ctx, chatMessages,\n\tllmconnector.WithChatModel(\"gpt-3.5-turbo\"),\n\tllmconnector.WithTemperature(0.7),\n\tllmconnector.WithMaxTokens(800),\n\tllmconnector.WithTopP(0.9),\n\tllmconnector.WithStop([]string{\"###\"}),\n)\n```\nFor embedding operations:\n\n```go\nembedResponse, err := modelContext.Embed(ctx, texts,\n\tllmconnector.WithEmbedModel(\"text-embedding-ada-002\"),\n\tllmconnector.WithEmbeddingType(\"document\"),\n)\n```\n\n### Advanced Configuration\n\n`llmconnector` now supports advanced HTTP client configuration through the `github.com/simp-lee/gohttpclient` package. You can configure:\n\n- Timeout\n- Retries\n- Rate Limiting\n- Proxy\n- Connection Pooling\n\nThese configurations are now part of the CommonConfig structure, which is shared across different providers:\n\n```go\ncommonConfig := llmconnector.CommonConfig{\n\tTimeout:                30 * time.Second,\n\tRetries:                3,\n\tMaxNumRequestPerSecond: 10,\n\tMaxNumRequestPerLimit:  10,\n\tProxyURL:               \"http://proxy.example.com:8080\",\n\tMaxIdleConns:           100,\n\tMaxConnsPerHost:        10,\n\tIdleConnTimeout:        90 * time.Second,\n}\n```\n\nThese configurations help in managing API rate limits, improving reliability with retries, and optimizing performance with connection pooling.\n\n### Best Practices\n\n- **API Key Security:** Never hardcode API keys in your source code. Use environment variables or secure configuration management.\n- **Context Usage:** Always pass a context to Chat and Embed operations for proper cancellation and timeout handling.\n- **Rate Limiting:** Be aware of the rate limits of the LLM providers you're using. Use the `MaxNumRequestPerSecond` \n  and `MaxNumRequestPerLimit` options to prevent rate limiting.\n- **Error Handling:** Always check for errors returned by the `llmconnector` functions and handle them appropriately.\n- **Proxy Usage:** If you're operating in an environment that requires a proxy, make sure to configure it correctly in the `CommonConfig`.\n\n### Supported Language Models\n\n- OpenAI: Supports chat and embedding operations using OpenAI's GPT models.\n- Alibaba Cloud: Supports chat and embedding operations using Alibaba Cloud's NLP services.\n\nAdding more models is straightforward. The `llmconnector` package provides simple interfaces (`ChatStrategy` and `EmbedStrategy`) for implementing new strategies.\n\n## Contributing\n\nContributions are welcome! Please open an issue or submit a pull request for any improvements or new features.\n\n## License\n\nThis project is licensed under the MIT License.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsimp-lee%2Fllmconnector","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsimp-lee%2Fllmconnector","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsimp-lee%2Fllmconnector/lists"}