{"id":14272291,"url":"https://github.com/teilomillet/gollm","last_synced_at":"2025-05-13T15:39:37.100Z","repository":{"id":249277523,"uuid":"830613483","full_name":"teilomillet/gollm","owner":"teilomillet","description":"Unified Go interface for Language Model (LLM) providers. Simplifies LLM integration with flexible prompt management and common task functions.","archived":false,"fork":false,"pushed_at":"2025-05-08T14:21:48.000Z","size":24995,"stargazers_count":465,"open_issues_count":8,"forks_count":44,"subscribers_count":15,"default_branch":"main","last_synced_at":"2025-05-08T15:25:51.976Z","etag":null,"topics":["ai","anthropic","dspy","genai","generative-ai","generative-ai-tools","go","golang","groq","language-model","llm","openai","prompt-engineering","prompt-optimization","structured-output"],"latest_commit_sha":null,"homepage":"https://docs.gollm.co","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/teilomillet.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-07-18T15:59:02.000Z","updated_at":"2025-05-08T14:12:48.000Z","dependencies_parsed_at":"2024-09-14T20:12:47.769Z","dependency_job_id":"f1cf2e4c-b219-4e79-974f-d9209799ee8a","html_url":"https://github.com/teilomillet/gollm","commit_stats":null,"previous_names":["teilomillet/goal","teilomillet/gollm"],"tags_count":28,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/teilomillet%2Fgollm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/teilomillet%2Fgollm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/teilomillet%2Fgollm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/teilomillet%2Fgollm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/teilomillet","download_url":"https://codeload.github.com/teilomillet/gollm/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253971362,"owners_count":21992692,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","anthropic","dspy","genai","generative-ai","generative-ai-tools","go","golang","groq","language-model","llm","openai","prompt-engineering","prompt-optimization","structured-output"],"created_at":"2024-08-22T20:01:13.544Z","updated_at":"2025-05-13T15:39:37.051Z","avatar_url":"https://github.com/teilomillet.png","language":"Go","funding_links":[],"categories":["Large Language Model","Go","Framework","Prompts"],"sub_categories":["DevTools"],"readme":"# gollm - Go Large Language Model\n\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"img/gopherrobot4s.jpg\" alt=\"Gophers building a robot by Renee French\"\u003e\n\u003c/div\u003e\n\n`gollm` is a Go package designed to help you build your own AI golems. Just as the mystical golem of legend was brought to life with sacred words, `gollm` empowers you to breathe life into your AI creations using the power of Large Language Models (LLMs). This package simplifies and streamlines interactions with various LLM providers, offering a unified, flexible, and powerful interface for AI engineers and developers to craft their own digital servants.\n\n\u003cdiv align=\"center\"\u003e\n  \u003ca href=\"https://www.youtube.com/watch?v=dDN2tUcgqII\"\u003e\n    \u003cimg src=\"https://img.youtube.com/vi/dDN2tUcgqII/0.jpg\" alt=\"Ed Zynda's Video\"\u003e\n  \u003c/a\u003e\n\u003c/div\u003e\n\n[Documentation](https://docs.gollm.co)\n\n## Table of Contents\n\n- [Key Features](#key-features)\n- [Real-World Applications](#real-world-applications)\n- [Installation](#installation)\n- [Quick Start](#quick-start)\n  - [Basic Usage](#basic-usage)\n- [Quick Reference](#quick-reference)\n  - [LLM Creation and Configuration](#llm-creation-and-configuration)\n  - [Prompt Creation](#prompt-creation)\n  - [Generate Response](#generate-response)\n  - [Chain of Thought](#chain-of-thought)\n  - [Prompt Optimization](#prompt-optimization)\n  - [Model Comparison](#model-comparison)\n- [Advanced Usage](#advanced-usage)\n  - [Prompt Engineering](#prompt-engineering)\n  - [Pre-built Functions (Chain of Thought)](#pre-built-functions-chain-of-thought)\n  - [Working with Examples](#working-with-examples)\n  - [Prompt Templates](#prompt-templates)\n  - [Structured Output (JSON Output Validation)](#structured-output-json-output-validation)\n  - [Prompt Optimizer](#prompt-optimizer)\n  - [Model Comparison](#model-comparison-1)\n  - [Memory Retention](#memory-retention)\n- [Best Practices](#best-practices)\n- [Examples and Tutorials](#examples-and-tutorials)\n- [Project Status](#project-status)\n- [Philosophy](#philosophy)\n- [Contributing](#contributing)\n- [License](#license)\n\n## Key Features\n\n- **Unified API for Multiple LLM Providers:** Interact seamlessly with various providers, including OpenAI, Anthropic, Groq, Ollama, and OpenRouter. Easily switch between models like GPT-4, Claude, and Llama-3.1.\n- **Easy Provider and Model Switching:** Configure preferred providers and models with simple options.\n- **Flexible Configuration Options:** Customize using environment variables, code-based configuration, or configuration files.\n- **Advanced Prompt Engineering:** Craft sophisticated instructions to guide your AI's responses effectively.\n- **Prompt Optimizer:** Automatically refine and improve your prompts for better results, with support for custom metrics and different rating systems.\n- **Memory Retention:** Maintain context across multiple interactions for more coherent conversations.\n- **Structured Output and Validation:** Ensure outputs are consistent and reliable with JSON schema generation and validation.\n- **Provider Comparison Tools:** Test performance across different LLM providers and models for the same task.\n- **High-Level AI Functions:** Use pre-built functions like `ChainOfThought` for complex reasoning tasks.\n- **Robust Error Handling and Retries:** Built-in retry mechanisms to handle API rate limits and transient errors.\n- **Extensible Architecture:** Easily expand support for new LLM providers and features.\n\n## Supported Providers\n\n`gollm` works with a variety of LLM providers:\n\n- **OpenAI**: GPT-4o, GPT-4, GPT-3.5 Turbo\n- **Anthropic**: Claude 3 (Opus, Sonnet, Haiku), Claude 2.1\n- **Groq**: Llama-3, Mixtral models with high-speed inference\n- **Ollama**: Local models support (Llama-3, Mistral, etc.)\n- **Mistral**: Mistral Large, Mistral Medium\n- **OpenRouter**: Access to multiple providers through a single API with:\n  - Model fallback capabilities\n  - Auto-routing between models\n  - Prompt caching\n  - Reasoning tokens\n  - Provider routing preferences\n\n## Real-World Applications\n\n`gollm` can handle a wide range of AI-powered tasks, including:\n\n- **Content Creation Workflows:** Generate research summaries, article ideas, and refined paragraphs.\n- **Complex Reasoning Tasks:** Use the `ChainOfThought` function to analyze complex problems step-by-step.\n- **Structured Data Generation:** Create and validate complex data structures with customizable JSON schemas.\n- **Model Performance Analysis:** Compare different models' performance for specific tasks.\n- **Prompt Optimization:** Automatically improve prompts for various tasks.\n- **Mixture of Agents:** Combine responses from multiple LLM providers.\n\n## Installation\n\n```bash\ngo get github.com/teilomillet/gollm\n```\n\n## Quick Start\n\n### Basic Usage\n\n```go\npackage main\n\nimport (\n    \"context\"\n    \"fmt\"\n    \"log\"\n    \"os\"\n\n    \"github.com/teilomillet/gollm\"\n)\n\nfunc main() {\n    // Load API key from environment variable\n    apiKey := os.Getenv(\"OPENAI_API_KEY\")\n    if apiKey == \"\" {\n        log.Fatalf(\"OPENAI_API_KEY environment variable is not set\")\n    }\n\n    // Create a new LLM instance with custom configuration\n    llm, err := gollm.NewLLM(\n        gollm.SetProvider(\"openai\"),\n        gollm.SetModel(\"gpt-4o-mini\"),\n        gollm.SetAPIKey(apiKey),\n        gollm.SetMaxTokens(200),\n        gollm.SetMaxRetries(3),\n        gollm.SetRetryDelay(time.Second*2),\n        gollm.SetLogLevel(gollm.LogLevelInfo),\n    )\n    if err != nil {\n        log.Fatalf(\"Failed to create LLM: %v\", err)\n    }\n\n    ctx := context.Background()\n\n    // Create a basic prompt\n    prompt := gollm.NewPrompt(\"Explain the concept of 'recursion' in programming.\")\n\n    // Generate a response\n    response, err := llm.Generate(ctx, prompt)\n    if err != nil {\n        log.Fatalf(\"Failed to generate text: %v\", err)\n    }\n    fmt.Printf(\"Response:\\n%s\\n\", response)\n}\n```\n\n## Quick Reference\n\nHere's a quick reference guide for the most commonly used functions and options in the `gollm` package:\n\n### LLM Creation and Configuration\n\n```go\n// Using OpenAI\nllm, err := gollm.NewLLM(\n    gollm.SetProvider(\"openai\"),\n    gollm.SetModel(\"gpt-4\"),\n    gollm.SetAPIKey(\"your-api-key\"),\n    gollm.SetMaxTokens(100),\n    gollm.SetTemperature(0.7),\n    gollm.SetMemory(4096),\n)\n\n// Using OpenRouter (with model fallback)\nllm, err := gollm.NewLLM(\n    gollm.SetProvider(\"openrouter\"),\n    gollm.SetModel(\"anthropic/claude-3-5-sonnet\"),\n    gollm.SetAPIKey(\"your-openrouter-api-key\"),\n    gollm.SetMaxTokens(500),\n)\n// Enable fallback models if primary model is unavailable\nllm.SetOption(\"fallback_models\", []string{\"openai/gpt-4o\", \"mistral/mistral-large\"})\n// Or use OpenRouter's auto-routing capability\n// llm.SetOption(\"auto_route\", true)\n```\n\n### Prompt Creation\n\n```go\nprompt := gollm.NewPrompt(\"Your prompt text here\",\n    gollm.WithContext(\"Additional context\"),\n    gollm.WithDirectives(\"Be concise\", \"Use examples\"),\n    gollm.WithOutput(\"Expected output format\"),\n    gollm.WithMaxLength(300),\n)\n```\n\n### Generate Response\n\n```go\nresponse, err := llm.Generate(ctx, prompt)\n```\n\n### Chain of Thought\n\n```go\nresponse, err := tools.ChainOfThought(ctx, llm, \"Your question here\")\n```\n\n### Prompt Optimization\n\n```go\noptimizer := optimizer.NewPromptOptimizer(llm, initialPrompt, taskDescription,\n    optimizer.WithCustomMetrics(/* custom metrics */),\n    optimizer.WithRatingSystem(\"numerical\"),\n    optimizer.WithThreshold(0.8),\n)\noptimizedPrompt, err := optimizer.OptimizePrompt(ctx)\n```\n\n### Model Comparison\n\n```go\nresults, err := tools.CompareModels(ctx, promptText, validateFunc, configs...)\n```\n\n## Advanced Usage\n\nThe `gollm` package offers a range of advanced features to enhance your AI applications:\n\n### Prompt Engineering\n\nCreate sophisticated prompts to guide the AI's responses:\n\n```go\nprompt := gollm.NewPrompt(\n    \"Explain the concept of recursion in programming\",\n    gollm.WithDirectives(\n        \"Be concise and clear\", \n        \"Include code examples in multiple languages\",\n        \"Provide a practical example.\",\n    ),\n    gollm.WithContext(\"This is for a beginner programmer who is just starting to learn.\"),\n    gollm.WithOutput(\"Structure your response with sections: Definition, Example, Pitfalls, Best Practices.\"),\n    gollm.WithMaxLength(1000),\n)\n```\n\n### Provider-Specific Features\n\n#### OpenRouter Special Features\n\nOpenRouter provides access to multiple LLM providers with additional capabilities:\n\n```go\n// Create OpenRouter client\nllm, err := gollm.NewLLM(\n    gollm.SetProvider(\"openrouter\"),\n    gollm.SetModel(\"anthropic/claude-3-5-sonnet\"),\n    gollm.SetAPIKey(apiKey),\n)\n\n// Enable model fallbacks (tried in order if primary model fails)\nllm.SetOption(\"fallback_models\", []string{\"openai/gpt-4o\", \"mistral/mistral-large\"})\n\n// Use auto-routing (automatically select best model)\nllm.SetOption(\"auto_route\", true)\n\n// Enable prompt caching (improves performance and reduces costs)\nllm.SetOption(\"enable_prompt_caching\", true)\n\n// Enable reasoning tokens (step-by-step thinking)\nllm.SetOption(\"enable_reasoning\", true)\n\n// Specify provider routing preferences\nllm.SetOption(\"provider_preferences\", map[string]interface{}{\n    \"openai\": map[string]interface{}{\n        \"weight\": 1.0,\n    },\n})\n```\n\n### Pre-built Functions (Chain of Thought)\n\nUse the `ChainOfThought` function for step-by-step reasoning:\n\n```go\nquestion := \"What is the result of 15 * 7 + 22?\"\nresponse, err := tools.ChainOfThought(ctx, llm, question)\nif err != nil {\n    log.Fatalf(\"Failed to perform chain of thought: %v\", err)\n}\nfmt.Printf(\"Chain of Thought:\\n%s\\n\", response)\n```\n\n### Working with Examples\n\nLoad examples directly from files:\n\n```go\nexamples, err := utils.ReadExamplesFromFile(\"examples.txt\")\nif err != nil {\n    log.Fatalf(\"Failed to read examples: %v\", err)\n}\n\nprompt := gollm.NewPrompt(\"Generate a similar example:\",\n    gollm.WithExamples(examples...),\n)\n\nresponse, err := llm.Generate(ctx, prompt)\nif err != nil {\n    log.Fatalf(\"Failed to generate example: %v\", err)\n}\nfmt.Printf(\"Generated Example:\\n%s\\n\", response)\n```\n\n### Prompt Templates\n\nCreate reusable prompt templates for consistent prompt generation:\n\n```go\n// Create a new prompt template\ntemplate := gollm.NewPromptTemplate(\n    \"AnalysisTemplate\",\n    \"A template for analyzing topics\",\n    \"Provide a comprehensive analysis of {{.Topic}}. Consider the following aspects:\\n\" +\n    \"1. Historical context\\n\" +\n    \"2. Current relevance\\n\" +\n    \"3. Future implications\",\n    gollm.WithPromptOptions(\n        gollm.WithDirectives(\n            \"Use clear and concise language\",\n            \"Provide specific examples where appropriate\",\n        ),\n        gollm.WithOutput(\"Structure your analysis with clear headings for each aspect.\"),\n    ),\n)\n\n// Use the template to create a prompt\ndata := map[string]interface{}{\n    \"Topic\": \"artificial intelligence in healthcare\",\n}\nprompt, err := template.Execute(data)\nif err != nil {\n    log.Fatalf(\"Failed to execute template: %v\", err)\n}\n\n// Generate a response using the created prompt\nresponse, err := llm.Generate(ctx, prompt)\nif err != nil {\n    log.Fatalf(\"Failed to generate response: %v\", err)\n}\n\nfmt.Printf(\"Analysis:\\n%s\\n\", response)\n```\n\n### Structured Output (JSON Output Validation)\n\nEnsure your LLM outputs are in a valid JSON format:\n\n```go\nprompt := gollm.NewPrompt(\"Analyze the pros and cons of remote work.\",\n    gollm.WithOutput(\"Respond in JSON format with 'topic', 'pros', 'cons', and 'conclusion' fields.\"),\n)\n\nresponse, err := llm.Generate(ctx, prompt, gollm.WithJSONSchemaValidation())\nif err != nil {\n    log.Fatalf(\"Failed to generate valid analysis: %v\", err)\n}\n\nvar result AnalysisResult\nif err := json.Unmarshal([]byte(response), \u0026result); err != nil {\n    log.Fatalf(\"Failed to parse response: %v\", err)\n}\n\nfmt.Printf(\"Analysis: %+v\\n\", result)\n```\n\n### Prompt Optimizer\n\nUse the `PromptOptimizer` to automatically refine and improve your prompts:\n\n```go\ninitialPrompt := gollm.NewPrompt(\"Write a short story about a robot learning to love.\")\ntaskDescription := \"Generate a compelling short story that explores the theme of artificial intelligence developing emotions.\"\n\noptimizerInstance := optimizer.NewPromptOptimizer(\n    llm,\n    initialPrompt,\n    taskDescription,\n    optimizer.WithCustomMetrics(\n        optimizer.Metric{Name: \"Creativity\", Description: \"How original and imaginative the story is\"},\n        optimizer.Metric{Name: \"Emotional Impact\", Description: \"How well the story evokes feelings in the reader\"},\n    ),\n    optimizer.WithRatingSystem(\"numerical\"),\n    optimizer.WithThreshold(0.8),\n    optimizer.WithVerbose(),\n)\n\noptimizedPrompt, err := optimizerInstance.OptimizePrompt(ctx)\nif err != nil {\n    log.Fatalf(\"Optimization failed: %v\", err)\n}\n\nfmt.Printf(\"Optimized Prompt: %s\\n\", optimizedPrompt.Input)\n```\n\n### Model Comparison\n\nCompare responses from different LLM providers or models:\n\n```go\nconfigs := []*gollm.Config{\n    {\n        Provider:  \"openai\",\n        Model:     \"gpt-4o-mini\",\n        APIKey:    os.Getenv(\"OPENAI_API_KEY\"),\n        MaxTokens: 500,\n    },\n    {\n        Provider:  \"anthropic\",\n        Model:     \"claude-3-5-sonnet-20240620\",\n        APIKey:    os.Getenv(\"ANTHROPIC_API_KEY\"),\n        MaxTokens: 500,\n    },\n    {\n        Provider:  \"groq\",\n        Model:     \"llama-3.1-70b-versatile\",\n        APIKey:    os.Getenv(\"GROQ_API_KEY\"),\n        MaxTokens: 500,\n    },\n}\n\npromptText := \"Tell me a joke about programming. Respond in JSON format with 'setup' and 'punchline' fields.\"\n\nvalidateJoke := func(joke map[string]interface{}) error {\n    if joke[\"setup\"] == \"\" || joke[\"punchline\"] == \"\" {\n        return fmt.Errorf(\"joke must have both a setup and a punchline\")\n    }\n    return nil\n}\n\nresults, err := tools.CompareModels(context.Background(), promptText, validateJoke, configs...)\nif err != nil {\n    log.Fatalf(\"Error comparing models: %v\", err)\n}\n\nfmt.Println(tools.AnalyzeComparisonResults(results))\n```\n\n### Memory Retention\n\nEnable memory to maintain context across multiple interactions:\n\n```go\nllm, err := gollm.NewLLM(\n    gollm.SetProvider(\"openai\"),\n    gollm.SetModel(\"gpt-3.5-turbo\"),\n    gollm.SetAPIKey(os.Getenv(\"OPENAI_API_KEY\")),\n    gollm.SetMemory(4096), // Enable memory with a 4096 token limit\n)\nif err != nil {\n    log.Fatalf(\"Failed to create LLM: %v\", err)\n}\n\nctx := context.Background()\n\n// First interaction\nprompt1 := gollm.NewPrompt(\"What's the capital of France?\")\nresponse1, err := llm.Generate(ctx, prompt1)\nif err != nil {\n    log.Fatalf(\"Failed to generate response: %v\", err)\n}\nfmt.Printf(\"Response 1: %s\\n\", response1)\n\n// Second interaction, referencing the first\nprompt2 := gollm.NewPrompt(\"What's the population of that city?\")\nresponse2, err := llm.Generate(ctx, prompt2)\nif err != nil {\n    log.Fatalf(\"Failed to generate response: %v\", err)\n}\nfmt.Printf(\"Response 2: %s\\n\", response2)\n```\n\n## Best Practices\n\n1. **Prompt Engineering**:\n   - Use `NewPrompt()` with options like `WithContext()`, `WithDirectives()`, and `WithOutput()` to create well-structured prompts.\n   - Example:\n     ```go\n     prompt := gollm.NewPrompt(\"Your main prompt here\",\n         gollm.WithContext(\"Provide relevant context\"),\n         gollm.WithDirectives(\"Be concise\", \"Use examples\"),\n         gollm.WithOutput(\"Specify expected output format\"),\n     )\n     ```\n\n2. **Utilize Prompt Templates**:\n   - For consistent prompt generation, create and use `","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fteilomillet%2Fgollm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fteilomillet%2Fgollm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fteilomillet%2Fgollm/lists"}