https://github.com/dmickelson/llmfactoryproject
Simple factory pattern that lets you choose different LLM providers on a dynamic basis
https://github.com/dmickelson/llmfactoryproject
chatgpt factory-pattern gemini llm openai python
Last synced: 2 months ago
JSON representation
Simple factory pattern that lets you choose different LLM providers on a dynamic basis
- Host: GitHub
- URL: https://github.com/dmickelson/llmfactoryproject
- Owner: dmickelson
- Created: 2024-08-25T15:52:33.000Z (almost 2 years ago)
- Default Branch: main
- Last Pushed: 2024-09-05T20:23:20.000Z (almost 2 years ago)
- Last Synced: 2025-01-12T10:09:10.194Z (over 1 year ago)
- Topics: chatgpt, factory-pattern, gemini, llm, openai, python
- Language: Python
- Homepage:
- Size: 11.7 KB
- Stars: 1
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# LLM Factory Project
This project implements a simple factory pattern that allows you to dynamically choose different Language Model (LLM) providers. It uses the Instructor module under the hood for getting structured output.
## Features
- Supports multiple LLM providers: OpenAI, Anthropic, Cohere, and Ollama
- Dynamic selection of LLM provider at runtime
- Structured output using Pydantic models
- Caching of settings for improved performance
- Environment-based configuration using .env files
## Usage
To use the LLM Factory, follow these steps:
1. Set up your environment variables in a `.env` file with the necessary API keys for each provider.
2. Import the `LLMFactory` and `CompletionModel` from the `llmfactory` module.
3. Create an instance of `LLMFactory` with your chosen provider.
4. Use the `create_completion` method to generate responses.
Example:
```python
from llmfactory import llm_factory
messages = [
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "If it takes 2 hours to dry 1 shirt out in the sun, how long will it take to dry 5 shirts?"},
]
llm = llm_factory.LLMFactory("cohere")
completion = llm.create_completion(
response_model=llm_factory.CompletionModel,
messages=messages,
)
print(f"Response: {completion.response}")
print(f"Reasoning: {completion.reasoning}")
```
## Project Structure
- llmfactory/llm_factory.py: Contains the main LLMFactory class and CompletionModel.
- llmfactory/settings.py: Defines the settings for each LLM provider using Pydantic.
- main.py: Provides an example of how to use the LLM Factory.
## Class Diagram
```mermaid
classDiagram
class LLMFactory {
+LLMProviders provider
+LLMSettings settings
+_initialize_client()
+create_completion()
}
class Settings {
+OpenAISettings openai
+AnthropicSettings anthropic
+OllamaSettings ollama
+CohereSettings cohere
}
class OpenAISettings {
+String api_key
+String default_model
+int max_tokens
}
class AnthropicSettings {
+String api_key
+String default_model
+int max_tokens
}
class CohereSettings {
+String api_key
+String default_model
+int max_tokens
}
class OllamaSettings {
+String api_key
+String default_model
+String base_url
}
class CompletionModel {
+String response
+String reasoning
}
LLMFactory --> Settings
LLMFactory --> CompletionModel
Settings --> OpenAISettings
Settings --> AnthropicSettings
Settings --> OllamaSettings
Settings --> CohereSettings
Settings <|-- get_settings
```