https://github.com/comfy-deploy/comfyui-llm-toolkit
https://github.com/comfy-deploy/comfyui-llm-toolkit
Last synced: about 1 month ago
JSON representation
- Host: GitHub
- URL: https://github.com/comfy-deploy/comfyui-llm-toolkit
- Owner: comfy-deploy
- Created: 2025-04-17T10:15:51.000Z (about 2 months ago)
- Default Branch: main
- Last Pushed: 2025-04-28T09:31:44.000Z (about 1 month ago)
- Last Synced: 2025-04-28T10:44:58.538Z (about 1 month ago)
- Language: Python
- Size: 151 KB
- Stars: 0
- Watchers: 1
- Forks: 2
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
- awesome-comfyui - **ComfyUI LLM Toolkit**
README
# ComfyUI LLM Toolkit
A custom node collection for integrating various LLM (Large Language Model) providers with ComfyUI.
## Features
## Features
- True Context-to-Context node connections one-input / one-output design
- Generators run even if nothing is conected and they are the only node in the workflow
- Streaming Output directly on the node UI
- Runs openai latest image model GPT-image-1 and we include various templates
- Text generation using various LLM providers (OpenAI and local models, etc.)
- Provider selection and configuration with dynamic model fetching
- API key management
- Seamless integration with ComfyUI workflows
## Installation
@@ -33,6 +43,7 @@ The following Python packages are required:
- pyyaml
- python-dotenv
- requests
- openai
## Installation
1. Clone this repository into your ComfyUI custom_nodes directory:
```bash
cd /path/to/ComfyUI/custom_nodes
git clone https://github.com/yourusername/comfyui-llm-toolkit.git
```
2. Install the required dependencies:
```bash
cd comfyui-llm-toolkit
pip install -r requirements.txt
```
3. Restart ComfyUI
## Dependencies
The following Python packages are required:
- aiohttp
- pyyaml
- python-dotenv
- requests
## Configuration
1. API keys for various providers can be stored in a `.env` file in the root directory:
```
OPENAI_API_KEY=your_openai_key
# Add other provider keys as needed
```
2. Alternatively, you can provide API keys directly in the node interface.
## Usage
### LLM Provider Selector Node
Use this node to select the LLM provider and model. It outputs provider configuration within the wildcard "context" output.
- When you change the provider, the model dropdown updates dynamically with available models
- IP and Port fields appear/disappear based on whether the provider needs them
- The output is a single "context" type that contains the provider configuration
### Generate Text Node
Basic text generation with the selected provider and model.
- Connect it to the LLM Provider Selector by connecting their "context" ports
- The node automatically detects if provider config is present in the input
- You can override the model with the dropdown
- The "context" output contains both the original input data and the LLM response
## Unified ONE-INPUT / ONE-OUTPUT Architecture
The LLM Toolkit uses a single "context" input/output approach for maximum flexibility:
1. **Single Connection Point**: Each node has just one wildcard input and one wildcard output named "context"
2. **Smart Data Handling**:
- Provider config is embedded within the "context" data structure
- Each node intelligently extracts the data it needs from the "context" input
- Nodes preserve all input data and add their own data to the "context" output
3. **Cascading Data Flow**: As data flows through nodes, it accumulates in the "context" structure
For example, with nodes A → B → C:
- Node A creates an "context" with provider config
- Node B receives A's "context", extracts the provider config, and adds LLM response to the "context"
- Node C receives B's "context" which now contains both provider config and LLM response
This allows you to:
- Chain multiple LLM operations with a single connection
- Preserve and accumulate data throughout the workflow
- Easily integrate with other ComfyUI nodes
## Supported Providers
- OpenAI (Default: gpt-4o-mini)
- Ollama (local)
- And more... (Comming Soon)
## Troubleshooting
If you encounter model list update issues:
1. Make sure ComfyUI is running with the correct server configuration
2. Check that JavaScript is enabled in your browser
3. Verify that your API keys are correctly set in the .env file or provided in the node
If you encounter import errors:
1. Make sure you've installed all dependencies: `pip install -r requirements.txt`
2. Verify that you've placed the custom node in the correct directory
3. Restart ComfyUI after installation
## Testing Your Installation
Run the included test script to verify your setup:
```bash
cd /path/to/ComfyUI/custom_nodes/comfyui-llm-toolkit
python test_js_integration.py
```