Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

https://github.com/comfy-addons/comfyui-sdk

SDK for ComfyUI
https://github.com/comfy-addons/comfyui-sdk

comfyui comfyui-manager comfyui-workflow sdk

Last synced: about 1 year ago
JSON representation

SDK for ComfyUI

Awesome Lists containing this project

README 

          
# ✨ ComfyUI SDK ✨



[![NPM Version](https://img.shields.io/npm/v/@saintno/comfyui-sdk?style=flat-square)](https://www.npmjs.com/package/@saintno/comfyui-sdk)

[![License](https://img.shields.io/npm/l/@saintno/comfyui-sdk?style=flat-square)](https://github.com/tctien342/comfyui-sdk/blob/main/LICENSE)

![CI](https://github.com/tctien342/comfyui-sdk/actions/workflows/release.yml/badge.svg)

[![Buy Me a Coffee](https://img.shields.io/badge/Buy%20Me%20a%20Coffee-donate-yellow.svg)](https://www.buymeacoffee.com/tctien342)



A robust and meticulously crafted TypeScript SDK 🚀 for seamless interaction with the [ComfyUI](https://github.com/comfyanonymous/ComfyUI) API. This SDK significantly simplifies the complexities of building, executing, and managing ComfyUI workflows, all while providing real-time updates and supporting multiple instances. 🖼️



## 🧭 Table of Contents



- [🌟 Key Features 🌟](#-key-features-)

- [📦 Installation 📦](#-installation-)

- [🚀 Getting Started 🚀](#-getting-started-)

  - [🎬 Basic Usage](#-basic-usage)

  - [🔄 Managing Multiple Instances with `ComfyPool`](#-managing-multiple-instances-with-comfypool)

  - [🔑 Authentication](#-authentication)

- [📚 API Reference 📚](#-api-reference-)

  - [`ComfyApi`](#comfyapi)

  - [`CallWrapper`](#callwrapper)

  - [`PromptBuilder`](#promptbuilder)

  - [`ComfyPool`](#comfypool)

  - [🗂️ Enums](#-enums)

  - [🗄️ Types](#-types)

  - [🧩 Features](#-features)

- [📂 Examples](#-examples)

- [🤝 Contributing](#-contributing)

- [📜 License](#-license)



## 🌟 Key Features 🌟



- **💎 TypeScript Powered**: Enjoy a fully typed codebase, ensuring enhanced development, maintainability, and type safety. 🛡️

- **🏗️ Workflow Builder**: Construct and manipulate intricate ComfyUI workflows effortlessly using a fluent, intuitive builder pattern. 🧩

- **🤹 Multi-Instance Management**: Handle a pool of ComfyUI instances with ease, employing flexible queueing strategies for optimal resource utilization. 🌐

- **⚡ Real-Time Updates**: Subscribe to WebSocket events for live progress tracking, image previews, and error notifications. 🔔

- **🔑 Authentication Flexibility**: Supports Basic Auth, Bearer Token, and Custom Authentication Headers, catering to diverse security requirements. 🔒

- **🔌 Extension Support**: Seamlessly integrate with ComfyUI Manager and leverage system monitoring through the ComfyUI-Crystools extension. 🛠️

- **🔀 Flexible Node Bypassing**: Strategically bypass specific nodes in your workflows during generation, enabling advanced customization. ⏭️

- **📚 Comprehensive Examples**: Includes practical examples for Text-to-Image (T2I), Image-to-Image (I2I), and complex multi-node workflows. 📝

- **🚨 Robust Error Handling**: Provides detailed error messages to facilitate debugging and graceful handling of API failures. 🐛

- **📝 Automatic Changelog**: Automatically generates a changelog with each release, utilizing `auto-changelog` for transparent version tracking. 🔄



## 📦 Installation 📦



```bash

bun add @saintno/comfyui-sdk

```



or



```bash

npm i @saintno/comfyui-sdk

```



## 🚀 Getting Started 🚀



### 🎬 Basic Usage



Here's a simplified example to quickly get you started:



```typescript

import { ComfyApi, CallWrapper, PromptBuilder, TSamplerName, TSchedulerName, seed } from "@saintno/comfyui-sdk";

import ExampleTxt2ImgWorkflow from "./example-txt2img-workflow.json";



const api = new ComfyApi("http://localhost:8189").init();

const workflow = new PromptBuilder(

  ExampleTxt2ImgWorkflow,

  ["positive", "negative", "checkpoint", "seed", "batch", "step", "cfg", "sampler", "sheduler", "width", "height"],

  ["images"]

)

  .setInputNode("checkpoint", "4.inputs.ckpt_name")

  .setInputNode("seed", "3.inputs.seed")

  .setInputNode("batch", "5.inputs.batch_size")

  .setInputNode("negative", "7.inputs.text")

  .setInputNode("positive", "6.inputs.text")

  .setInputNode("cfg", "3.inputs.cfg")

  .setInputNode("sampler", "3.inputs.sampler_name")

  .setInputNode("sheduler", "3.inputs.scheduler")

  .setInputNode("step", "3.inputs.steps")

  .setInputNode("width", "5.inputs.width")

  .setInputNode("height", "5.inputs.height")

  .setOutputNode("images", "9")

  .input("checkpoint", "SDXL/realvisxlV40_v40LightningBakedvae.safetensors", api.osType)

  .input("seed", seed())

  .input("step", 6)

  .input("cfg", 1)

  .input("sampler", "dpmpp_2m_sde_gpu")

  .input("sheduler", "sgm_uniform")

  .input("width", 1024)

  .input("height", 1024)

  .input("batch", 1)

  .input("positive", "A picture of cute dog on the street");



new CallWrapper(api, workflow)

  .onFinished((data) => console.log(data.images?.images.map((img: any) => api.getPathImage(img))))

  .run();

```



#### 🔍 Breakdown



- Import essential components from the SDK.

- Create and initialize the `ComfyApi` instance.

- Use `PromptBuilder` to define the workflow structure and set input nodes.

- Set specific input values, including the checkpoint path, seed, and prompt.

- Execute the workflow and log the generated image URLs using the `CallWrapper`.



### 🔄 Managing Multiple Instances with `ComfyPool`



```typescript

import {

  ComfyApi,

  CallWrapper,

  ComfyPool,

  EQueueMode,

  PromptBuilder,

  seed,

  TSamplerName,

  TSchedulerName

} from "@saintno/comfyui-sdk";

import ExampleTxt2ImgWorkflow from "./example-txt2img-workflow.json";



const ApiPool = new ComfyPool(

  [new ComfyApi("http://localhost:8188"), new ComfyApi("http://localhost:8189")],

  EQueueMode.PICK_ZERO

)

  .on("init", () => console.log("Pool in initializing"))

  .on("add_job", (ev) => console.log("Job added at index", ev.detail.jobIdx, "weight:", ev.detail.weight))

  .on("added", (ev) => console.log("Client added", ev.detail.clientIdx));



const generateFn = async (api: ComfyApi, clientIdx?: number) => {

  const workflow = new PromptBuilder(

    ExampleTxt2ImgWorkflow,

    ["positive", "negative", "checkpoint", "seed", "batch", "step", "cfg", "sampler", "sheduler", "width", "height"],

    ["images"]

  )

    .setInputNode("checkpoint", "4.inputs.ckpt_name")

    .setInputNode("seed", "3.inputs.seed")

    .setInputNode("batch", "5.inputs.batch_size")

    .setInputNode("negative", "7.inputs.text")

    .setInputNode("positive", "6.inputs.text")

    .setInputNode("step", "3.inputs.steps")

    .setInputNode("width", "5.inputs.width")

    .setInputNode("height", "5.inputs.height")

    .setInputNode("cfg", "3.inputs.cfg")

    .setInputNode("sampler", "3.inputs.sampler_name")

    .setInputNode("scheduler", "3.inputs.scheduler")

    .setOutputNode("images", "9")

    .input("checkpoint", "SDXL/realvisxlV40_v40LightningBakedvae.safetensors", api.osType)

    .input("seed", seed())

    .input("step", 6)

    .input("width", 512)

    .input("height", 512)

    .input("batch", 2)

    .input("cfg", 1)

    .input("sampler", "dpmpp_2m_sde_gpu")

    .input("scheduler", "sgm_uniform")

    .input("positive", "A close up picture of cute Cat")

    .input("negative", "text, blurry, bad picture, nsfw");



  return new Promise((resolve) => {

    new CallWrapper(api, workflow)

      .onFinished((data) => {

        const url = data.images?.images.map((img: any) => api.getPathImage(img));

        resolve(url as string[]);

      })

      .run();

  });

};



const jobA = ApiPool.batch(Array(5).fill(generateFn), 10).then((res) => {

  console.log("Batch A done");

  return res.flat();

});



const jobB = ApiPool.batch(Array(5).fill(generateFn), 0).then((res) => {

  console.log("Batch B done");

  return res.flat();

});



console.log(await Promise.all([jobA, jobB]).then((res) => res.flat()));

```



#### 🔍 Breakdown



- Create a `ComfyPool` with multiple `ComfyApi` instances.

- Set up event listeners for pool initialization, job additions, and client connections.

- Define an async function (`generateFn`) that creates a workflow, sets its inputs, and executes it with a `CallWrapper`.

- Use `ApiPool.batch` to run multiple jobs and wait for all batches to complete.



### 🔑 Authentication



```typescript

import { ComfyApi, BasicCredentials, BearerTokenCredentials, CustomCredentials } from "@saintno/comfyui-sdk";



// Basic Authentication

const basicAuth = new ComfyApi("http://localhost:8189", "node-id", {

  credentials: { type: "basic", username: "username", password: "password" } as BasicCredentials

}).init();



// Bearer Token Authentication

const bearerAuth = new ComfyApi("http://localhost:8189", "node-id", {

  credentials: { type: "bearer_token", token: "your_bearer_token" } as BearerTokenCredentials

}).init();



// Custom Header Authentication

const customAuth = new ComfyApi("http://localhost:8189", "node-id", {

  credentials: { type: "custom", headers: { "X-Custom-Header": "your_custom_header" } } as CustomCredentials

}).init();

```



#### 🔍 Breakdown



- Import the necessary types from the SDK.

- Create `ComfyApi` instances using the corresponding credential types: `BasicCredentials`, `BearerTokenCredentials`, and `CustomCredentials`..



## 📚 API Reference 📚



### `ComfyApi`



#### 🏗️ Constructor



```typescript

constructor(host: string, clientId: string, opts?: { forceWs?: boolean, wsTimeout?: number, credentials?: BasicCredentials | BearerTokenCredentials | CustomCredentials; })

```



- `host`: The base URL of your ComfyUI server.

- `clientId`: A unique ID for WebSocket communication (optional). Defaults to a generated ID.

- `opts`: Optional settings:

  - `forceWs`: Boolean to force WebSocket usage.

  - `wsTimeout`: Timeout for WebSocket connections (milliseconds).

  - `credentials`: Optional authentication credentials.



#### ⚙️ Methods



- `init(maxTries?: number, delayTime?: number)`: Initializes the client and establishes connection.

- `on(type: K, callback: (event: TComfyAPIEventMap[K]) => void, options?: AddEventListenerOptions | boolean)`: Attach an event listener.

- `off(type: K, callback: (event: TComfyAPIEventMap[K]) => void, options?: EventListenerOptions | boolean)`: Detach an event listener.

- `removeAllListeners()`: Detach all event listeners.

- `fetchApi(route: string, options?: FetchOptions)`: Fetch data from the API endpoint.

- `pollStatus(timeout?: number)`: Polls the ComfyUI server status.

- `queuePrompt(number: number | null, workflow: object)`: Queues a prompt for processing.

- `appendPrompt(workflow: object)`: Adds a prompt to the workflow queue.

- `getQueue()`: Retrieves the current state of the queue.

- `getHistories(maxItems?: number)`: Retrieves the prompt execution history.

- `getHistory(promptId: string)`: Retrieves a specific history entry by ID.

- `getSystemStats()`: Retrieves system and device statistics.

- `getExtensions()`: Retrieves a list of installed extensions.

- `getEmbeddings()`: Retrieves a list of available embeddings.

- `getCheckpoints()`: Retrieves a list of available checkpoints.

- `getLoras()`: Retrieves a list of available Loras.

- `getSamplerInfo()`: Retrieves sampler and scheduler information.

- `getNodeDefs(nodeName?: string)`: Retrieves node object definitions.

- `getUserConfig()`: Get user configuration data.

- `createUser(username: string)`: Create new user.

- `getSettings()`: Get all setting values for the current user.

- `getSetting(id: string)`: Get a specific setting for the current user.

- `storeSettings(settings: Record)`: Store setting for the current user.

- `storeSetting(id: string, value: unknown)`: Store a specific setting for the current user.

- `uploadImage(file: Buffer | Blob, fileName: string, config?: { override?: boolean; subfolder?: string })`: Uploads an image file.

- `uploadMask(file: Buffer | Blob, originalRef: ImageInfo)`: Uploads a mask file.

- `freeMemory(unloadModels: boolean, freeMemory: boolean)`: Frees memory by unloading models.

- `getPathImage(imageInfo: ImageInfo)`: Returns the path to an image.

- `getImage(imageInfo: ImageInfo)`: Returns the blob data of image.

- `getUserData(file: string)`: Get a user data file.

- `storeUserData(file: string, data: unknown, options?: RequestInit & { overwrite?: boolean, stringify?: boolean, throwOnError?: boolean })`: Store a user data file.

- `deleteUserData(file: string)`: Delete a user data file.

- `moveUserData(source: string, dest: string, options?: RequestInit & { overwrite?: boolean })`: Move a user data file.

- `listUserData(dir: string, recurse?: boolean, split?: boolean)`: List a user data file.

- `interrupt()`: Interrupts the execution of the running prompt.

- `reconnectWs(opened?: boolean)`: Reconnects to the WebSocket server.



### `CallWrapper`



#### 🏗️ Constructor



```typescript

constructor(client: ComfyApi, workflow: PromptBuilder)

```



- `client`: An instance of the `ComfyApi` client.

- `workflow`: An instance of `PromptBuilder` defining the workflow.



#### ⚙️ Methods



- `onPreview(fn: (ev: Blob, promptId?: string) => void)`: Set callback for preview events.

- `onPending(fn: (promptId?: string) => void)`: Set callback when job is queued.

- `onStart(fn: (promptId?: string) => void)`: Set callback when the job is started.

- `onOutput(fn: (key: keyof PromptBuilder["mapOutputKeys"], data: any, promptId?: string) => void)`: Sets a callback for when an output node is executed.

- `onFinished(fn: (data: Record["mapOutputKeys"], any>, promptId?: string) => void)`: Set callback when the job is finished.

- `onFailed(fn: (err: Error, promptId?: string) => void)`: Set callback when the job failed.

- `onProgress(fn: (info: NodeProgress, promptId?: string) => void)`: Set callback for progress updates.

- `run()`: Executes the workflow.



### `PromptBuilder`



#### 🏗️ Constructor



```typescript

constructor(prompt: T, inputKeys: I[], outputKeys: O[])

```



- `prompt`: The initial workflow data object.

- `inputKeys`: An array of input node keys.

- `outputKeys`: An array of output node keys.



#### ⚙️ Methods



- `clone()`: Creates a new `PromptBuilder` instance with the same configuration.

- `bypass(node: keyof T | (keyof T)[]): PromptBuilder`: Marks node(s) to be bypassed at generation.

- `reinstate(node: keyof T | (keyof T)[]): PromptBuilder`: Unmarks node(s) from bypass at generation.

- `setInputNode(input: I, key: DeepKeys | Array>)`: Sets input node path for a key.

- `setRawInputNode(input: I, key: string | string[])`: Sets raw input node path for a key.

- `appendInputNode(input: I, key: DeepKeys | Array>)`: Appends a node to the input node path.

- `appendRawInputNode(input: I, key: string | string[])`: Appends a node to the raw input node path.

- `setOutputNode(output: O, key: DeepKeys)`: Sets output node path for a key.

- `setRawOutputNode(output: O, key: string)`: Sets raw output node path for a key.

- `input(key: I, value: V, encodeOs?: OSType)`: Sets an input value.

- `inputRaw(key: string, value: V, encodeOs?: OSType)`: Sets a raw input value with dynamic key.

- `get workflow`: Retrieves the workflow object.

- `get caller`: Retrieves current `PromptBuilder` object.



### `ComfyPool`



#### 🏗️ Constructor



```typescript

constructor(clients: ComfyApi[], mode: EQueueMode = EQueueMode.PICK_ZERO)

```



- `clients`: Array of `ComfyApi` instances.

- `mode`: The queue mode using `EQueueMode` enum values.



#### ⚙️ Methods



- `on(type: K, callback: (event: TComfyPoolEventMap[K]) => void, options?: AddEventListenerOptions | boolean)`: Attach an event listener.

- `off(type: K, callback: (event: TComfyPoolEventMap[K]) => void, options?: EventListenerOptions | boolean)`: Detach an event listener.

- `addClient(client: ComfyApi)`: Adds a new client to the pool.

- `removeClient(client: ComfyApi)`: Removes a client from the pool.

- `removeClientByIndex(index: number)`: Removes a client by index.

- `changeMode(mode: EQueueMode)`: Changes the queue mode.

- `pick(idx?: number)`: Picks a client by index.

- `pickById(id: string)`: Picks a client by ID.

- `run(job: (client: ComfyApi, clientIdx?: number) => Promise, weight?: number, clientFilter?: { includeIds?: string[]; excludeIds?: string[] })`: Run a job with priority on an available client.

- `batch(jobs: Array<(client: ComfyApi, clientIdx?: number) => Promise>, weight?: number, clientFilter?: { includeIds?: string[]; excludeIds?: string[] })`: Run multiple jobs concurrently.



### 🗂️ Enums



- `EQueueMode`:

  - `PICK_ZERO`: Selects the client with zero remaining queue.

  - `PICK_LOWEST`: Selects the client with the lowest remaining queue.

  - `PICK_ROUTINE`: Selects clients in a round-robin manner.



### 🗄️ Types



- `OSType`:

  - `POSIX`: For Unix-like systems.

  - `NT`: For Windows systems.

  - `JAVA`: For Java virtual machine.

- `TSamplerName`: A union type of all available sampler names.

- `TSchedulerName`: A union type of all available scheduler names.



### 🧩 Features



- `ManagerFeature`: Provides methods to manage ComfyUI Manager Extension.



  ```typescript

  const api = new ComfyApi("http://localhost:8189").init();

  await api.waitForReady();



  if (api.ext.manager.isSupported) {

    await api.ext.manager.getExtensionList().then(console.log);

    // Check api.ext.manager for more methods

  }

  ```



- `MonitoringFeature`: Provides methods to monitor system resources using ComfyUI-Crystools Extension.



  ```typescript

  const api = new ComfyApi("http://localhost:8189").init();

  await api.waitForReady();



  if (api.ext.monitor.isSupported) {

    // For subscribing to system monitor events

    api.ext.monitor.on("system_monitor", (ev) => {

      console.log(ev.detail);

    });



    // For getting current monitor data

    console.log(api.ext.monitor.monitorData);

  }

  ```



> Note: Features require respective extensions ([ComfyUI-Manager](https://github.com/ltdrdata/ComfyUI-Manager) and [ComfyUI-Crystools](https://github.com/crystian/ComfyUI-Crystools)) to be installed.



## 📂 Examples



The `examples` directory contains practical demonstrations of SDK usage:



- `example-i2i.ts`: Demonstrates image-to-image generation.

- `example-pool.ts`: Demonstrates how to manage multiple ComfyUI instances using `ComfyPool`.

- `example-pool-basic-auth.ts`: Demonstrates how to use `ComfyPool` with HTTP Basic Authentication.

- `example-t2i.ts`: Demonstrates text-to-image generation.

- `example-t2i-upscaled.ts`: Demonstrates text-to-image generation with upscaling.

- `example-img2img-workflow.json`: Example workflow for image-to-image.

- `example-txt2img-workflow.json`: Example workflow for text-to-image.

- `example-txt2img-upscaled-workflow.json`: Example workflow for text-to-image with upscaling.



## 🤝 Contributing



Contributions are always welcome! Feel free to submit pull requests or create issues for bug reports and feature enhancements. 🙏



## 📜 License



This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for more details. 📄