{"id":27949393,"url":"https://github.com/comfy-org/nimnodes","last_synced_at":"2025-05-07T15:27:25.756Z","repository":{"id":290448512,"uuid":"901647659","full_name":"Comfy-Org/NIMnodes","owner":"Comfy-Org","description":"Custom node for interacting with NVIDIA NIMs","archived":false,"fork":false,"pushed_at":"2025-04-28T21:28:24.000Z","size":2450,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-28T22:47:53.466Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Comfy-Org.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-12-11T03:51:55.000Z","updated_at":"2025-04-28T21:28:23.000Z","dependencies_parsed_at":"2025-04-28T22:47:56.981Z","dependency_job_id":"a679f121-1757-4c48-97c1-6f9ea1a193ec","html_url":"https://github.com/Comfy-Org/NIMnodes","commit_stats":null,"previous_names":["comfy-org/nimnodes"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Comfy-Org%2FNIMnodes","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Comfy-Org%2FNIMnodes/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Comfy-Org%2FNIMnodes/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Comfy-Org%2FNIMnodes/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Comfy-Org","download_url":"https://codeload.github.com/Comfy-Org/NIMnodes/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252904681,"owners_count":21822654,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-05-07T15:27:21.777Z","updated_at":"2025-05-07T15:27:25.744Z","avatar_url":"https://github.com/Comfy-Org.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# NVIDIA FLUX NIM\n\n### What is the FLUX NIM?\n\nThe FLUX NIM is an NVIDIA Microservices container designed to run Black Forest Lab's FLUX models in the most optimal manner. This NIM provides support for the following models from Black Forest Lab's:  Flux.1 dev, Flux.1-Depth-dev, and Flux.1-Canny-dev. \n\n## Getting Started with the FLUX NIM in ComfyUI\n\nBefore installing, ensure your system meets the following requirements:  \nOperating System: Windows 11 (22H1 or later)  \nGPU: RTX 5090, RTX 5080, RTX 4090, RTX 4090 Laptop, RTX 4080, RTX 6000 Ada \nGPU Driver: Version 572.83 or later  \nVirtualization Settings: Enabled in SBIOS - [Instructions to enable virtualization if it is not enabled](https://support.microsoft.com/en-gb/windows/enable-virtualization-on-windows-c5578302-6e43-4b4b-a449-8ced115f58e1)\n\n\nThe node can automatically detect if you have already set up NVIDIA NIM, if not, it will navigate to the NIMsetup.exe download page where you can download and install NIMs.  \nHowever it is recommended to installer from [here](https://assets.ngc.nvidia.com/products/api-catalog/rtx/NIM_Prerequisites_Installer_03052025.zip) unzip the downloaded file and run NIMSetup.exe. \n\nAfter the NIM setup has completed, please perform the following steps to start NIMs in Comfy UI:\n\n1. Install ComfyUI following [this](https://github.com/comfyanonymous/ComfyUI?tab=readme-ov-file#installing) note the special instructions for installing for NVIDIA 50-Series (Blackwell) and prepare the running enviorment for ComfyUI\n2. Open ComfyUI folder, clone this repo and put it under `...\\ComfyUI\\custom_nodes\\`\n3. Go to `...\\ComfyUI\\custom_nodes\\comfyui_nim\\`and install dependency with `pip install -r requirements.txt`\n4. If using the windows standalone ComfyUI install use this command `..\\..\\..\\python_embeded\\python -m pip install -r requirements.txt`\n5. A HuggingFace API token is required to access the Flux models. For information on how to create an access token see [here](https://huggingface.co/docs/hub/en/security-tokens)\n6. To avoid having to input your token into the NIM everytime you can set the HF_TOKEN environment variable.\n7. Open a command prompt and type `setx HF_TOKEN \u003chftoken_info\u003e` where \u003chftoken_info\u003e represents your actual Hugging Face Token string.\n8. Accept the use agreement for the FLUX models on Hugging Face\n    1. Login into [Hugging Face](https://huggingface.co/login) using the account associated with your access token\n  \nNavigate to: [https://huggingface.co/black-forest-labs/FLUX.1-dev](https://huggingface.co/black-forest-labs/FLUX.1-dev) \n\n![image](https://github.com/user-attachments/assets/73206800-87cc-4bdf-a4bd-c0dde8730161)\nAccept the license agreement.\n\nRepeat this step for the following FLUX model variants:\n| Model      |URL |\n| ----------- | ----------- |\n| FLUX.1-Canny-dev      | [https://huggingface.co/black-forest-labs/FLUX.1-Canny-dev](https://huggingface.co/black-forest-labs/FLUX.1-Canny-dev) |\n| FLUX.1-Depth-dev      | [https://huggingface.co/black-forest-labs/FLUX.1-Depth-dev](https://huggingface.co/black-forest-labs/FLUX.1-Depth-dev) |\n| FLUX.1-dev-onnx       | [https://huggingface.co/black-forest-labs/FLUX.1-dev-onnx](https://huggingface.co/black-forest-labs/FLUX.1-dev-onnx) |\n| FLUX.1-Canny-dev-onnx | [https://huggingface.co/black-forest-labs/FLUX.1-Canny-dev-onnx](https://huggingface.co/black-forest-labs/FLUX.1-Canny-dev-onnx) |\n| FLUX.1-Depth-dev-onnx | [https://huggingface.co/black-forest-labs/FLUX.1-Depth-dev-onnx](https://huggingface.co/black-forest-labs/FLUX.1-Depth-dev-onnx) |\n\n## Start ComfyUI\n1. Run ComfyUI APP with `python main.py` under `...\\ComfyUI\\`\n2. For ComfyUI standalone, run ComfyUI using the run_nvidia_gpu.bat file. \n3. Open ComfyUI in browser, and import workflow `...\\ComfyUI\\custom_nodes\\comfyui_nim\\example_workflows\\FLUX_Dev_NIM_Workflow.json`\n4. Run the workflow. *The first time you run this it will download and configure the container, this may take a while.*\n5. ![flux_dev nim workflow](assets/Flux.1_dev_NIM.png)\n6. ![flux_depth_dev nim workflow](assets/Flux.1_depth_dev_NIM.png)\n7. ![flux_canny_dev nim workflow](assets/Flux.1_canny_dev_NIM.png) \n8. When ComfyUI is shutdown, the running NIMs will also be stopped  \n\n### Install node in ComfyUI\nThe recommended way to install these nodes is to use the [ComfyUI Manager](https://github.com/ltdrdata/ComfyUI-Manager) to easily install them to your ComfyUI instance.  \nYou can also manually install them by git cloning the repo to your ComfyUI/custom_nodes folder.\n\n\n### Node Details\n![Install NIM Node](assets/Install_NIM_Node.png)\n\nThe **Install NIM Node** checks to verify that the NIMs have been setup on the system, if the NIM setup has been completed this node returns *TRUE*. If NIM setup has not be completed, this node returns *FALSE* and will download the NIMSetup package.\n\nThis node must be connected to the **is_nim_installed** input on the Load NIM Node\n\n![Load NIM Node](assets/Load_NIM_Node.png)\n\nThe **Load NIM Node** is responsible for loading the requested NIM. \n\nInputs:\n\n*model_type*: [Flux Dev, Flux Canny, or Flux Depth] Determines which Flux model is loaded.\n\n*operation*: [Start, Stop]. **Start** is used to load and start the requested model in the NIM.  **Stop** will stop the NIM and unload loaded models, when switching between NIM models, any running models should be stopped before starting a new model.\n\n*offloading_policy*: [None, System RAM, Disk]. The offloading policy determine how models should be offloaded from VRAM.\n\n**None** indicates that models will not be offloaded, if the models exceed the available VRAM then generation will fail, it is recommended to only use **None** on GPUs with 24GB or more VRAM. If supported by the GPU, **None** offers the best performance.\n\n**System RAM** will move models to System RAM. The **System RAM** option provides a good mix of performance and flexibility, but may not be the best option for systems with limited system RAM.\n\n**Disk** will move offloaded models to disk. Offloading to disk impacts the overall performance but provides a viable option for GPUs with less than 24GB on systems with limited system RAM.\n\n*hf_token*: This field is used to provide the users Hugging Face API token, it is recommended to store the Hugging Face API token to the HF_TOKEN environment variable and use the Use **HF_TOKEN EnVar Node** to provide this input. *This field is required and must provide a valid HF API Token*.\n\n*is_nim_installed*: This input takes the output from the **Install NIM Node** is_nim_install output.\n\nOutputs:\n\n*is_nim_started*: This output sends information on whether the NIM has been started and is ready to recieved input. If the NIM has started and is ready it will return **True**. If the NIM fails to start it will return **False**.\n\n![FLUX NIM Node](assets/Flux_NIM_Node.png)\n\nThe **NIM FLUX NODE** allows the user to configure the options used by the FLUX NIM to generate images.\n\nInputs:\n\n*image*: When the FLUX Canny or FLUX Depth models are used, an image needs to be used to guide the image output. The Image input takes regular images as input and will be converted to *Depth* or *Canny* images within the NIM. \n\n*is_nim_started*: This input takes the output from the **is_nim_started** output from the *Load NIM Node*.\n\n*width*: The image width. Valid ranges are \"768\", \"832\", \"896\", \"960\", \"1024\", \"1088\", \"1152\", \"1216\", \"1280\", \"1344\"\n\n*height*: The image height. Valid ranges are \"768\", \"832\", \"896\", \"960\", \"1024\", \"1088\", \"1152\", \"1216\", \"1280\", \"1344\"\n\n*prompt*: The text description of the desired image output\n\n*cfg_scale*: The cfg scale determines how closely the output adheres to the prompt input, higher values will cause more adherence. \n\n*seed*: The seed used for noise generation.\n\n*control after generate*: [fixed, increment, decrement, randomize]\n\n*steps*: The number of generation steps used per image.\n\nOutput:\n\n*image*: The generated image, this output should be connected to a Preview Image Node or Save Image Node.\n\n![HF_TOKEN Node](assets/HF_TOKEN_Node.png)\n\nThe **Use HF_TOKEN Node** willread the HF_TOKEN environment variable and pass it as an output which can be connected to the **hf_token** input on the *Load NIM Node*\n\nInputs:\n\nNONE\n\nOutput:\n\n*hf_token*: Outputs the contents of the HF_TOKEN environment variable, will generate a failure if the environment variable does not exist.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcomfy-org%2Fnimnodes","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcomfy-org%2Fnimnodes","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcomfy-org%2Fnimnodes/lists"}