{"id":30687320,"url":"https://github.com/1038lab/ComfyUI-MiniCPM","last_synced_at":"2025-09-02T00:04:00.343Z","repository":{"id":311810650,"uuid":"965699231","full_name":"1038lab/ComfyUI-MiniCPM","owner":"1038lab","description":"A ComfyUI custom node for MiniCPM vision-language models, enabling high-quality image captioning and analysis.","archived":false,"fork":false,"pushed_at":"2025-08-26T18:30:52.000Z","size":1764,"stargazers_count":4,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-08-27T01:57:39.758Z","etag":null,"topics":["comfyui","custom-nodes","gguf","llama-cpp","minicpm","minicpm-v","muti-models","stable-diffusion"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/1038lab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-04-13T18:13:16.000Z","updated_at":"2025-08-27T00:26:45.000Z","dependencies_parsed_at":"2025-08-27T01:57:44.452Z","dependency_job_id":"714cbbb9-f58a-4498-92bf-15c39480cf3d","html_url":"https://github.com/1038lab/ComfyUI-MiniCPM","commit_stats":null,"previous_names":["1038lab/comfyui-minicpm"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/1038lab/ComfyUI-MiniCPM","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/1038lab%2FComfyUI-MiniCPM","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/1038lab%2FComfyUI-MiniCPM/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/1038lab%2FComfyUI-MiniCPM/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/1038lab%2FComfyUI-MiniCPM/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/1038lab","download_url":"https://codeload.github.com/1038lab/ComfyUI-MiniCPM/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/1038lab%2FComfyUI-MiniCPM/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":273208777,"owners_count":25064204,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-01T02:00:09.058Z","response_time":120,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["comfyui","custom-nodes","gguf","llama-cpp","minicpm","minicpm-v","muti-models","stable-diffusion"],"created_at":"2025-09-02T00:01:52.396Z","updated_at":"2025-09-02T00:04:00.315Z","avatar_url":"https://github.com/1038lab.png","language":"Python","funding_links":[],"categories":["Workflows (3395) sorted by GitHub Stars"],"sub_categories":[],"readme":"# ComfyUI-MiniCPM\r\n\r\nA custom ComfyUI node for MiniCPM vision-language models, supporting v4, v4.5, and v4 GGUF formats, enabling high-quality image captioning and visual analysis.\r\n\r\n**🎉 Now supports MiniCPM-V-4.5! The latest model with enhanced capabilities.**\r\n\r\n---\r\n## News \u0026 Updates\r\n- **2025/08/28**: Update ComfyUI-MIniCPM to **v1.1.1** ( [update.md](update.md#v111-2025-08-28) )\r\n- **2025/08/27**: Update ComfyUI-MIniCPM to **v1.1.0** ( [update.md](update.md#v110-2025-08-27) )\r\n[![MiniCPM v4 VS v45](example_workflows/MiniCPM_v4VSv45.jpg)](example_workflows/MiniCPM_v4VSv45.json)\r\n- Added support for **MiniCPM-V-4.5** models (Transformers)\r\n  \r\n## Features\r\n- MiniCPM-V-4 GGUF\r\n[![MiniCPM-V-4-GGUF](example_workflows/MiniCPM-V-4-GGUF.jpg)](example_workflows/MiniCPM-V-4-GGUF.json)\r\n- MiniCPM-V-4 Batch Images\r\n[![MiniCPM-V-4_batchImages](example_workflows/MiniCPM-V-4_batchImages.jpg)](example_workflows/MiniCPM-V-4_batchImages.json)\r\n- MiniCPM-V-4 video\r\n[![MiniCPM-V-4_video](example_workflows/MiniCPM-V-4_video.jpg)](example_workflows/MiniCPM-V-4_video.json)\r\n\r\n- Supports **MiniCPM-V-4.5 (Transformers)** and **MiniCPM-V-4.0 (GGUF)** models\r\n- **Latest MiniCPM-V-4.5** with enhanced capabilities via Transformers\r\n- Multiple caption types to suit different use cases (Describe, Caption, Analyze, etc.)\r\n- Memory management options to balance VRAM usage and speed\r\n- Auto-downloads model files on first use for easy setup\r\n- Customizable parameters: max tokens, temperature, top-p/k sampling, repetition penalty\r\n- Advanced node with full parameter control\r\n- Legacy node for backward compatibility\r\n- Comprehensive GGUF quantization options for V4.0 models\r\n\r\n---\r\n\r\n## Installation\r\n\r\nClone the repo into your ComfyUI custom nodes folder:\r\n\r\n```bash\r\ncd ComfyUI/custom_nodes\r\ngit clone https://github.com/1038lab/comfyui-minicpm.git\r\n```\r\n\r\nInstall required dependencies:\r\n\r\n```bash\r\ncd ComfyUI/custom_nodes/comfyui-minicpm\r\nComfyUI\\python_embeded\\python pip install -r requirements.txt\r\nComfyUI\\python_embeded\\python llama_cpp_install.py\r\n```\r\n\r\n\u003e [!note]\r\n\u003e `llama-cpp-python` CUDA Installation for ComfyUI Portable\r\n\u003e - [llama_cpp_install.md](llama_cpp_install/llama_cpp_install.md)\r\n---\r\n\r\n## Supported Models\r\n\r\n### Transformers Models\r\n| Model                | Description                                    |\r\n| -------------------- | ---------------------------------------------- |\r\n| **MiniCPM-V-4.5**        | 🌟 **Latest V4.5 version with enhanced capabilities** |\r\n| **MiniCPM-V-4.5-int4**   | 🌟 **V4.5 4-bit quantized version, smaller memory footprint** |\r\n| MiniCPM-V-4          | V4.0 full precision version, higher quality   |\r\n| MiniCPM-V-4-int4     | V4.0 4-bit quantized version, smaller memory footprint |\r\n\r\nhttps://huggingface.co/openbmb/MiniCPM-V-4_5  \r\nhttps://huggingface.co/openbmb/MiniCPM-V-4_5-int4  \r\nhttps://huggingface.co/openbmb/MiniCPM-V-4\r\nhttps://huggingface.co/openbmb/MiniCPM-V-4-int4\r\n\r\n### GGUF Models\r\n\r\n\u003e **Note**: MiniCPM-V-4.5 GGUF models are temporarily unavailable due to llama-cpp-python compatibility issues. Please use MiniCPM-V-4.5 Transformers models or MiniCPM-V-4.0 GGUF models.\r\n\r\n#### MiniCPM-V-4.0 (Fully Supported)\r\n| Model                | Size      | Description                           |\r\n| -------------------- | --------- | ------------------------------------- |\r\n| **MiniCPM-V-4 (Q4_K_M)** | ~2.19GB   | **Recommended balance of quality/size** |\r\n| MiniCPM-V-4 (Q4_0)      | ~2.08GB   | Standard 4-bit quantization          |\r\n| MiniCPM-V-4 (Q4_1)      | ~2.29GB   | 4-bit quantization improved          |\r\n| MiniCPM-V-4 (Q4_K_S)    | ~2.09GB   | 4-bit K-quants small                 |\r\n| MiniCPM-V-4 (Q5_0)      | ~2.51GB   | 5-bit quantization                   |\r\n| MiniCPM-V-4 (Q5_1)      | ~2.72GB   | 5-bit quantization improved          |\r\n| MiniCPM-V-4 (Q5_K_M)    | ~2.56GB   | 5-bit K-quants medium                |\r\n| MiniCPM-V-4 (Q5_K_S)    | ~2.51GB   | 5-bit K-quants small                 |\r\n| MiniCPM-V-4 (Q6_K)      | ~2.96GB   | Very high quality                    |\r\n| MiniCPM-V-4 (Q8_0)      | ~3.83GB   | Highest quality quantized            |\r\n\r\nhttps://huggingface.co/openbmb/MiniCPM-V-4-gguf\r\n\r\n\u003e The models will be automatically downloaded on first run.\r\n\u003e Manual download and placement into `models/LLM` (transformers) or `models/LLM/GGUF` (GGUF) is also supported.\r\n\r\n---\r\n\r\n## Available Nodes\r\n\r\n### 1. MiniCPM-4-V-Transformers\r\n- Basic transformers-based node with essential parameters\r\n- Supports image and video input\r\n- Memory management options\r\n- Preset prompt types\r\n\r\n### 2. MiniCPM-4-V-Transformers Advanced\r\n- Full-featured transformers-based node\r\n- All parameters customizable\r\n- System prompt support\r\n- Advanced video processing options\r\n\r\n### 3. MiniCPM-4-V-GGUF\r\n- GGUF-based node with essential parameters\r\n- Optimized for performance\r\n\r\n### 4. MiniCPM-4-V-GGUF Advanced\r\n- Full-featured GGUF-based node\r\n- All parameters customizable\r\n\r\n### 5. MiniCPM (Legacy)\r\n- Original node for backward compatibility\r\n- Basic functionality\r\n\r\n---\r\n\r\n## Usage\r\n\r\n1. Add the **MiniCPM** node from the `🧪AILab` category in ComfyUI.\r\n2. Connect an image or video input node to the MiniCPM node.\r\n3. Select the model variant (default is MiniCPM-V-4-int4 for transformers).\r\n4. Choose caption type and adjust parameters as needed.\r\n5. Execute your workflow to generate captions or analysis.\r\n\r\n---\r\n\r\n## Configuration Defaults\r\n\r\n```json\r\n{\r\n  \"context_window\": 4096,\r\n  \"gpu_layers\": -1,\r\n  \"cpu_threads\": 4,\r\n  \"default_max_tokens\": 1024,\r\n  \"default_temperature\": 0.7,\r\n  \"default_top_p\": 0.9,\r\n  \"default_top_k\": 100,\r\n  \"default_repetition_penalty\": 1.10,\r\n  \"default_system_prompt\": \"You are MiniCPM-V, a helpful, concise and knowledgeable vision-language assistant. Answer directly and stay on task.\"\r\n}\r\n```\r\n\r\n---\r\n\r\n## Caption Types\r\n\r\n* **Describe:** Describe this image in detail.\r\n* **Caption:** Write a concise caption for this image.\r\n* **Analyze:** Analyze the main elements and scene in this image.\r\n* **Identify:** What objects and subjects do you see in this image?\r\n* **Explain:** Explain what's happening in this image.\r\n* **List:** List the main objects visible in this image.\r\n* **Scene:** Describe the scene and setting of this image.\r\n* **Details:** What are the key details in this image?\r\n* **Summarize:** Summarize the key content of this image in 1-2 sentences.\r\n* **Emotion:** Describe the emotions or mood conveyed by this image.\r\n* **Style:** Describe the artistic or visual style of this image.\r\n* **Location:** Where might this image be taken? Analyze the setting or location.\r\n* **Question:** What question could be asked based on this image?\r\n* **Creative:** Describe this image as if writing the beginning of a short story.\r\n\r\n---\r\n\r\n## Memory Management Options\r\n\r\n* **Keep in Memory:** Model stays loaded for faster subsequent runs\r\n* **Clear After Run:** Model is unloaded after each run to save memory\r\n* **Global Cache:** Model is cached globally and shared between nodes\r\n\r\n---\r\n\r\n## Tips\r\n\r\n### VRAM Requirements\r\n* **4-6GB VRAM**: Use MiniCPM-V-4-int4 or GGUF Q4 models\r\n* **8GB VRAM**: Use MiniCPM-V-4.5-int4 (recommended)\r\n* **12GB+ VRAM**: Can use full MiniCPM-V-4.5\r\n* **CUDA OOM Error**: Try int4 quantized models or CPU mode\r\n\r\n### General Tips\r\n* 🌟 **Try MiniCPM-V-4.5 Transformers first** - enhanced capabilities over V4.0\r\n* For **best balance**: use MiniCPM-V-4 (Q4_K_M) GGUF model\r\n* For **highest quality**: use MiniCPM-V-4.5 Transformers\r\n* For **low VRAM**: use MiniCPM-V-4.5-int4 or MiniCPM-V-4 (Q4_0) GGUF\r\n* Adjust temperature (0.6–0.8) for balancing creativity and coherence.\r\n* Use top-p (0.9) and top-k (80) sampling for natural output diversity.\r\n* Lower max tokens or precision (bf16/fp16) for faster generation on less powerful GPUs.\r\n* Memory modes help optimize VRAM usage: default, balanced, max savings.\r\n* Transformers models offer better quality but use more memory.\r\n* GGUF models are more memory-efficient but may have slightly lower quality.\r\n\r\n---\r\n\r\n## License\r\n\r\n\r\nGPL-3.0 License\r\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2F1038lab%2FComfyUI-MiniCPM","html_url":"https://awesome.ecosyste.ms/projects/github.com%2F1038lab%2FComfyUI-MiniCPM","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2F1038lab%2FComfyUI-MiniCPM/lists"}