Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/huggingface/visual-blocks-custom-components
Custom Hugging Face Nodes for Google Visual Blocks for ML
https://github.com/huggingface/visual-blocks-custom-components
Last synced: 3 months ago
JSON representation
Custom Hugging Face Nodes for Google Visual Blocks for ML
- Host: GitHub
- URL: https://github.com/huggingface/visual-blocks-custom-components
- Owner: huggingface
- License: apache-2.0
- Created: 2024-01-05T13:13:10.000Z (11 months ago)
- Default Branch: main
- Last Pushed: 2024-05-16T06:43:15.000Z (7 months ago)
- Last Synced: 2024-05-23T02:39:39.177Z (7 months ago)
- Language: TypeScript
- Homepage: https://huggingface.co/hf-vb
- Size: 472 KB
- Stars: 61
- Watchers: 6
- Forks: 7
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Hugging Face + Visual Blocks Custom Components
Visual blocks is an amazing tool from our friends at [Google](https://github.com/google/visualblocks)
that allows you to easily create and experiment with machine learning pipelines using a visual interface.
This repository contains the source code for custom components that allow you to use Hugging Face client and server models in your Visual Blocks pipelines.
We've created a few nodes supporting different tasks and models following our [Tasks](https://huggingface.co/tasks) definitions.
> [!NOTE]
> Visual Blocks seems to be mostly working in Chrome. If you are having trouble with the interface, try using Chrome, and please submit an [issue](https://github.com/google/visualblocks/issues) to the Visual Blocks team.
Important links:
- https://visualblocks.withgoogle.com/
- https://www.npmjs.com/package/huggingface-visualblocks-nodes
- https://cdn.jsdelivr.net/npm/huggingface-visualblocks-nodes@latest
Table of Contents
- [How to use the custom components](#how-to-use-the-custom-components)
- [Nodes and Examples](#nodes-and-examples)
- [Client Nodes](#client-nodes)
- [Translation](#translation)
- [Token Classification](#token-classification)
- [Text Classification](#text-classification)
- [Object Detection](#object-detection)
- [Image Segmentation](#image-segmentation)
- [Image Classification](#image-classification)
- [Depth Estimation](#depth-estimation)
- [Background Removal](#background-removal)
- [Server Nodes](#server-nodes)
- [Text Generation and Chat Completion](#text-generation-and-chat-completion)
- [Fill Mask](#fill-mask)
- [Image Classification](#image-classification-1)
- [Summarization](#summarization)
- [Text Classification](#text-classification-1)
- [Text Generation](#text-generation)
- [Text to Image](#text-to-image)
- [Token Classification](#token-classification-1)
- [Extra Examples](#extra-examples)
- [Local Development](#local-development)
## How to use the custom components
To start playing with our custom components you need to **Add a custom node** to your Visual Blocks project. First you need to start a new project [https://visualblocks.withgoogle.com/#/edit/new](https://visualblocks.withgoogle.com/#/edit/new), then click on the "+" button in the bottom left corner to add a new node.
Then input the pre-bundled code from our npm package. You can do this by pasting the following link into the input field and clicking "Submit":
```
https://cdn.jsdelivr.net/npm/huggingface-visualblocks-nodes@latest
```
Then you will be able to see three Hugging Face Collections: Client, Server and Common.
# Nodes and Examples
## Client Nodes
Client nodes are nodes running tranformers pipelines on the client side using [Transformers.js](https://github.com/xenova/transformers.js). All Client nodes have WASM and WebGPU (experimental) backend support, and you can find web-compatible models by visiting https://huggingface.co/models?library=transformers.js.
> [!NOTE]
> WebGPU support in transformers.js is still experimental and may not work on all devices. Not all models are supported by WebGPU backend yet.
### Translation
[**Translation Node Example**](https://visualblocks.withgoogle.com/#/edit/new_hfdemo?project_json=https://cdn.jsdelivr.net/npm/huggingface-visualblocks-nodes@latest/examples/client/translation_client_.json)
More info:
- https://huggingface.co/tasks/translation
- https://huggingface.co/docs/transformers.js/api/pipelines#module_pipelines.TranslationPipeline
### Token Classification
[**Token Classification Node Exampl**e](https://visualblocks.withgoogle.com/#/edit/new_hfdemo?project_json=https://cdn.jsdelivr.net/npm/huggingface-visualblocks-nodes@latest/examples/client/token_classification_client_.json)
More info:
- https://huggingface.co/tasks/token-classification
- https://huggingface.co/docs/transformers.js/api/pipelines#module_pipelines.TokenClassificationPipeline
### Text Classification
[**Text Classification Node Example**](https://visualblocks.withgoogle.com/#/edit/new_hfdemo?project_json=https://cdn.jsdelivr.net/npm/huggingface-visualblocks-nodes@latest/examples/client/text_classification_client_.json)
More info:
- https://huggingface.co/tasks/text-classification
- https://huggingface.co/docs/transformers.js/api/pipelines#module_pipelines.TextClassificationPipeline
### Object Detection
[**Object Detection Node Example**](https://visualblocks.withgoogle.com/#/edit/new_hfdemo?project_json=https://cdn.jsdelivr.net/npm/huggingface-visualblocks-nodes@latest/examples/client/object_detection_client_.json)
More info:
- https://huggingface.co/tasks/object-detection
- https://huggingface.co/docs/transformers.js/api/pipelines#module_pipelines.ObjectDetectionPipeline
### Image Segmentation
[**Image Segmentation Node Example**](https://visualblocks.withgoogle.com/#/edit/new_hfdemo?project_json=https://cdn.jsdelivr.net/npm/huggingface-visualblocks-nodes@latest/examples/client/image_segmentation_client_.json)
More info:
- https://huggingface.co/tasks/image-segmentation
- https://huggingface.co/docs/transformers.js/api/pipelines#module_pipelines.ImageSegmentationPipeline
### Image Classification
[**Image Classification Node Example**](https://visualblocks.withgoogle.com/#/edit/new_hfdemo?project_json=https://cdn.jsdelivr.net/npm/huggingface-visualblocks-nodes@latest/examples/client/image_classification_client_.json)
More info:
- https://huggingface.co/tasks/image-classification
- https://huggingface.co/docs/transformers.js/api/pipelines#module_pipelines.ImageClassificationPipeline
### Depth Estimation
[Depth Estimation Node Example](https://visualblocks.withgoogle.com/#/edit/new_hfdemo?project_json=https://cdn.jsdelivr.net/npm/huggingface-visualblocks-nodes@latest/examples/client/depth_estimation_client_.json)
More info:
- https://huggingface.co/tasks/depth-estimation
- https://huggingface.co/docs/transformers.js/api/pipelines#module_pipelines.DepthEstimationPipeline
### Background Removal
[**Background Removal Node Example**](https://visualblocks.withgoogle.com/#/edit/new_hfdemo?project_json=https://cdn.jsdelivr.net/npm/huggingface-visualblocks-nodes@latest/examples/client/background_removal_client_.json)
## Server Nodes
Server nodes are nodes running Transformers pipeline tasks using the [Hugging Face Serverless API](https://huggingface.co/docs/api-inference/en/index). For a few selected LLM models, it's running using our hosted [Text Generation Inference](https://huggingface.co/docs/text-generation-inference/en/index), our fast, optimized inference for LLMs.
> [!NOTE]
> You can use the Hugging Face Serverless API for free with limited usage, after which you'll be rate limited. If you need more usage, you can create an account at https://huggingface.co/join and get an API token at https://huggingface.co/settings/tokens or log in using the Hugging Face Login node.
For server nodes you have the option to Login using your Hugging Face account to get more usage and access to private models. Using **Hugging Face Hub Login**
If successful, you can obtain your **Apikey** directly from the **Hugging Face Hub Login** node.
### Text Generation and Chat Completion
[**Chat Template Text Generation Node Example**](https://visualblocks.withgoogle.com/#/edit/new_hfdemo?project_json=https://cdn.jsdelivr.net/npm/huggingface-visualblocks-nodes@latest/examples/server/chat_template_text_generation_server_.json)
[**Chat Completion Node Example**](https://visualblocks.withgoogle.com/#/edit/new_hfdemo?project_json=https://cdn.jsdelivr.net/npm/huggingface-visualblocks-nodes@latest/examples/server/chat_completion_server_.json)
More info:
- https://huggingface.co/tasks/text-generation
### Fill Mask
[**Fill Mask Node Example**](https://visualblocks.withgoogle.com/#/edit/new_hfdemo?project_json=https://cdn.jsdelivr.net/npm/huggingface-visualblocks-nodes@latest/examples/server/fill_mask_server_.json)
More info:
- https://huggingface.co/tasks/fill-mask
### Image Classification
[**Image Classification Node Example**](https://visualblocks.withgoogle.com/#/edit/new_hfdemo?project_json=https://cdn.jsdelivr.net/npm/huggingface-visualblocks-nodes@latest/examples/server/image_classification_server_.json)
More info:
- https://huggingface.co/tasks/image-classification
### Summarization
[**Summarization Node Example**](https://visualblocks.withgoogle.com/#/edit/new_hfdemo?project_json=https://cdn.jsdelivr.net/npm/huggingface-visualblocks-nodes@latest/examples/server/summarization_server_.json)
More info:
- https://huggingface.co/tasks/summarization
### Text Classification
[**Text Classification Node Example**](https://visualblocks.withgoogle.com/#/edit/new_hfdemo?project_json=https://cdn.jsdelivr.net/npm/huggingface-visualblocks-nodes@latest/examples/server/text_classification_server_.json)
More info:
- https://huggingface.co/tasks/text-classification
### Text Generation
[**Text Generation Node Example**](https://visualblocks.withgoogle.com/#/edit/new_hfdemo?project_json=https://cdn.jsdelivr.net/npm/huggingface-visualblocks-nodes@latest/examples/server/text_generation_server_.json)
More info:
- https://huggingface.co/tasks/text-generation
### Text to Image
[**Text to Image Node Example**](https://visualblocks.withgoogle.com/#/edit/new_hfdemo?project_json=https://cdn.jsdelivr.net/npm/huggingface-visualblocks-nodes@latest/examples/server/text_to_image_server_.json)
More info:
- https://huggingface.co/tasks/text-to-image
### Token Classification
[**Token Classification Node Example**](https://visualblocks.withgoogle.com/#/edit/new_hfdemo?project_json=https://cdn.jsdelivr.net/npm/huggingface-visualblocks-nodes@latest/examples/server/token_classification_server_.json)
More info:
- https://huggingface.co/tasks/token-classification
## Extra Examples
[**Background Removal Text to Image**](https://visualblocks.withgoogle.com/#/edit/new_hfdemo?project_json=https://cdn.jsdelivr.net/npm/huggingface-visualblocks-nodes@latest/examples/extras/background_removal_text_to_image.json)
[**Chat Completion Text to Image Depth**](https://visualblocks.withgoogle.com/#/edit/new_hfdemo?project_json=https://cdn.jsdelivr.net/npm/huggingface-visualblocks-nodes@latest/examples/extras/chat_completion_txt2img_depth.json)
[**Image Segmentation Webcam Client**](https://visualblocks.withgoogle.com/#/edit/new_hfdemo?project_json=https://cdn.jsdelivr.net/npm/huggingface-visualblocks-nodes@latest/examples/extras/image_segmentation_webcam_client.json)
# Local Development
1. Clone the repository
```bash
git clone https://github.com/huggingface/visual-blocks-custom-components.git
cd visual-blocks-custom-components
```
2. Install the dependencies
```bash
npm i
```
3. Run the development server
```bash
npm run dev
```
4. Visit Google's staging server
5. Click the + in the bottom left corner to add the custom nodes.
6. Paste in the link to the script (e.g., http://localhost:8080/index.js) and click "Submit".