https://github.com/modelscope/imagepulse
Open Image Curation Tools
https://github.com/modelscope/imagepulse
Last synced: 6 months ago
JSON representation
Open Image Curation Tools
- Host: GitHub
- URL: https://github.com/modelscope/imagepulse
- Owner: modelscope
- License: apache-2.0
- Created: 2025-03-31T08:36:04.000Z (9 months ago)
- Default Branch: main
- Last Pushed: 2025-04-22T11:19:08.000Z (8 months ago)
- Last Synced: 2025-04-22T12:30:39.554Z (8 months ago)
- Language: Python
- Size: 54.7 KB
- Stars: 9
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# ImagePulse
ImagePulse project aims to provide dataset support for the next generation of image understanding and generation models, by atomizing the capabilities of these models and constructing atomic capability datasets.
[Switch to Chinese](./README_zh.md)
## Atomic Capability Datasets
### 1. Change, Add, Remove
* Dataset: [https://www.modelscope.cn/datasets/DiffSynth-Studio/ImagePulse-ChangeAddRemove](https://www.modelscope.cn/datasets/DiffSynth-Studio/ImagePulse-ChangeAddRemove)
* Dataset Construction Script: [./scripts/change_add_remove.py](./scripts/change_add_remove.py)
|image_1|image_2|mask|editing_instruction|reverse_editing_instruction|
|-|-|-|-|-|
||||Remove the mustache and beard, change the white shirt to a blue turtleneck sweater, and remove the glass of milk.|Add a mustache and beard, change the blue turtleneck sweater to a white shirt, and add a glass of milk.|
||||Add a silver butterfly to the glowing golden lace on her face.|Remove the silver butterfly from the glowing golden lace on her face.|
||||Remove the necklace.|Add a necklace.|
### 2. Zoom In, Zoom Out
* Dataset: [https://www.modelscope.cn/datasets/DiffSynth-Studio/ImagePulse-ZoominZoomout](https://www.modelscope.cn/datasets/DiffSynth-Studio/ImagePulse-ZoominZoomout)
* Dataset Construction Script: [./scripts/zoomin_zoomout.py](./scripts/zoomin_zoomout.py)
|image_1|image_2|image_cropped|mask|editing_instruction|reverse_editing_instruction|
|-|-|-|-|-|-|
|||||Zoom in to focus on the headband.|Zoom out to show the full view of the anime girl.|
|||||Remove the superhero costume and replace it with a red shirt. Adjust the lighting to highlight the man's face.|Add a superhero costume with a red and yellow emblem on the chest and a red cape. Adjust the lighting to emphasize the costume.|
|||||Remove the elephant and replace it with a large rock.|Replace the large rock with an elephant.|
### 3. Style Transfer
* Dataset: [https://www.modelscope.cn/datasets/DiffSynth-Studio/ImagePulse-StyleTransfer](https://www.modelscope.cn/datasets/DiffSynth-Studio/ImagePulse-StyleTransfer)
* Dataset Construction Script: [./scripts/style_transfer.py](./scripts/style_transfer.py)
|image_1|image_2|image_3|image_4|editing_instruction|reverse_editing_instruction|
|-|-|-|-|-|-|
|||||transform the image into a cartoon style with vibrant colors and a confident expression.|transform the image into a realistic portrait with a serious expression and subtle lighting.|
|||||transform the image to have a brighter, more colorful palette and a clear blue sky.|transform the image to have a more muted color palette and an overcast sky.|
|||||transform the style of the image to an anime illustration, change the jacket to red, and add a cityscape background.|transform the style of the image to a digital painting, change the jacket to black, and remove the cityscape background.|
### 4. Face ID
* Dataset: [https://www.modelscope.cn/datasets/DiffSynth-Studio/ImagePulse-FaceID](https://www.modelscope.cn/datasets/DiffSynth-Studio/ImagePulse-FaceID)
* Dataset Construction Script: [./scripts/faceid.py](./scripts/faceid.py)
|image_face|image_1|image_2|editing_instruction|reverse_editing_instruction|
|-|-|-|-|-|
||||Change the woman's white t-shirt to a white tank top.|Change the woman's white tank top to a white t-shirt.|
||||Add a nighttime street scene with bokeh lights in the background.|Remove the nighttime street scene and bokeh lights from the background.|
||||Change the background to a warmly lit room with lamps, change the suit to maroon, and add a sweater under the suit.|Change the background to a dimly lit room with red lighting, change the suit to black, and remove the sweater.|
## Running Dataset Generation
```bash
pip install -r requirements.txt
```
```bash
python change_add_remove.py \
--target_dir "data/dataset" \
--cache_dir "data/cache" \
--dashscope_api_key "sk-xxxxxxxxxxxxxxxx" \
--qwenvl_model_id "qwen-vl-max" \
--modelscope_access_token "xxxxxxxxxxxxxxx" \
--modelscope_dataset_id "DiffSynth-Studio/ImagePulse-ChangeAddRemove" \
--num_data 1000000 \
--max_num_files_per_folder 1000
```
* `target_dir`: Path to store the dataset
* `cache_dir`: Cache path
* `dashscope_api_key`: [DashScope](https://DashScope.console.aliyun.com/#/home) API Key, required when calling DashScope API
* `qwenvl_model_id`: ID of the Qwen-VL model on [DashScope](https://DashScope.console.aliyun.com/#/home), required when calling DashScope API
* `modelscope_access_token`: Access token from [ModelScope](https://modelscope.cn/my/myaccesstoken), required when uploading datasets to ModelScope
* `modelscope_dataset_id`: Dataset ID on [ModelScope](https://modelscope.cn), required when uploading datasets to ModelScope
* `num_data`: Total number of data samples
* `max_num_files_per_folder`: Number of files per packaged folder
## Acknowledgements
* [DiffSynth-Studio](https://github.com/modelscope/DiffSynth-Studio): Provided Diffusion model inference support for this project
* [ModelScope](https://modelscope.cn): Provided storage and download support for models and datasets in this project
* [DashScope](https://DashScope.console.aliyun.com/#/home): Provided inference API support for large language models in this project