https://github.com/insdet/instance-detection

https://github.com/insdet/instance-detection

Last synced: about 1 month ago
JSON representation

• Host: GitHub
• URL: https://github.com/insdet/instance-detection
• Owner: insdet
• Created: 2023-06-10T09:38:39.000Z (over 1 year ago)
• Default Branch: main
• Last Pushed: 2024-06-14T06:07:12.000Z (7 months ago)
• Last Synced: 2024-06-14T07:27:23.476Z (7 months ago)
• Language: Jupyter Notebook
• Homepage:
• Size: 26.9 MB
• Stars: 4
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • Citation: CITATION.cff

Awesome Lists containing this project

• Awesome-Segment-Anything - [code

README

        
## A High-Resolution Dataset for Instance Detection with Multi-View Instance Capture

NeurIPS (Datasets and Benchmarks) 2023

**Authors**: Qianqian Shen\*, Yunhan Zhao\*, Nahyun Kwon, Jeeeun Kim, Yanan Li, Shu Kong

If you find our model/method/dataset useful, please cite our work ([NeurIPS version on arxiv](https://arxiv.org/abs/2310.19257)):

    @article{shen2024high,

      title={A High-Resolution Dataset for Instance Detection with Multi-View Object Capture},

      author={Shen, Qianqian and Zhao, Yunhan and Kwon, Nahyun and Kim, Jeeeun and Li, Yanan and Kong, Shu},

      journal={Advances in Neural Information Processing Systems},

      volume={36},

      year={2024}

    }

The InsDet datase is a high-resolution real-world dataset for **Instance Detection** with **Multi-view Instance Capture**.


We provide an [InsDet-mini](https://drive.google.com/drive/folders/1X8MT5JuLq0Vjq1jNE1I9h3q_JGolNJsI?usp=sharing) for demo and visualization, and the full dataset [InsDet-FULL](https://drive.google.com/drive/folders/1rIRTtqKJGCTifcqJFSVvFshRb-sB0OzP?usp=sharing).

## Dataset

The full dataset contains 100 objects with multi-view profile images in 24 rotation positions (per 15°), 160 testing scene images with high-resolution, and 200 pure background images. The mini version contains 5 objects, 10 testing scene images, and 10 pure background images.

### Details

The **Objects** contains:

- 000_aveda_shampoo

  - images: raw RGB images (e.g., "images/001.jpg")

  - masks: segmentation masks generated by [GrabCut Annotation Toolbox](https://github.com/Kazuhito00/GrabCut-Annotation-Tool) (e.g., "masks/001.png")

- 
 $\vdots$ 

- 099_mug_blue


![vis-objects](/assets/vis-objects.png)

Tip: The first three digits specify the instance id.

The **Scenes** contains:

- easy

  - leisure\_zone

    - raw RGB images with 6144×8192 pixels (e.g. “office001/rgb\_000.jpg”)

    - bounding box annotation for objects in test scenes generated by labelImg toolbox and using PascalVOC format (e.g. “office\_001/rgb\_000.xml”)

  - meeting\_room

  - office\_002

  - pantry\_room\_002

  - sink

- hard

  - office\_001

  - pantry\_room\_001

![vis-scenes](/assets/vis-scenes.png)

Tip: Each bounding box is specified by [xmin, ymin, xmax, ymax].

The **Background** contains 200 pure background images that do not include any instances from **Objects** folder.

![vis-background](/assets/vis-background.png)

## Code

The project is built on [detectron2](https://github.com/facebookresearch/detectron2), [segment-anything](https://github.com/facebookresearch/segment-anything), and [DINOv2](https://github.com/facebookresearch/dinov2).


### Demo

The Jupyter notebooks files demonstrate our non-learned method using SAM and DINOv2. We choose light pretrained models of SAM (vit_l) and DINOv2 (dinov2_vits14) for efficiency.