https://github.com/insdet/instance-detection

Last synced: 5 months ago
JSON representation

Host: GitHub
URL: https://github.com/insdet/instance-detection
Owner: insdet
Created: 2023-06-10T09:38:39.000Z (over 2 years ago)
Default Branch: main
Last Pushed: 2024-06-14T06:07:12.000Z (over 1 year ago)
Last Synced: 2024-06-14T07:27:23.476Z (over 1 year ago)
Language: Jupyter Notebook
Homepage:
Size: 26.9 MB
Stars: 4
Watchers: 1
Forks: 0
Open Issues: 0
Metadata Files:
- Readme: README.md
- Citation: CITATION.cff

Awesome Lists containing this project

Awesome-Segment-Anything - [code

README

          


A High-Resolution Dataset for Instance Detection with Multi-View Instance Capture


[**Qianqian Shen**](https://shenqq377.github.io/)¹ · [**Yunhan Zhao**](https://yunhan-zhao.github.io/)² ·  [**Nahyun Kwon**](https://nahyunkwon.github.io/)³ · [**Jeeeun Kim**](https://github.com/qubick)³ · [**Yanan Li**](https://yananlix1.github.io/)¹ · [**Shu Kong**](https://aimerykong.github.io/)³^,⁴^,⁵

¹Zhejiang Lab ²UC Irvine ³Texas A&M University ⁴University of Macau ⁵Institute of Collaborative











The paper has been accepted by **NeurIPS (Datasets and Benchmarks) 2023**.

![InsDet](assets/object-insdet.png)

## Dataset

The InsDet datase is a high-resolution real-world dataset for **Instance Detection** with **Multi-view Instance Capture**.


We provide an [InsDet-mini](https://drive.google.com/drive/folders/1X8MT5JuLq0Vjq1jNE1I9h3q_JGolNJsI?usp=sharing) for demo and visualization, and the full dataset [InsDet-FULL](https://drive.google.com/drive/folders/1rIRTtqKJGCTifcqJFSVvFshRb-sB0OzP?usp=sharing).

The full dataset contains 100 objects with multi-view profile images in 24 rotation positions (per 15°), 160 testing scene images with high-resolution, and 200 pure background images. The mini version contains 5 objects, 10 testing scene images, and 10 pure background images.

### Details

The **Objects** contains:

- 000_aveda_shampoo

  - images: raw RGB images (e.g., "images/001.jpg")

  - masks: segmentation masks generated by [GrabCut Annotation Toolbox](https://github.com/Kazuhito00/GrabCut-Annotation-Tool) (e.g., "masks/001.png")

- 
 $\vdots$ 

- 099_mug_blue

![vis-objects](/assets/vis-objects.png)

Tip: The first three digits specify the instance id.

The **Scenes** contains:

- easy

  - leisure\_zone

    - raw RGB images with 6144×8192 pixels (e.g. “office001/rgb\_000.jpg”)

    - bounding box annotation for objects in test scenes generated by labelImg toolbox and using PascalVOC format (e.g. “office\_001/rgb\_000.xml”)

  - meeting\_room

  - office\_002

  - pantry\_room\_002

  - sink

- hard

  - office\_001

  - pantry\_room\_001

![vis-scenes](/assets/vis-scenes.png)

Tip: Each bounding box is specified by [xmin, ymin, xmax, ymax].

The **Background** contains 200 pure background images that do not include any instances from **Objects** folder.

![vis-background](/assets/vis-background.png)

## Code

The project is built on [detectron2](https://github.com/facebookresearch/detectron2), [segment-anything](https://github.com/facebookresearch/segment-anything), and [DINOv2](https://github.com/facebookresearch/dinov2).


### Demo

The Jupyter notebooks files demonstrate our non-learned method using SAM and DINOv2. We choose light pretrained models of SAM (vit_l) and DINOv2 (dinov2_vits14) for efficiency.

## Citation

If you find our project useful, please consider citing:

```bibtex

@inproceedings{shen2023high,

        title={A high-resolution dataset for instance detection with multi-view object capture},

        author={Shen, Qianqian and Zhao, Yunhan and Kwon, Nahyun and Kim, Jeeeun and Li, Yanan and Kong, Shu},

        booktitle={NeurIPS Datasets & Benchmark Track},

        year={2023}

```