https://github.com/ChengShiest/Zip-Your-CLIP

https://github.com/ChengShiest/Zip-Your-CLIP

Last synced: about 2 months ago
JSON representation

• Host: GitHub
• URL: https://github.com/ChengShiest/Zip-Your-CLIP
• Owner: ChengShiest
• License: apache-2.0
• Created: 2023-09-30T02:57:01.000Z (over 1 year ago)
• Default Branch: main
• Last Pushed: 2024-03-14T07:26:23.000Z (11 months ago)
• Last Synced: 2024-03-15T05:30:43.726Z (11 months ago)
• Language: Jupyter Notebook
• Size: 29.1 MB
• Stars: 1
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE.txt

Awesome Lists containing this project

• Awesome-Segment-Anything - [code

README

        





  
Zip-Your-CLIP

  


	The Devil is in the Object Boundary: Towards Annotation-free Instance Segmentation using Foundation Models

	Zip: CLIP Itself is a Good Object-detector

    ICLR, 2024

    


    Cheng Shi

    ·

    Sibei Yang†

  


  


    

      

  




![image-20230924124604776](__assets__/fig1.png)

## Installation

The code requires `python>=3.8`, as well as `pytorch>=1.7` and `torchvision>=0.8`. Please follow the instructions [here](https://pytorch.org/get-started/locally/) to install both PyTorch and TorchVision dependencies.

Install Segment Anything:

```

pip install git+https://github.com/facebookresearch/segment-anything.git

```

Install CLIP and CLIP_Surgery

```

pip install git+https://github.com/openai/CLIP.git

git clone https://github.com/xmed-lab/CLIP_Surgery.git

```

Make sure that you correctly install the dependencies of CLIP_Surgery

```

### Test

device = 'cuda' if torch.cuda.is_available() else 'cpu'

model, preprocess = clip.load("CS-ViT-B/16", device=device)

```

## Getting Started

![image-20230924124604777](__assets__/fig2.png)

## License

This Project is licensed under the [Apache 2.0 license](__assets__/LICENSE.txt).

## Citation

```

@inproceedings{

shi2024the,

title={The Devil is in the Object Boundary: Towards Annotation-free Instance Segmentation using Foundation Models},

author={Cheng Shi and Sibei Yang},

booktitle={The Twelfth International Conference on Learning Representations},

year={2024},

url={https://openreview.net/forum?id=4JbrdrHxYy}

}

```