https://github.com/joisino/clear

A fully user-side image search engine. Accepted to CIKM 2022 demo track.
https://github.com/joisino/clear

demo flickr search search-engine

Last synced: 6 days ago
JSON representation

A fully user-side image search engine. Accepted to CIKM 2022 demo track.

• Host: GitHub
• URL: https://github.com/joisino/clear
• Owner: joisino
• License: mit
• Created: 2022-06-17T02:11:43.000Z (about 2 years ago)
• Default Branch: main
• Last Pushed: 2022-08-18T02:18:39.000Z (almost 2 years ago)
• Last Synced: 2024-02-29T23:33:43.338Z (4 months ago)
• Topics: demo, flickr, search, search-engine
• Language: JavaScript
• Homepage: https://clear.joisino.net/
• Size: 1020 KB
• Stars: 248
• Watchers: 4
• Forks: 23
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Lists

• awesome-stars - joisino/clear - A fully user-side image search engine. Accepted to CIKM 2022 demo track. (JavaScript)

README

        


  



A fully user-side image search engine.


## ✨ Highlights

* Flickr does not provide an official similar image search engine nor corresponding API. Nevertheless, CLEAR realizes it on a user-side.

* CLEAR runs totally on a client side. It does not use a backend server at all.

* CLEAR does not store any images nor build search indices, while traditional search systems require to build search indices.

## 📜 Paper

The paper describes CLEAR in detail.

arXiv: https://arxiv.org/abs/2206.08521

## 💻 Online Demo

https://clear.joisino.net/



Interface of CLEAR. Upload an image, and then CLEAR retrieves similar images from Flickr. The functionality of CLEAR is simple. The highlight lies rather in how it is realized and how easy the deployment is.

## 💿 Run your own CLEAR

```

$ git clone [email protected]:joisino/clear.git

$ cd clear

$ echo 'REACT_APP_FLICKR_KEY="0123456789abcdef"' > .env.local

$ npm install

$ npm start

```

Or you can try it with Docker.

```

$ git clone [email protected]:joisino/clear.git

$ cd clear

$ echo 'REACT_APP_FLICKR_KEY="0123456789abcdef"' > .env.local

$ docker run -it --rm -v $PWD:/app -p 13000:3000 -w /app -u node node /bin/bash -c "npm install && npm start"

```

Replace `0123456789abcdef` with a Flickr API key.

You can get a key at https://www.flickr.com/services/api/misc.api_keys.html. It takes a few minutes to get a key.

## 🛠️ Customize CLEAR

The feature extractor and score function are defined in `src/score.js`.

* `getFeature` computes feature vectors for both the source image and retrieved images. The embedding layer is defined in `embeddingName`. You can try other layers, e.g., `'module_apply_default/MobilenetV2/Logits/AvgPool'`.

* `embs2score` computes scores. The higher the better. You can try other functions, e.g., the Gaussian kernel `Math.exp(- emb1.squaredDifference(emb2).sum().dataSync()[0] / 1000)`.

As CLEAR does not rely on any backend servers or search indices, you can seamlessly use the system after you change the score function.

You can also change the search target from Flickr to other services by writing a wrapper in `src/flickr.js`.

## 🖋️ Citation

```

@inproceedings{sato2022clear,

  author    = {Ryoma Sato},

  title     = {CLEAR: A Fully User-side Image Search System},

  booktitle = {The 31st {ACM} International Conference on Information

               and Knowledge Management, {CIKM}},

  year      = {2022},

}

```