{"id":17490274,"url":"https://github.com/ryogrid/anime-illust-image-searcher","last_synced_at":"2025-04-22T19:09:10.230Z","repository":{"id":258075877,"uuid":"868452931","full_name":"ryogrid/anime-illust-image-searcher","owner":"ryogrid","description":"Anime Style Illustration Specific Image Search App with ViT Tagger x BM25/Doc2Vec","archived":false,"fork":false,"pushed_at":"2025-03-02T02:10:56.000Z","size":354,"stargazers_count":8,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-22T19:08:52.420Z","etag":null,"topics":["anime","anime-style","bm25","deep-learning","doc2vec","gensim","illustration","image-search","machine-learning","onnxruntime","python","pytorch","search-engine","streamlit","transformer","vector-search","vision-transformer"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ryogrid.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-10-06T12:26:27.000Z","updated_at":"2025-04-16T23:17:31.000Z","dependencies_parsed_at":"2024-11-08T12:21:05.716Z","dependency_job_id":"39861445-efaa-40a2-a411-bf6e15fce908","html_url":"https://github.com/ryogrid/anime-illust-image-searcher","commit_stats":{"total_commits":110,"total_committers":1,"mean_commits":110.0,"dds":0.0,"last_synced_commit":"f4b82791cc5909b3d953fd16a62ba25cb3635d57"},"previous_names":["ryogrid/anime-illust-image-searcher"],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ryogrid%2Fanime-illust-image-searcher","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ryogrid%2Fanime-illust-image-searcher/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ryogrid%2Fanime-illust-image-searcher/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ryogrid%2Fanime-illust-image-searcher/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ryogrid","download_url":"https://codeload.github.com/ryogrid/anime-illust-image-searcher/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250306639,"owners_count":21408926,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["anime","anime-style","bm25","deep-learning","doc2vec","gensim","illustration","image-search","machine-learning","onnxruntime","python","pytorch","search-engine","streamlit","transformer","vector-search","vision-transformer"],"created_at":"2024-10-19T06:06:45.699Z","updated_at":"2025-04-22T19:09:10.185Z","avatar_url":"https://github.com/ryogrid.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Anime Style Illustration Specific Image Search App with ViT Tagger x BM25/Doc2Vec\n![image](https://github.com/user-attachments/assets/3b95b3b4-db6d-483f-8bd1-8d2203c16792)\n\n## What's This?\n- Anime Style Illustration Specific Image Search App with ML Technique\n  - can be used for photos. but flexible photo search is offered by Google Photos or etc :)\n- Search capabilities of cloud photo album services towards illustration image files are poor for some reason\n- So, I wrote simple scripts\n\n## Method\n- Search Images Matching with Query Texts on Latent Semantic Representation Vector Space and with BM25\n  - Vectors are generated with embedding model: Tagger Using Visual Transformar (ViT) Internally x Doc2Vec\n  - Scores which is calculated with [BM25](https://en.wikipedia.org/wiki/Okapi_BM25) is used in combination\n  - Internal re-ranking method is also introduced\n    - Assumption: Users make queries better asymptotically according to top search results and find appropriate queries eventually\n    - If you wan to know detail of the method, please read webui.py :)\n- Doc2Vec is Mainly Used for Covering Tagging Presision\n  - Simple search logic can be implemented with BM25 only\n  - But, you can use tags to search which are difficult for tagging because the index data which is composed of vectors generated with Doc2Vec model\n    - implemented with Gensim lib\n- ( Web UI is implemented with StreamLit )\n\n## Usage\n- (collect working confirmed environment)\n  - (Windows 11 Pro 64bit 23H2 X86_64)\n  - Python 3.12.7\n  - pip 22.0.4\n- $ pip install -r requirements.txt\n- (If use you want use GPU from tagging.py, please install appropriate pytorch pypi module like below)\n  - $ pip install torch==2.5.1+cu121 torchaudio==2.5.1+cu121 torchvision==0.20.1+cu121 --index-url https://download.pytorch.org/whl/cu121\n    - CUDA version should be changed to matche with your PC environment and change index-url option for the CUDA version\n- $ python tagging.py --dir \"IMAGE FILES CONTAINED DIR PATH\"\n  - The script searches directory structure recursively :)\n  - This takes quite a while...\n    - About 1.7 sec/file at middle spec desktop PC (GPU is not used)\n      - AMD Ryzen 7 5700X 8-Core Processor 4.50 GHz\n    - About 0.5sec/file with GPU\n      - GeForce GTX 1660 SUPER\n        - Released at 2009/10\n        - VRAM: 6GB\n        - CUDA core: 1408 units\n        - Core frequrncy: 1785MHz (boost mode)\n        - Theorical peak flops (fp32): about 5.02TFLOPS\n  - Pathes and tags of image files are saved to tags-wd-tagger.txt\n- $ python genmodel.py\n  - This takes quite a while...\n- $ streamlit run webui.py\n  - Search app is opend on your web browser\n\n## Use Character Image Feture Based Reranking Mode (Optional)\n- Reranking based on similarity calculation with [Quantized CCIP(Contrastive Anime Character Image Pre-Training) model](https://huggingface.co/deepghs/ccip_onnx)\n  - When index data described below exists, this mode becomes selectable at webui.py\n- **Additional index data preparation is needed**\n  - $ python gen_cfeatures.py --dir \"IMAGE FILES CONTAINED DIR PATH\"\n    - If 'onnx-runtime-gpu' module does not work, please uninstall it and install normal 'onnx-runtime'...\n    - Best of luck!\n\n## Index Data Updating\n- When you get and store new image files, you should update index data for adding the files to be hitted at search on webui.py\n- Procedure\n  - 1 Backup all files genarated by scripts on this repo!\n    - Model files on your home directory is exception :)\n  - 2 $ python tagging.py --dir \"IMAGE FILES CONTAINED DIR PATH\" **--after \"YYYY-MM-DD\"**\n    - Param of --dir doesn't have to be changed\n    - Adding --after option is needed. Please specify date after last index data creation or update\n      - Tagging target is filtered by specified date: added date (cdate attribute) \u003c= YYYY-MM-DD\n  - 3 $ python genmodel.py --update\n  - Optional\n    - 4 $ python gen_cfeatures.py --dir \"IMAGE FILES CONTAINED DIR PATH\" **--after \"YYYY-MM-DD\"**\n  - Thats's all!\n\n## Usage (Binary Package of Windows at Release Page)\n- Same with above except that you need not to execute python and execution path (current path) is little bit different :)\n- First, unzip package and launch command prompt or PowerShell :)\n- $ cd anime-illust-image-searcher-pkg\n- $ .\\cmd_run\\cmd_run.exe tagging --dir \"IMAGE FILES CONTAINED DIR PATH\"\n- $ .\\cmd_run\\cmd_run.exe genmodel\n  - Same with above :)\n- $ .\\run_webui.exe\n  - Search app is opend on your web browser!\n\n## Tips (Attention)\n- Words (tags) which were not apeeared at tagging are not usable on query\n  - Solution\n    - Search words you want to use from taggs-wd-tagger.txt with grep, editor or something for existance checking\n    - If exist, there is no problem. If not, you should think similar words and search it in same manner :)\n- **Specifying Eath Tag Weight (format -\u003e TAG:WEIGHT, WEIGHT shoud be integer)**\n  - Examples\n    - \"girl:3 dragon\"\n    - \"girl:2 boy:3\"\n  - **Exclude tag marking**\n    - **Weight specification which starts with '-' indicates that images tagged it should be excluded**\n    - **ex: \"girl boy:-3\"**\n      - **Images tagged 'boy' are removed from results. Numerical weight value is ignored but can't be omitted :)** \n  - **Required tag marking**\n    - **Weight specification which starts with '+' indicates the tag is required**\n    - **ex: \"girl:+3 dragon\"**\n      - **Images not tagged 'girl' are removed from results**\n      - **Weight value is NOT ignored at calculation of scores**\n- **Search Result Exporting feature**\n  - You can export file paths list which is hitted at search\n  - Pressing 'Export' button saves the list as text file to path Web UI executed at\n  - File name is query text with timestamp and contents is line break delimited\n    - Some viewer tools such as [Irfan View](https://www.irfanview.com/) can load image files with passing a text file contains path list :)\n    - Irfan View can slideshow also. It's nice :)\n  - At Windows, charactor code is sjis. At other OSes, charactor code is utf-8\n- Character code of file pathes \n  - If file path contains characters which can't be converted to Unicode or utf-8, scripts may ouput error message at processing the file\n  - But, it doesn't mean that your script usage is wrong. Though these files is ignored or not displayed at Web UI :|\n    - This is problem of current implentation. When you use scripts on Windows and charactor code of directory/file names isn't utf-8, the problem may occur\n\n## Information Related to Copyrights\n- Tagger\n  - [this code](https://huggingface.co/spaces/SmilingWolf/wd-tagger/blob/main/app.py) was used as reference wheh implmenting tagger script\n  - [\"WD EVA02-Large Tagger v3\" model](https://huggingface.co/SmilingWolf/wd-eva02-large-tagger-v3) is used for image file tagging\n  - **I thank to great works of SmilingWolf**\n- Character visual similarity calculation\n  - [this code](https://huggingface.co/spaces/deepghs/ccip/blob/f7d50a4f5dd3d4681984187308d70839ff0d3f5b/ccip.py) was used as reference when implemnting model execution\n  - [Quantized CCIP(Contrastive anime Character Image Pre-training) model](https://huggingface.co/deepghs/ccip_onnx) is used\n  - **I thank to great works of deepghs community members**\n\n## For Busy People\n- **Tagging using Google Colab env !**\n  - 1 Make preprocessed data with [utility/make_tensor_files.py](./utility/make_tensor_files.py)\n  - 2 Zip the output dir\n  - 3 Upload zipped file to Google Drive\n  - 4 Use Google Colab env like [this](https://github.com/ryogrid/ryogridJupyterNotebooks/blob/master/tagging_colab-241104-T4-with-Tensor-files.ipynb)\n  - 5 Get tags-wd-tagger.txt and replace file pathes on it to be matched with your image files existing pathes :)\n  - 6 Execute genmodel.py !\n\n## TODO\n- [ ] \u003cdel\u003eSearch on latent representation generated by CLIP model\u003c/del\u003e\n  - **This method was alredy tried but precition was not good because current public available CLIP models are not fitting for anime style illust :|**\n    - If CLIP models which are fine tuned with anime style illust images are available, this method is better than current one\n- [x] Weight specifying to tags like prompt format of Stable Diffusion Web UI\n  - Current implemenataion uses all tags faialy. But there is many cases that users want to emphasize specific tags and can't get appropriate results without that!\n- [x] Fix a bug: some type of tags on tags-wd-tagger.txt can't be used on query \n- [x] Incremental index updating at image files increasing\n- [x] Similar image search with specifying a image file\n  - This is realized at 'Character Image Feture Based Reranking Mode' practically :)\n- [x] Exporting found files list feature\n  - In text file. Once you get list, many other tools and viewer you like can be used :)\n- [x] Making binary package of this app which doesn't need python environment building\n\n\n## Screenshots of Demo\n- I used about 1000 image files collected from [Irasutoya](https://www.irasutoya.com/) which offers free image materials as search target example\n  - Note: image materials of Irasutoya have restrictions at commercial purposes use\n- Partial tagging result: [./tagging_example.txt](/tagging_example.txt)\n  - Generation script was executed in Windows\n  - File paths in linked file have been partially masked \n\n\n- Search \"standing\"\n  - ![image](https://github.com/user-attachments/assets/6e324a1e-ae49-40de-9dbd-d040e153b837)\n- Search \"standing animal\"\n  - ![image](https://github.com/user-attachments/assets/cd2862e3-e7e2-42fe-b830-705c778e10b8)\n- Image info page\n  - ![image](https://github.com/user-attachments/assets/78898162-ac6d-4fdf-b806-798f2f52a8d5)\n- Slideshow feature\n  - Auto slide in 5 sec period (roop)\n  - ![image](https://github.com/user-attachments/assets/ea42336f-6b59-402b-b19e-f10610e4b200)\n  \n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fryogrid%2Fanime-illust-image-searcher","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fryogrid%2Fanime-illust-image-searcher","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fryogrid%2Fanime-illust-image-searcher/lists"}