{"id":24082913,"url":"https://github.com/parthapray/document-image-matching-system","last_synced_at":"2026-07-18T17:34:09.023Z","repository":{"id":269673191,"uuid":"908130109","full_name":"ParthaPRay/Document-Image-Matching-System","owner":"ParthaPRay","description":"This repo contains implementation of searching documents in book form from a library by using an image by using openAI api","archived":false,"fork":false,"pushed_at":"2026-04-26T07:54:37.000Z","size":9604,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-07-18T17:34:07.523Z","etag":null,"topics":["book","document","gpt5","gradio","image","large-language-models","library","library-automation","open-library","openai-api","openai-vision","openlibrary-api","search","searching"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ParthaPRay.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2024-12-25T08:04:32.000Z","updated_at":"2026-04-26T07:54:41.000Z","dependencies_parsed_at":"2025-08-24T00:04:13.457Z","dependency_job_id":"4b1c5457-3e23-43fa-bd68-8f65beab7e17","html_url":"https://github.com/ParthaPRay/Document-Image-Matching-System","commit_stats":null,"previous_names":["parthapray/-document-image-matching-system"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/ParthaPRay/Document-Image-Matching-System","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ParthaPRay%2FDocument-Image-Matching-System","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ParthaPRay%2FDocument-Image-Matching-System/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ParthaPRay%2FDocument-Image-Matching-System/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ParthaPRay%2FDocument-Image-Matching-System/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ParthaPRay","download_url":"https://codeload.github.com/ParthaPRay/Document-Image-Matching-System/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ParthaPRay%2FDocument-Image-Matching-System/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35626138,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-07-18T02:00:07.223Z","response_time":61,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["book","document","gpt5","gradio","image","large-language-models","library","library-automation","open-library","openai-api","openai-vision","openlibrary-api","search","searching"],"created_at":"2025-01-09T23:54:52.079Z","updated_at":"2026-07-18T17:34:09.018Z","avatar_url":"https://github.com/ParthaPRay.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 📚 Document–Image Matching System for Open Library using GPT 5.4\n\nAI-powered **Image → Main Entity Identification → Open Library Book Retrieval** using OpenAI Vision Models and the Open Library API.\n\nDeveloped by **Partha Pratim Ray**  \n📧 parthapratimray1986@gmail.com  \n📧 ppray@cus.ac.in\n\n---\n\n## 🚀 Project Overview\n\nThis project is an intelligent multimodal retrieval system that accepts an uploaded image and automatically identifies the **single most important searchable entity** from that image (such as a famous person, book title, monument, historical event, scientific concept, cultural object, or academic subject).\n\nThe system then converts that entity into a clean bibliographic search query and retrieves the most relevant books from the **Open Library API**, presenting results in a structured tabular format along with detailed API timing analysis.\n\nAdditionally, all results are automatically saved into a local `.csv` file for future research analysis, benchmarking, and documentation.\n\nThis system is especially useful for:\n\n- Academic demonstrations\n- Research prototypes\n- Digital library systems\n- Educational retrieval systems\n- Multimodal search experiments\n- AI-assisted bibliographic discovery\n\n---\n\n## 🔍 Core Workflow\n\n```text\nImage Upload\n   ↓\nImage Encoding (Base64)\n   ↓\nAI Vision Analysis (GPT-5.4-mini)\n   ↓\nMain Entity Detection\n   ↓\nTextual Description Generation\n   ↓\nKeyword Extraction (GPT-5.4-nano)\n   ↓\nBibliographic Query Generation\n   ↓\nOpen Library Search API\n   ↓\nBook Retrieval\n   ↓\nResult Display + CSV Logging\n---\n\n# 🧠 Model Strategy\n\n| Task           | Model/API Used   |\n| -------------- | ---------------- |\n| Image Analysis | GPT-5.4-mini     |\n| Query Cleaning | GPT-5.4-nano     |\n| Book Retrieval | Open Library API |\n| User Interface | Gradio           |\n| Result Storage | CSV + Pandas     |\n\n---\n\n# ✨ Key Features\n\n## 1. Intelligent Image Understanding\n\nInstead of generating generic image descriptions, the system identifies:\n\n* Famous personalities\n* Authors\n* Historical figures\n* Religious figures\n* Monuments\n* Artworks\n* Scientific concepts\n* Cultural themes\n* Academic subjects\n* Book titles\n* Recognizable named entities\n\nThis significantly improves search quality.\n\n---\n\n## 2. Strict Entity-Based Retrieval\n\nThe system avoids vague descriptions such as:\n\n❌ elderly man\n❌ black-and-white photo\n❌ traditional dress\n❌ seated person\n\nand prioritizes:\n\n✅ C. V. Raman\n✅ Buddha\n✅ White Tiger\n✅ Bharatanatyam\n✅ Taj Mahal\n\nThis improves Open Library matching accuracy.\n\n---\n\n## 3. Structured Open Library Search\n\nRetrieved metadata includes:\n\n* Title\n* Author\n* First Publish Year\n* Edition Count\n* Languages\n* ISBN\n* Open Library Work ID\n\n---\n\n## 4. Automatic CSV Logging\n\nEvery processed image is permanently saved in:\n\n```text\nimage_open_library_results.csv\n```\n\nThis supports:\n\n* research reproducibility\n* performance benchmarking\n* large-scale experiments\n* longitudinal evaluation\n\n---\n\n## 5. API Performance Benchmarking\n\nThe system records:\n\n* Image Encoding Time\n* Vision API Time\n* Keyword API Time\n* Open Library API Time\n* Internal Processing Delay\n* Total End-to-End Processing Time\n\nThis enables system-level performance analysis.\n\n---\n\n# 📦 Required Installation\n\nInstall dependencies using:\n\n```bash\npip install openai gradio pandas requests python-dotenv -q\n```\n\n---\n\n# 🔐 OpenAI API Setup (Google Colab)\n\nThis project is designed for **Google Colab**.\n\nStore your API key securely using:\n\n```python\nfrom google.colab import userdata\n```\n\nAdd your OpenAI key inside:\n\n```text\nColab Secrets → OPENAI_API_KEY\n```\n\nThe system automatically loads:\n\n```python\napi_key = userdata.get(\"OPENAI_API_KEY\")\n```\n\n---\n\n# ▶️ How to Run\n\n## Step 1\n\nOpen the notebook in **Google Colab**\n\n---\n\n## Step 2\n\nInstall required packages\n\n```bash\npip install openai gradio pandas requests python-dotenv -q\n```\n\n---\n\n## Step 3\n\nAdd your OpenAI API Key to Colab Secrets\n\n```text\nOPENAI_API_KEY\n```\n\n---\n\n## Step 4\n\nRun all cells\n\n---\n\n## Step 5\n\nUpload an image and click:\n\n```text\n🚀 Analyze Image and Find Books\n```\n\n---\n\n## Step 6\n\nView:\n\n* AI Analysis Result\n* Open Library Results\n* Timing Summary\n* Saved CSV File\n\n---\n\n# 📊 Example Output\n\n## Image Analysis\n\n```text\nMain Entity: C. V. Raman\nConfidence: High\nReason: The portrait closely matches the well-known Indian physicist C. V. Raman in formal attire and turban.\n```\n\n---\n\n## Final Open Library Search Query\n\n```text\nC. V. Raman\n```\n\n---\n\n## Open Library Results\n\n| Title                   | Author  | Year |\n| ----------------------- | ------- | ---- |\n| The Life of C. V. Raman | Various | 1985 |\n\n---\n\n## Timing Summary\n\n```text\nVision API Time: 1.53 seconds\nKeyword API Time: 0.76 seconds\nOpen Library API Time: 0.41 seconds\nTotal Processing Time: 2.71 seconds\n```\n\n---\n\n# 🎯 Research Significance\n\nThis work demonstrates a practical bridge between:\n\n## Computer Vision\n\nand\n\n## Bibliographic Information Retrieval\n\nusing lightweight LLM-based semantic reasoning.\n\nIt is especially valuable for:\n\n* Digital Humanities\n* Smart Libraries\n* Educational AI Systems\n* Visual Knowledge Retrieval\n* AI-powered Cataloguing\n* Library Science Automation\n\n---\n\n# 🖥️ Interface Design\n\nThe system uses a professionally designed **Gradio Interface** with:\n\n* sober academic color scheme\n* structured workflow display\n* wide result tables\n* clean result visualization\n* CSV download support\n* professional research presentation aesthetics\n\nSuitable for:\n\n* conferences\n* thesis demonstrations\n* congress presentations\n* research exhibitions\n\n---\n\n# 🔬 Future Improvements\n\nPossible future extensions include:\n\n* multi-book relevance ranking\n* PDF retrieval integration\n* Google Books API support\n* CrossRef API support\n* Semantic Scholar integration\n* OCR for scanned documents\n* multilingual search support\n* domain-specific academic retrieval\n* local vector database integration\n* RAG-based scholarly retrieval\n\n---\n\n# 👨‍💻 Author\n\n## Dr. Partha Pratim Ray\n\nAssistant Professor (Stage II)\nDepartment of Computer Applications\nSchool of Physical Sciences\nSikkim University\n\nResearch Areas:\n\n* Internet of AI Things (IoAT)\n* Edge Computing\n* Large Language Models\n* Multimodal Retrieval Systems\n* Generative AI\n* Pervasive Biomedical Informatics\n\nRecognitions:\n\n* Stanford Top 2% Scientist\n* Fellow of IETE\n* Senior Member of INAE\n\n---\n\n# 📜 License\n\nThis project is developed for:\n\n## Academic Research and Educational Demonstration\n\nFeel free to use, extend, and cite for non-commercial academic purposes.\n\n---\n\n# ⭐ If You Like This Project\n\nPlease consider:\n\n* starring the repository\n* citing the work\n* sharing with researchers\n* extending the system for your domain\n\n---\n\n# धन्यवाद\n\nBuilding intelligent bridges between images and knowledge.\n\n```\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fparthapray%2Fdocument-image-matching-system","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fparthapray%2Fdocument-image-matching-system","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fparthapray%2Fdocument-image-matching-system/lists"}