{"id":24647137,"url":"https://github.com/oztrkoguz/visquerypdf","last_synced_at":"2025-03-20T16:22:58.561Z","repository":{"id":246633925,"uuid":"821666709","full_name":"oztrkoguz/VisQueryPDF","owner":"oztrkoguz","description":"It automatically describes images in PDF files and generates questions from these descriptions. With its advanced RAG structure, it directs these questions directly to PDF text content, providing comprehensive information extraction and analysis.","archived":false,"fork":false,"pushed_at":"2024-06-29T06:57:15.000Z","size":1638,"stargazers_count":11,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-01-25T15:15:08.460Z","etag":null,"topics":["agent","clip","langchain","langchain-agent","langchain-python","langchain-rag","llm-agent","multi-agent-systems","ollama"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/oztrkoguz.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-06-29T05:17:51.000Z","updated_at":"2024-10-29T14:54:23.000Z","dependencies_parsed_at":"2024-06-29T09:22:06.521Z","dependency_job_id":"d0cebc5c-58f8-4d81-8a89-f6ac197b9381","html_url":"https://github.com/oztrkoguz/VisQueryPDF","commit_stats":null,"previous_names":["oztrkoguz/visquerypdf"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oztrkoguz%2FVisQueryPDF","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oztrkoguz%2FVisQueryPDF/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oztrkoguz%2FVisQueryPDF/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oztrkoguz%2FVisQueryPDF/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/oztrkoguz","download_url":"https://codeload.github.com/oztrkoguz/VisQueryPDF/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244647506,"owners_count":20487093,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["agent","clip","langchain","langchain-agent","langchain-python","langchain-rag","llm-agent","multi-agent-systems","ollama"],"created_at":"2025-01-25T15:15:10.704Z","updated_at":"2025-03-20T16:22:58.540Z","avatar_url":"https://github.com/oztrkoguz.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# VisQueryPDF\n### The aim of the project is to verify the alignment between images and texts in documents.\n### 2 different methods were discussed.\n## 1.Method \n\n![yontem1](https://github.com/oztrkoguz/VisQueryPDF/assets/101019436/65b62ab9-c98a-44db-bee1-abd71e6d0714)\n\nImages automatically extracted from the document were described using a VLM agent structure. Using the description results, questions were generated with a question generation agent. Subsequently, these questions were posed to the document using the RAG system, and answers were verified.\n\n## 2.Method\n\n![Adsz-2024-06-29-0711](https://github.com/oztrkoguz/VisQueryPDF/assets/101019436/2a0dd56e-8839-446c-b42b-3758c577cf86)\n\nImages and texts are automatically extracted from the document. Text data undergoes processing using a summarization agent to obtain a concise summary. Subsequently, embeddings of images and texts are extracted using the CLIP model, and their similarities are compared.\n\n### The first method achieved a similarity rate of 60%, whereas the other method showed similarities around 33%.\n\n## Usage\n```\ngit clone https://github.com/oztrkoguz/VisQueryPDF.git\ncd VisQueryPDF\npython main.py\n```\n## Requirements\n```\nPython \u003e 3.10\nlangchain==0.2.6\nlangchain-chroma==0.1.1\nlangchain-community==0.0.38\nlangchain-core==0.1.52\nlangchain-openai==0.0.5\nlangchain-text-splitters==0.2.1\nlangsmith==0.1.82\nollama==0.2.1\n\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foztrkoguz%2Fvisquerypdf","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Foztrkoguz%2Fvisquerypdf","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foztrkoguz%2Fvisquerypdf/lists"}