{"id":15839365,"url":"https://github.com/mindsdb/aipdf","last_synced_at":"2025-04-09T23:20:54.503Z","repository":{"id":257809563,"uuid":"867544533","full_name":"mindsdb/aipdf","owner":"mindsdb","description":" A tool to OCR PDFs using gen-AI models","archived":false,"fork":false,"pushed_at":"2025-01-31T19:51:45.000Z","size":53,"stargazers_count":37,"open_issues_count":2,"forks_count":5,"subscribers_count":12,"default_branch":"main","last_synced_at":"2025-04-02T12:12:45.777Z","etag":null,"topics":["ai","gpt","groq","hacktoberfest","ocr","ollama","openai","pdf"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mindsdb.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-10-04T09:10:54.000Z","updated_at":"2025-03-10T12:02:23.000Z","dependencies_parsed_at":null,"dependency_job_id":"2530d436-84b0-4976-9263-351591309f0a","html_url":"https://github.com/mindsdb/aipdf","commit_stats":{"total_commits":25,"total_committers":3,"mean_commits":8.333333333333334,"dds":0.24,"last_synced_commit":"0fbf28cb76e88334be8237ecffc77cb840ae3ec5"},"previous_names":["mindsdb/aipdf"],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mindsdb%2Faipdf","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mindsdb%2Faipdf/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mindsdb%2Faipdf/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mindsdb%2Faipdf/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mindsdb","download_url":"https://codeload.github.com/mindsdb/aipdf/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248126333,"owners_count":21051905,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","gpt","groq","hacktoberfest","ocr","ollama","openai","pdf"],"created_at":"2024-10-05T16:04:15.216Z","updated_at":"2025-04-09T23:20:54.483Z","avatar_url":"https://github.com/mindsdb.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# AIPDF: Minimalistic PDF to Markdown (and others), with GPT-like Multimodal Models\n\nAIPDF is a stand-alone, minimalistic, yet powerful pure Python library that leverages multi-modal gen AI models (OpenAI, llama3 or compatible alternatives) to extract data from PDFs and convert it into various formats such as Markdown or JSON. \n\n## Installation\n\n```bash\npip install aipdf\n```\n\nin macOS you will need to install poppler\n```bash\nbrew install poppler \n```\n\n## Quick Start\n\n\n\n```python\nfrom aipdf import ocr\n\n# Your OpenAI API key   \napi_key = 'your_openai_api_key'\n\nfile = open('somepdf.pdf', 'rb')\nmarkdown_pages = ocr(file, api_key)\n\n```\n\n##  Ollama\n\nYou can use with any ollama multi-modal models \n\n```python\nocr(pdf_file, api_key='ollama', model=\"llama3.2\", base_url= 'http://localhost:11434/v1', prompt=...)\n```\n## Any file system\n\nWe chose that you pass a file object, because that way it is flexible for you to use this with any type of file system, s3, localfiles, urls etc\n\n### From url\n```python\n\npdf_file = io.BytesIO(requests.get('https://arxiv.org/pdf/2410.02467').content)\n\n# extract\npages = ocr(pdf_file, api_key, prompt=\"extract tables, return each table in json\")\n\n```\n### From S3\n\n```python\n\ns3 = boto3.client('s3', config=Config(signature_version='s3v4'),\n                  aws_access_key_id=access_token,\n                  aws_secret_access_key='', # Not needed for token-based auth\n                  aws_session_token=access_token)\n\n\npdf_file = io.BytesIO(s3.get_object(Bucket=bucket_name, Key=object_key)['Body'].read())\n# extract \npages = ocr(pdf_file, api_key, prompt=\"extract charts data, turn it into tables that represent the variables in the chart\")\n```\n\n\n## Why AIPDF?\n\n1. **Simplicity**: AIPDF provides a straightforward function, it requires minimal setup, dependencies and configuration.\n2. **Flexibility**: Extract data into Markdown, JSON, HTML, YAML, whatever... file format and schema.\n3. **Power of AI**: Leverages state-of-the-art multi modal models (gpt, llama, ..).\n4. **Customizable**: Tailor the extraction process to your specific needs with custom prompts.\n5. **Efficient**: Utilizes parallel processing for faster extraction of multi-page PDFs.\n\n## Requirements\n\n- Python 3.7+\n\nWe will keep this super clean, only 3 required libraries:\n\n- openai library to talk to completion endpoints\n- pdf2image library (for PDF to image conversion)\n- Pillow (PIL) library\n\n## License\n\nThis project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.\n\n## Contributing\n\nContributions are welcome! Please feel free to submit a Pull Request.\n\n## Support\n\nIf you encounter any problems or have any questions, please open an issue on the GitHub repository.\n\n---\n\nAIPDF makes PDF data extraction simple, flexible, and powerful. Try it out and simplify your PDF processing workflow today!\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmindsdb%2Faipdf","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmindsdb%2Faipdf","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmindsdb%2Faipdf/lists"}