{"id":25053304,"url":"https://github.com/Goldziher/kreuzberg","last_synced_at":"2025-10-21T09:30:27.631Z","repository":{"id":275266270,"uuid":"925434317","full_name":"Goldziher/kreuzberg","owner":"Goldziher","description":"Document intelligence framework for Python - Extract text, metadata, and structured data   from PDFs, images, Office documents, and more. Built on Pandoc, PDFium, and Tesseract.","archived":false,"fork":false,"pushed_at":"2025-10-15T19:34:00.000Z","size":284571,"stargazers_count":2435,"open_issues_count":6,"forks_count":103,"subscribers_count":10,"default_branch":"main","last_synced_at":"2025-10-15T21:08:43.392Z","etag":null,"topics":["async","document-intelligence","mcp","metadata-extraction","ocr","pandoc","pdf-extraction","pdfium","python","rag","table-extraction","tesseract","text-extraction"],"latest_commit_sha":null,"homepage":"https://kreuzberg.dev/","language":"HTML","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Goldziher.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"docs/contributing.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-01-31T21:50:02.000Z","updated_at":"2025-10-15T18:02:13.000Z","dependencies_parsed_at":"2025-02-15T14:18:25.463Z","dependency_job_id":"93cfc0b9-3a30-45d0-8222-4ad5ef52bfe3","html_url":"https://github.com/Goldziher/kreuzberg","commit_stats":null,"previous_names":["goldziher/kreuzberg"],"tags_count":66,"template":false,"template_full_name":null,"purl":"pkg:github/Goldziher/kreuzberg","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Goldziher%2Fkreuzberg","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Goldziher%2Fkreuzberg/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Goldziher%2Fkreuzberg/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Goldziher%2Fkreuzberg/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Goldziher","download_url":"https://codeload.github.com/Goldziher/kreuzberg/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Goldziher%2Fkreuzberg/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279361415,"owners_count":26155579,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-17T02:00:07.504Z","response_time":56,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["async","document-intelligence","mcp","metadata-extraction","ocr","pandoc","pdf-extraction","pdfium","python","rag","table-extraction","tesseract","text-extraction"],"created_at":"2025-02-06T11:02:04.111Z","updated_at":"2025-10-21T09:30:22.620Z","avatar_url":"https://github.com/Goldziher.png","language":"HTML","funding_links":[],"categories":["📚 Projects (1974 total)","HTML","Python","Uncategorized"],"sub_categories":["MCP Servers","Uncategorized"],"readme":"# Kreuzberg\n\nKreuzberg is a library for simplified text extraction from PDF files. It's meant to offer simple, hassle free text\nextraction.\n\nWhy?\n\nI am building, like many do now, a RAG focused service (checkout https://grantflow.ai). I have text extraction needs.\nThere are quite a lot of commercial options out there, and several open-source + paid options.\nBut I wanted something simple, which does not require expansive round-trips to an external API.\nFurthermore, I wanted something that is easy to run locally and isn't very heavy / requires a GPU.\n\nHence, this library.\n\n## Features\n\n- Extract text from PDFs, images, office documents and more (see supported formats below)\n- Use modern Python with async (via `anyio`) and proper type hints\n- Extensive error handling for easy debugging\n\n## Installation\n\n1. Begin by installing the python package:\n\n   ```shell\n\n   pip install kreuzberg\n\n   ```\n\n2. Install the system dependencies:\n\n- [pandoc](https://pandoc.org/installing.html) (non-pdf text extraction, GPL v2.0 licensed but used via CLI only)\n- [tesseract-ocr](https://tesseract-ocr.github.io/) (for image/PDF OCR, Apache License)\n\n## Dependencies and Philosophy\n\nThis library is built to be minimalist and simple. It also aims to utilize OSS tools for the job. Its fundamentally a\nhigh order async abstraction on top of other tools, think of it like the library you would bake in your code base, but\npolished and well maintained.\n\n### Dependencies\n\n- PDFs are processed using pdfium2 for searchable PDFs + Tesseract OCR for scanned documents\n- Images are processed using Tesseract OCR\n- Office documents and other formats are processed using Pandoc\n- PPTX files are converted using python-pptx\n- HTML files are converted using html-to-markdown\n- Plain text files are read directly with appropriate encoding detection\n\n### Roadmap\n\nV1:\n\n- [x] - html file text extraction\n- [ ] - better PDF table extraction\n- [ ] - TBD\n\nV2:\n\n- [ ] - extra install groups (to make dependencies optional)\n- [ ] - metadata extraction (possible breaking change)\n- [ ] - TBD\n\n### Feature Requests\n\nFeel free to open a discussion in GitHub or an issue if you have any feature requests\n\n### Contribution\n\nIs welcome! Read guidelines below.\n\n## Supported File Types\n\nKreuzberg supports a wide range of file formats:\n\n### Document Formats\n\n- PDF (`.pdf`) - both searchable and scanned documents\n- Word Documents (`.docx`, `.doc`)\n- Power Point Presentations (`.pptx`)\n- OpenDocument Text (`.odt`)\n- Rich Text Format (`.rtf`)\n\n### Image Formats\n\n- JPEG, JPG (`.jpg`, `.jpeg`, `.pjpeg`)\n- PNG (`.png`)\n- TIFF (`.tiff`, `.tif`)\n- BMP (`.bmp`)\n- GIF (`.gif`)\n- WebP (`.webp`)\n- JPEG 2000 (`.jp2`, `.jpx`, `.jpm`, `.mj2`)\n- Portable Anymap (`.pnm`)\n- Portable Bitmap (`.pbm`)\n- Portable Graymap (`.pgm`)\n- Portable Pixmap (`.ppm`)\n\n#### Text and Markup Formats\n\n- HTML (`.html`, `.htm`)\n- Plain Text (`.txt`)\n- Markdown (`.md`)\n- reStructuredText (`.rst`)\n- LaTeX (`.tex`)\n\n#### Data Formats\n\n- Comma-Separated Values (`.csv`)\n- Tab-Separated Values (`.tsv`)\n\n## Usage\n\nKreuzberg exports two async functions:\n\n- Extract text from a file (string path or `pathlib.Path`) using `extract_file()`\n- Extract text from a byte-string using `extract_bytes()`\n\n### Extract from File\n\n```python\nfrom pathlib import Path\nfrom kreuzberg import extract_file\n\n\n# Extract text from a PDF file\nasync def extract_pdf():\n    result = await extract_file(\"document.pdf\")\n    print(f\"Extracted text: {result.content}\")\n    print(f\"Output mime type: {result.mime_type}\")\n\n\n# Extract text from an image\nasync def extract_image():\n    result = await extract_file(\"scan.png\")\n    print(f\"Extracted text: {result.content}\")\n\n\n# or use Path\n\nasync def extract_pdf():\n    result = await extract_file(Path(\"document.pdf\"))\n    print(f\"Extracted text: {result.content}\")\n    print(f\"Output mime type: {result.mime_type}\")\n```\n\n### Extract from Bytes\n\n```python\nfrom kreuzberg import extract_bytes\n\n\n# Extract text from PDF bytes\nasync def process_uploaded_pdf(pdf_content: bytes):\n    result = await extract_bytes(pdf_content, mime_type=\"application/pdf\")\n    return result.content\n\n\n# Extract text from image bytes\nasync def process_uploaded_image(image_content: bytes):\n    result = await extract_bytes(image_content, mime_type=\"image/jpeg\")\n    return result.content\n```\n\n### Forcing OCR\n\nWhen extracting a PDF file or bytes, you might want to force OCR - for example, if the PDF includes images that have text that should be extracted etc.\nYou can do this by passing `force_ocr=True`:\n\n```python\nfrom kreuzberg import extract_bytes\n\n\n# Extract text from PDF bytes and force OCR\nasync def process_uploaded_pdf(pdf_content: bytes):\n    result = await extract_bytes(pdf_content, mime_type=\"application/pdf\", force_ocr=True)\n    return result.content\n```\n\n### Error Handling\n\nKreuzberg raises two exception types:\n\n#### ValidationError\n\nRaised when there are issues with input validation:\n\n- Unsupported mime types\n- Undetectable mime types\n- Path doesn't point at an exist file\n\n#### ParsingError\n\nRaised when there are issues during the text extraction process:\n\n- PDF parsing failures\n- OCR errors\n- Pandoc conversion errors\n\n```python\nfrom kreuzberg import extract_file\nfrom kreuzberg.exceptions import ValidationError, ParsingError\n\n\nasync def safe_extract():\n    try:\n        result = await extract_file(\"document.doc\")\n        return result.content\n    except ValidationError as e:\n        print(f\"Validation error: {e.message}\")\n        print(f\"Context: {e.context}\")\n    except ParsingError as e:\n        print(f\"Parsing error: {e.message}\")\n        print(f\"Context: {e.context}\")  # Contains detailed error information\n```\n\nBoth error types include helpful context information for debugging:\n\n```python\ntry:\n    result = await extract_file(\"scanned.pdf\")\nexcept ParsingError as e:\n# e.context might contain:\n# {\n#    \"file_path\": \"scanned.pdf\",\n#    \"error\": \"Tesseract OCR failed: Unable to process image\"\n# }\n```\n\n### ExtractionResult\n\nAll extraction functions return an ExtractionResult named tuple containing:\n\n- `content`: The extracted text as a string\n- `mime_type`: The mime type of the output (either \"text/plain\" or, if pandoc is used- \"text/markdown\")\n\n```python\nfrom kreuzberg import ExtractionResult\n\n\nasync def process_document(path: str) -\u003e str:\n    result: ExtractionResult = await extract_file(path)\n    return result.content\n\n\n# or access the result as tuple\n\nasync def process_document(path: str) -\u003e str:\n    content, mime_type = await extract_file(path)\n    # do something with mime_type\n    return content\n```\n\n## Contribution\n\nThis library is open to contribution. Feel free to open issues or submit PRs. Its better to discuss issues before\nsubmitting PRs to avoid disappointment.\n\n### Local Development\n\n1. Clone the repo\n2. Install the system dependencies\n3. Install the full dependencies with `uv sync`\n4. Install the pre-commit hooks with:\n   ```shell\n   pre-commit install \u0026\u0026 pre-commit install --hook-type commit-msg\n   ```\n5. Make your changes and submit a PR\n\n## License\n\nThis library uses the MIT license.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FGoldziher%2Fkreuzberg","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FGoldziher%2Fkreuzberg","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FGoldziher%2Fkreuzberg/lists"}