{"id":26541383,"url":"https://github.com/revankumard/llamarker","last_synced_at":"2026-02-07T00:37:57.253Z","repository":{"id":268548071,"uuid":"904699942","full_name":"RevanKumarD/LlaMarker","owner":"RevanKumarD","description":"Your ultimate tool for effortlessly converting and parsing documents into clean, well-structured Markdown—fast, reliable, and 100% local! 💻✨","archived":false,"fork":false,"pushed_at":"2025-01-19T16:55:44.000Z","size":6561,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-20T17:04:31.464Z","etag":null,"topics":["document-parser","llama-ai","llamarker","local-parsing-tool","marker"],"latest_commit_sha":null,"homepage":"https://www.being-rev.me","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/RevanKumarD.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-12-17T11:44:01.000Z","updated_at":"2025-01-19T16:55:20.000Z","dependencies_parsed_at":null,"dependency_job_id":"76d59889-38a1-42fc-85db-376f96bd3d81","html_url":"https://github.com/RevanKumarD/LlaMarker","commit_stats":null,"previous_names":["revankumard/revparser","revankumard/llamarker"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RevanKumarD%2FLlaMarker","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RevanKumarD%2FLlaMarker/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RevanKumarD%2FLlaMarker/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RevanKumarD%2FLlaMarker/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/RevanKumarD","download_url":"https://codeload.github.com/RevanKumarD/LlaMarker/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244890827,"owners_count":20527155,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["document-parser","llama-ai","llamarker","local-parsing-tool","marker"],"created_at":"2025-03-22T01:20:22.303Z","updated_at":"2026-02-07T00:37:57.221Z","avatar_url":"https://github.com/RevanKumarD.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\" style=\"margin: 0; padding: 0;\"\u003e\n  \u003cimg src=\"llamarker/assets/Llamarker_logo.png\" alt=\"LlaMarker Logo\" width=\"200\"\u003e\n\u003c/p\u003e\n\n\u003ch1 align=\"center\"\u003e🖍️ LlaMarker\u003c/h1\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003cb\u003eYour go-to tool for converting and parsing documents into clean, well-structured Markdown!\u003c/b\u003e\u003cbr\u003e\n  \u003ci\u003eFast, intuitive, and entirely local 💻🚀.\u003c/i\u003e\n\u003c/p\u003e\n\n\u003cdiv align=\"center\"\u003e\n  \u003cspan\u003e\n    \u003ca href=\"https://www.python.org/downloads/release/python-380/\"\u003e\n      \u003cimg src=\"https://img.shields.io/badge/Python-3.10%2B-blue.svg\" alt=\"Python Versions\"\u003e\n    \u003c/a\u003e\n  \u003c/span\u003e\n  \u003cspan\u003e\n    \u003ca href=\"https://github.com/VikParuchuri/marker\"\u003e\n      \u003cimg src=\"https://img.shields.io/badge/License-Refer%20Marker%20Repo-lightgrey.svg\" alt=\"License\"\u003e\n    \u003c/a\u003e\n  \u003c/span\u003e\n  \u003cspan\u003e\n    \u003ca href=\"https://pypi.org/project/llamarker/\"\u003e\n      \u003cimg src=\"https://img.shields.io/pypi/v/llamarker\" alt=\"PyPI version\"\u003e\n    \u003c/a\u003e\n  \u003c/span\u003e\n\u003c/div\u003e\n\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"llamarker/assets/llamarker_demo.gif\" alt=\"LlaMarker Demo\" width=\"1200\"\u003e\n\u003c/p\u003e\n\n\n## ✨ Key Features\n\n- ✨ **All-in-One Parsing**  \n  Supports **TXT**, **DOCX**, **PDF**, **PPT**, **XLSX**, and more—even processes images inside documents.\n\n- 🖼️ **Visual Content Extraction**  \n  Utilizes **Llama 3.2 Vision** to detect images, tables, charts, and diagrams, converting them into rich Markdown.\n\n- 🏗️ **Built with Marker**  \n  Extends the open-source [Marker](https://github.com/VikParuchuri/marker) parser to handle complex content types **locally**.\n\n- 🛡️ **Local-First Privacy**  \n  No cloud, no external servers—**all processing** happens on your machine.\n\n---\n\n## 🚀 How It Works\n\n1. **Parsing \u0026 Conversion**\n\n   - Parses and converts multiple file types (.txt, .docx, .pdf, .ppt, .xlsx, etc.) into Markdown.\n   - Leverages **Marker** for accurate and efficient parsing of both text and visual elements.\n   - Extracts images, charts, and tables, embedding them in Markdown.\n   - _(Optional)_ Converts documents into PDFs using **LibreOffice** for easy viewing.\n\n2. **Visual Analysis**\n\n   - Distinguishes logos from content-rich images.\n   - Extracts and preserves the original language from images.\n   - Uses multiple agents to extract useful information from the images.\n\n3. **Fast \u0026 Efficient**\n\n   - Supports parallel processing for faster handling of large folders.\n\n4. **Streamlit GUI**\n   - A user-friendly interface to upload and parse files (or multiple files at once!) or entire directories.\n   - Download results directly from the GUI.\n\n---\n\n## 📑 Table of Contents\n\n1. [Features](#features)\n2. [Prerequisites](#prerequisites)\n3. [Installation Options](#installation-options)\n   - [Install via PyPI](#install-via-pypi)\n   - [Local Development Setup](#local-development-setup)\n4. [Basic Usage](#basic-usage)\n   - [CLI Usage](#cli-usage)\n   - [Streamlit GUI](#streamlit-gui)\n5. [Advanced Usage](#advanced-usage)\n   - [Command-Line Arguments](#command-line-arguments)\n   - [Example Commands](#example-commands)\n6. [Output Structure](#output-structure)\n7. [Code Example](#code-example)\n8. [Contributing](#contributing)\n9. [License](#license)\n10. [Acknowledgments](#acknowledgments)\n\n---\n\n## ✨ Features\n\n- 📄 **Document Conversion**  \n  Converts `.txt`, `.docx`, and other supported file types into `.pdf` using **LibreOffice** (optional if you only need to parse PDFs).\n\n- 📊 **Page Counting**  \n  Automatically counts pages in PDFs using **PyPDF2**.\n\n- 🖼️ **Image Processing**  \n  Analyzes images to differentiate logos from content-rich images. Extracts relevant data and updates the corresponding Markdown file.\n\n- ✍️ **Markdown Parsing**  \n  Uses **Marker** to generate clean, structured Markdown files from parsed PDFs.\n\n- 🌐 **Multilingual Support**  \n  Maintains the original language of the content during extraction.\n\n- 📈 **Data Visualization**  \n  Generates analysis plots based on the page counts of processed documents.\n\n---\n\n## ⚙️ Prerequisites\n\nBefore installing or running **LlaMarker**, please ensure you meet the following requirements:\n\n1. **Python 3.10+**\n\n   - Core language for running **LlaMarker**.\n   - Verify your Python version:\n     ```bash\n     python --version\n     ```\n\n2. **Marker**\n\n   - [Marker](https://github.com/VikParuchuri/marker) is an open-source parser that **LlaMarker** extends.\n   - To install Marker, follow these steps:\n     1. Clone the repository:\n        ```bash\n        git clone https://github.com/VikParuchuri/marker.git\n        cd marker\n        ```\n     2. Install Marker in editable mode:\n        ```bash\n        pip install -e .\n        ```\n     3. Verify the installation:\n        ```bash\n        marker --help\n        ```\n   - **GPU Support**: If you plan to leverage GPUs, ensure **PyTorch** is installed with **CUDA** support (e.g., via `pytorch-cuda` or the official PyTorch distribution).\n   - **Path Configuration**: If Marker is not in your `PATH`, ensure you specify its location with the `--marker_path` argument.\n\n3. **LibreOffice**\n\n   - Required for converting `.docx`, `.ppt`, `.xlsx`, etc., into `.pdf` before parsing.\n   - **Linux** (Ubuntu/Debian example):\n     ```bash\n     sudo apt update\n     sudo apt install libreoffice\n     ```\n   - **Windows**:  \n     [Download the installer](https://www.libreoffice.org/download/download/) and consider adding LibreOffice to your system `PATH`.\n   - **macOS**:\n     - Download from [LibreOffice’s website](https://www.libreoffice.org/download/download/) or\n     - Use Homebrew:\n       ```bash\n       brew install --cask libreoffice\n       ```\n\n4. **Ollama \u0026 Vision Models**\n\n   - [Install Ollama](https://github.com/jmorganca/ollama) for your OS.\n   - Pull the required model:\n     ```bash\n     ollama pull llama3.2-vision\n     ```\n   - Test run to ensure your model is set up correctly.\n\n5. **Poetry** (for local development only)\n   - Recommended dependency manager if you’re cloning the repository to develop or modify **LlaMarker**.\n   - **Linux/Mac**:\n     ```bash\n     curl -sSL https://install.python-poetry.org | python3 -\n     # (If not added to PATH automatically)\n     export PATH=\"$HOME/.local/bin:$PATH\"\n     ```\n   - **macOS (Homebrew)**:\n     ```bash\n     brew install poetry\n     ```\n   - **Windows**:  \n     Follow instructions on [Poetry’s official site](https://python-poetry.org/docs/#installation).\n\n---\n\n## 🚀 Installation Options\n\n### 1. Install via PyPI\n\nThe simplest approach—ideal if you just want to **use** LlaMarker rather than develop it:\n\n```bash\npip install llamarker\n```\n\n- **Requires**: Python 3.10+\n- After installing, you have access to two main commands:\n  1. `llamarker` — CLI tool.\n  2. `llamarker_gui` — Streamlit-based GUI for interactive use.\n\n\u003e **Note**: LibreOffice, Marker, and any optional OCR components need to be installed separately, if you plan to use their respective features.\n\n---\n\n### 2. Local Development Setup\n\nIf you plan to contribute or dive into the source code:\n\n1. **Clone the repository**:\n   ```bash\n   git clone https://github.com/RevanKumarD/LlaMarker.git\n   cd LlaMarker\n   ```\n2. **Install dependencies** using **Poetry**:\n   ```bash\n   poetry install\n   ```\n3. **Run LlaMarker locally**:\n   - **CLI**:\n     ```bash\n     poetry run python llamarker/llamarker.py --directory \u003cdirectory_path\u003e\n     ```\n   - **GUI**:\n     ```bash\n     poetry run streamlit run llamarker/llamarker_gui.py\n     ```\n\n\u003e No `requirements.txt` is provided; **Poetry** is the recommended (and supported) method for local development.\n\n---\n\n## 📌 Basic Usage\n\n### CLI Usage\n\n#### Installed via PyPI\n\n- **Process a folder**:\n  ```bash\n  llamarker --directory \u003cdirectory_path\u003e\n  ```\n- **Process a single file**:\n  ```bash\n  llamarker --file \u003cfile_path\u003e\n  ```\n\n#### Local Development\n\n- **CLI**:\n  ```bash\n  poetry run python llamarker/llamarker.py --directory \u003cdirectory_path\u003e\n  ```\n\n---\n\n### Streamlit GUI\n\nA user-friendly interface to upload files/directories, parse them, and download results.\n\n- **Installed via PyPI**:\n  ```bash\n  llamarker_gui\n  ```\n- **Local Development**:\n  ```bash\n  poetry run streamlit run llamarker/llamarker_gui.py\n  ```\n\nOpen the link (e.g., `http://localhost:8501`) in your browser to start using **LlaMarker** via GUI.\n\n---\n\n## 🔧 Advanced Usage\n\n### Command-Line Arguments\n\n| Argument         | Description                                                                                                                                          |\n| ---------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- |\n| `--directory`    | **Root directory** containing documents to process.                                                                                                  |\n| `--file`         | Path to a single file to process (optional).                                                                                                         |\n| `--temp_dir`     | Temporary directory for intermediate files (optional).                                                                                               |\n| `--save_pdfs`    | Flag to **save PDFs** in a separate directory (`PDFs`) under the root directory.                                                                     |\n| `--output`       | Directory to **save output** files (optional). By default, parsed Markdown files are stored in `ParsedFiles` and images go under `ParsedFiles/pics`. |\n| `--marker_path`  | Path to the **Marker** executable (optional). Auto-detects if `Marker` is in your `PATH`.                                                            |\n| `--force_ocr`    | Force **OCR** on all pages, even if text is extractable. Useful for poorly formatted PDFs or PPTs.                                                   |\n| `--languages`    | Comma-separated list of languages for OCR (default: `\"en\"`).                                                                                         |\n| `--qa_evaluator` | Enable **QA Evaluator** for selecting the best response during image processing.                                                                     |\n| `--verbose`      | Set verbosity level: **0** = WARNING, **1** = INFO, **2** = DEBUG (default: **0**).                                                                  |\n| `--model`        | **Ollama** model for image analysis (default: `llama3.2-vision`). A local vision model is required for this to work.                                 |\n\n---\n\n### Example Commands\n\n1. **Directory processing**:\n   ```bash\n   llamarker --directory /path/to/documents\n   ```\n2. **Single file with verbose output**:\n   ```bash\n   llamarker --file /path/to/document.docx --verbose 2\n   ```\n3. **Parsing with OCR in multiple languages**:\n   ```bash\n   llamarker --directory /path/to/docs --force_ocr --languages \"en,de,fr\"\n   ```\n4. **Save parsed PDFs to a custom folder**:\n   ```bash\n   llamarker --directory /path/to/docs --save_pdfs --output /path/to/output\n   ```\n\n---\n\n## Output Structure\n\nAfter processing, **LlaMarker** organizes files as follows:\n\n- **`ParsedFiles`**\n  - Contains the generated Markdown files.\n  - **`pics`** — subfolder for extracted images.\n- **`PDFs`**\n  - Stores converted PDF files (if `--save_pdfs` is used).\n- **`OutDir`**\n  - Contains processed PDF files (used by the GUI).\n- **`logs`**\n  - Holds log files for each run (processing status, errors, etc.).\n\n---\n\n## Code Example\n\nFor local development, you can programmatically use **LlaMarker**:\n\n```python\nfrom llamarker import LlaMarker\n\nllamarker = LlaMarker(\n    input_dir=\"/path/to/documents\",\n    save_pdfs=True,\n    output_dir=\"/path/to/output\",\n    verbose=1\n)\n\n# Process all documents in the specified directory\nllamarker.process_documents()\n\n# Generate summary info\nresults = llamarker.generate_summary()\nfor file, page_count in results:\n    print(f\"{file}: {page_count} pages\")\n\n# Generate analysis plots\nllamarker.plot_analysis(llamarker.parent_dir)\n```\n\n---\n\n## 🚧 Shortcomings \u0026 Future Updates\n\n### Current Shortcomings:\n\n1. **Limited OCR Accuracy for Complex Documents**  \n   - While OCR works well for most cases, it may struggle with highly complex layouts or poorly scanned documents.\n2. **No Direct Cloud Integration**  \n   - Currently, LlaMarker only supports local processing. There’s no option to process files directly from cloud storage services like Google Drive or Dropbox.\n3. **Basic Support for PPT and XLSX Parsing**  \n   - Parsing of **PPT** and **XLSX** files is available but lacks advanced formatting support (e.g., slide layouts, complex charts).\n4. **Poor XLSX to PDF Conversion**  \n   - The current conversion of **XLSX** files to **PDF** results in poorly formatted output. Improvements are needed to handle large spreadsheets and complex tables.\n5. **Manual Setup for Marker and LibreOffice**  \n   - Users must manually install **Marker** and **LibreOffice**, which can be cumbersome for those unfamiliar with the setup process.\n\n---\n\n### Planned Future Updates:\n\n1. **Enhanced OCR Capabilities**  \n   - Improve OCR performance by integrating additional vision models for better handling of complex document layouts and multi-column formats.\n2. **Cloud Storage Integration**  \n   - Add support for uploading documents directly from cloud services (Google Drive, Dropbox, OneDrive).\n3. **Improved PPT \u0026 XLSX Handling**  \n   - Enhance parsing accuracy for **PPT** and **XLSX** files by adding better support for slides, tables, and embedded charts.\n4. **Better XLSX to PDF Conversion**  \n   - Improve the **XLSX to PDF** conversion process to handle large sheets, complex tables, and maintain proper formatting.\n5. **Cross-Platform Installation Script**  \n   - Provide an easy-to-use installation script for all platforms (Linux, Windows, macOS) to automate the setup of dependencies like **Marker** and **LibreOffice**.\n\n---\n\n## Contributing\n\nContributions are welcome! Feel free to open an issue or submit a pull request. Let’s make **LlaMarker** even more powerful—together. 🤝\n\n---\n\n## License\n\nThis project references the [Marker](https://github.com/VikParuchuri/marker) repository, which comes with its own license. Please review the Marker repo for licensing restrictions and guidelines.\n\n© 2025 Revan Kumar Dhanasekaran. Released under the GPLv3 License.\n\n---\n\n## Acknowledgments\n\n- **Huge thanks** to the [Marker](https://github.com/VikParuchuri/marker) project for providing an excellent foundation for parsing.\n- **Special thanks** to the open-source community for continuous support and contributions.\n\n---\n\n\u003cp align=\"center\"\u003e\n  \u003cb\u003eHappy Parsing!\u003c/b\u003e 🌟\n\u003c/p\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frevankumard%2Fllamarker","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frevankumard%2Fllamarker","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frevankumard%2Fllamarker/lists"}