{"id":50972300,"url":"https://github.com/danindiana/pdf_validator_rs","last_synced_at":"2026-06-19T03:31:50.303Z","repository":{"id":323614043,"uuid":"1093984296","full_name":"danindiana/pdf_validator_rs","owner":"danindiana","description":"Pdf validator written in Rust.","archived":false,"fork":false,"pushed_at":"2025-11-11T07:07:03.000Z","size":158,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"master","last_synced_at":"2025-11-11T07:13:10.285Z","etag":null,"topics":["file","file-handling","pdf","pdf-document-processor","rustlang","validator"],"latest_commit_sha":null,"homepage":"https://calisota.ai","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/danindiana.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-11-11T05:14:41.000Z","updated_at":"2025-11-11T07:07:06.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/danindiana/pdf_validator_rs","commit_stats":null,"previous_names":["danindiana/pdf_validator_rs"],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/danindiana/pdf_validator_rs","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/danindiana%2Fpdf_validator_rs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/danindiana%2Fpdf_validator_rs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/danindiana%2Fpdf_validator_rs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/danindiana%2Fpdf_validator_rs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/danindiana","download_url":"https://codeload.github.com/danindiana/pdf_validator_rs/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/danindiana%2Fpdf_validator_rs/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34516549,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-19T02:00:06.005Z","response_time":61,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["file","file-handling","pdf","pdf-document-processor","rustlang","validator"],"created_at":"2026-06-19T03:31:49.153Z","updated_at":"2026-06-19T03:31:50.291Z","avatar_url":"https://github.com/danindiana.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n\n```\n███████╗██████╗ ███████╗    ██╗   ██╗ █████╗ ██╗     ██╗██████╗  █████╗ ████████╗ ██████╗ ██████╗\n██╔════╝██╔══██╗██╔════╝    ██║   ██║██╔══██╗██║     ██║██╔══██╗██╔══██╗╚══██╔══╝██╔═══██╗██╔══██╗\n███████╗██║  ██║█████╗      ██║   ██║███████║██║     ██║██║  ██║███████║   ██║   ██║   ██║██████╔╝\n╚════██║██║  ██║██╔══╝      ╚██╗ ██╔╝██╔══██║██║     ██║██║  ██║██╔══██║   ██║   ██║   ██║██╔══██╗\n███████║██████╔╝██║          ╚████╔╝ ██║  ██║███████╗██║██████╔╝██║  ██║   ██║   ╚██████╔╝██║  ██║\n╚══════╝╚═════╝ ╚═╝           ╚═══╝  ╚═╝  ╚═╝╚══════╝╚═╝╚═════╝ ╚═╝  ╚═╝   ╚═╝    ╚═════╝ ╚═╝  ╚═╝\n```\n\n\u003cpicture\u003e\n  \u003csource media=\"(prefers-color-scheme: dark)\" srcset=\"https://img.shields.io/badge/Version-1.0.0-FF6B35?style=for-the-badge\u0026logo=semantic-release\u0026logoColor=white\"\u003e\n  \u003cimg alt=\"Version\" src=\"https://img.shields.io/badge/Version-1.0.0-FF6B35?style=for-the-badge\u0026logo=semantic-release\u0026logoColor=white\"\u003e\n\u003c/picture\u003e\n\u003cpicture\u003e\n  \u003csource media=\"(prefers-color-scheme: dark)\" srcset=\"https://img.shields.io/badge/Rust-1.90.0-FF6B35?style=for-the-badge\u0026logo=rust\u0026logoColor=white\"\u003e\n  \u003cimg alt=\"Rust Version\" src=\"https://img.shields.io/badge/Rust-1.90.0-FF6B35?style=for-the-badge\u0026logo=rust\u0026logoColor=white\"\u003e\n\u003c/picture\u003e\n\u003cpicture\u003e\n  \u003csource media=\"(prefers-color-scheme: dark)\" srcset=\"https://img.shields.io/badge/Ubuntu-22.04_LTS-FF6B35?style=for-the-badge\u0026logo=ubuntu\u0026logoColor=white\"\u003e\n  \u003cimg alt=\"Ubuntu\" src=\"https://img.shields.io/badge/Ubuntu-22.04_LTS-FF6B35?style=for-the-badge\u0026logo=ubuntu\u0026logoColor=white\"\u003e\n\u003c/picture\u003e\n\u003cpicture\u003e\n  \u003csource media=\"(prefers-color-scheme: dark)\" srcset=\"https://img.shields.io/badge/Parallel-Rayon-FF6B35?style=for-the-badge\u0026logo=apache-spark\u0026logoColor=white\"\u003e\n  \u003cimg alt=\"Parallel Processing\" src=\"https://img.shields.io/badge/Parallel-Rayon-FF6B35?style=for-the-badge\u0026logo=apache-spark\u0026logoColor=white\"\u003e\n\u003c/picture\u003e\n\u003cpicture\u003e\n  \u003csource media=\"(prefers-color-scheme: dark)\" srcset=\"https://img.shields.io/badge/License-MIT_OR_Apache--2.0-FF6B35?style=for-the-badge\"\u003e\n  \u003cimg alt=\"License\" src=\"https://img.shields.io/badge/License-MIT_OR_Apache--2.0-FF6B35?style=for-the-badge\"\u003e\n\u003c/picture\u003e\n\n```\n╔═══════════════════════════════════════════════════════════════════════════════╗\n║                                                                               ║\n║   🦀 HIGH-PERFORMANCE PARALLEL PDF VALIDATION TOOL WRITTEN IN RUST 🦀        ║\n║                                                                               ║\n║   ⚡ Parallel Processing  │  🔍 Duplicate Detection  │  📊 Detailed Reports  ║\n║                                                                               ║\n╚═══════════════════════════════════════════════════════════════════════════════╝\n```\n\n\u003c/div\u003e\n\n---\n\n## 🚀 Overview\n\nA high-performance, parallel PDF validation tool written in Rust. Validates PDF files for corruption, detects duplicates, and provides comprehensive reporting.\n\n## Features\n\n- **Parallel Processing**: Uses Rayon for multi-threaded validation\n- **Recursive Directory Scanning**: Process entire directory trees\n- **Duplicate Detection**: SHA-256 based duplicate file detection\n- **Multiple Validation Modes**:\n  - Strict validation (default)\n  - Lenient mode for PDFs with minor issues\n  - Optional rendering validation\n- **Batch Operations**:\n  - Delete invalid PDFs automatically\n  - Remove duplicate files\n- **Progress Tracking**: Real-time progress bar with indicatif\n- **Detailed Reporting**: Comprehensive validation reports\n\n## Installation\n\n### Prerequisites\n\n- Rust 1.70 or higher\n- Cargo (comes with Rust)\n\n### Build from Source\n\n```bash\ngit clone https://github.com/danindiana/pdf_validator_rs.git\ncd pdf_validator_rs\ncargo build --release\n```\n\nThe compiled binary will be at `target/release/pdf_validator_rs`\n\n## Usage\n\n### Basic Usage\n\n```bash\n# Validate PDFs in a single directory\ncargo run --release -- /path/to/pdfs\n\n# Recursive validation through subdirectories\ncargo run --release -- /path/to/pdfs --recursive\n```\n\n### Command-Line Options\n\n```\npdf_validator_rs [OPTIONS] \u003cDIRECTORY\u003e\n\nArguments:\n  \u003cDIRECTORY\u003e  Target directory to scan for PDF files\n\nOptions:\n  -r, --recursive              Scan directories recursively\n  -w, --workers \u003cWORKERS\u003e      Number of parallel worker threads (default: number of CPUs)\n  -o, --output \u003cOUTPUT\u003e        Output report filename [default: validation_report_rust.txt]\n      --delete-invalid         Delete invalid/corrupted PDF files\n  -v, --verbose                Verbose output\n      --detect-duplicates      Detect and report duplicate files\n      --delete-duplicates      Delete duplicate PDF files (requires --detect-duplicates)\n      --batch                  Run in batch mode (no interactive prompts, no progress bar)\n      --no-render-check        Skip rendering quality checks (faster validation)\n      --lenient                Use lenient parsing mode (accept more PDFs with minor issues)\n  -h, --help                   Print help\n```\n\n### Examples\n\n**Validate recursively with progress bar:**\n```bash\ncargo run --release -- /path/to/pdfs --recursive\n```\n\n**Detect and remove duplicates:**\n```bash\ncargo run --release -- /path/to/pdfs -r --detect-duplicates --delete-duplicates\n```\n\n**Lenient mode for edge cases:**\n```bash\ncargo run --release -- /path/to/pdfs -r --lenient\n```\n\n**Delete invalid PDFs automatically:**\n```bash\ncargo run --release -- /path/to/pdfs -r --delete-invalid\n```\n\n**Batch mode for scripting:**\n```bash\ncargo run --release -- /path/to/pdfs -r --batch --output batch_report.txt\n```\n\n**Custom worker threads:**\n```bash\ncargo run --release -- /path/to/pdfs -r --workers 16\n```\n\n## Validation Report\n\nThe tool generates a detailed report (default: `validation_report_rust.txt`) containing:\n\n- Total files processed\n- Valid vs invalid file counts\n- List of invalid files with full paths\n- Duplicate file groups (if duplicate detection enabled)\n- Processing statistics\n\n## Architecture \u0026 Design\n\n### Overall Program Flow\n\nThe PDF validator follows a systematic workflow from CLI input to final report generation:\n\n```mermaid\n%%{init: {'theme':'base', 'themeVariables': { 'primaryColor':'#fff0','primaryTextColor':'#333','primaryBorderColor':'#333','lineColor':'#333','secondaryColor':'#fff0','tertiaryColor':'#fff0','background':'#fff0','mainBkg':'#fff0','secondBkg':'#fff0'}}}%%\nflowchart TD\n    Start([User Runs CLI]) --\u003e Parse[Parse Command Line Args]\n    Parse --\u003e ThreadPool[Initialize Rayon Thread Pool]\n    ThreadPool --\u003e Scan[Scan Directory for PDFs]\n    Scan --\u003e CheckRecursive{Recursive Mode?}\n    CheckRecursive --\u003e|Yes| WalkDir[WalkDir: Traverse All Subdirectories]\n    CheckRecursive --\u003e|No| ReadDir[ReadDir: Scan Single Directory]\n    WalkDir --\u003e Collect[Collect PDF File Paths]\n    ReadDir --\u003e Collect\n    Collect --\u003e CheckEmpty{Files Found?}\n    CheckEmpty --\u003e|No| Exit1([Exit: No Files])\n    CheckEmpty --\u003e|Yes| Progress[Initialize Progress Bar]\n    Progress --\u003e Parallel[Parallel Validation with Rayon]\n    Parallel --\u003e ValidateLoop[For Each PDF File]\n    ValidateLoop --\u003e SelectMode{Validation Mode}\n    SelectMode --\u003e|Lenient| Lenient[validate_pdf_lenient]\n    SelectMode --\u003e|Render Check| RenderCheck[validate_pdf + validate_pdf_rendering]\n    SelectMode --\u003e|Normal| Normal[validate_pdf]\n    Lenient --\u003e UpdateProgress[Update Progress Bar]\n    RenderCheck --\u003e UpdateProgress\n    Normal --\u003e UpdateProgress\n    UpdateProgress --\u003e MoreFiles{More Files?}\n    MoreFiles --\u003e|Yes| ValidateLoop\n    MoreFiles --\u003e|No| Complete[Validation Complete]\n    Complete --\u003e CheckDuplicates{Detect Duplicates?}\n    CheckDuplicates --\u003e|Yes| HashFiles[Hash Valid Files with SHA-256]\n    CheckDuplicates --\u003e|No| Summarize\n    HashFiles --\u003e FindDups[Find Duplicate Groups]\n    FindDups --\u003e DeleteDups{Delete Duplicates?}\n    DeleteDups --\u003e|Yes| RemoveDups[Delete Duplicate Files]\n    DeleteDups --\u003e|No| Summarize\n    RemoveDups --\u003e Summarize\n    Summarize[Generate Summary Statistics] --\u003e CheckInvalid{Delete Invalid?}\n    CheckInvalid --\u003e|Yes| RemoveInvalid[Delete Invalid Files]\n    CheckInvalid --\u003e|No| Report\n    RemoveInvalid --\u003e Report\n    Report[Write Detailed Report] --\u003e Display[Display Summary to User]\n    Display --\u003e Exit2([Program Complete])\n```\n\n[View detailed flow diagram](docs/diagrams/overall-flow.md)\n\n### Validation Strategy\n\nThe validator employs multiple validation strategies with fallback mechanisms:\n\n```mermaid\n%%{init: {'theme':'base', 'themeVariables': { 'primaryColor':'#fff0','primaryTextColor':'#333','primaryBorderColor':'#333','lineColor':'#333','secondaryColor':'#fff0','tertiaryColor':'#fff0','background':'#fff0','mainBkg':'#fff0','secondBkg':'#fff0'}}}%%\nflowchart TD\n    Start([PDF File Path]) --\u003e ModeCheck{Validation Mode}\n    ModeCheck --\u003e|Normal/Strict| NormalPath[validate_pdf]\n    ModeCheck --\u003e|Lenient| LenientPath[validate_pdf_lenient]\n    ModeCheck --\u003e|Rendering| RenderPath[validate_pdf + validate_pdf_rendering]\n    NormalPath --\u003e Lopdf1[Try: validate_pdf_with_lopdf]\n    Lopdf1 --\u003e LopdfCheck1{Success?}\n    LopdfCheck1 --\u003e|Yes| PageCheck1{Has Pages?}\n    PageCheck1 --\u003e|Yes| Valid1([Return: VALID])\n    PageCheck1 --\u003e|No| Invalid1([Return: INVALID])\n    LopdfCheck1 --\u003e|No| Fallback1[Try: validate_pdf_basic]\n    Fallback1 --\u003e BasicResult1{Basic Valid?}\n    BasicResult1 --\u003e|Yes| Valid2([Return: VALID])\n    BasicResult1 --\u003e|No| Invalid2([Return: INVALID])\n```\n\n[View complete validation strategy](docs/diagrams/validation-strategy.md)\n\n### Parallel Processing Architecture\n\nThe tool leverages Rayon for efficient parallel processing:\n\n```mermaid\n%%{init: {'theme':'base', 'themeVariables': { 'primaryColor':'#fff0','primaryTextColor':'#333','primaryBorderColor':'#333','lineColor':'#333','secondaryColor':'#fff0','tertiaryColor':'#fff0','background':'#fff0','mainBkg':'#fff0','secondBkg':'#fff0'}}}%%\ngraph TB\n    subgraph Main[\"Main Thread\"]\n        CLI[CLI Argument Parser]\n        Setup[Thread Pool Setup]\n        Scanner[File Scanner]\n        Progress[Progress Bar]\n    end\n    subgraph RayonPool[\"Rayon Thread Pool\"]\n        Splitter[Work Splitter]\n        subgraph Worker1[\"Worker Thread 1\"]\n            V1[Validate PDF 1]\n            V2[Validate PDF 2]\n        end\n        subgraph Worker2[\"Worker Thread 2\"]\n            V4[Validate PDF 3]\n            V5[Validate PDF 4]\n        end\n        Collector[Result Collector]\n    end\n    CLI --\u003e Setup --\u003e Scanner --\u003e Splitter\n    Splitter --\u003e Worker1\n    Splitter --\u003e Worker2\n    Worker1 --\u003e Collector\n    Worker2 --\u003e Collector\n```\n\n[View detailed parallel architecture](docs/diagrams/parallel-architecture.md)\n\n### Module Structure\n\n```mermaid\n%%{init: {'theme':'base', 'themeVariables': { 'primaryColor':'#fff0','primaryTextColor':'#333','primaryBorderColor':'#333','lineColor':'#333','secondaryColor':'#fff0','tertiaryColor':'#fff0','background':'#fff0','mainBkg':'#fff0','secondBkg':'#fff0'}}}%%\ngraph LR\n    subgraph Binary[\"Binary Crate\"]\n        Main[main.rs]\n    end\n    subgraph Library[\"Library: pdf_validator_rs\"]\n        subgraph CoreMod[\"core::\"]\n            Validator[validator.rs]\n        end\n        subgraph ScannerMod[\"scanner::\"]\n            FileScanner[file_scanner.rs]\n            DupDetector[duplicate_detector.rs]\n        end\n        subgraph ReportingMod[\"reporting::\"]\n            ReportWriter[report_writer.rs]\n        end\n    end\n    Main --\u003e Validator\n    Main --\u003e FileScanner\n    Main --\u003e DupDetector\n    Main --\u003e ReportWriter\n```\n\n[View detailed module structure](docs/diagrams/module-structure.md)\n\n## Project Structure\n\n```\npdf_validator_rs/\n├── src/\n│   ├── main.rs              # CLI entry point\n│   ├── lib.rs               # Library exports\n│   ├── core/                # Core validation logic\n│   │   ├── mod.rs\n│   │   └── validator.rs     # PDF validation functions\n│   ├── scanner/             # File scanning \u0026 duplicate detection\n│   │   ├── mod.rs\n│   │   ├── file_scanner.rs\n│   │   └── duplicate_detector.rs\n│   └── reporting/           # Report generation\n│       ├── mod.rs\n│       └── report_writer.rs\n├── docs/\n│   └── diagrams/            # Architecture diagrams\n│       ├── overall-flow.md\n│       ├── validation-strategy.md\n│       ├── parallel-architecture.md\n│       └── module-structure.md\n├── examples/\n│   └── diagnose_discrepancies.rs  # Diagnostic tool\n├── Cargo.toml\n└── README.md\n```\n\n## Performance\n\n- **Parallel Processing**: Utilizes all available CPU cores\n- **Memory Efficient**: Streaming validation without loading entire PDFs\n- **Optimized Build**: Release builds use LTO and aggressive optimization\n- **Progress Tracking**: Minimal overhead with batched updates\n\n## Dependencies\n\n- **clap**: Command-line argument parsing\n- **rayon**: Data parallelism library\n- **lopdf**: PDF parsing and validation\n- **walkdir**: Recursive directory traversal\n- **sha2**: SHA-256 hashing for duplicate detection\n- **indicatif**: Progress bars and spinners\n- **anyhow**: Error handling\n\n## Optional Features\n\n### Rendering Validation (Coming Soon)\n\nEnable the `rendering` feature for additional validation:\n\n```bash\ncargo build --release --features rendering\n```\n\nThis adds pdfium-render support to validate that PDFs can be rendered correctly.\n\n## Error Handling\n\nThe validator uses multiple validation strategies:\n\n1. **Basic Validation**: Checks PDF structure and syntax\n2. **Lenient Mode**: More permissive parsing for edge cases\n3. **Rendering Check**: Optional page rendering validation (when feature enabled)\n\n## Examples\n\n### Diagnostic Tool\n\nCheck for discrepancies between validation results:\n\n```bash\ncargo run --example diagnose_discrepancies -- file1.pdf file2.pdf file3.pdf\n```\n\n## Contributing\n\nContributions are welcome! Please feel free to submit a Pull Request.\n\n## License\n\nThis project is open source. See LICENSE file for details.\n\n## Acknowledgments\n\nBuilt with Rust and powered by:\n- The amazing Rust PDF ecosystem\n- Rayon for fearless parallelism\n- The Rust community\n\n---\n\n## Build Information\n\n### Development Environment\n- **Rust Version**: rustc 1.90.0 (1159e78c4 2025-09-14)\n- **Cargo Version**: cargo 1.90.0 (840b83a10 2025-07-30)\n- **Operating System**: Ubuntu 22.04.5 LTS (Jammy Jellyfish)\n- **Linux Kernel**: 6.8.0-87-generic\n- **Build Date**: Mon Nov 10 23:22:26 CST 2025\n- **Unix Timestamp**: 1762838546\n\n### System Requirements\n- **Minimum Rust**: 1.70 or higher\n- **Recommended OS**: Linux (Ubuntu 20.04+), macOS, Windows 10+\n- **Architecture**: x86_64, aarch64\n\n### Tested Platforms\n- ✅ Ubuntu 22.04 LTS (Primary development environment)\n- ✅ Linux Kernel 6.8.0+\n- ✅ Rust 1.90.0\n\n---\n\n**Generated with [Claude Code](https://claude.com/claude-code)**\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdanindiana%2Fpdf_validator_rs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdanindiana%2Fpdf_validator_rs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdanindiana%2Fpdf_validator_rs/lists"}