https://github.com/hiborn4/kpidociq

AI-powered document intelligence system for extracting KPIs from industrial reports, PDFs, and scanned tables using computer vision, OCR, and LLMs.
https://github.com/hiborn4/kpidociq

Last synced: 12 months ago
JSON representation

AI-powered document intelligence system for extracting KPIs from industrial reports, PDFs, and scanned tables using computer vision, OCR, and LLMs.

Host: GitHub
URL: https://github.com/hiborn4/kpidociq
Owner: HiBorn4
Created: 2024-05-30T06:33:27.000Z (about 2 years ago)
Default Branch: main
Last Pushed: 2025-08-06T18:31:23.000Z (12 months ago)
Last Synced: 2025-08-06T20:36:12.637Z (12 months ago)
Language: Python
Homepage:
Size: 51.6 MB
Stars: 1
Watchers: 1
Forks: 1
Open Issues: 0
Metadata Files:
- Readme: README.md

Awesome Lists containing this project

README

# 🧠 i4Ideas: AI-Powered Document Processing & Extraction System

![Banner](/public/test1.png)

> An advanced AI-based system that extracts complex tabular and structured information from technical documents with unmatched precision. Built for manufacturing, steel, and QA/QC workflows.

---

### 📸 Visual Workflow — Document ➝ Table ➝ KPI Extraction

📄 Raw Document

✂️ Cropped Table

🧠 KPIs

---

## 🔍 Overview

**i4Ideas** is a state-of-the-art, production-grade document parsing system designed to extract meaningful data from industrial reports like tensile strength sheets, quality certificates, and lab reports. The system leverages deep learning, OCR, and vision-language models to output structured, human-verified JSON for downstream analytics or compliance.

This solution has been **field-tested across diverse document formats**, including JSW, Tata, Hyundai, and Jamshedpur technical data sheets.

---

## 🧠 Core Capabilities

- 📄 **PDF → Table/Image Extraction**
- 🔍 **Object Detection (YOLOv5 / ResNet50)** to locate tabular blocks
- 🧾 **PaddleOCR / Adobe OCR** integration for clean text extraction
- 🤖 **LLM-Powered Parsing (GPT)** using domain-specific prompts
- 📊 **Aesthetic JSON Output** with units and data validation
- 🔄 **Multi-format switch handling** (JSW, Tata, Hyundai, etc.)

---

## 📸 Supported Formats

| Company | Keys Extracted |
|----------------|------------------------------------------------------------------------------|
| JSW Steel | `Coil No`, `YS`, `UTS`, `EL`, `Ra` |
| Tata Steel | `Mother Coil`, `YS`, `UTS`, `EL`, `rBAR`, `n`, `Ra` |
| Jamshedpur | `Coil No`, `YS`, `UTS`, `EL`, `rVALUE`, `NVALM`, `RaMICROM` |
| Hyundai | `Dimension`, `YP`, `TS`, `EL`, `Ra` |
| Maharashtra | `Product No`, `YP`, `TS`, `EL`, `Ra` |

---

## 🚀 Installation

```bash
git clone https://github.com/HiBorn4/AI-powered-document-processing-and-extraction-system.git
cd AI-powered-document-processing-and-extraction-system
pip install -r requirements.txt
````

---

## 🎥 Demo

*(Add video preview here)*

---

## 🧪 Example Output

```json
{
"NC65081000": {
"YS": "347 MPa",
"UTS": "294 MPa",
"EL": "45%",
"Ra": "67.39 μm"
},
"NC65082000": {
"YS": "364 MPa",
"UTS": "470 MPa",
"EL": "80%",
"Ra": "28.10 μm"
}
}
```

---

## 📁 Project Structure

```
📦 AI-powered-document-processing-and-extraction-system
├── bulk_table_extraction.py # Batch document processing
├── extract_keys.py # Prompt-based LLM key-value extraction
├── table_detection.py # YOLO/ResNet table detection pipeline
├── pdf_to_image.py # Converts PDF to image for OCR
├── paddleocr_implemented/ # PaddleOCR model directory
├── jcap_tsr.py, tata_tsr.py, jsw_tsr.py # Custom flows per company
├── image_postprocess.py # Image cleaning and noise reduction
├── summarize_prompts.py # Prompt engineering and format matching
```

---

## 📦 Tech Stack

* 🧠 **OpenAI GPT (Function Calling + Vision)**
* 🧾 **PaddleOCR / Adobe Acrobat OCR**
* 🖼️ **YOLOv5 / ResNet50** for layout detection
* 🧪 **Prompt Engineering** for reliable key mapping
* 📄 **PDF2Image**, **Pillow**, **Pandas**

---

## 🔐 Use Cases

* Steel Manufacturing QA Automation
* Compliance & Lab Test Report Digitization
* Quality Certificate Parsing & Validation
* Document Indexing for Enterprise Search

---

## 💼 Portfolio & Freelance Suitability

This project is ideal to showcase:

* LLMOps / Document AI skills
* OCR + Vision + GPT integration
* Custom multi-format prompt engineering
* PDF/Scan processing pipelines

---

## 📬 Contact

For collaboration or freelance work:

* 💼 **Upwork**: [HiBorn4](https://www.upwork.com/freelancers/~hiborn4)
* 💻 **GitHub**: [HiBorn4](https://github.com/HiBorn4)
* 📫 **Email**: [reach.hiborn@gmail.com](mailto:reach.hiborn4@gmail.com)

---

## 📝 License

ecosyste.ms

Data

Tools

Indexes

Applications

Experiments

Awesome

https://github.com/hiborn4/kpidociq

Awesome Lists containing this project

README