https://github.com/stabrise/scaledp-tutorials

Tutorials for ScaleDP library. ScaleDP is an Open-Source Library for Processing Documents in Apache Spark.
https://github.com/stabrise/scaledp-tutorials

ner nlp ocr ocr-python pdf spark

Last synced: 15 days ago
JSON representation

Tutorials for ScaleDP library. ScaleDP is an Open-Source Library for Processing Documents in Apache Spark.

• Host: GitHub
• URL: https://github.com/stabrise/scaledp-tutorials
• Owner: StabRise
• License: agpl-3.0
• Created: 2024-11-05T07:14:29.000Z (3 months ago)
• Default Branch: master
• Last Pushed: 2024-12-03T05:30:14.000Z (2 months ago)
• Last Synced: 2025-01-30T20:54:11.448Z (15 days ago)
• Topics: ner, nlp, ocr, ocr-python, pdf, spark
• Language: Jupyter Notebook
• Homepage: https://stabrise.com/scaledp/
• Size: 5.3 MB
• Stars: 1
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        


  


    

  






    An Open-Source Library for Processing Documents in Apache Spark.





    

    

    



# ScaleDP Tutorials

[ScaleDP](https://github.com/StabRise/scaledp/) is a library for processing documents using Apache Spark.

And this repository contains jupyter notebooks with tutorials and examples of usage ScaleDP library.

ScaleDP includes the following features:

- Load PDF documents/Images

- Extract text from PDF documents/Images

- Extract images from PDF documents

- OCR Images/PDF documents

- Run NER on text extracted from PDF documents/Images

- Visualize NER results

## Table of Contents

We advise to run all examples through Google Colab for the easiest setup. Google Colab allows you to run it for free. All examples were tested using Google Colab, so it should be the most stable platform. However, any other cloud provider or local run should work. 

| Chapter                           | Notebook  |

|-----------------------------------|---|

| 1: QuickStart with ScaleDP        | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/StabRise/scaledp-tutorials/blob/master/1.QuickStart.ipynb)   |

| 2: Example of usage different OCR | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/StabRise/scaledp-tutorials/blob/master/2.Ocr.ipynb)  |

| 3: NER Examples                   | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/StabRise/scaledp-tutorials/blob/master/3.Ner.ipynb)  |

## Local Run

### Requirements

- Python 3.11

- Apache Spark 3.5 or higher

- Java 8, 11

- Tesseract 5.0 or higher

- Jupyter Lab | Google Colab