{"id":22301787,"url":"https://github.com/stabrise/scaledp-tutorials","last_synced_at":"2025-10-12T07:10:18.994Z","repository":{"id":266213051,"uuid":"883558242","full_name":"StabRise/ScaleDP-Tutorials","owner":"StabRise","description":"Tutorials for ScaleDP library. ScaleDP is an Open-Source Library for Processing Documents in Apache Spark.","archived":false,"fork":false,"pushed_at":"2025-03-19T10:56:43.000Z","size":11694,"stargazers_count":4,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-06-24T16:51:21.731Z","etag":null,"topics":["ner","nlp","ocr","ocr-python","pdf","spark"],"latest_commit_sha":null,"homepage":"https://stabrise.com/scaledp/","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"agpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/StabRise.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-11-05T07:14:29.000Z","updated_at":"2025-06-11T10:38:08.000Z","dependencies_parsed_at":"2025-03-19T11:27:15.879Z","dependency_job_id":"76910a6c-640e-4578-a67c-cd73d8ffaacd","html_url":"https://github.com/StabRise/ScaleDP-Tutorials","commit_stats":null,"previous_names":["stabrise/scaledp-tutorials"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/StabRise/ScaleDP-Tutorials","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/StabRise%2FScaleDP-Tutorials","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/StabRise%2FScaleDP-Tutorials/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/StabRise%2FScaleDP-Tutorials/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/StabRise%2FScaleDP-Tutorials/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/StabRise","download_url":"https://codeload.github.com/StabRise/ScaleDP-Tutorials/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/StabRise%2FScaleDP-Tutorials/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279010641,"owners_count":26084783,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-12T02:00:06.719Z","response_time":53,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ner","nlp","ocr","ocr-python","pdf","spark"],"created_at":"2024-12-03T18:31:00.337Z","updated_at":"2025-10-12T07:10:18.963Z","avatar_url":"https://github.com/StabRise.png","language":"Jupyter Notebook","readme":"\u003cp align=\"center\"\u003e\n  \u003cbr/\u003e\n    \u003ca href=\"https://stabrise.com/scaledp/\"\u003e\u003cimg alt=\"ScaleDP\" src=\"https://raw.githubusercontent.com/StabRise/ScaleDP/refs/heads/master/images/scaledp.webp\" width=\"450\" style=\"max-width: 100%;\"\u003e\u003c/a\u003e\n  \u003cbr/\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n    \u003ci\u003eAn Open-Source Library for Processing Documents in Apache Spark.\u003c/i\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n    \u003ca href=\"https://pypi.org/project/scaledp/\" alt=\"Package on PyPI\"\u003e\u003cimg src=\"https://img.shields.io/pypi/v/scaledp.svg\" /\u003e\u003c/a\u003e\n    \u003ca href=\"https://github.com/stabrise/spark-pdf/blob/main/LICENSE\"\u003e\u003cimg alt=\"GitHub\" src=\"https://img.shields.io/github/license/stabrise/spark-pdf.svg?color=blue\"\u003e\u003c/a\u003e\n    \u003ca href=\"https://stabrise.com\"\u003e\u003cimg alt=\"StabRise\" src=\"https://img.shields.io/badge/powered%20by-StabRise-orange.svg?style=flat\u0026colorA=E1523D\u0026colorB=007D8A\"\u003e\u003c/a\u003e\n\u003c/p\u003e\n\n\n# ScaleDP Tutorials\n\n[ScaleDP](https://github.com/StabRise/scaledp/) is a library for processing documents using Apache Spark.\nAnd this repository contains jupyter notebooks with tutorials and examples of usage ScaleDP library.\n\nScaleDP includes the following features:\n\n- Load PDF documents/Images\n- Extract text from PDF documents/Images\n- Extract text Images/PDF documents, OCR\n- Zero-shot Data extraction from the Images/PDF documents using LLM\n- Object detection over Images/PDF documents\n- NER/LLM over Images/PDF documents \n- Visualize results on the images\n- Compatible with Spark PDF Datasource\n\n## Table of Contents\n\nWe advise to run all examples through Google Colab for the easiest setup. Google Colab allows you to run it for free. All examples were tested using Google Colab, so it should be the most stable platform. However, any other cloud provider or local run should work. \n\n| Chapter                           | Notebook                                                                                                                                                                           |\n|-----------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| 1: QuickStart with ScaleDP        | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/StabRise/scaledp-tutorials/blob/master/1.QuickStart.ipynb)   |\n| 2: Example of usage different OCR | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/StabRise/scaledp-tutorials/blob/master/2.Ocr.ipynb)          |\n| 3: NER Examples                   | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/StabRise/scaledp-tutorials/blob/master/3.Ner.ipynb)          |\n| 4: LLM NER Examples               | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/StabRise/scaledp-tutorials/blob/master/4.LLMNer.ipynb)       |\n| 5: With Spark PDF                 | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/StabRise/scaledp-tutorials/blob/master/5.WithSparkPdf.ipynb) |\n\n## Local Run\n\n### Requirements\n\n- Python 3.11\n- Apache Spark 3.5 or higher\n- Java 8, 11\n- Tesseract 5.0 or higher\n- Jupyter Lab | Google Colab\n\n![](./images/ner_visualize.png)\n\n![](./images/ner_visualize_1.png)\n\n\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstabrise%2Fscaledp-tutorials","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fstabrise%2Fscaledp-tutorials","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstabrise%2Fscaledp-tutorials/lists"}