https://github.com/sshniro/line-segmentation-algorithm-to-gcp-vision

Line segmentation algorithm for Google Vision API.
https://github.com/sshniro/line-segmentation-algorithm-to-gcp-vision

data-extraction google-vision invoice proposed-algorithm segmentation

Last synced: 5 days ago
JSON representation

Line segmentation algorithm for Google Vision API.

• Host: GitHub
• URL: https://github.com/sshniro/line-segmentation-algorithm-to-gcp-vision
• Owner: sshniro
• License: apache-2.0
• Created: 2018-01-14T09:26:48.000Z (almost 7 years ago)
• Default Branch: master
• Last Pushed: 2022-11-08T08:16:50.000Z (almost 2 years ago)
• Last Synced: 2024-10-30T09:35:55.750Z (7 days ago)
• Topics: data-extraction, google-vision, invoice, proposed-algorithm, segmentation
• Language: Kotlin
• Homepage:
• Size: 2.76 MB
• Stars: 94
• Watchers: 12
• Forks: 37
• Open Issues: 16
• Metadata Files:
  • Readme: README.md
  • License: LICENSE.txt

Awesome Lists containing this project

README

        
[![Hex.pm](https://img.shields.io/hexpm/l/plug.svg)](https://github.com/sshniro/line-segmentation-algorithm-to-gcp-vision/blob/master/LICENSE)

[![Build Status](https://travis-ci.org/sshniro/line-segmentation-algorithm-to-gcp-vision.svg?branch=master)](https://travis-ci.com/sshniro/line-segmentation-algorithm-to-gcp-vision)

# Introduction

Google Vision provides 2 options for optical character recognition(OCR).

````

- Option 1: TEXT_DETECTION - Words with coordinates

- Option 2: DOCUMENT_TEXT_DETECTION - OCR on dense text to extract lines and paragraph information

````

The second option is suitable for data extraction from articles (Dense Text such as News Papers/Books). This option has an 

intelligent segmentation method to merge words which are nearby and form lines and paragraphs.

 

This feature is not desirable for images with sparse text content such as retail invoices, where the data relevant to the same line

resides in two corners (A huge gap/whitespace between the product name and price). For these images the OCR segments the 

lines in a different order. If the distance of two words in a single line is too far apart then google vision identifies 

them as two separate paragraphs/lines. 

The below images shows the sample output for a typical invoice from google vision.



This behaviour creates a problem in information extraction scenarios. For example, to extract a price of a product from a 

retail invoice the system needs to find a way to match the words in the same line. The algorithm proposed below performs 

line segmentation based on characters polygon coordinates for data extraction.

## Usage Guide

Usage instruction for each programing language is located in the ReadMe files inside the relevant folders.

### Proposed Algorithm

The implemented algorithm runs in two stages

- Stage 1 - Groups nearby words to generate a longer strip of line

- Stage 2 - Connects words which are far apart using the bounding polygon approach



## Explanation.

Stage one helps to reduce the computations needed for the second phase of the algorithm. In the first phase the algorithms

tries to merge words/characters which are very near. Stage 1 should be completed because for price related text like $3.40 is presented as 2 words by 

Google Vision (word 1: `$3.` word 2:`,40`). The first stage helps to concat nearby characters to form a text-block/word. 

This step helps reduces the computation needed for the second phase.

The stage 2 algorithm draws an imaginary bounding polygon (with a threshold) over the words and computes the 

words which belongs to each line.

## Issues.

The algorithm successfully works for most of the slanted and slightly crumpled images. But it will fail to highly 

crumpled or folded images.

## Test 

##### Node JS

- cd nodejs

- npm install

- npm test

## Future Work

Try to implement the water-flow algorithm for line segmentation and measure accuracies with bounding polygon approach.