https://github.com/wzbsocialsciencecenter/pdf2xml-viewer
A simple viewer and inspection tool for text boxes in PDF documents
https://github.com/wzbsocialsciencecenter/pdf2xml-viewer
d3 ocr pdf pdf-document viewer xml
Last synced: 4 months ago
JSON representation
A simple viewer and inspection tool for text boxes in PDF documents
- Host: GitHub
- URL: https://github.com/wzbsocialsciencecenter/pdf2xml-viewer
- Owner: WZBSocialScienceCenter
- License: apache-2.0
- Created: 2016-07-08T09:18:01.000Z (almost 10 years ago)
- Default Branch: master
- Last Pushed: 2022-03-07T10:45:34.000Z (over 4 years ago)
- Last Synced: 2025-04-12T20:36:24.729Z (about 1 year ago)
- Topics: d3, ocr, pdf, pdf-document, viewer, xml
- Language: HTML
- Size: 110 KB
- Stars: 95
- Watchers: 9
- Forks: 20
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# pdf2xml-viewer - A simple viewer and inspection tool for text boxes in PDF documents
July 2016 / Feb. 2017, Markus Konrad / / [Berlin Social Science Center](https://www.wzb.eu/en)
**This project is currently not maintained.**
## Introduction
This is a small tool with which it is possible to view and examine individual text boxes in PDF documents. This is
very helpful for analyzing the distribution of texts across a page, especially in the case of
[OCR-processed PDFs](https://en.wikipedia.org/wiki/Optical_character_recognition) (so called "sandwich PDFs") from
which you might want to extract structured information (see
[pdftabextract](https://github.com/WZBSocialScienceCenter/pdftabextract) for this). With this viewer, you can examine
such PDFs and have a look at the properties of individual text boxes, like position, width, height or font
specification. In combination with pdftabextract, you can view the grids that were generated for the detected columns and rows. [This blog post](https://datascience.blog.wzb.eu/2017/02/16/data-mining-ocr-pdfs-using-pdftabextract-to-liberate-tabular-data-from-scanned-documents/) shows an example usage.
The viewer requires you to convert your PDFs to the *pdf2xml* format. Afterwards
you can start up a local webserver, display this XML file in the viewer (as seen below) and examine the individual
text boxes with your browsers developer console.
The created file in pdf2xml format can later also be used to extract structured information, which I explain in my
series of blog posts about [data mining PDFs](https://datascience.blog.wzb.eu/category/pdfs/).
## How to use it
### 1. Convert a PDF to pdf2xml format
At first, you need to convert your PDFs using the **poppler-utils**, a package which is part of most Linux distributions
and is also available for OSX via Homebrew or MacPorts. From this package we need the command `pdftohtml` and can create
an XML file in pdf2xml format in the following way using the Terminal:
```
pdftohtml -c -hidden -xml input.pdf output.xml
```
The arguments *input.pdf* and *output.xml* are your input PDF file and the created XML file in pdf2xml format
respectively. It is important that you specifiy the *-hidden* parameter when you're dealing with OCR-processed
("sandwich") PDFs. You can furthermore add the parameters *-f n* and *-l n* to set only a range of pages to be
converted.
### 2. Start a minimal local webserver to display the text boxes in the PDF with the viewer
Now that you have your file(s) in pdf2xml format, change to the directory where pdf2xml-viewer resides (where it's
*index.html* file is). You should also copy the generated XML files to this location. Now let's start up a minimal
local webserver. This can be done very easily with Python, which is installed on Linux and Mac OSX by default.
You can do so in the Terminal with Python 2.x:
```
python -m SimpleHTTPServer 8080
```
Or with Python 3:
```
python3 -m http.server 8080 --bind 127.0.0.1
```
Now you open your browser and go to the adress http://127.0.0.1:8080. The viewer shows up and you can now enter the
file name of your file to load (it must be relative to the directory in which pdf2xml-viewer resides). If you just
want to see an example, type in *example/ocr-output.pdf.xml* and load this file. Now you browse through the pages of
your PDF document and you'll see the text boxes with red frames. You can further examine these boxes by using your
browser's inspection tools (right click on element and select "Inspect" in Chrome or Firefox) as seen below:
### 3. Use the advanced features of the viewer
You can load a page grid JSON file that was generated with [pdftabextract](https://github.com/WZBSocialScienceCenter/pdftabextract) (function `common.save_page_grids`):
### 4. Extract data from your PDFs
If you want to extract structured data from the PDFs, you should have a look at the
[pdftabextract](https://github.com/WZBSocialScienceCenter/pdftabextract) package.
## Technical details
This viewer uses [d3.js](https://d3js.org) to display the pdf2xml file. I chose this approach because it is the fastest
and simples in order to inspect individual elements of a (OCR-processed) PDF document, without using expensive special
software. Furthermore it allows to add additional features such as displaying overlays of calculated lines or grids.
## License
Apache License 2.0. See LICENSE file.