https://github.com/ds4sd/doclaynet
DocLayNet: A Large Human-Annotated Dataset for Document-Layout Analysis
https://github.com/ds4sd/doclaynet
Last synced: 5 months ago
JSON representation
DocLayNet: A Large Human-Annotated Dataset for Document-Layout Analysis
- Host: GitHub
- URL: https://github.com/ds4sd/doclaynet
- Owner: DS4SD
- License: other
- Created: 2022-02-09T16:07:30.000Z (over 4 years ago)
- Default Branch: main
- Last Pushed: 2023-02-01T06:44:02.000Z (over 3 years ago)
- Last Synced: 2025-08-10T23:51:47.532Z (12 months ago)
- Size: 59.8 MB
- Stars: 365
- Watchers: 7
- Forks: 21
- Open Issues: 3
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# DocLayNet
DocLayNet is a human-annotated document layout segmentation dataset containing `80863` pages from a broad variety of document sources.
## News
| | |
| :---------- | :----------------------------------- |
| **Jan. 26th, 2023** | The DocLayNet dataset is available on Hugging Face [ds4sd/DocLayNet](https://huggingface.co/datasets/ds4sd/DocLayNet). |
| **Jan. 13th, 2023** | We are hosting a competition on layout segmentation in corporate documents in ICDAR 2023. Find the details in the [competition website](https://ds4sd.github.io/icdar23-doclaynet/). |
## Overview
DocLayNet provides page-by-page layout segmentation ground-truth using bounding-boxes for 11 distinct class labels on 80863 unique pages from 6 document categories. It provides several unique features compared to related work such as PubLayNet or DocBank:
1. *Human Annotation*: DocLayNet is hand-annotated by well-trained experts, providing a gold-standard in layout segmentation through human recognition and interpretation of each page layout
2. *Large layout variability*: DocLayNet includes diverse and complex layouts from a large variety of public sources in Finance, Science, Patents, Tenders, Law texts and Manuals
3. *Detailed label set*: DocLayNet defines 11 class labels to distinguish layout features in high detail.
4. *Redundant annotations*: A fraction of the pages in DocLayNet are double- or triple-annotated, allowing to estimate annotation uncertainty and an upper-bound of achievable prediction accuracy with ML models
5. *Pre-defined train- test- and validation-sets*: DocLayNet provides fixed sets for each to ensure proportional representation of the class-labels and avoid leakage of unique layout styles across the sets.
## Using the dataset with :hugs: Hugging Face
The DocLayNet dataset is available on Hugging Face at [ds4sd/DocLayNet](https://huggingface.co/datasets/ds4sd/DocLayNet).
```py
>>> from datasets import load_dataset
>>> dataset = load_dataset("ds4sd/DocLayNet")
>>> dataset
DatasetDict({
train: Dataset({
features: ['image_id', 'image', 'width', 'height', 'doc_category', 'collection', 'doc_name', 'page_no', 'objects'],
num_rows: 69375
})
validation: Dataset({
features: ['image_id', 'image', 'width', 'height', 'doc_category', 'collection', 'doc_name', 'page_no', 'objects'],
num_rows: 6489
})
test: Dataset({
features: ['image_id', 'image', 'width', 'height', 'doc_category', 'collection', 'doc_name', 'page_no', 'objects'],
num_rows: 4999
})
})
```
## Download
| Dataset | Records | Size(GB) | URL |
|------------------|---------|----------|-------------|
| DocLayNet core dataset| 80,863 | 28 GiB | [Download](https://codait-cos-dax.s3.us.cloud-object-storage.appdomain.cloud/dax-doclaynet/1.0.0/DocLayNet_core.zip)|
| DocLayNet extra files| 80,863 | 7.5 GiB | [Download](https://codait-cos-dax.s3.us.cloud-object-storage.appdomain.cloud/dax-doclaynet/1.0.0/DocLayNet_extra.zip)|
Additionally, we provide the labeling guideline used for training of the annotation experts [here](https://raw.githubusercontent.com/DS4SD/DocLayNet/main/assets/DocLayNet_Labeling_Guide_Public.pdf).
## Dataset structure
DocLayNet provides four types of data assets:
1. PNG images of all pages, resized to square `1025 x 1025px`
2. Bounding-box annotations in COCO format for each PNG image
3. Extra: Single-page PDF files matching each PNG image
4. Extra: JSON file matching each PDF page, which provides the digital text cells with coordinates and content
The dataset is organized in the following directory structure:
*Doclaynet core dataset*
```
├── COCO
│ ├── test.json
│ ├── train.json
│ └── val.json
├── PNG
│ ├── .png
│ ├── ...
```
*Doclaynet extra files*
```
├── PDF
│ ├── .pdf
│ ├── ...
├── JSON
│ ├── .json
│ ├── ...
```
## Data format details
### Pages
Document pages are provided as bitmap images (PNG) and in original PDF format. The layout annotations in COCO format are referring to the PNG images only.
### COCO annotations
For bounding-box annotations, DocLayNet provides standard COCO format annotations as defined [here](https://cocodataset.org/#format-data).
Each COCO image record contains additional custom fields to allow data sub-selection and provide provenance.
#### Example page with bounding-box annotations
#### Example COCO image record
```js
...
{
"id": 1,
"width": 1025,
"height": 1025,
"file_name": "132a855ee8b23533d8ae69af0049c038171a06ddfcac892c3c6d7e6b4091c642.png",
// Custom fields:
"doc_category": "financial_reports" // high-level document category
"collection": "ann_reports_00_04_fancy", // sub-collection name
"doc_name": "NASDAQ_FFIN_2002.pdf", // original document filename
"page_no": 9, // page number in original document
"precedence": 0, // Annotation order, non-zero in case of redundant double- or triple-annotation
},
...
```
The `doc_category` field uses one of the following constants:
```
financial_reports,
scientific_articles,
laws_and_regulations,
government_tenders,
manuals,
patents
```
#### Example COCO annotation records
The annotation records shown below all belong to the page image shown above. Every bounding-box is a separate record, matched to the common `image_id`.
Click to expand...
```
"annotations": [
{
"id": 8,
"image_id": 1,
"category_id": 1,
"bbox": [
210.06018382352943,
31.14536268939389,
173.9850743464052,
39.270946654040586
],
"segmentation": [
[
210.06018382352943,
31.14536268939389,
210.06018382352943,
70.41630934343448,
384.04525816993464,
70.41630934343448,
384.04525816993464,
31.14536268939389
]
],
"area": 6832.558573256964,
"iscrowd": 0,
"precedence": 0
},
{
"id": 9,
"image_id": 1,
"category_id": 7,
"bbox": [
434.9334063800317,
-0.4906348977078778,
589.8585905504372,
590.2337819428021
],
"segmentation": [
[
434.9334063800317,
-0.4906348977078778,
434.9334063800317,
589.7431470450942,
1024.791996930469,
589.7431470450942,
1024.791996930469,
-0.4906348977078778
]
],
"area": 348154.46671203536,
"iscrowd": 0,
"precedence": 0
},
{
"id": 10,
"image_id": 1,
"category_id": 8,
"bbox": [
66.99346405228758,
112.10344760101009,
290.869358251634,
13.66279703282828
],
"segmentation": [
[
66.99346405228758,
112.10344760101009,
66.99346405228758,
125.76624463383837,
357.8628223039216,
125.76624463383837,
357.8628223039216,
112.10344760101009
]
],
"area": 3974.0890048610913,
"iscrowd": 0,
"precedence": 0
},
{
"id": 11,
"image_id": 1,
"category_id": 10,
"bbox": [
66.99346405228758,
133.5865287247475,
325.3560694444444,
131.31064046717177
],
"segmentation": [
[
66.99346405228758,
133.5865287247475,
66.99346405228758,
264.89716919191926,
392.34953349673196,
264.89716919191926,
392.34953349673196,
133.5865287247475
]
],
"area": 42722.71385863161,
"iscrowd": 0,
"precedence": 0
},
{
"id": 12,
"image_id": 1,
"category_id": 10,
"bbox": [
66.99346405228758,
272.84557828282834,
325.4857017973857,
131.3025
],
"segmentation": [
[
66.99346405228758,
272.84557828282834,
66.99346405228758,
404.14807828282835,
392.4791658496732,
404.14807828282835,
392.4791658496732,
272.84557828282834
]
],
"area": 42737.08636025123,
"iscrowd": 0,
"precedence": 0
},
{
"id": 13,
"image_id": 1,
"category_id": 10,
"bbox": [
66.99346405228758,
414.8919160353536,
325.69678145424837,
80.83059406565656
],
"segmentation": [
[
66.99346405228758,
414.8919160353536,
66.99346405228758,
495.72251010101013,
392.6902455065359,
495.72251010101013,
392.6902455065359,
414.8919160353536
]
],
"area": 26326.26433021921,
"iscrowd": 0,
"precedence": 0
},
{
"id": 14,
"image_id": 1,
"category_id": 10,
"bbox": [
112.37566899509804,
626.7556887626263,
863.111772998366,
310.9754294507576
],
"segmentation": [
[
112.37566899509804,
626.7556887626263,
112.37566899509804,
937.7311182133839,
975.487441993464,
937.7311182133839,
975.487441993464,
626.7556887626263
]
],
"area": 268406.55427217163,
"iscrowd": 0,
"precedence": 0
},
{
"id": 15,
"image_id": 1,
"category_id": 5,
"bbox": [
18.32874183006536,
1005.2406660353536,
7.4496732026143775,
10.353535094696968
],
"segmentation": [
[
18.32874183006536,
1005.2406660353536,
18.32874183006536,
1015.5942011300506,
25.77841503267974,
1015.5942011300506,
25.77841503267974,
1005.2406660353536
]
],
"area": 77.13045294729152,
"iscrowd": 0,
"precedence": 0
},
...
]
```
### Extra: JSON files
DocLayNet provides auxiliary JSON files which contain the bounding-box coordinates and text for every PDF cell, and additional metadata. This data is generated only from the PDFs and is independent from the annotations.
#### Example JSON data
The snippet below shows part of the JSON data for the page shown further above. The text cell shown is the section heading (index: 3).
```js
{
"metadata": {
"page_hash": "132a855ee8b23533d8ae69af0049c038171a06ddfcac892c3c6d7e6b4091c642", // unique identifier, equal to filename
"original_filename": "NASDAQ_FFIN_2002.pdf", // original document filename
"page_no": 9, // page number in original document
"num_pages": 28, // total pages in original document
"original_width": 612, // width in pixels @72 ppi
"original_height": 792, // height in pixels @72 ppi
"coco_width": 1025, // with in pixels in PNG and COCO format
"coco_height": 1025, // with in pixels in PNG and COCO format
"collection": "ann_reports_00_04_fancy", // sub-collection name
"doc_category": "financial_reports" // high-level document category
},
"cells": [ // all text cells in the digital PDF data
{
// Bounding-box coordinates of text cells,
// formatted as [x,y,w,h] (same as COCO annotations)
// where (x,y) is the upper-left corner and
// (w,h) is the width and height
// in the coordinate space of (0,0, coco_width, coco_height)
"bbox": [
66.99346405228758,
112.10344760101009,
290.869358251634,
13.66279703282828
],
"text": "Leigh Taliaferro, M.D., values consistency.", // string content of cell
"font": {
"color": [
12,
72,
142,
255
],
"name": "/AAAAAC+HelveticaNeue-Medium",
"size": 1
}
},
...
```
## Paper
**"DocLayNet: A Large Human-Annotated Dataset for Document-Layout Analysis"** (KDD 2022).
- Birgit Pfitzmann (bpf@zurich.ibm.com)
- Christoph Auer (cau@zurich.ibm.com)
- Michele Dolfi (dol@zurich.ibm.com)
- Ahmed Nassar (ahn@zurich.ibm.com)
- Peter Staar (taa@zurich.ibm.com)
ArXiv link: https://arxiv.org/abs/2206.01062
**Citation:**
```
@article{doclaynet2022,
title = {DocLayNet: A Large Human-Annotated Dataset for Document-Layout Analysis},
doi = {10.1145/3534678.353904},
url = {https://arxiv.org/abs/2206.01062},
author = {Pfitzmann, Birgit and Auer, Christoph and Dolfi, Michele and Nassar, Ahmed S and Staar, Peter W J},
year = {2022}
}
```