{"id":13469360,"url":"https://github.com/invoice-x/invoice2data","last_synced_at":"2025-05-14T02:09:29.106Z","repository":{"id":30111003,"uuid":"33660820","full_name":"invoice-x/invoice2data","owner":"invoice-x","description":"Extract structured data from PDF invoices","archived":false,"fork":false,"pushed_at":"2025-05-12T07:11:14.000Z","size":2347,"stargazers_count":1965,"open_issues_count":59,"forks_count":501,"subscribers_count":65,"default_branch":"master","last_synced_at":"2025-05-12T08:37:25.655Z","etag":null,"topics":["data-mining","python"],"latest_commit_sha":null,"homepage":"https://invoice2data.readthedocs.io/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/invoice-x.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE.md","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2015-04-09T09:37:41.000Z","updated_at":"2025-05-12T07:11:18.000Z","dependencies_parsed_at":"2023-10-21T11:56:03.594Z","dependency_job_id":"3708d039-66bd-4876-8f7b-042c19b14c43","html_url":"https://github.com/invoice-x/invoice2data","commit_stats":{"total_commits":410,"total_committers":51,"mean_commits":8.03921568627451,"dds":0.697560975609756,"last_synced_commit":"0f50bc34d1ad04408b0c3eda961e559302ecc5fe"},"previous_names":[],"tags_count":95,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/invoice-x%2Finvoice2data","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/invoice-x%2Finvoice2data/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/invoice-x%2Finvoice2data/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/invoice-x%2Finvoice2data/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/invoice-x","download_url":"https://codeload.github.com/invoice-x/invoice2data/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253702579,"owners_count":21949999,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-mining","python"],"created_at":"2024-07-31T15:01:35.797Z","updated_at":"2025-05-14T02:09:24.088Z","avatar_url":"https://github.com/invoice-x.png","language":"Python","readme":"# Data extractor for PDF invoices - invoice2data\n\n[![Read the documentation at https://invoice2data.readthedocs.io/](https://img.shields.io/readthedocs/invoice2data/latest.svg?label=Read%20the%20Docs)][read the docs]\n[![invoice2data build status on GitHub Actions](https://github.com/invoice-x/invoice2data/workflows/Test/badge.svg)](https://github.com/invoice-x/invoice2data/actions)\n[![Version](https://img.shields.io/pypi/v/invoice2data.svg)](https://pypi.python.org/pypi/invoice2data)\n[![Support Python versions](https://img.shields.io/pypi/pyversions/invoice2data.svg)](https://pypi.python.org/pypi/invoice2data)\n[![License](https://img.shields.io/pypi/l/invoice2data)][license]\n[![Tests](https://github.com/invoice-x/invoice2data/workflows/Tests/badge.svg)][tests]\n[![Codecov](https://codecov.io/gh/invoice-x/invoice2data/branch/main/graph/badge.svg)][codecov]\n[![pre-commit](https://img.shields.io/badge/pre--commit-enabled-brightgreen?logo=pre-commit\u0026logoColor=white)][pre-commit]\n[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)\n\n[pypi status]: https://pypi.org/project/invoice2data/\n[read the docs]: https://invoice2data.readthedocs.io/\n[tests]: https://github.com/invoice-x/invoice2data/actions?workflow=Tests\n[codecov]: https://app.codecov.io/gh/invoice-x/invoice2data\n[pre-commit]: https://github.com/pre-commit/pre-commit\n[ruff badge]: https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json\n[ruff project]: https://github.com/charliermarsh/ruff\n\nA command line tool and Python library that automates the extraction of key information from invoices to support your accounting\nprocess. The library is very flexible and can be used on other types of business documents as well.\n\nIn essence, invoice2data simplifies the process of getting data from invoices by:\n\nAutomating text extraction: No more manual copying and pasting.\nUsing templates for structure: Handles different invoice layouts.\nProviding structured output: Makes the data ready for analysis or further processing.\nThis makes it a valuable tool for businesses and developers dealing with a large volume of invoices, saving time and reducing errors associated with manual data entry.\n\n1. extracts text from PDF files using different techniques, like\n   `pdftotext`, `text`, `ocrmypdf`, `pdfminer`, `pdfplumber` or OCR -- `tesseract`, or\n   `gvision` (Google Cloud Vision).\n2. searches for regex in the result using a YAML or JSON-based template system\n3. saves results as CSV, JSON or XML or renames PDF files to match the content.\n\nWith the flexible template system you can:\n\n- precisely match content PDF files\n- plugins available to match line items and tables\n- define static fields that are the same for every invoice\n- define custom fields needed in your organisation or process\n- have multiple regex per field (if layout or wording changes)\n- define currency\n- extract invoice-items using the `lines`-plugin developed by [Holger\n  Brunn](https://github.com/hbrunn)\n\nGo from PDF files to this:\n\n    {'date': (2014, 5, 7), 'invoice_number': '30064443', 'amount': 34.73, 'desc': 'Invoice 30064443 from QualityHosting', 'lines': [{'price': 42.0, 'desc': u'Small Business StandardExchange 2010\\nGrundgeb\\xfchr pro Einheit\\nDienst: OUDJQ_office\\n01.05.14-31.05.14\\n', 'pos': u'7', 'qty': 1.0}]}\n    {'date': (2014, 6, 4), 'invoice_number': 'EUVINS1-OF5-DE-120725895', 'amount': 35.24, 'desc': 'Invoice EUVINS1-OF5-DE-120725895 from Amazon EU'}\n    {'date': (2014, 8, 3), 'invoice_number': '42183017', 'amount': 4.11, 'desc': 'Invoice 42183017 from Amazon Web Services'}\n    {'date': (2015, 1, 28), 'invoice_number': '12429647', 'amount': 101.0, 'desc': 'Invoice 12429647 from Envato'}\n\n\n## Usage\n\nBasic usage. Process PDF files and write result to CSV.\nPlease see the [Command-line Reference] for details.\n\n- `invoice2data invoice.pdf`\n- `invoice2data invoice.txt`\n- `invoice2data *.pdf`\n\nChoose any of the following input readers:\n\n- pdftotext `invoice2data --input-reader pdftotext invoice.pdf`\n- pdftotext `invoice2data --input-reader text invoice.txt`\n- tesseract `invoice2data --input-reader tesseract invoice.pdf`\n- pdfminer.six `invoice2data --input-reader pdfminer invoice.pdf`\n- pdfplumber `invoice2data --input-reader pdfplumber invoice.pdf`\n- ocrmypdf `invoice2data --input-reader ocrmypdf invoice.pdf`\n- gvision `invoice2data --input-reader gvision invoice.pdf` (needs `GOOGLE_APPLICATION_CREDENTIALS` env var and a Google Cloud Bucket name. The bucket name can be set as an argument to the function ``to_text`` or as an Environment variable named ``GOOGLE_CLOUD_BUCKET_NAME`` )\n\nChoose any of the following output formats:\n\n- csv `invoice2data --output-format csv invoice.pdf`\n- json `invoice2data --output-format json invoice.pdf`\n- xml `invoice2data --output-format xml invoice.pdf`\n\nSave output file with custom name or a specific folder\n\n`invoice2data --output-format csv --output-name myinvoices/invoices.csv invoice.pdf`\n\n**Note:** You must specify the `output-format` in order to create\n`output-name`\n\nSpecify folder with yml templates. (e.g. your suppliers)\n\n`invoice2data --template-folder ACME-templates invoice.pdf`\n\nOnly use your own templates and exclude built-ins\n\n`invoice2data --exclude-built-in-templates --template-folder ACME-templates invoice.pdf`\n\nProcesses a folder of invoices and copies renamed invoices to new\nfolder.\n\n`invoice2data --copy new_folder folder_with_invoices/*.pdf`\n\nProcesses a single file and dumps whole file for debugging (useful when\nadding new templates in templates.py)\n\n`invoice2data --debug my_invoice.pdf`\n\nRecognize test invoices: `invoice2data invoice2data/test/pdfs/* --debug`\n\n### Use as Python Library\n\nYou can easily add `invoice2data` to your own Python scripts as library.\n\n    from invoice2data import extract_data\n    result = extract_data('path/to/my/file.pdf')\n\nUsing in-house templates\n\n    from invoice2data import extract_data\n    from invoice2data.extract.loader import read_templates\n\n    templates = read_templates('/path/to/your/templates/')\n    result = extract_data(filename, templates=templates)\n\n## Template system\n\nSee `invoice2data/extract/templates` for existing templates. Just extend\nthe list to add your own. If deployed by a bigger organisation, there\nshould be an interface to edit templates for new suppliers. 80-20 rule.\nFor a short tutorial on how to add new templates, see [tutorial.md](../docs/tutorial.md).\n\nTemplates are based on Yaml or JSON. They define one or more keywords to find\nthe right template, one or more exclude_keywords to further narrow it down\nand regexp for fields to be extracted. They could also be a static value,\nlike the full company name.\n\nTemplate files are tried in alphabetical order.\n\nWe may extend them to feature options to be used during invoice\nprocessing.\n\nExample:\n\n````yaml\n    issuer: Amazon Web Services, Inc.\n    keywords:\n    - Amazon Web Services\n    exclude_keywords:\n    - San Jose\n    fields:\n      amount: TOTAL AMOUNT DUE ON.*\\$(\\d+\\.\\d+)\n      amount_untaxed: TOTAL AMOUNT DUE ON.*\\$(\\d+\\.\\d+)\n      date: Invoice Date:\\s+([a-zA-Z]+ \\d+ , \\d+)\n      invoice_number: Invoice Number:\\s+(\\d+)\n      partner_name: (Amazon Web Services, Inc\\.)\n    options:\n      remove_whitespace: false\n      currency: HKD\n      date_formats:\n        - '%d/%m/%Y'\n    lines:\n        start: Detail\n        end: \\* May include estimated US sales tax\n        first_line: ^    (?P\u003cdescription\u003e\\w+.*)\\$(?P\u003cprice_unit\u003e\\d+\\.\\d+)\n        line: (.*)\\$(\\d+\\.\\d+)\n        skip_line: Note\n        last_line: VAT \\*\\*\n````\n\nThe lines package has multiple settings:\n\n- start \u003e The pattern where the lines begin. This is typically the header row of the table. This row is not included in the line matching.\n- end \u003e The pattern denoting where the lines end. Typically some text at the very end or immediately below the table. Also not included in the line matching.\n- first_line \u003e Optional. This is the primary line item for each entry.\n- line \u003e If first_line is not provided, this will be used as the primary line pattern. If first_line is provided, this is the pattern for any sub-lines such as line item details.\n- skip_line \u003e Optional. If first_line is passed, this pattern indicates which sub-lines will be skipped and their data not recorded. This is useful if tables span multiple pages and you need to skip over page numbers or headers that appear mid-table.\n- last_line \u003e Optional. If first_line is passed, this pattern denotes the final line of the sub-lines and is included in the output data.\n\n:warning: Invoice2data uses a yaml templating system. The yaml templates are loaded with [pyyaml](https://github.com/yaml/pyyaml) which is a pure python implementation. (thus rather slow)\nAs an alternative json templates can be used. Which are natively better supported by python.\n\nThe performance with yaml templates can be greatly increased **10x** by using [libyaml](https://github.com/yaml/libyaml)\nIt can be installed on most distributions by:\n`sudo apt-get libyaml-dev`\n\n## Development\n\nIf you are interested in improving this project, have a look at our\n[developer guide](../docs/contributing.md) to get you started quickly.\n\n## Roadmap and open tasks\n\n- integrate with online OCR?\n- try to 'guess' parameters for new invoice formats.\n- apply machine learning to guess new parameters / template creation\n- Data cleanup per field\n- advanced table parsing with [pypdf_table_extraction](https://github.com/py-pdf/pypdf_table_extraction)\n\n## Maintainers\n\n- [Manuel Riel](https://github.com/m3nu)\n- [Alexis de Lattre](https://github.com/alexis-via)\n- [bosd](https://github.com/bosd)\n\n## Contributors and Credits\n\n- [Harshit Joshi](https://github.com/duskybomb): As Google Summer of\n  Code student.\n- [Holger Brunn](https://github.com/hbrunn): Add support for parsing\n  invoice items.\n\n[pypi]: https://pypi.org/\n[file an issue]: https://github.com/invoice-x/invoice2data/issues\n[pip]: https://pip.pypa.io/\n\nContributions are very welcome.\nTo learn more, see the [Contributor Guide].\n\n## Used By\n\n- Odoo, OCA module [account_invoice_import_invoice2data](https://github.com/OCA/edi)\n\n## Related Projects\n\n- [OCR-Invoice](https://github.com/robela/OCR-Invoice) (FOSS \\| C\\#)\n- [DeepLogic AI](https://deeplogicai.tech/case_list/automatic-key-information-extraction-business-documents/) (Commercial \\| SaaS)\n- [Docparser](https://docparser.com/) (Commercial \\| Web Service)\n- [A-PDF](http://www.a-pdf.com/data-extractor/index.htm) (Commercial)\n- [PDFdeconstruct](http://www.glyphandcog.com/PDFdeconstruct.html?g6)\n  (Commercial)\n- [CVision](http://www.cvisiontech.com/library/document-automation/forms-processing/extract-data-from-invoice.html)\n  (Commercial)\n\n\u003c!-- github-only --\u003e\n\n[license]: https://github.com/invoice-x/invoice2data/blob/main/LICENSE\n[contributor guide]: https://github.com/invoice-x/invoice2data/blob/main/CONTRIBUTING.md\n[command-line reference]: https://invoice2data.readthedocs.io/en/latest/usage.html\n","funding_links":[],"categories":["Import Transactions","Python","python"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Finvoice-x%2Finvoice2data","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Finvoice-x%2Finvoice2data","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Finvoice-x%2Finvoice2data/lists"}