{"id":21289440,"url":"https://github.com/hansmi/paperminer","last_synced_at":"2025-03-15T16:10:12.583Z","repository":{"id":222912000,"uuid":"758717359","full_name":"hansmi/paperminer","owner":"hansmi","description":"Amend Paperless documents with extracted information","archived":false,"fork":false,"pushed_at":"2025-03-13T15:11:09.000Z","size":246,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-13T16:25:22.424Z","etag":null,"topics":["paperless","paperless-ngx","pdf"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/hansmi.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-02-16T22:58:01.000Z","updated_at":"2025-03-13T15:11:11.000Z","dependencies_parsed_at":"2024-02-21T20:29:21.713Z","dependency_job_id":"c1be51ce-14f9-45c0-8993-af6b9152ef24","html_url":"https://github.com/hansmi/paperminer","commit_stats":null,"previous_names":["hansmi/paperminer"],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hansmi%2Fpaperminer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hansmi%2Fpaperminer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hansmi%2Fpaperminer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hansmi%2Fpaperminer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/hansmi","download_url":"https://codeload.github.com/hansmi/paperminer/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243754094,"owners_count":20342542,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["paperless","paperless-ngx","pdf"],"created_at":"2024-11-21T12:40:19.826Z","updated_at":"2025-03-15T16:10:12.559Z","avatar_url":"https://github.com/hansmi.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Amend Paperless documents with extracted information\n\n[![Latest release](https://img.shields.io/github/v/release/hansmi/paperminer)][releases]\n[![CI workflow](https://github.com/hansmi/paperminer/actions/workflows/ci.yaml/badge.svg)](https://github.com/hansmi/paperminer/actions/workflows/ci.yaml)\n[![Go reference](https://pkg.go.dev/badge/github.com/hansmi/paperminer.svg)](https://pkg.go.dev/github.com/hansmi/paperminer)\n\nPaperminer is a system for amending documents stored in\n[Paperless-ngx][paperless] with additional information (\"facts\") extracted from\nthe documents themselves or other sources.\n\nThe [`hansmi/dossier` package][dossier] is called to parse PDF documents (other\nformats could be implemented).\n\nThe Go programming language's [`plugin` package][gopkgplugin] comes with\na number of caveats which make it unsuitable. Compile-time plugins via the\n[`hansmi/staticplug` package][staticplug] are used instead. It's therefore\nnecessary to set up your own build. An example for a program with a plugin can\nbe found in the [`example/myminer` directory](./example/myminer).\n\nPlugins may use [dossier sketches][dossiersketch] to look for specific regular\nexpressions at absolute or relative positions on pages. The [`sketchfacts`\npackage](./pkg/sketchfacts/) is often sufficient even though it ignores pages\nbeyond the first. Custom logic can produce document facts from the findings.\n\nPlugins may also extract arbitrary document pages and implement their own data\nextraction. External APIs may also be involved.\n\nNormalizing extracted text before parsing it further is generally recommended,\nnot just for date and time: remove extraneous whitespace and separators, etc.\nRegular expressions should also be written to be flexible where possible.\nOCR-derived text is often not exactly the same as the original.\n\nUseful packages for writing document facters:\n\n* [`hansmi/zyt`][zyt]: Parse language/locale-specific date and time formats.\n* [`hansmi/aurum`][aurum]: Golden tests. Used for generic document facter tests\n  by the [`factertest` package](./pkg/factertest).\n\n[aurum]: https://github.com/hansmi/aurum/\n[dossier]: https://github.com/hansmi/dossier/\n[dossiersketch]: https://github.com/hansmi/dossier/#sketches\n[gopkgplugin]: https://pkg.go.dev/plugin@go1.22.0\n[paperless]: https://docs.paperless-ngx.com/\n[releases]: https://github.com/hansmi/paperminer/releases/latest\n[staticplug]: https://github.com/hansmi/staticplug/\n[zyt]: https://github.com/hansmi/zyt/\n\n\u003c!-- vim: set sw=2 sts=2 et : --\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhansmi%2Fpaperminer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhansmi%2Fpaperminer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhansmi%2Fpaperminer/lists"}