{"id":50536731,"url":"https://github.com/fairdataihub/poster-repo-to-json","last_synced_at":"2026-06-03T17:01:03.035Z","repository":{"id":358144978,"uuid":"1122497400","full_name":"fairdataihub/poster-repo-to-json","owner":"fairdataihub","description":"Extract scientific poster content to machine-actionable JSON using LLMs. Part of the Machine-Actionable Poster Initiative (Beta).","archived":false,"fork":false,"pushed_at":"2026-05-15T22:28:45.000Z","size":271,"stargazers_count":1,"open_issues_count":1,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-16T00:56:10.691Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/fairdataihub.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-12-24T21:40:01.000Z","updated_at":"2026-05-15T22:28:49.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/fairdataihub/poster-repo-to-json","commit_stats":null,"previous_names":["fairdataihub/poster-repo-to-json"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/fairdataihub/poster-repo-to-json","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fairdataihub%2Fposter-repo-to-json","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fairdataihub%2Fposter-repo-to-json/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fairdataihub%2Fposter-repo-to-json/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fairdataihub%2Fposter-repo-to-json/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/fairdataihub","download_url":"https://codeload.github.com/fairdataihub/poster-repo-to-json/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fairdataihub%2Fposter-repo-to-json/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33874679,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-03T02:00:06.370Z","response_time":59,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-06-03T17:01:01.879Z","updated_at":"2026-06-03T17:01:03.026Z","avatar_url":"https://github.com/fairdataihub.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# poster-repo-to-json (staging)\n\n\u003e **Part of the Machine-Actionable Scientific Poster Initiative**\n\nFull pipeline for converting scientific posters into machine-actionable JSON. Uses [poster2json](https://github.com/fairdataihub/poster2json) for extraction (pdfalto + Qwen2-VL + Llama 3.1 8B), then enriches with Zenodo/Figshare repository metadata. Output conforms to the [poster-json-schema](https://posters.science/schema/v0.1/poster_schema.json) (DataCite 4.6 with poster extensions).\n\n**This `staging` branch** contains the refactored, production-hardened pipeline used to process the full Zenodo+Figshare corpus (~24K posters) on a multi-GPU node.\n\n## Pipeline Position\n\n1. **Collect** posters ([poster-repo-scraper](https://github.com/fairdataihub/poster-repo-scraper))\n2. **Validate** and classify ([poster-repo-qc](https://github.com/fairdataihub/poster-repo-qc))\n3. **Extract and enrich** to machine-actionable JSON (this package)\n\n## How It Works\n\n```\nposter.pdf ──► poster2json ──► scaffold JSON ──► merge ──► final JSON\n                                                   ▲\n                                                   │\nZenodo/Figshare metadata ──► SchemaConverter ──────┘\n                                               (backfill)\n```\n\n**poster2json output is PRIMARY.** Repository metadata only fills in fields that poster2json left empty or is authoritative for (conference name/dates, DOIs, licenses, funding, etc.). The extracted poster content (title, abstract, sections, captions) is gospel.\n\n### Merge Rules (staging branch)\n\n| Field | Source of truth |\n|-------|----------------|\n| `titles`, `descriptions`, `content`, `imageCaptions`, `tableCaptions` | **Extraction only** (never overwritten) |\n| `conference` | **Metadata supersedes extraction** (LLM hallucinates conferences; repositories have authoritative data) |\n| `identifiers` | **Metadata only** (real poster DOI + record ID). DOIs found in poster text get moved to `relatedIdentifiers` with `relationType: \"References\"` |\n| `dates`, `language`, `types`, `rightsList`, `fundingReferences`, `publisher` | Extraction base, metadata backfill if missing |\n| `creators` | Extraction base; metadata backfills ORCIDs, affiliations |\n| `subjects` | Union of both (deduped, case-insensitive) |\n\n### Quality Enforcement\n\n- **Single description only** — schema expects one abstract; if the LLM dumps multiple description entries, the merger keeps only the first `Abstract`-typed one.\n- **Post-batch QC** (`scripts/post_batch_qc.py`) writes a TSV of failed extractions for retry (multi-description, no-content, corrupt JSON, OCR failures).\n\n## Installation\n\n```bash\n# Clone with submodules\ngit clone https://github.com/fairdataihub/poster-repo-to-json.git\ncd poster-repo-to-json\ngit checkout staging\n\n# Install this package\npip install -e .\n\n# Install poster2json as a sibling directory (patched version)\ngit clone https://github.com/fairdataihub/poster2json.git ../poster2json\ncd ../poster2json \u0026\u0026 pip install -e .\n```\n\n### Prerequisites\n\n- **CUDA GPU** with 16GB+ VRAM (or 6GB+ with 4-bit quantization)\n- **pdfalto** for PDF layout analysis\n\n```bash\ngit clone https://github.com/kermitt2/pdfalto.git\ncd pdfalto \u0026\u0026 mkdir build \u0026\u0026 cd build \u0026\u0026 cmake .. \u0026\u0026 make\nsudo cp pdfalto /usr/local/bin/\n```\n\n## Single-Poster Usage\n\n```bash\n# One-shot extraction via poster2json + merge with metadata\npython run_extraction.py \\\n    --posters /path/to/poster/pdfs \\\n    --metadata /path/to/repo/metadata \\\n    --output ./output\n```\n\n## Multi-GPU Batch Pipeline\n\nThe `scripts/` directory contains the production batch pipeline used for large corpora:\n\n```\nscripts/\n├── batch_extract_v2.py      # Two-phase extraction (text first, LLM second)\n├── run_2gpu.sh              # Launcher for 2-GPU parallel processing\n├── post_batch_qc.py         # Quality check + failed extraction log\n├── run_merge.py             # Convert metadata + merge incrementally\n└── clean_stale_errors.py    # Remove stale error files with recoverable cached text\n```\n\n### Architecture: Two-Phase Extraction\n\n**Phase 1 (CPU+GPU):** Extract raw text from all posters.\n- `pdfalto` for text-based PDFs (fast, CPU-only)\n- Qwen2-VL OCR fallback for image-based PDFs (GPU)\n- Raw text cached to disk (`_raw_text/`) so it survives crashes\n\n**Phase 2 (GPU):** Load Llama 3.1 8B once, process all texts sequentially.\n- Model stays resident (no reload overhead between posters)\n- 4-bit NF4 quantization by default (~5GB VRAM, quality preserved)\n- Individual saves per poster (crash-safe resume)\n\n### Running on 2 GPUs\n\n```bash\n# Set poster2json location\nexport POSTER2JSON_PATH=/path/to/poster2json\n\n# Split input across GPUs (zenodo first, figshare second)\n# Place symlinks in ./gpu_splits/gpu0, gpu1, gpu2, gpu3\n\n# Launch (runs gpu0+gpu1 first, then gpu2+gpu3 after they finish)\nbash scripts/run_2gpu.sh\n```\n\n### Running on 3 GPUs (with 4-bit quantization)\n\nWith the JSON model in 4-bit NF4 (~5GB VRAM, default in poster2json 0.4.x), three RTX 3090s at 250W each pull ~600W combined under load — within the same circuit budget as the old 2-GPU bfloat16 setup. The third GPU adds ~50% throughput.\n\n```bash\n# Cap all four GPUs to 250W (idempotent, runs from Windows side on WSL)\nnvidia-smi -i 0 -pl 250\nnvidia-smi -i 1 -pl 250\nnvidia-smi -i 2 -pl 250\nnvidia-smi -i 3 -pl 250\n\n# Launch: 3 GPUs in parallel on splits 0,1,2; then GPU 0 picks up split 3\nbash scripts/run_3gpu.sh\n```\n\n**Why not 4 GPUs?** Running all four RTX 3090s at full load (350W each) on a single consumer circuit trips overcurrent protection. Three at 250W stays within a standard 15A/20A circuit. Four at 250W is right at the edge — possible, not advised. `nvidia-smi -pl 250` limits draw.\n\n### Post-Batch Quality Check (default)\n\nQC runs **automatically after every batch** in both `run_2gpu.sh` and `run_3gpu.sh` — no manual step required. Each run writes the failure list to `failed_extractions.tsv` so you can retry the bad ones.\n\nYou can also invoke it standalone:\n\n```bash\npython scripts/post_batch_qc.py\n# Writes: /path/to/output/extractions/../failed_extractions.tsv\n```\n\nChecks for:\n- `multi_description` — LLM dumped section content into descriptions instead of `content.sections`\n- `no_content` — no sections extracted\n- `corrupt_json` — mid-write crash\n- `extraction_error` — Phase 1 failures (OCR couldn't recover)\n\n### Backfilling New poster2json Features\n\nWhen poster2json ships new enrichment features (e.g. SPDX license normalization, ROR affiliation IDs, heuristic language detection, `researchField` on the OpenAlex 4 domains), existing extraction JSONs can be updated in-place without re-running the LLM:\n\n```bash\nexport POSTER2JSON_PATH=/path/to/poster2json\npython scripts/backfill_features.py --extractions /path/to/output/extractions\n```\n\nThe backfill is idempotent and reuses cached raw text from `_raw_text/` for language detection. It applies:\n\n- SPDX license normalization (`rightsList` entries get canonical SPDX IDs + URIs)\n- Subject dedupe + NFKC cleanup\n- ROR enrichment on `creators.affiliation` and `publisher` (canonical names + ROR IDs)\n- Heuristic language detection from raw text (overrides LLM-hallucinated language)\n- `researchField` placeholder strip (drops \"Other\", \"Unknown\", etc. → `null`)\n- NFKC normalization on titles and descriptions\n\nThe feature modules (`normalize.py`, `ror.py`, `language.py`) are vendored in `vendor/poster2json_features/` for reference.\n\n### Stale Error Recovery\n\nIf earlier runs had bugs (OOM crashes, broken pdfalto flag, etc.), those stale error JSONs block re-processing. This script removes them if the raw text is cached (Phase 1 succeeded but Phase 2 failed):\n\n```bash\npython scripts/clean_stale_errors.py\n# Re-running the pipeline will reprocess them with instant Phase 1 (cache hit)\n```\n\n### Incremental Merging\n\n```bash\npython scripts/run_merge.py\n# Converts repository metadata + merges with available extractions.\n# Safe to re-run — only processes new files.\n```\n\n## CLI Commands\n\n```bash\n# Single-step commands (wraps the Python API)\nposter-to-json extract --input ./posters --output ./extractions\nposter-to-json convert --input ./metadata/zenodo --output ./converted --source zenodo\nposter-to-json merge --extractions ./extractions --metadata ./converted --output ./merged\nposter-to-json pipeline --posters ./posters --metadata ./metadata --output ./output\n```\n\n## Python API\n\n```python\nfrom poster_to_json import PosterExtractor, SchemaConverter, MetadataMerger\n\nextractor = PosterExtractor()\nextraction = extractor.extract(\"poster.pdf\")\n\nconverter = SchemaConverter()\nmetadata = converter.convert_zenodo(raw_zenodo_record)\n\nmerger = MetadataMerger()\ncomplete = merger.merge(extraction, metadata)\n```\n\n## Output Schema\n\nConforms to the **poster-json-schema** (DataCite 4.6 with poster extensions):\n\n```json\n{\n  \"$schema\": \"https://posters.science/schema/v0.1/poster_schema.json\",\n  \"identifiers\": [{\"identifier\": \"10.5281/zenodo.12345678\", \"identifierType\": \"DOI\"}],\n  \"titles\": [{\"title\": \"...\"}],\n  \"creators\": [{\"name\": \"Smith, John\", \"nameIdentifiers\": [...], \"affiliation\": [...]}],\n  \"descriptions\": [{\"descriptionType\": \"Abstract\", \"description\": \"...\"}],\n  \"content\": {\n    \"sections\": [\n      {\"sectionTitle\": \"Introduction\", \"sectionContent\": \"...\"},\n      {\"sectionTitle\": \"Methods\", \"sectionContent\": \"...\"},\n      {\"sectionTitle\": \"Results\", \"sectionContent\": \"...\"}\n    ]\n  },\n  \"imageCaptions\": [{\"caption\": \"Figure 1: ...\"}],\n  \"tableCaptions\": [{\"caption\": \"Table 1: ...\"}],\n  \"conference\": {\"conferenceName\": \"...\", \"conferenceYear\": 2025},\n  \"relatedIdentifiers\": [{\"relatedIdentifier\": \"10.1234/...\", \"relatedIdentifierType\": \"DOI\", \"relationType\": \"References\"}]\n}\n```\n\n## Directory Structure\n\n```\nposter-repo-to-json/\n├── src/poster_to_json/\n│   ├── extractor.py         # Thin wrapper around poster2json\n│   ├── schema_converter.py  # Zenodo/Figshare → poster_schema.json\n│   ├── merger.py             # Extraction base + metadata backfill (conference superseding, single description)\n│   ├── cli.py                # CLI commands\n│   └── poster_schema.json    # Bundled schema\n├── scripts/                  # Production batch pipeline\n│   ├── batch_extract_v2.py\n│   ├── run_2gpu.sh\n│   ├── post_batch_qc.py\n│   ├── run_merge.py\n│   └── clean_stale_errors.py\n├── run_extraction.py         # Single-instance batch extraction\n├── posters/                  # Input: classified poster PDFs\n├── metadata/                 # Input: per-record JSON metadata\n└── output/\n    ├── extractions/          # poster2json raw output (+ _raw_text/ cache)\n    ├── converted/            # Metadata converted to schema\n    └── merged/               # Final merged records\n```\n\n## Operational Notes\n\n- **Windows Task Scheduler** is used to launch `run_2gpu.sh` so it survives SSH disconnects: `schtasks /create /tn PosterExtraction /tr \"wsl.exe -d Ubuntu-24.04 -e bash /path/to/run_2gpu.sh\" /sc once /st 00:00 /f /rl highest \u0026\u0026 schtasks /run /tn PosterExtraction`\n- **Power limits** (`nvidia-smi -pl 250`) reset on reboot — re-apply after any power event.\n- **Vision OCR needs torchvision** and cuDNN is disabled at the module level to avoid a driver incompatibility on CUDA 12.9 + cuDNN 9.19.\n\n## Related Packages\n\n| Package | Purpose |\n|---------|---------|\n| [poster2json](https://github.com/fairdataihub/poster2json) | Core extraction engine (pdfalto + Qwen2-VL + Llama 3.1) |\n| [poster-repo-scraper](https://github.com/fairdataihub/poster-repo-scraper) | Scrape poster metadata from Zenodo/Figshare |\n| [poster-repo-qc](https://github.com/fairdataihub/poster-repo-qc) | Validate and classify posters with PosterSentry |\n| [poster-json-schema](https://posters.science/schema/v0.1/poster_schema.json) | DataCite 4.6-based schema for scientific posters |\n\n## License\n\nMIT License - See [LICENSE](LICENSE) for details.\n\n## Citation\n\n```bibtex\n@software{poster_repo_to_json,\n  title = {poster-repo-to-json: Machine-Actionable Scientific Poster Pipeline},\n  author = {{FAIR Data Innovations Hub}},\n  year = {2026},\n  url = {https://github.com/fairdataihub/poster-repo-to-json}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffairdataihub%2Fposter-repo-to-json","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffairdataihub%2Fposter-repo-to-json","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffairdataihub%2Fposter-repo-to-json/lists"}