{"id":28555121,"url":"https://github.com/pguso/pdf-extract-text","last_synced_at":"2025-07-06T09:31:33.752Z","repository":{"id":296210611,"uuid":"992609274","full_name":"pguso/pdf-extract-text","owner":"pguso","description":null,"archived":false,"fork":false,"pushed_at":"2025-05-29T13:04:23.000Z","size":26,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-06-10T05:44:46.967Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pguso.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-05-29T12:36:54.000Z","updated_at":"2025-05-29T13:04:26.000Z","dependencies_parsed_at":"2025-06-03T02:33:43.273Z","dependency_job_id":null,"html_url":"https://github.com/pguso/pdf-extract-text","commit_stats":null,"previous_names":["pguso/pdf-extract-text"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/pguso/pdf-extract-text","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pguso%2Fpdf-extract-text","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pguso%2Fpdf-extract-text/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pguso%2Fpdf-extract-text/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pguso%2Fpdf-extract-text/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pguso","download_url":"https://codeload.github.com/pguso/pdf-extract-text/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pguso%2Fpdf-extract-text/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":263208111,"owners_count":23430679,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-06-10T05:35:07.024Z","updated_at":"2025-07-06T09:31:33.747Z","avatar_url":"https://github.com/pguso.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# PDF Text Extractor\n\nA fast, native Node.js module to extract and process text from PDF files using Rust and N-API. Built with [Tokio](https://tokio.rs/), [`pdf-extract`](https://docs.rs/pdf-extract), and [`text-splitter`](https://crates.io/crates/text-splitter), this package provides efficient and easy-to-use async APIs.\n\n## Features\n- High-performance native code (Rust)\n- Asynchronous functions (non-blocking I/O)\n- Useful for LLM pipelines and search indexing\n- Extract cleaned text from PDF files\n- Split PDF text into pages with automatic page number detection\n- Generate overlapping text chunks with configurable sizes\n- TypeScript support\n\n## Installation\n\n```bash\nnpm install pdf-extract-text\n# or\nyarn add pdf-extract-text\n```\n\n## Usage\n\n### Basic Text Extraction\n\n**JavaScript**\n```javascript\nconst { extractTextFromPdf } = require('pdf-extract-text');\n\nasync function main() {\n  try {\n    const text = await extractTextFromPdf('./document.pdf');\n    console.log('Cleaned PDF text:', text);\n  } catch (error) {\n    console.error('Error:', error.message);\n  }\n}\n\nmain();\n```\n\n**TypeScript**\n```typescript\nimport { extractTextFromPdf } from 'pdf-extract-text';\n\nasync function main() {\n  try {\n    const text: string = await extractTextFromPdf('./document.pdf');\n    console.log('Cleaned PDF text:', text);\n  } catch (error) {\n    console.error('Error:', (error as Error).message);\n  }\n}\n\nmain();\n```\n\n### Page-based Extraction\n\n**JavaScript**\n```javascript\nconst { extractTextPages } = require('pdf-extract-text');\n\nasync function extractPages() {\n  try {\n    const pages = await extractTextPages('./document.pdf');\n    pages.forEach(page =\u003e {\n      console.log(`Page ${page.page}:`);\n      console.log(page.text);\n      console.log('\\n---\\n');\n    });\n  } catch (error) {\n    console.error('Error:', error.message);\n  }\n}\n\nextractPages();\n```\n**TypeScript**\n```typescript\nimport { extractTextPages, Page } from 'pdf-extract-text';\n\nasync function extractPages() {\n  try {\n    const pages: Page[] = await extractTextPages('./document.pdf');\n    pages.forEach((page: Page) =\u003e {\n      console.log(`Page ${page.page}:`);\n      console.log(page.text);\n      console.log('\\n---\\n');\n    });\n  } catch (error) {\n    console.error('Error:', (error as Error).message);\n  }\n}\n\nextractPages();\n```\n\n### Text Chunking with Overlaps\n\n**JavaScript**\n```javascript\nconst { extractTextChunks } = require('pdf-extract-text');\n\nasync function chunkText() {\n  try {\n    const chunks = await extractTextChunks('./document.pdf', 1000, 200);\n    chunks.forEach(chunk =\u003e {\n      console.log(`Chunk ${chunk.id}:`);\n      console.log(chunk.text);\n      console.log('\\n=====\\n');\n    });\n  } catch (error) {\n    console.error('Error:', error.message);\n  }\n}\n\nchunkText();\n```\n\n**TypeScript**\n```typescript\nimport { extractTextChunks, TextChunk } from 'pdf-extract-text';\n\nasync function chunkText() {\n  try {\n    const chunks: TextChunk[] = await extractTextChunks('./document.pdf', 1000, 200);\n    chunks.forEach((chunk: TextChunk) =\u003e {\n      console.log(`Chunk ${chunk.id}:`);\n      console.log(chunk.text);\n      console.log('\\n=====\\n');\n    });\n  } catch (error) {\n    console.error('Error:', (error as Error).message);\n  }\n}\n\nchunkText();\n```\n\n## Types\n\n```typescript\ntype Page = {\n  page: number;\n  text: string;\n};\n\ntype TextChunk = {\n  id: number;\n  text: string;\n};\n```\n\n## API Documentation\n\n### `extractTextFromPdf(path: string): Promise\u003cstring\u003e`\nExtracts and cleans text from a PDF file\n- `path`: Path to PDF file\n- Returns: Cleaned text with numeric-only lines removed\n\n### `extractTextPages(path: string): Promise\u003cPage[]\u003e`\nExtracts text split into pages\n```typescript\ninterface Page {\n  page: number;\n  text: string;\n}\n```\n\n### `extractTextChunks(path: string, chunkSize: number, chunkOverlap: number): Promise\u003cTextChunk[]\u003e`\nGenerates overlapping text chunks\n```typescript\ninterface TextChunk {\n  id: number;\n  text: string;\n}\n```\n- `chunkSize`: Target chunk size in characters\n- `chunkOverlap`: Overlap between chunks (must be \u003c chunkSize)\n\n## Error Handling\nAll functions throw errors with descriptive messages for:\n- File not found or read errors\n- PDF parsing failures\n- Invalid chunk configurations (overlap \u003e= chunk size)\n\n## Use Cases\n- Document understanding and chunking for LLMs\n- PDF content extraction for chatbots or search\n- Indexing and pre-processing for embeddings\n\n## Processing Details\n1. **Text Cleaning**:\n   - Removes lines containing only numeric characters\n   - Preserves original line breaks and formatting\n\n2. **Page Detection**:\n   - Splits text at lines containing only page numbers\n   - Handles variable page number positions\n\n3. **Chunking**:\n   - Uses semantic-aware splitting (paragraphs/sentences)\n   - Maintains context with overlapping chunks\n   - Configurable through simple parameters\n\n## Requirements\n- Node.js 16+\n- Rust (for building from source)\n\n## License\nMIT","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpguso%2Fpdf-extract-text","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpguso%2Fpdf-extract-text","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpguso%2Fpdf-extract-text/lists"}