{"id":50349404,"url":"https://github.com/labrodev/document-sampler","last_synced_at":"2026-05-29T20:30:23.053Z","repository":{"id":353538306,"uuid":"1219847290","full_name":"labrodev/document-sampler","owner":"labrodev","description":"Pure PHP library that extracts a structured, representative sample from a document of any length. No framework dependency, no HTTP calls, no AI — just text processing.","archived":false,"fork":false,"pushed_at":"2026-04-24T09:32:51.000Z","size":11,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-04-24T11:33:36.943Z","etag":null,"topics":["ai","knowledge","knowledge-base","md","php","php-library","php-package","php-utility","rag-pipeline"],"latest_commit_sha":null,"homepage":"","language":"PHP","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/labrodev.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-04-24T09:25:33.000Z","updated_at":"2026-04-24T09:32:55.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/labrodev/document-sampler","commit_stats":null,"previous_names":["labrodev/document-sampler"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/labrodev/document-sampler","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/labrodev%2Fdocument-sampler","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/labrodev%2Fdocument-sampler/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/labrodev%2Fdocument-sampler/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/labrodev%2Fdocument-sampler/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/labrodev","download_url":"https://codeload.github.com/labrodev/document-sampler/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/labrodev%2Fdocument-sampler/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33670211,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-05-29T02:00:06.066Z","response_time":107,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","knowledge","knowledge-base","md","php","php-library","php-package","php-utility","rag-pipeline"],"created_at":"2026-05-29T20:30:22.264Z","updated_at":"2026-05-29T20:30:23.047Z","avatar_url":"https://github.com/labrodev.png","language":"PHP","funding_links":[],"categories":[],"sub_categories":[],"readme":"# DocumentSampler\n\nPure PHP library that extracts a structured, representative sample from a document of any length. No framework dependency, no HTTP calls, no AI — just text processing.\n\nDesigned as the input layer for downstream AI-powered packages such as relevance checkers, prompt injection detectors, and depersonalisation services.\n\n---\n\n## Requirements\n\n- PHP `^8.5`\n\n---\n\n## Installation\n\n```bash\ncomposer require labrodev/document-sampler\n```\n\n---\n\n## Basic usage\n\n```php\nuse Labrodev\\DocumentSampler\\DocumentSampler;\n\n$result = (new DocumentSampler())-\u003esample($rawText);\n\n$result-\u003eintro             // opening chars — title and introduction\n$result-\u003eoutline           // extracted section headings from anywhere in the document\n$result-\u003emiddle            // fixed window centred on the document midpoint\n$result-\u003etail              // closing chars — conclusion and sign-off\n$result-\u003etext              // all samples joined with separators\n$result-\u003echarCount         // character count of the combined sample\n$result-\u003eoriginalCharCount // character count of the original document\n```\n\n---\n\n## Custom window sizes\n\nBy default each zone uses the window defined on the `DocumentPart` enum. Pass any subset to the constructor to override:\n\n```php\n// Override specific zones — unset zones use the enum defaults\n$sampler = new DocumentSampler(\n    intro:   2000,\n    middle:  300,\n);\n\n$result = $sampler-\u003esample($rawText);\n```\n\n---\n\n## How it works\n\nThe sampler partitions every document into four fixed-size windows regardless of document length:\n\n| Zone | Default window | What it captures |\n|------|---------------|-----------------|\n| `intro` | 1000 chars | Title, abstract, opening paragraphs |\n| `outline` | 500 chars | Section headings (`# Markdown`, `1.1 Numbered`, `ALL-CAPS` lines) |\n| `middle` | 500 chars | Window centred on the document midpoint |\n| `tail` | 500 chars | Closing paragraphs, conclusion, signature |\n\nWindows are fixed — a 400-page PDF gets the same sized sample as a one-page memo. The goal is a compact, representative fingerprint of the document, not a summary.\n\n---\n\n## Exporting results\n\n### JSON\n\n```php\n$result-\u003etoJson();\n```\n\n```json\n{\n    \"meta\": {\n        \"originalCharCount\": 50000,\n        \"sampledCharCount\": 2300\n    },\n    \"samples\": {\n        \"intro\": \"...\",\n        \"outline\": \"...\",\n        \"middle\": \"...\",\n        \"tail\": \"...\"\n    }\n}\n```\n\n### Markdown\n\n```php\n$result-\u003etoMd();\n```\n\n```markdown\n## Document Sample\n\n**Original size:** 50,000 chars\n**Sampled size:** 2,300 chars\n\n### Intro\n...\n\n### Outline\n...\n\n### Middle\n...\n\n### Tail\n...\n```\n\nEmpty zones are omitted from both outputs.\n\n---\n\n## Default window sizes\n\nWindow sizes are defined on the `DocumentPart` enum and can be read at runtime:\n\n```php\nuse Labrodev\\DocumentSampler\\Enums\\DocumentPart;\n\nDocumentPart::Intro-\u003echars();   // 1000\nDocumentPart::Outline-\u003echars(); // 500\nDocumentPart::Middle-\u003echars();  // 500\nDocumentPart::Tail-\u003echars();    // 500\n```\n\n---\n\n## When to use this\n\n- **Before calling an AI API** — reduce a large document to a structured excerpt that fits in a context window without losing structural information.\n- **Relevance checking** — feed `$result-\u003etext` to a classifier to decide whether a document is relevant before processing it in full.\n- **Prompt injection detection** — scan a compact sample for malicious instructions before passing untrusted documents to an LLM.\n- **Depersonalisation** — run PII detection over a representative sample before deciding whether to redact the full document.\n- **Document classification** — use the outline and intro zones to classify document type without reading the entire file.\n\n---\n\n## Testing\n\n```bash\ncomposer test\n```\n\n## Static analysis\n\n```bash\ncomposer analyse\n```\n\n---\n\n## Author\n\n**Petro Lashyn** — [contact@labrodev.com](mailto:contact@labrodev.com)\n\n---\n\n## License\n\nMIT\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flabrodev%2Fdocument-sampler","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flabrodev%2Fdocument-sampler","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flabrodev%2Fdocument-sampler/lists"}