{"id":34698170,"url":"https://github.com/aget-framework/template-document-processor-aget","last_synced_at":"2026-05-03T00:04:39.432Z","repository":{"id":321005907,"uuid":"1083866175","full_name":"aget-framework/template-document-processor-AGET","owner":"aget-framework","description":"Production-ready template for creating document processing agents with LLM pipelines, security protocols, and multi-provider support","archived":false,"fork":false,"pushed_at":"2026-04-26T05:20:13.000Z","size":417,"stargazers_count":0,"open_issues_count":1,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-04-26T07:18:18.656Z","etag":null,"topics":["aget-framework","document-processing","llm","python","template"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/aget-framework.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":"AGENTS.md","dco":null,"cla":null}},"created_at":"2025-10-26T21:28:58.000Z","updated_at":"2026-04-26T05:20:16.000Z","dependencies_parsed_at":"2025-10-27T09:32:35.781Z","dependency_job_id":null,"html_url":"https://github.com/aget-framework/template-document-processor-AGET","commit_stats":null,"previous_names":["aget-framework/template-document-processor-aget"],"tags_count":6,"template":true,"template_full_name":null,"purl":"pkg:github/aget-framework/template-document-processor-AGET","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aget-framework%2Ftemplate-document-processor-AGET","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aget-framework%2Ftemplate-document-processor-AGET/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aget-framework%2Ftemplate-document-processor-AGET/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aget-framework%2Ftemplate-document-processor-AGET/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/aget-framework","download_url":"https://codeload.github.com/aget-framework/template-document-processor-AGET/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aget-framework%2Ftemplate-document-processor-AGET/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32553693,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-02T22:28:24.418Z","status":"ssl_error","status_checked_at":"2026-05-02T22:28:14.225Z","response_time":132,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aget-framework","document-processing","llm","python","template"],"created_at":"2025-12-24T22:47:54.690Z","updated_at":"2026-05-03T00:04:39.426Z","avatar_url":"https://github.com/aget-framework.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# template-document-processor-AGET\n\n\u003e **Status**: 🟡 DORMANT (stable template - v2.8.0 released 2025-11-03)\n\u003e\n\u003e Production-ready template. Activated when new document processing agents are needed.\n\n**Template Version**: 2.8.0\n**AGET Framework**: v2.7.0\n**Type**: AGET Template (specialized from worker template)\n**Domain**: Document Processing\n\nA production-ready template for creating document processing agents with LLM pipelines, security protocols, format preservation, and multi-provider support.\n\n**Note**: This template is based on `template-worker-aget` v2.7.0 with specialized document processing capabilities. Template version (v2.8.0) tracks template-specific features independently from AGET framework version (v2.7.0).\n\n## Overview\n\nThis template provides a complete foundation for agents that:\n- ✅ Process documents using LLM assistance (OpenAI, Anthropic, Google)\n- ✅ Preserve DOCX formatting (Track Changes, comments, annotations)\n- ✅ Prevent catastrophic format loss (L245-type failures)\n- ✅ Support batch operations with validation pipelines\n- ✅ Implement security protocols (injection prevention, content filtering)\n- ✅ Provide caching, metrics, and observability\n- ✅ Enable task decomposition for large documents\n- ✅ Support rollback and version management\n\n## Based on L208\n\nThis template is based on **L208: Document Processing Agent Template Pattern**, which analyzed production document processing agents to extract common patterns and best practices.\n\n**Time Savings**: 60-70% reduction in new agent setup (3-5 hours → 1-2 hours)\n\n## Quick Start\n\n### 1. Clone Template\n\n```bash\ngh repo clone aget-framework/template-document-processor-AGET my-document-agent\ncd my-document-agent\n```\n\n### 2. Customize Configuration\n\nUpdate these files:\n\n**`.aget/version.json`**:\n```json\n{\n  \"agent_name\": \"my-document-agent\",\n  \"domain\": \"your-domain\"\n}\n```\n\n**`configs/validation_rules.yaml`**:\n```yaml\nmax_file_size_mb: 10\nallowed_extensions: [\".pdf\", \".docx\", \".txt\", \".md\"]\nrequired_validations:\n  - file_size\n  - file_format\n  - content_safety\n```\n\n**`configs/llm_providers.yaml`**:\n```yaml\nproviders:\n  openai:\n    api_key_env: OPENAI_API_KEY\n    enabled: true\n  anthropic:\n    api_key_env: ANTHROPIC_API_KEY\n    enabled: false\nbudget:\n  monthly_limit_usd: 300.0\n```\n\n### 3. Update Mission\n\nEdit `AGENTS.md` to describe your agent's specific purpose and domain.\n\n### 4. Run Tests\n\n```bash\npython3 -m pytest tests/ -v\n```\n\n### 5. Deploy\n\n```bash\ngit remote set-url origin \u003cyour-repo-url\u003e\ngit add .\ngit commit -m \"feat: Initialize from template-document-processor-AGET v2.7.0\"\ngit push -u origin main\n```\n\n## Architecture\n\n### Core Components\n\n- **`src/ingestion/`** - Queue management, validation, batch processing\n- **`src/processing/`** - LLM providers, model routing, caching, schema validation\n- **`src/output/`** - Publishing, version management, rollback\n- **`src/security/`** - Input sanitization, content filtering, resource limiting\n- **`src/pipeline/`** - Task decomposition, orchestration, metrics\n- **`src/wikitext/`** - Document format support (docx, wikitext, extensible for additional formats)\n\n### Configuration Files (9 YAMLs)\n\n- **`configs/validation_rules.yaml`** - Document validation criteria\n- **`configs/llm_providers.yaml`** - LLM provider configuration\n- **`configs/model_routing.yaml`** - Model selection strategy\n- **`configs/models.yaml`** - Model definitions and capabilities\n- **`configs/security_policy.yaml`** - Security and content filtering\n- **`configs/processing_limits.yaml`** - Resource limits (tokens, time, cost)\n- **`configs/caching.yaml`** - Cache settings and TTL\n- **`configs/metrics.yaml`** - Metrics collection and alerts\n- **`configs/orchestration.yaml`** - Task decomposition and pipeline\n\n## Customization Points\n\n### 1. Document Validation\n\nCustomize `configs/validation_rules.yaml` for your document format:\n- File extensions\n- Size limits\n- Format-specific validation rules\n\n### 2. LLM Providers\n\nSet up providers in `configs/llm_providers.yaml`:\n- API keys (use environment variables)\n- Model selection (cost vs quality)\n- Fallback chain\n- Budget limits\n\n### 3. Security\n\nConfigure security in `configs/security_policy.yaml`:\n- Input sanitization rules\n- Content filtering (PII detection)\n- Resource limits (tokens, time, cost)\n\n### 4. Metrics\n\nDefine metrics in `configs/metrics.yaml`:\n- Accuracy tracking\n- Latency monitoring (p50/p95/p99)\n- Cost tracking\n- Alert thresholds\n\n## Protocols\n\nThe template includes 10 operational protocols in `.aget/docs/protocols/`:\n\n1. **Format Preservation** - Prevent L245-type catastrophic failures (Track Changes loss)\n2. **Queue Management** - Managing document queues\n3. **Processing Authorization** - Approval gates and STOP protocol\n4. **Validation Pipeline** - Pre/post validation\n5. **Rollback** - Version management and recovery\n6. **Security Validation** - Input/output sanitization\n7. **Task Decomposition** - Breaking large documents into subtasks\n8. **Model Routing** - Selecting optimal LLM for each task\n9. **Caching** - LLM response caching for cost/speed\n10. **Metrics Collection** - Tracking accuracy/latency/cost\n\nEach protocol includes bash commands and code examples.\n\n**Format Preservation Guide**: See `.aget/docs/FORMAT_PRESERVATION_GUIDE.md` for complete usage guide.\n\n## Scripts\n\nThe template provides 17 operational tools (15 scripts + 2 helper tools):\n\n**Session Management**:\n- `session_protocol.py` - Wake up/wind down/sign off\n- `queue_status.py` - Queue management CLI\n- `health_check.py` - System diagnostics\n\n**Core Operations** (`scripts/`):\n- `validate.py` - Document validation CLI\n- `process.py` - End-to-end processing pipeline\n- `queue_status.py` - Queue status and management\n- `rollback.py` - Version rollback operations\n- `cache_setup.py` - Cache initialization\n- `metrics.py` - Metrics display and export\n\n**Supporting Operations** (`scripts/`):\n- `health_check.py` - System health diagnostics\n- `security_check.py` - Security validation\n- `audit.py` - Audit trail viewer\n- `model_router.py` - Model routing recommendations\n- `cache_stats.py` - Cache statistics\n- `cache_clear.py` - Cache clearing\n\n**Specialized Tools** (`scripts/` and `.aget/tools/`):\n- `session_protocol.py` - Session lifecycle (wake/wind-down/sign-off)\n- `checkpoint.py` - Checkpoint save/load/list\n- `task_planner.py` - Task decomposition planning\n- `.aget/tools/analyze_agent_fit.py` - Use case fit analysis\n- `.aget/tools/instantiate_template.py` - Template instantiation helper\n\n## Testing\n\nTemplate includes 30 contract tests (100% passing):\n\n```bash\n# Run all tests\npython3 -m pytest tests/ -v\n\n# Run specific test category\npython3 -m pytest tests/test_processing.py -v\n```\n\nTest coverage:\n- **Smoke tests** (20 tests): All 20 modules tested\n- **Integration tests** (10 tests): End-to-end workflows, script integration, contract validation\n\n```bash\n# Run smoke tests only\npython3 tests/smoke_test.py\n\n# Run integration tests only\npython3 tests/test_integration.py\n```\n\n## Helper Tools\n\nTwo helper tools are provided for template users:\n\n**Analyze Agent Fit**:\n```bash\n# Check if use case fits this template\npython3 .aget/tools/analyze_agent_fit.py \"Process legal contracts and extract structured data\"\n\n# Interactive mode\npython3 .aget/tools/analyze_agent_fit.py --interactive\n```\n\n**Instantiate Template**:\n```bash\n# Create new agent from template\npython3 .aget/tools/instantiate_template.py invoice-processor ~/github/invoice-processor-AGET\n\n# Verify instantiation\npython3 .aget/tools/instantiate_template.py --check ~/github/invoice-processor-AGET\n```\n\n## Version History\n\n### Template Versioning\n\nThis template uses dual versioning:\n- **Template Version** (v2.8.0): Template-specific features and enhancements\n- **AGET Framework** (v2.7.0): Framework compliance version\n\n**Template v2.8.0** tracks format preservation capabilities added to this specialized template.\n**AGET v2.7.0** indicates compliance with AGET framework standards and base worker template.\n\n---\n\n**Template v2.8.0** (2025-11-02) - AGET Framework v2.7.0\n- **Format Preservation Framework**: Prevent L245-type catastrophic failures\n- **New capabilities**:\n  - OOXML verification for Track Changes, comments, annotations\n  - Round-trip validation (before → process → after)\n  - Multi-stage checkpoint system for pipeline verification\n  - L245 failure detection (100% format loss prevention)\n- **Documentation**:\n  - FORMAT_PRESERVING_DECISION_PROTOCOL.md (5-question architecture checklist)\n  - FORMAT_PRESERVATION_GUIDE.md (implementation guide)\n- **Test coverage**: 17 tests, 38% coverage (critical paths validated)\n- **API**: 5-module framework with simple and advanced usage patterns\n\n**Template v2.7.0** (2025-10-26) - AGET Framework v2.7.0\n- Initial template release\n- Based on L208 document processing pattern analysis\n- **20 source modules** (Gate 2A: 8, Gate 2B: 7, Gate 2C: 5)\n- **9 configuration files** (YAML-based)\n- **9 operational protocols** (tested and validated)\n- **3 formal specifications** (168 EARS requirements)\n- **17 operational tools** (15 scripts + 2 helper tools)\n- **30 contract tests** (20 smoke + 10 integration, 100% passing)\n- Multi-provider LLM support (OpenAI, Anthropic, Google)\n- Security protocols (input sanitization, content filtering, resource limits)\n- Document format support (docx, wikitext, extensible to additional formats)\n\n## Support\n\nFor template issues or questions:\n- Issue tracker: https://github.com/aget-framework/aget/issues\n- Template repository: https://github.com/aget-framework/template-document-processor-AGET\n- AGET framework: https://github.com/aget-framework\n\n## License\n\nLicensed under the Apache License, Version 2.0. See [LICENSE](LICENSE) file for details.\n\nCopyright 2025 AGET Framework Contributors\n\n---\n\n*Generated by AGET v2.7.0*\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faget-framework%2Ftemplate-document-processor-aget","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Faget-framework%2Ftemplate-document-processor-aget","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faget-framework%2Ftemplate-document-processor-aget/lists"}