{"id":30981016,"url":"https://github.com/benjaminr/statutory-duty-extractor","last_synced_at":"2025-09-12T08:07:09.622Z","repository":{"id":311672229,"uuid":"1028888545","full_name":"benjaminr/statutory-duty-extractor","owner":"benjaminr","description":null,"archived":false,"fork":false,"pushed_at":"2025-08-04T15:30:24.000Z","size":1056,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-08-25T22:38:25.514Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/benjaminr.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-07-30T07:58:10.000Z","updated_at":"2025-08-04T15:38:15.000Z","dependencies_parsed_at":"2025-08-25T22:38:27.477Z","dependency_job_id":"f60d8bc4-5bd3-47ca-a1bd-505a7b2f8f27","html_url":"https://github.com/benjaminr/statutory-duty-extractor","commit_stats":null,"previous_names":["benjaminr/statutory-duty-extractor"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/benjaminr/statutory-duty-extractor","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/benjaminr%2Fstatutory-duty-extractor","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/benjaminr%2Fstatutory-duty-extractor/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/benjaminr%2Fstatutory-duty-extractor/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/benjaminr%2Fstatutory-duty-extractor/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/benjaminr","download_url":"https://codeload.github.com/benjaminr/statutory-duty-extractor/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/benjaminr%2Fstatutory-duty-extractor/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":274777730,"owners_count":25347652,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-12T02:00:09.324Z","response_time":60,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-09-12T08:07:07.021Z","updated_at":"2025-09-12T08:07:09.584Z","avatar_url":"https://github.com/benjaminr.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Statutory Duty Extractor 📜🤖\n\n**Note: This repository contains proprietary materials for authorised use only. Do not share further. See [LICENSE](./LICENSE) for important usage restrictions.**\n\n## The Mission 🎯\n\nThe Prime Minister's Office has issued an urgent request: Local Authorities across the UK are drowning in statutory instruments, struggling to identify their legal obligations buried within thousands of pages of legislation. Your mission is to build an AI-powered solution that can automatically extract and index statutory duties from these documents, helping councils understand exactly what they're legally required to do.\n\nThis prototype tool uses cutting-edge LLMs to transform impenetrable legal PDFs into clear, structured data about who must do what under UK law.\n\n## Overview\n\nThis tool processes UK statutory instruments (in PDF format) and extracts:\n- **Duty descriptions**: The specific legal obligations\n- **Duty holders**: Who must fulfil each duty (e.g., \"local authority\", \"Secretary of State\")\n- **Legislative references**: Where the duty appears (e.g., \"regulation 3\")\n\n## Quick Start\n\n1. **Set up environment**\n   ```bash\n   # Clone the repository\n   git clone \u003crepository-url\u003e\n   cd statutory-duty-extractor\n\n   # Install dependencies using UV\n   uv sync\n\n   # Copy and configure environment variables\n   cp .env.example .env\n   # Edit .env with your Azure OpenAI credentials\n   ```\n\n2. **Run extraction**\n   ```bash\n   # Extract from a single PDF\n   uv run statutory-duty-extractor data/statutory_instruments_pdf/2089.pdf\n   ```\n\n## Project Structure\n\n```\nstatutory-duty-extractor/\n├── src/statutory_duty_extractor/\n│   ├── models.py          # Pydantic models for duties\n│   ├── extractor.py       # Core extraction logic\n│   └── cli.py            # Command-line interface\n├── data/\n│   ├── statutory_instruments_pdf/  # Original PDF documents to process\n│   ├── ground_truth_json/          # Simplified examples (matches data model)\n├── prompts/                        # Extraction prompts\n│   ├── system_prompt.txt\n│   └── user_prompt.txt\n└── tests/                         # Unit tests\n```\n\n### Data Directory Structure\n\n- **`statutory_instruments_pdf/`**: The original UK statutory instrument PDFs that need processing\n- **`ground_truth_json/`**: Simplified ground truth examples in JSON format that match our minimal data model (3 fields per duty)\n\n## Current Implementation\n\nThe current approach:\n1. Extracts text from PDFs using PyMuPDF\n2. Sends the full text to Azure OpenAI with a prompt\n3. Uses structured outputs to get back `StatutoryInstrument` objects\n4. Displays results in a formatted table\n\n## Development\n\n```bash\n# Run tests\nuv run pytest\n\n# Format code\nuv run ruff format .\n\n# Type check\nuv run mypy src/\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbenjaminr%2Fstatutory-duty-extractor","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbenjaminr%2Fstatutory-duty-extractor","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbenjaminr%2Fstatutory-duty-extractor/lists"}