{"id":51631241,"url":"https://github.com/mcaimi/flan-t5-frontend","last_synced_at":"2026-07-13T07:30:59.583Z","repository":{"id":364986726,"uuid":"1220116200","full_name":"mcaimi/flan-t5-frontend","owner":"mcaimi","description":"Frontend application that completes the FLAN-T5 Anonymization Finetuning on OCP AI Example","archived":false,"fork":false,"pushed_at":"2026-06-15T10:43:10.000Z","size":197,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-06-15T12:20:31.700Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://codeberg.org/mcaimi/flan-t5-frontend.git","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mcaimi.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-04-24T15:03:31.000Z","updated_at":"2026-06-15T10:43:14.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/mcaimi/flan-t5-frontend","commit_stats":null,"previous_names":["mcaimi/flan-t5-frontend"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/mcaimi/flan-t5-frontend","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mcaimi%2Fflan-t5-frontend","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mcaimi%2Fflan-t5-frontend/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mcaimi%2Fflan-t5-frontend/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mcaimi%2Fflan-t5-frontend/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mcaimi","download_url":"https://codeload.github.com/mcaimi/flan-t5-frontend/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mcaimi%2Fflan-t5-frontend/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35414732,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-07-13T02:00:06.543Z","response_time":119,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-07-13T07:30:59.494Z","updated_at":"2026-07-13T07:30:59.574Z","avatar_url":"https://github.com/mcaimi.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# FLAN-T5 Frontend\n\nThis repository contains a Streamlit application for interacting with a KServe-based inference pipeline that processes text and PDF documents for anonymization, translation, and summarization tasks using configurable models.\n\n## Features\n\n- 🤖 Interactive web interface for FLAN-T5 model inference\n- 📄 **PDF Anonymization**: Upload PDFs, extract text with docling, split into sentences using NLTK, anonymize each sentence, and download anonymized PDF with two modes:\n  - **Presidio Mode** (Local PII Masking): Privacy-preserving local PII detection and masking using Microsoft Presidio with **multi-language support** (English, Italian, Spanish, German)\n  - **KServe Mode** (AI Model): Context-aware anonymization using external AI model\n- 📝 **PDF Summarization**: Upload PDFs, extract text with docling, split into sentences using NLTK, summarize each sentence, and download as PDF or TXT\n- 🔒 **Text Anonymization** (optional): Single-sentence anonymization, translation, and summarization for quick text processing\n- 🔄 Support for individual or batch processing of all tasks\n- 📈 Real-time payload preview and response visualization\n- 📜 Request history tracking\n- 🔍 Automatic model discovery from KServe endpoints\n- 💾 Response download capability (JSON and PDF)\n- 🧠 **NLTK-based sentence splitting** for improved sentence boundary detection\n- 🎨 **Unified diff display**: Red/green Git-style diff view showing before/after changes without truncation\n\n## Files\n\n- `main_streamlit.py`: Main Streamlit application entry point\n- `config.yaml`: Feature flag and Presidio configuration file\n- `libs/apis.py`: API-related functions for KServe v2 inference endpoints\n- `libs/ui.py`: UI widget components for the Streamlit application\n- `libs/pdf_processor.py`: PDF processing module using docling for text extraction, NLTK for sentence splitting, and KServe API for anonymization and summarization\n- `libs/presidio.py`: Presidio integration for local PII detection and anonymization\n- `libs/diff_utils.py`: Unified diff generation with red/green highlighting for before/after comparisons\n- `pyproject.toml`: Project dependencies and metadata (UV-managed)\n\n## Installation\n\nThis is a UV-managed project. To install dependencies:\n\n```bash\nuv sync\n```\n\n## Usage\n\nStart the Streamlit application:\n```bash\nuv run streamlit run main_streamlit.py\n```\n\nThe available features are controlled by the `config.yaml` file in the project root.\n\n## Configuration\n\nThe application uses `config.yaml` to enable/disable features and configure Presidio:\n\n```yaml\nfeatures:\n  text_anonymization: false  # Enable sentence-level text anonymization tab\n  pdf_anonymization: true    # Enable PDF document anonymization tab\n  pdf_summarization: true    # Enable PDF document summarization tab\n\npresidio:\n  language: en                 # Language for PII detection: en, it, es, de\n  model_size: large            # Model size: small, medium, large\n  spacy_model: en_core_web_lg  # (Optional) Explicit spaCy model (overrides language + model_size)\n  gpu_enabled: auto            # GPU acceleration: auto, true, false\n  device: auto                 # Device selection: auto, cuda, cuda:0, mps, cpu\n```\n\n### Feature Flags\n\n- **`text_anonymization`**: When `true`, shows the Text Anonymization interface with support for single-sentence anonymization, translation, and summarization tasks.\n- **`pdf_anonymization`**: When `true`, shows the PDF Anonymization interface for uploading and processing PDF documents.\n- **`pdf_summarization`**: When `true`, shows the PDF Summarization interface for uploading PDFs and generating AI-powered summaries.\n\n### Feature Combinations\n\nThe application dynamically creates tabs based on enabled features. You can enable any combination:\n\n1. **Single Feature** (shows single container, no tabs):\n   ```yaml\n   features:\n     text_anonymization: false\n     pdf_anonymization: true\n     pdf_summarization: false\n   ```\n\n2. **Two Features** (shows two tabs):\n   ```yaml\n   features:\n     text_anonymization: false\n     pdf_anonymization: true\n     pdf_summarization: true\n   ```\n\n3. **All Features** (shows three tabs):\n   ```yaml\n   features:\n     text_anonymization: true\n     pdf_anonymization: true\n     pdf_summarization: true\n   ```\n\n4. **No Features**:\n   ```yaml\n   features:\n     text_anonymization: false\n     pdf_anonymization: false\n     pdf_summarization: false\n   ```\n   Displays a courtesy message: \"There are no features enabled for this instance\"\n\n### Multi-Language Support for Presidio\n\nPresidio now supports PII detection in multiple languages with configurable spaCy models:\n\n**Configuration options:**\n\n- **`language`**: Default language for PII detection\n  - `en` (English, default)\n  - `it` (Italian)\n  - `es` (Spanish)\n  - `de` (German)\n\n- **`model_size`**: spaCy model size (quality vs speed tradeoff)\n  - `small`: Fastest, ~25-30MB, ~70-80% accuracy\n  - `medium`: Balanced, ~40-50MB, ~85-90% accuracy\n  - `large` (default): Most accurate, ~100-560MB, ~90-95% accuracy\n\n- **`spacy_model`**: (Optional) Explicit spaCy model name\n  - If specified, overrides `language` and `model_size` settings\n  - Useful for backward compatibility or advanced configurations\n\n**Available spaCy models by language:**\n\n| Language | Small | Medium | Large |\n|----------|-------|--------|-------|\n| 🇬🇧 English | en_core_web_sm | en_core_web_md | en_core_web_lg |\n| 🇮🇹 Italian | it_core_news_sm | it_core_news_md | it_core_news_lg |\n| 🇪🇸 Spanish | es_core_news_sm | es_core_news_md | es_core_news_lg |\n| 🇩🇪 German | de_core_news_sm | de_core_news_md | de_core_news_lg |\n\n**Runtime language selection:**\n\nThe PDF Anonymization interface (Presidio mode) provides dropdown selectors to choose:\n- **Language**: Select document language (English, Italian, Spanish, German)\n- **Model Size**: Choose quality vs speed tradeoff (Small/Fast, Medium/Balanced, Large/Accurate)\n\nModels are downloaded automatically on first use with progress feedback in the UI.\n\n### GPU Acceleration for Presidio\n\nPresidio supports GPU acceleration on both NVIDIA GPUs (CUDA) and Apple Silicon (MPS) for faster PII detection across all supported languages:\n\n**Configuration options:**\n\n- **`gpu_enabled`**: Controls GPU usage\n  - `auto` (default): Automatically use GPU if available, fallback to CPU\n  - `true`: Force GPU usage, show warning if not available\n  - `false`: Always use CPU even if GPU is available\n\n- **`device`**: Specific device to use\n  - `auto` (default): Auto-detect best device (CUDA \u003e MPS \u003e CPU)\n  - `cuda` or `cuda:0`: NVIDIA GPU (first GPU)\n  - `cuda:1`, `cuda:2`, etc.: Specific NVIDIA GPU (multi-GPU systems)\n  - `mps`: Apple Silicon GPU (M1/M2/M3/M4 Macs)\n  - `cpu`: CPU only\n\n**Performance expectations:**\n\n| Hardware | Speedup vs CPU | Best for |\n|----------|----------------|----------|\n| NVIDIA GPU | 2-3x (standard models)\u003cbr\u003e5-10x (transformer models) | Large PDFs, transformer models (`en_core_web_trf`) |\n| Apple Silicon (MPS) | 1.5-2.5x (standard models)\u003cbr\u003e3-5x (transformer models) | M1/M2/M3/M4 Macs with macOS 12.3+ |\n| CPU | Baseline | Small documents, development |\n\n**Requirements:**\n\n- **NVIDIA GPU**: CUDA-capable GPU with drivers installed (`nvidia-smi` works)\n- **Apple Silicon**: M1/M2/M3/M4 Mac with macOS 12.3 or later\n- PyTorch with CUDA/MPS support (already included in dependencies)\n\n**GPU memory usage:**\n\n| Model Size | NVIDIA GPU | Apple Silicon (MPS) |\n|------------|------------|---------------------|\n| Small (`*_sm`) | ~0.5GB | Shares unified memory |\n| Medium (`*_md`) | ~1.0GB | Shares unified memory |\n| Large (`*_lg`) | ~2.0GB | Shares unified memory |\n\n*Memory usage is similar across all supported languages (en/it/es/de)*\n\n**Recommended configurations:**\n\n```yaml\n# NVIDIA GPU system - English documents\npresidio:\n  language: en\n  model_size: large\n  gpu_enabled: auto\n  device: cuda:0\n\n# Apple Silicon Mac - Italian documents\npresidio:\n  language: it\n  model_size: medium  # Balanced for MPS\n  gpu_enabled: auto\n  device: mps\n\n# CPU-only - Spanish documents (e.g., cloud deployment without GPU)\npresidio:\n  language: es\n  model_size: small  # Faster on CPU\n  gpu_enabled: false\n  device: cpu\n\n# Multi-language with explicit model (backward compatibility)\npresidio:\n  spacy_model: de_core_news_lg  # German, large model\n  gpu_enabled: auto\n  device: auto\n```\n\n**Monitoring GPU usage:**\n\nThe application displays device status in the sidebar under \"🔧 Presidio Status\":\n- NVIDIA GPU: Shows GPU name and memory usage\n- Apple Silicon: Shows \"Apple Silicon GPU (MPS)\"\n- CPU: Shows \"Device: CPU\"\n- Also displays: Current spaCy model and selected language\n\n**Error handling:**\n\nIf GPU runs out of memory, the application automatically:\n1. Clears GPU cache\n2. Falls back to CPU\n3. Retries the operation\n4. Logs the fallback for debugging\n\n## Application Interface\n\nThe Streamlit application provides a comprehensive web interface with:\n\n### Configuration Panel (Sidebar)\n- Base server URL configuration for KServe endpoint\n- Model discovery and selection\n- Request history tracking\n- Clear history functionality\n\n### Main Interface\n\n**Text Anonymization Tab** (when `text_anonymization: true`):\n- Task selection (Anonymize, Translate, Summarize, or All tasks)\n- Dedicated text input areas for each task with character counting\n- Live payload preview showing the exact JSON sent to the inference endpoint\n- Send request button with real-time feedback\n- Response visualization with metrics (status code, response time, size)\n- Downloadable response data\n\n**PDF Anonymization Tab** (when `pdf_anonymization: true`):\n- PDF file upload (max 10MB)\n- Anonymization mode selector:\n  - **Presidio Mode**: Local privacy-preserving PII detection with:\n    - **Multi-language support**: English, Italian, Spanish, German\n    - **Language selector**: Choose document language with flag emojis (🇬🇧 🇮🇹 🇪🇸 🇩🇪)\n    - **Model size selector**: Balance quality vs speed (Small/Fast, Medium/Balanced, Large/Accurate)\n    - **Entity type configuration**: Select which PII types to detect (PERSON, EMAIL_ADDRESS, PHONE_NUMBER, etc.)\n    - **Auto-download models**: First-time model download with progress feedback\n  - **KServe Mode**: AI model-based context-aware anonymization\n- Automatic text extraction using docling\n- Intelligent sentence splitting using NLTK (KServe mode only)\n- Sentence-by-sentence anonymization via AI model (KServe mode) or full-text PII masking (Presidio mode)\n- Progress tracking with real-time status\n- **Unified diff view**: Red/green Git-style comparison showing all changes without truncation\n- PII detection summary table (Presidio mode only)\n- Download anonymized PDF\n- Error reporting for failed sentences\n\n**PDF Summarization Tab** (when `pdf_summarization: true`):\n- PDF file upload (max 10MB)\n- Automatic text extraction using docling\n- Intelligent sentence splitting using NLTK\n- Sentence-by-sentence summarization via AI model\n- Progress tracking with real-time status\n- Summary display in text area\n- Download summary as PDF or TXT format\n- Error reporting and detailed status messages\n\n## PDF Processing Pipelines\n\n### KServe Mode (AI Model) Pipeline\n\nFor PDF anonymization and summarization using the AI model:\n\n1. **PDF → Markdown**: Extract text content using docling\n2. **Markdown → Sentences**: Split text into sentences using NLTK's punkt tokenizer\n3. **Process Sentences**: Each sentence is sent to the KServe AI model (anonymize or summarize)\n4. **Sentences → Markdown**: Rebuild the document from processed sentences\n5. **Markdown → PDF**: Convert back to PDF using reportlab\n\n### Presidio Mode (Local PII Masking) Pipeline\n\nFor privacy-preserving local anonymization:\n\n1. **PDF → Markdown**: Extract text content using docling\n2. **Analyze Text**: Presidio analyzer detects PII entities (PERSON, EMAIL_ADDRESS, etc.) using spaCy NER\n3. **Anonymize Text**: Presidio anonymizer replaces detected entities with placeholders (e.g., `\u003cPERSON\u003e`, `\u003cEMAIL_ADDRESS\u003e`)\n4. **Markdown → PDF**: Convert anonymized text back to PDF using reportlab\n5. **Generate Diff**: Create unified red/green diff showing all changes\n\n### Key Technologies\n\n- **docling**: High-quality PDF text extraction that preserves document structure\n- **NLTK**: Advanced sentence tokenization that handles edge cases (abbreviations, decimals, etc.) better than regex-based approaches\n- **KServe v2 API**: Scalable inference endpoint for processing individual sentences (KServe mode)\n- **Presidio**: Microsoft's open-source PII detection and anonymization framework (Presidio mode)\n  - **presidio-analyzer**: Detects PII entities using spaCy NER models\n  - **presidio-anonymizer**: Masks detected entities with configurable operators\n- **spaCy**: Industrial-strength NLP for named entity recognition (powers Presidio)\n- **reportlab**: PDF generation from processed text\n- **difflib**: Python standard library for generating unified diffs with red/green highlighting\n\n### Benefits of Different Anonymization Modes\n\n**Presidio Mode (Local PII Masking)**:\n- **Privacy-preserving**: All processing happens locally, no data sent to external APIs\n- **Multi-language**: Supports English, Italian, Spanish, and German documents\n- **Fast**: No network latency, instant PII detection\n- **Configurable**: Choose which entity types to detect and mask, select model size for quality/speed tradeoff\n- **Transparent**: See exactly what was detected with entity type and confidence scores\n- **Deterministic**: Same input always produces the same output\n- **GPU accelerated**: Optional GPU support (NVIDIA CUDA, Apple Silicon MPS) for faster processing\n\n**KServe Mode (AI Model)**:\n- **Context-aware**: AI model understands context for more nuanced anonymization\n- **Sentence-level processing**: Each sentence processed independently for better quality\n- **Progress tracking**: Real-time feedback on which sentence is being processed\n- **Error resilience**: Failed sentences don't block the entire document\n- **Flexible**: Can be fine-tuned for specific domains or use cases\n\n## Runtime Configuration\n\n### KServe Endpoint Configuration\n\nDefault server URL is `http://localhost:8080`. You can change this in the sidebar configuration panel.\n\nThe application automatically discovers available models from the KServe v2 discovery endpoint (`/v2/models`) and allows you to select which model to use for inference.\n\n### Deployment in Containerized Environments\n\nThe default behaviour can be also modified via environment variables: this is useful when deploying in containerized environments (e.g. k8s)\n\n#### Available config variables:\n\n- KSERVE_ENDPOINT_ADDRESS: The address of the KServe v2 inference endpoint. Defaults to \"http://localhost:8080\"\n- MODEL_NAME: The selected model name. This must match the name of the deployed model in the KServe Engine. Defaults to \"None\" (e.g. you need to perform discovery before inference)\n\n## Requirements\n\n- Python 3.13+\n- UV package manager\n- Access to a running KServe inference server with FLAN-T5 models (for KServe mode)\n- Dependencies:\n  - streamlit\u003e=1.56.0\n  - requests\u003e=2.33.1\n  - docling\u003e=2.0.0 (for PDF text extraction)\n  - pypdf\u003e=5.1.0 (for PDF manipulation)\n  - reportlab\u003e=4.2.0 (for PDF generation)\n  - nltk\u003e=3.8.0 (for sentence tokenization)\n  - presidio-analyzer\u003e=2.2.354 (for local PII detection)\n  - presidio-anonymizer\u003e=2.2.354 (for local PII masking)\n\n### First-Time Setup\n\nThe application automatically downloads required models on first run:\n- **NLTK punkt tokenizer**: Downloaded automatically for sentence splitting\n- **spaCy language models**: Downloaded automatically for Presidio NER based on language selection\n\n**Configuration options:**\n\n```yaml\n# Recommended: Use language + model_size (automatically generates correct model)\npresidio:\n  language: en        # en, it, es, de\n  model_size: large   # small, medium, large\n\n# Alternative: Explicit model name (backward compatibility)\npresidio:\n  spacy_model: it_core_news_lg  # Explicit Italian large model\n```\n\n**Available spaCy models:**\n\n| Language | Small (~25-30MB) | Medium (~40-50MB) | Large (~100-560MB) |\n|----------|------------------|-------------------|---------------------|\n| English  | en_core_web_sm   | en_core_web_md    | en_core_web_lg      |\n| Italian  | it_core_news_sm  | it_core_news_md   | it_core_news_lg     |\n| Spanish  | es_core_news_sm  | es_core_news_md   | es_core_news_lg     |\n| German   | de_core_news_sm  | de_core_news_md   | de_core_news_lg     |\n\n**Model quality tradeoffs:**\n- **Small**: 2-3x faster, ~70-80% accuracy\n- **Medium**: Balanced, ~85-90% accuracy\n- **Large**: Most accurate, ~90-95% accuracy (default)\n\n**Runtime model selection:**\n\nUsers can also select language and model size directly in the PDF Anonymization UI. Models are downloaded on-demand when first selected, with progress feedback shown in the interface.\n\n## Building the image on Openshift using OCP BuildConfigs\n\nA Containerfile is provided for automated builds on OCP.\n\n1. Generate a token to access the target container registry (Optional) (e.g. a Robot Account for Quay.io)\n\n2. Create a push secret to access the target registry (Optional)\n\n```bash\n$ oc create secret docker-registry push-secret --docker-server=\u003cSERVER URL\u003e --docker-username=\u003cUSERNAME\u003e --docker-password=\u003cPASSWORD\u003e -n \u003cNAMESPACE\u003e\n```\n\n3. Create a build config\n\n```yaml\n#\n# Private Git: create a Secret (basic-auth or ssh) and set spec.source.sourceSecret.\n#\napiVersion: build.openshift.io/v1\nkind: BuildConfig\nmetadata:\n  name: flan-t5-frontend\nspec:\n  runPolicy: Serial\n  source:\n    type: Git\n    git:\n      uri: https://codeberg.org/mcaimi/flan-t5-frontend\n      ref: main\n    # sourceSecret:\n    #   name: git-credentials\n  strategy:\n    type: Docker\n    dockerStrategy:\n      dockerfilePath: Containerfile\n  output:\n    to:\n      kind: DockerImage\n      name: \u003cTARGET CONTAINER REGISTRY IMAGE URL+TAG\u003e # fill in the required information\n    pushSecret:\n      name: push-secret # optional secret created at step 2\n```\n\n4. Run the build on OCP\n\n```bash\n$ oc apply -f buildconfig.yaml\n$ oc start-build flan-t5-frontend\n```\n\n## Deploy on OCP\n\n```bash\n$ oc new-app -e KSERVE_ENDPOINT_ADDRESS=\u003cKSERVE_ENDPOINT\u003e --image=\u003cIMAGE:TAG\u003e --name=flan-t5-frontend\n$ oc create route edge flan-t5-frontend --service=flan-t5-frontend --port=8501-tcp\n```\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmcaimi%2Fflan-t5-frontend","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmcaimi%2Fflan-t5-frontend","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmcaimi%2Fflan-t5-frontend/lists"}