{"id":33329365,"url":"https://github.com/abdulvahapmutlu/mediscan-iq","last_synced_at":"2026-05-01T21:04:58.302Z","repository":{"id":324001923,"uuid":"1095555704","full_name":"abdulvahapmutlu/mediscan-iq","owner":"abdulvahapmutlu","description":"Mediscan-IQ is a new developed tool to de-identify clinical text, summarize findings, and tag case risk (low / moderate / high) — via CLI, REST API, or Streamlit.","archived":false,"fork":false,"pushed_at":"2025-11-13T08:13:56.000Z","size":52,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-11-20T16:02:42.740Z","etag":null,"topics":["clinical-nlp","fastapi","healthcare","natural-language-processing","nltk","privacy-engineering","pydantic-v2","python","streamlit","text-classification","text-summarization","transformers","typer","uvicorn","zero-shot-classification"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/abdulvahapmutlu.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-11-13T07:56:22.000Z","updated_at":"2025-11-13T08:29:57.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/abdulvahapmutlu/mediscan-iq","commit_stats":null,"previous_names":["abdulvahapmutlu/mediscan-iq"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/abdulvahapmutlu/mediscan-iq","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/abdulvahapmutlu%2Fmediscan-iq","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/abdulvahapmutlu%2Fmediscan-iq/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/abdulvahapmutlu%2Fmediscan-iq/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/abdulvahapmutlu%2Fmediscan-iq/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/abdulvahapmutlu","download_url":"https://codeload.github.com/abdulvahapmutlu/mediscan-iq/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/abdulvahapmutlu%2Fmediscan-iq/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32512687,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-30T13:12:12.517Z","status":"online","status_checked_at":"2026-05-01T02:00:05.856Z","response_time":64,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["clinical-nlp","fastapi","healthcare","natural-language-processing","nltk","privacy-engineering","pydantic-v2","python","streamlit","text-classification","text-summarization","transformers","typer","uvicorn","zero-shot-classification"],"created_at":"2025-11-20T16:02:22.619Z","updated_at":"2026-05-01T21:04:58.283Z","avatar_url":"https://github.com/abdulvahapmutlu.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003c!-- Hero banner --\u003e\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"mediscan-iq-banner-readme.svg\"\n       alt=\"MediScan-IQ — De-identify • Summarize • Risk-tag clinical text\"\n       width=\"100%\"\u003e\n\u003c/p\u003e\n\n# MediScan-IQ\n\n**De-identify clinical text, summarize findings, and tag case risk (low / moderate / high)** — via CLI, REST API, or Streamlit.\n\n\u003e **Not a medical device.** For demonstration and clinical decision support research only.\n\n---\n\n## What it does\n\n1. **Ingest \u0026 validate** clinical text (radiology / pathology / discharge / ecg / echo / others).  \n2. **PHI anonymization** (emails, phones, MRNs/IDs, address \u0026 name hints, dates\\*).  \n3. **Sentence splitting** for downstream NLP.  \n4. **Summarization** (abstractive): 2–3 sentence summary of key findings.  \n5. **Risk tagging** (hybrid): **NLI model** + **heuristics** → `low | moderate | high` with probabilities.  \n6. **Explainability**: UI highlights risk-related keywords; meta includes model names/modes \u0026 PHI counts.\n\n\\* Date redaction is configurable.\n\n---\n\n## Features\n\n- **Three interfaces:** `CLI` • `REST API (FastAPI)` • `Streamlit` web app  \n- **PHI handling:** mask or salted-hash; counts reported in responses  \n- **Models (baseline):**\n  - **Summarizer:** `google/flan-t5-base`\n  - **Risk NLI:** `facebook/bart-large-mnli` (configurable)\n- **Config via `.env`** (limits, model names, thresholds, device)\n- **CPU-only friendly**; optional CUDA/MPS if available\n\n---\n\n## Tech stack\n\n**Language \u0026 Runtime**\n- Python **3.9+** (tested on 3.11), CPU-first with optional **CUDA/MPS** via PyTorch\n\n**Core Libraries**\n- **FastAPI** + **Uvicorn** — REST API server\n- **Typer** — CLI entrypoints\n- **Pydantic v2** + **pydantic-settings** — schema \u0026 config management\n- **regex**, **nltk**, **langdetect**, **numpy** — preprocessing, sentence split, language hints\n\n**NLP / Models (Hugging Face)**\n- **transformers**, **torch**, **accelerate**, **sentencepiece**\n- **Summarizer:** `google/flan-t5-base` (abstractive)\n- **Risk Tagger (NLI):** `facebook/bart-large-mnli` (zero-shot label probabilities)\n- **scikit-learn** — small utilities (e.g., softmax/normalization if needed)\n\n**UI**\n- **Streamlit**, **Altair**, **pandas** — interactive web app, charts, and tables\n\n**Build \u0026 Quality**\n- **pyproject.toml** (setuptools) — packaging\n- Optional dev extras: **pytest**, **pytest-cov**, **ruff**, **mypy**\n\n**Storage/State**\n- Stateless API; models cached locally by `transformers` in the user HF cache\n\n---\n\n## Quickstart\n\n### Windows (PowerShell)\n\n```\n# 1) Project root\nSet-Location C:\\path\\to\\mediscan-iq\n\n# 2) Create \u0026 activate venv\npy -3.11 -m venv .venv\n.\\.venv\\Scripts\\Activate.ps1\npython -m pip install -U pip wheel setuptools\n\n# 3) Install package (editable) + dev extras\npip install -e .[dev]\n\n# 4) NLTK punkt (once)\npython - \u003c\u003c'PY'\nimport nltk; nltk.download('punkt'); print(\"punkt ready\")\nPY\n```\n\n### macOS / Linux\n\n```\ncd /path/to/mediscan-iq\npython3 -m venv .venv\nsource .venv/bin/activate\npython -m pip install -U pip wheel setuptools\npip install -e .[dev]\npython - \u003c\u003c'PY'\nimport nltk; nltk.download('punkt'); print(\"punkt ready\")\nPY\n```\n\n### .env configuration\n\nCopy the example and adjust values:\n\n```\ncp .env.example .env\n```\n\nMost baselines declare `accepted_report_types` as a **List[str]** in `config.py`, so use **JSON** in `.env`:\n\n```\n# Server\nAPI_HOST=0.0.0.0\nAPI_PORT=8000\n\n# Limits\nMAX_CHARS=20000\n\n# Accepted report types (JSON array)\nACCEPTED_REPORT_TYPES=[\"radiology\",\"pathology\",\"discharge\",\"ecg\",\"echo\",\"others\"]\n\n# PHI\nANONYMIZE_STRATEGY=hash      # mask | hash\nMASK_CHAR=█\nHASH_SALT=mediscan\nKEEP_DATES=false\nREDUCE_WHITESPACE=true\n\n# Models\nSUMMARIZER_MODEL=google/flan-t5-base\nRISK_NLI_MODEL=facebook/bart-large-mnli\nRISK_THRESHOLD_HIGH=0.64\nRISK_THRESHOLD_MODERATE=0.42\n\n# Runtime\nDEVICE_PREFERENCE=auto       # auto | cpu | cuda | mps\nSEED=42\n```\n\n\u003e If your `config.py` uses a `*_raw` CSV string with a parsing property, change `ACCEPTED_REPORT_TYPES` to:\n\u003e `radiology,pathology,discharge,ecg,echo,others`.\n\n---\n\n## CLI usage\n\n```\n# Ingest (validate + anonymize + split)\npython -m mediscan_iq.cli ingest .\\sample_data\\example_radiology.txt -t radiology\n\n# Summarize (anonymize → abstractive summary)\npython -m mediscan_iq.cli summarize .\\sample_data\\example_radiology.txt -t radiology\n\n# Risk tagging (anonymize → NLI + heuristics)\npython -m mediscan_iq.cli risk .\\sample_data\\example_radiology.txt\n\n# Full pipeline (anonymize → split → summarize → risk)\npython -m mediscan_iq.cli analyze .\\sample_data\\example_radiology.txt -t radiology\n```\n\nIf installed as a console script:\n\n```\nmediscan-iq analyze path/to/report.txt -t radiology\n```\n\n---\n\n## REST API\n\n### Run server\n\n```\npython -m uvicorn mediscan_iq.api:app --host 0.0.0.0 --port 8000 --reload\n```\n\nHealth probe:\n\n```\nInvoke-RestMethod http://localhost:8000/health\n```\n\n### Endpoints\n\n* `GET /health` → `{\"status\":\"ok\",\"version\":\"\u003csemver\u003e\"}`\n* `POST /ingest` → anonymized text, sentences, PHI counts\n* `POST /analyze` → summary, risk level + probabilities, anonymized text, sentences, meta\n\n### Example requests\n\n**PowerShell — `/ingest`**\n\n```\n$body = @{ text = \"Patient John Doe ...\"; report_type = \"radiology\" } | ConvertTo-Json\nInvoke-RestMethod -Uri \"http://localhost:8000/ingest\" -Method Post -ContentType \"application/json\" -Body $body\n```\n\n**PowerShell — `/analyze` (multi-line)**\n\n```\n$text = @\"\nFINDINGS: Mild cardiomegaly. No acute pulmonary edema or effusion.\nIMPRESSION: Cardiomegaly without acute cardiopulmonary process.\n\"@\n$body = @{ text = $text; report_type = \"radiology\" } | ConvertTo-Json -Depth 4\nInvoke-RestMethod -Uri \"http://localhost:8000/analyze\" -Method Post -ContentType \"application/json\" -Body $body\n```\n\n**curl — `/analyze`**\n\n```\ncurl -s -X POST http://localhost:8000/analyze \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"text\":\"FINDINGS: Mild cardiomegaly. IMPRESSION: No acute disease.\",\"report_type\":\"radiology\"}'\n```\n\n---\n\n## Streamlit app (optional)\n\n```\npip install streamlit altair pandas\nstreamlit run .\\app_streamlit.py\n```\n\n* Paste or upload a note → click **Analyze**.\n* See **summary**, **risk badge**, **probability chart**, **anonymized text with highlights**, and **meta**.\n\n---\n\n## Realistic test cases\n\nCreate files quickly (PowerShell):\n\n```\n@'\nFINDINGS: Mild cardiomegaly. No acute pulmonary edema or pleural effusion.\nIMPRESSION: Cardiomegaly without acute cardiopulmonary process.\n'@ | Set-Content .\\CXR_mild_cardiomegaly.txt -Encoding UTF8\n\n@'\nFINDINGS: Acute subarachnoid hemorrhage within the suprasellar cisterns and anterior interhemispheric fissure.\nIMPRESSION: Acute subarachnoid hemorrhage. Urgent neurosurgical evaluation recommended.\n'@ | Set-Content .\\HeadCT_SAH.txt -Encoding UTF8\n\n@'\nHISTORY: Fever, productive cough.\nFINDINGS: New right upper lobe air-space consolidation with air bronchograms.\nIMPRESSION: Right upper lobe pneumonia. Recommend therapy.\n'@ | Set-Content .\\CXR_RUL_pneumonia.txt -Encoding UTF8\n\n@'\nADMISSION: Acute decompensated heart failure with volume overload.\nCOURSE: IV diuresis net -2.5L in 48h; renal function improved; EF 40-45%; small pleural effusions.\nDISCHARGE: Furosemide 40 mg, Spironolactone 25 mg, low-sodium diet, cardiology follow-up.\n'@ | Set-Content .\\Discharge_ADHF_long.txt -Encoding UTF8\n```\n\n---\n\n## Success criteria\n\n* **CXR_mild_cardiomegaly** → Summary mentions *mild cardiomegaly; no acute process*. Risk typically **moderate** (keyword “cardiomegaly”) or **low** if thresholds stricter—OK if the summary is correct.\n* **HeadCT_SAH** → Summary includes *acute subarachnoid hemorrhage*. **Risk = high**; `high risk` probability on top.\n* **CXR_RUL_pneumonia** → Summary includes *RUL pneumonia / consolidation*. **Risk = moderate**; `moderate risk` on top.\n* **Discharge_ADHF_long** → Summary captures *ADHF, diuresis, EF ~40–45%, small effusions, meds*. **Risk = moderate** (may be low if conservative). Summary should remain coherent despite length.\n\nFocus on **correct clinical concepts** and **top-ranked risk bucket** rather than exact probability numbers.\n\n---\n\n## Performance tips\n\n* First run downloads models; subsequent runs are fast.\n* CPU-only works; set `DEVICE_PREFERENCE=cuda` if you have a GPU.\n* For richer summaries, increase:\n\n  * `SUMMARIZER_MAX_OUTPUT_TOKENS=160`\n  * `SUMMARIZER_NUM_BEAMS=6` (when using seq2seq)\n* For stricter triage tuning:\n\n  * `RISK_THRESHOLD_HIGH=0.58–0.65`\n  * `RISK_THRESHOLD_MODERATE=0.40–0.50`\n\n---\n\n## Troubleshooting\n\n* **`BaseSettings has moved to pydantic-settings`**\n  Install \u0026 update import:\n\n  ```powershell\n  pip install pydantic-settings\n  ```\n\n  In `config.py`:\n\n  ```python\n  from pydantic_settings import BaseSettings\n  ```\n\n* **`Unsupported report type` even though value looks correct**\n  Check `.env` format for `ACCEPTED_REPORT_TYPES`:\n\n  * If `List[str]` field → **JSON array**\n  * If `*_raw: str` + property → **CSV**\n\n* **`422 Unprocessable Entity` on `/analyze`**\n  Ensure JSON body has both fields and correct header:\n\n  ```json\n  {\"text\":\"...\",\"report_type\":\"radiology\"}\n  ```\n\n* **“temperature ignored” notice**\n  Harmless on some HF models; disappears if temperature is only passed when sampling.\n\n---\n\n## Security \u0026 privacy\n\n* The anonymizer removes common PHI **heuristically** (emails, phones, dates*, IDs, name/address hints).\n* Do **not** ingest directly identifiable patient data without reviewing your PHI policy.\n* This repository is **not a medical device**; clinicians remain responsible for decisions.\n\n---\n\n## License\n\nThis project is licensed under MIT License.\n\n---\n\n### Maintainers\n\n* Abdulvahap Mutlu — *MediScan-IQ*\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fabdulvahapmutlu%2Fmediscan-iq","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fabdulvahapmutlu%2Fmediscan-iq","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fabdulvahapmutlu%2Fmediscan-iq/lists"}