{"id":50138097,"url":"https://github.com/junjslee/neonatal-ai-reliability","last_synced_at":"2026-05-23T23:10:59.164Z","repository":{"id":332008517,"uuid":"981493766","full_name":"junjslee/neonatal-ai-reliability","owner":"junjslee","description":"MRMC crossover study: expertise-dependent automation bias and sentinel behavior in human-AI collaborative neonatal diagnosis","archived":false,"fork":false,"pushed_at":"2026-04-10T20:05:43.000Z","size":44840,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2026-04-10T21:34:31.830Z","etag":null,"topics":["artificial-intelligence","cnn","computer-vision","deep-learning","fine-tuning","foundation-models","human-ai-interaction","medical-ai","medical-imaging","medical-research","neonatal-radiology","pediatrics","rad-dino"],"latest_commit_sha":null,"homepage":"https://neonatal-ai-sandbox.pages.dev/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/junjslee.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-05-11T08:32:27.000Z","updated_at":"2026-04-10T20:05:48.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/junjslee/neonatal-ai-reliability","commit_stats":null,"previous_names":["junjslee/neonatal-ai-reliability"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/junjslee/neonatal-ai-reliability","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/junjslee%2Fneonatal-ai-reliability","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/junjslee%2Fneonatal-ai-reliability/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/junjslee%2Fneonatal-ai-reliability/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/junjslee%2Fneonatal-ai-reliability/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/junjslee","download_url":"https://codeload.github.com/junjslee/neonatal-ai-reliability/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/junjslee%2Fneonatal-ai-reliability/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33415127,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-23T22:14:44.296Z","status":"ssl_error","status_checked_at":"2026-05-23T22:14:43.778Z","response_time":53,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["artificial-intelligence","cnn","computer-vision","deep-learning","fine-tuning","foundation-models","human-ai-interaction","medical-ai","medical-imaging","medical-research","neonatal-radiology","pediatrics","rad-dino"],"created_at":"2026-05-23T23:10:58.524Z","updated_at":"2026-05-23T23:10:59.157Z","avatar_url":"https://github.com/junjslee.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Expertise modulates automation bias and sentinel behavior in human-AI collaborative diagnosis of neonatal pneumoperitoneum\n\n[![Status](https://img.shields.io/badge/Status-In_Revision-orange)]()\n[![Venue](https://img.shields.io/badge/Venue-npj_Digital_Medicine-blueviolet)]()\n[![Study Design](https://img.shields.io/badge/Study-MRMC_Crossover-blue)]()\n[![Analysis](https://img.shields.io/badge/Statistics-GLMM_Crossed_Random_Effects-green)]()\n[![Educational Tool](https://img.shields.io/badge/Educational_Sandbox-Live-brightgreen)](https://neonatal-ai-sandbox.pages.dev/)\n\n**Lee et al.** — submitted to *npj Digital Medicine*; currently in revision (round 1).\n\n\u003e **One-line takeaway:** *AI reliability does not translate linearly into clinical benefit.* In 1,750 interpretation events across a multi-reader multi-center crossover study, high AI reliability paradoxically induced **automation bias in trainees**, while error-prone AI triggered **sentinel (vigilant) behavior in experts** — demonstrating that adversarial resilience, not standalone accuracy, is the defining metric of human-AI team performance.\n\n---\n\n## Abstract\n\nMedical AI is often validated under an additive assumption that algorithmic sensitivity and clinician oversight will combine to improve care. We tested this assumption in the high-stakes diagnosis of neonatal pneumoperitoneum, a time-critical surgical emergency. In a multi-reader crossover study analyzing 1,750 interpretation events, clinicians reviewed radiographs aided by either a high-reliability model or a systematically error-injected model. We found that high AI reliability paradoxically induces automation bias in trainees, who accepted 52.0% of incorrect suggestions, while offering limited gains to experts. Conversely, when challenged by flawed AI, the three participating neonatologists exhibited a \"sentinel behavior\" phenotype, correctly overriding 91.7% of errors (Wilson 95% CI 83.0–96.0%) consistent with increased deliberation; given the small specialist cohort, this finding is hypothesis-generating and warrants prospective replication. We operationalize systemic resilience as the capacity to maintain diagnostic integrity under algorithmic failure and demonstrate that clinical validity depends on the human-AI team's adversarial resilience rather than standalone accuracy. To mitigate the risk of deskilling and never-skilling, we release an open-source educational sandbox designed to inoculate clinicians against automated errors.\n\n**Keywords:** Neonatal pneumoperitoneum · Automation bias · Sentinel behavior · Artificial intelligence · Deep learning · Multi-reader multi-case study · Human-AI interaction · Radiology\n\n---\n\n## Educational Sandbox\n\n\u003e **Try it live:** [neonatal-ai-sandbox.pages.dev](https://neonatal-ai-sandbox.pages.dev/)\n\nAn open-source, web-based educational tool designed to inoculate clinicians against automated errors — simulating both reliable and error-prone AI assistance to build adversarial resilience in trainees and practicing clinicians.\n\n---\n\n## Model Architecture\n\n![Model Arch](figures/rdino%20model_arch.png)\n\nTo ensure that differences in reader behavior were driven solely by **AI reliability** (and not model capacity), both the Reliable and Error-Injected assistants use the same underlying architecture:\n\n- **Backbone:** RAD-DINO (ViT-B/14) — a vision foundation model pre-trained on large-scale radiology datasets\n- **Adaptation (LoRA):** Parameter-efficient fine-tuning via Low-Rank Adaptation ($r=12, \\alpha=24$) injected into Query/Value projections and the MLP layer; only **1.36%** of parameters were trainable\n- **Sampling Strategy (RFBS):** A custom Representation-Focused Batch Sampler enforcing diversity and exposure to uncommon pneumoperitoneum distributions during training\n\n\u003e **Error-Injected model:** Same architecture, trained on systematically poisoned labels. False positives engineered by mislabeling clinically plausible confounders (iatrogenic devices, portal venous gas, pneumatosis intestinalis, abdominal drains) as pneumoperitoneum — curated by a board-certified pediatric radiologist to simulate realistic deployment failures rather than random noise.\n\n---\n\n## Why This Matters\n\nNeonatal pneumoperitoneum is a time-critical surgical emergency. Integrating AI into this workflow is not just about model accuracy. Clinicians interact with advice, confidence cues, and time pressure.\n\nThis study investigates the **Human-AI Interaction (HAI) layer**:\n1. **Automation Bias:** When AI is *highly capable*, does it help — or does it reduce human vigilance?\n2. **The Sentinel Effect:** When AI is *systematically wrong*, do clinicians disengage, blindly follow, or become hyper-vigilant?\n3. **Expertise Gradient:** Do neonatologists, radiologists, and residents react differently to the same AI signals?\n4. **Never-Skilling Risk:** Does early-career reliance on reliable AI prevent trainees from developing independent pattern recognition?\n\n---\n\n## Study Overview\n\n### Cohorts\n\n| Cohort | Radiographs | Positive Cases | Source |\n| :--- | :--- | :--- | :--- |\n| Internal Development | 688 (from 216 patients) | 310 | Asan Medical Center |\n| External Validation (Reader Study) | 125 | 40 | 11 tertiary hospitals via AI-Hub |\n\n### Reader Study Design\n\n- **Participants (N=14):**\n  - Pediatric Radiologists: $n=6$ (mean experience 16.2 ± 4.2 years)\n  - Neonatologists: $n=3$ (mean experience 10.3 ± 1.5 years)\n  - Radiology Residents: $n=5$ (mean experience 2.2 ± 1.3 years)\n- **Design:** Two-session, counterbalanced MRMC crossover with 6-week washout; double-masked\n- **Total interpretation events:** 1,750\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"figures/reader_study_diagram.png\" width=\"85%\" title=\"Reader Study Design\"\u003e\n\u003c/p\u003e\n\n**Case Allocation (Stratified, N=125):**\n\n| Condition | Cases |\n| :--- | :--- |\n| Unaided | 41 |\n| Reliable AI | 40 |\n| Error-Injected AI | 44 |\n\n\u003e Reliability was fixed at the *case level*. Readers were blinded to the reference standard and unaware of the two distinct AI reliability conditions.\n\n---\n\n## AI Tools Evaluated\n\n| Model | Performance | Engineering | Purpose |\n| :--- | :--- | :--- | :--- |\n| **Reliable AI** | AUC 0.861 (study subset); AUC 0.948 (full external validation) | Standard training on clean labels | Test automation bias |\n| **Error-Injected AI** | Balanced accuracy 0.44 (sensitivity 0.40; specificity 0.47) | Systematic label poisoning via clinically plausible confounders | Test sentinel / adversarial resilience |\n\n---\n\n## Statistical Methodology\n\nPrimary analysis: **Crossed Random-Effects GLMM** (logit link)\n\n- Random intercept for `Case_ID` — controls for intrinsic image difficulty (variance 3.83 on log-odds scale)\n- Random intercept for `Reader_ID` — controls for individual competence (variance 0.15)\n- Covariates: gestational age, birth weight (both non-significant: P=0.542, P=0.969)\n- No session-order effects (Session 2 vs 1: OR 1.33, P=0.448)\n- Post-hoc contrasts adjusted via Holm-Bonferroni\n\n---\n\n## Key Findings\n\n### 1. Expertise-Stratified Interaction\n\nThe primary GLMM identified a significant Condition × Expertise interaction for neonatologists under the Error-Injected AI condition:\n\n| Contrast (vs Pediatric Radiologist) | OR | 95% CI | P-value |\n| :--- | :--- | :--- | :--- |\n| **Error-Injected AI × Neonatologist** | **4.16** | **1.26–13.77** | **0.020** |\n\nConfirmed by GEE (P=0.018) and Leave-One-Neonatologist-Out sensitivity analysis (ORs 2.04–2.64 across all leave-one-out configurations).\n\nPediatric Radiologists maintained stable accuracy across all conditions (no significant gains or losses). Radiology Residents showed patterns consistent with automation bias.\n\n### 2. Unaided Baseline Performance\n\n| Group | Unaided Accuracy |\n| :--- | :--- |\n| Pediatric Radiologists | 90.2% |\n| Radiology Residents | 85.9% |\n| Neonatologists | 85.4% |\n\n### 3. Error Acceptance (Automation Bias)\n\nWhen AI was incorrect — rate at which readers accepted the wrong suggestion:\n\n| Group | Acceptance of Incorrect AI (Reliable AI condition) |\n| :--- | :--- |\n| **Radiology Residents** | **52.0% (13/25)** |\n| Neonatologists | 33.3% (5/15) |\n| Pediatric Radiologists | 20.0% (6/30) |\n\nResidents vs Radiologists: P=0.016 (significant after Bonferroni correction).\n\n### 4. Sentinel Behavior (Correct Override of Flawed AI)\n\nWhen the Error-Injected AI was wrong — rate at which readers successfully overrode it:\n\n| Group | Correct Override Rate | Wilson 95% CI |\n| :--- | :--- | :--- |\n| **Neonatologists** | **91.7% (66/72)** | **83.0–96.0%** |\n| Pediatric Radiologists | 85.4% (123/144) | 78.6–90.4% |\n| Radiology Residents | 81.7% (98/120) | 73.8–87.6% |\n\n\u003e The 91.7% neonatologist override rate comes from three participating specialists (75 reader-case rows on the Error-Injected arm) and is presented as a **hypothesis-generating** observation that warrants prospective replication in a larger specialist cohort.\n\n### 5. Verification Effort (Deliberation Time)\n\nReading time was modeled with a linear mixed-effects model on the aided set:\n\n```\nlog(reading_time_sec) ~ disagree * reliability * group + pgy_within_5\n                       + (1 | reader) + (1 | case)\n```\n\nHeadline fixed effects (REML, Satterthwaite df via lmerTest; full table in Supplementary Table 5):\n\n| Term | β (log s) | SE | t (df) | P | Time ratio (95% CI) |\n| :--- | :--- | :--- | :--- | :--- | :--- |\n| Discordance (main) | 0.892 | 0.174 | 5.12 (1043.0) | \u003c0.001 | 2.44 (1.74–3.43) |\n| Discordance × Reliability[Unreliable] | -0.610 | 0.215 | -2.84 (948.6) | 0.005 | — |\n| Reliability[Unreliable] (main) | 0.172 | 0.131 | 1.32 (200.9) | 0.189 | — |\n\nRandom-effect variances: σ²(reader)=0.096, σ²(case)=0.148, σ²(residual)=0.665.\nMarginal R²=0.101, Conditional R²=0.342 (Nakagawa–Schielzeth).\n\nGroup-specific simple slopes (disagree vs agree, marginalized over reliability):\n\n| Group | Time ratio (disagree/agree) | 95% CI | P |\n| :--- | :--- | :--- | :--- |\n| Pediatric Radiologists | 1.77× | 1.44–2.18 | \u003c0.001 |\n| Neonatologists | 1.95× | 1.46–2.59 | \u003c0.001 |\n| Radiology Residents | 1.49× | 1.18–1.88 | \u003c0.001 |\n\n\u003e **Statistical caveat (revised in round 1):** the omnibus disagree × group interaction was not significant (F(2, 1096)=1.32, P=0.267) — the model does not provide evidence that the *magnitude* of the slowdown differs across groups. We interpret the per-group multiplicative slowdowns as evidence that AI-induced verification effort imposed a similar workflow cost across expertise levels, with the between-group differential being operationally negligible.\n\u003e\n\u003e **Note on prior reporting:** an earlier draft summarized this analysis using absolute second-level differences (+1.2 / +3.1 / +4.6 s). Those raw second-level values were a technical error introduced during initial drafting and were not reproducible from the analytic dataset. They have been removed in favor of the model-adjusted multiplicative slowdowns above, which are traceable to Supplementary Table 5.\n\n\u003e **Within-session trust recalibration.** The significant `Discordance × Reliability[Unreliable]` interaction (β=-0.610, P=0.005) indicates that on the log-time scale, the verification cost of disagreeing with the AI was substantially attenuated under the Error-Injected condition. Once readers encountered a low-reliability AI, the cognitive cost of rejecting its output decreased — consistent with empirical recalibration of trust during the session.\n\n### 6. Error-Type Stratification (Error-Injected AI arm; added in revision)\n\nThe Error-Injected AI was wrong on **18 unique false-positive (FP)** and **6 unique false-negative (FN)** cases (336 reader-case rows across the 14 readers in the aided arm). Agreement with the wrong AI, stratified:\n\n| Group | FP rate (n agreed / n) | FN rate (n agreed / n) | FP→FN drop |\n| :--- | :--- | :--- | :--- |\n| Pediatric Radiologists | 19.4% (21/108) | **0.0%** (0/36) | -19.4 pp |\n| Neonatologists | 11.1% (6/54) | **0.0%** (0/18) | -11.1 pp |\n| **Radiology Residents** | 18.9% (17/90) | **16.7%** (5/30) | **-2.2 pp** |\n\nWilson 95% CIs and full counts in `quantitative_analysis/revision_analyses/r2_8_error_type_stratification.py`.\n\nThe conventional GLMM (`agree_with_ai ~ group * error_type + (1|reader) + (1|case)`) converged but exhibited **practical separation** in the FN cell (two of three groups had zero events). We therefore report Firth penalized logistic regression as the primary inferential model:\n\n| Term | OR | 95% CI | P (Firth penalized LRT) |\n| :--- | :--- | :--- | :--- |\n| **error_type[FN]** (Ped Rad ref) | **0.06** | 0.00–0.42 | **0.001** |\n| group[Radiology Resident] × error_type[FN] | **16.25** | 1.51–2239 | **0.017** |\n| group[Neonatologist] × error_type[FN] | 3.62 | 0.02–721 | 0.54 |\n\nPediatric Radiologists and Neonatologists overrode every FN case. Residents accepted the AI's \"all clear\" verdict on 5/30 FN reader-case rows — the FP-to-FN drop is largely absent in trainees, suggesting that the FN cases are precisely where novice over-reliance is most clinically dangerous.\n\n\u003e **Exploratory caveat:** only 6 unique FN cases are shared across all 14 readers; the Resident × FN interaction CI spans 1.5 to 2239, directionally informative but with wide uncertainty.\n\n### 7. Saliency Map Usage\n\n| Group | Usage Rate (AI-incorrect cases) | Accuracy with Map | Interpretation |\n| :--- | :--- | :--- | :--- |\n| Radiology Residents | 53.8% (78/145) | 78.2% (vs 73.1% without; P=0.61) | Confirmatory — reinforces over-reliance |\n| Pediatric Radiologists | 34.5% (60/174) | Trending lower (81.7% vs 86.0%; P=0.58) | Intermediate |\n| **Neonatologists** | **17.2% (15/87)** | **100% (15/15; Wilson 95% CI 79.6–100%)** | **Refutation utility (exploratory)** |\n\n\u003e Neonatologist 100% accuracy is from 15 user-initiated map views and is presented as a **descriptive observation** — user-initiated access creates selection effects, and the wide Wilson CI (79.6–100%) reflects the limited subset.\n\nExperts used explainability maps selectively to *refute* the AI; trainees used them indiscriminately, often reinforcing over-reliance.\n\n---\n\n## Conclusion\n\n\u003e *\"Ultimately, in neonatal pneumoperitoneum, AI reliability affects clinicians through verification behavior and error phenotypes rather than accuracy alone. Highly reliable AI tends to induce automation bias in trainees, whereas intentionally error-injected AI can trigger vigilance in experts. Future evaluation and deployment frameworks must explicitly measure expertise-dependent behaviors to ensure resilience in time-critical emergencies.\"*\n\n---\n\n## Limitations\n\n1. **Simulated environment** — Cannot fully replicate the time pressures of a live NICU\n2. **Small neonatologist cohort** (n=3) — Mitigated by 375 independent decision points for the subgroup and LONO sensitivity analysis\n3. **Saliency map analysis is exploratory** — User-initiated access creates selection effects; randomized exposure required for causal inference\n4. **Generalizability** — Replication in larger, multicenter specialist cohorts needed\n\n---\n\n## Code and Data Availability\n\n- **Source code** (preprocessing, model, training, evaluation, saliency, statistical analysis): [github.com/junjslee/neonatal-ai-reliability](https://github.com/junjslee/neonatal-ai-reliability)\n- **Educational sandbox:** [neonatal-ai-sandbox.pages.dev](https://neonatal-ai-sandbox.pages.dev/)\n- **Primary statistical pipeline:** `quantitative_analysis/reader_study_full_analysis.py` — GLMM crossed random effects, GEE sensitivity, time/CAM mechanism analyses, HCI-type stacked bars (Type 1–4 derivation including the **Type 4 automation-bias** and **Type 3 sentinel-override** counts that underlie §3–§4).\n- **Revision-round analyses:** `quantitative_analysis/revision_analyses/` — R2-6 (reading-time LMM full output), R2-7 (between-group differential omnibus + simple slopes), R2-8 (FP-vs-FN error-type stratification with Firth sensitivity).\n- **Model checkpoints:** Both the Reliable AI and Error-Injected AI weights are included in this repository under `quantitative_analysis/standalone_model_performance/rad_dino/`. These are the exact checkpoints used in the reader study and can be used to reproduce inference results without retraining. Weights are derived from [microsoft/rad-dino](https://huggingface.co/microsoft/rad-dino) (MIT License, research use only — not for clinical practice).\n- **Raw image data:** Cannot be publicly redistributed (IRB/licensing); external validation set available via [AI-Hub](https://www.aihub.or.kr/)\n- **De-identified derived data** (reader metrics, AI predictions, consensus labels): Available upon request to corresponding authors\n\n---\n\n## Citation\n\nIf you use the code, findings, or the error-injection validation framework, please cite:\n\n\u003e Lee, J., Kim, Y., Kim, V., Park, C., Song, J. M., Kwon, J., Nam, Y., Lenehan, P., Kim, D. Y., Cho, Y. A., Kim, P. H., Hwang, J.-Y., Lee, J., Lee, B. S., Jung, E., Jung, A. Y., Choi, J., Kim, N.\\* \u0026 Yoon, H. M.\\* *Expertise modulates automation bias and sentinel behavior in human-AI collaborative diagnosis of neonatal pneumoperitoneum.* **npj Digit. Med.** (in revision, 2026).\n\u003e\n\u003e \\*Co-corresponding authors: Namkug Kim, PhD (namkugkim@gmail.com); Hee Mang Yoon, MD, PhD (espoirhm@gmail.com).\n\nMachine-readable metadata: see [`CITATION.cff`](CITATION.cff). A final BibTeX entry will be added on acceptance.\n\n---\n\n## Correspondence\n\n- **Namkug Kim, PhD** — namkugkim@gmail.com (MI2RL, Asan Medical Center)\n- **Hee Mang Yoon, MD, PhD** — espoirhm@gmail.com (Massachusetts General Hospital / Asan Medical Center)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjunjslee%2Fneonatal-ai-reliability","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjunjslee%2Fneonatal-ai-reliability","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjunjslee%2Fneonatal-ai-reliability/lists"}