awesome-safety-critical-ai
When the stakes are high, intelligence is only half the equation - reliability is the other ⚠️
https://github.com/jgalego/awesome-safety-critical-ai
Last synced: 15 days ago
JSON representation
-
About Us
-
Bleeding Edge ⚗️
- Critical Software - and mission-critical software.
- open roles
-
-
<a id="articles"></a>📝 Articles
-
AI in Critical Systems
- Trustworthy AI - Part I - AI-Part-II-Mariani-Rossi/9f354b3a88e6d6512d22ec152e6c6131a1e44cab) and [III](https://www.semanticscholar.org/paper/Trustworthy-AI-Part-III-Mariani-Rossi/ff446b46c5b9b4c0d18849d479fe5645f6182a36)
- Lessons From Red Teaming 100 Generative AI Products
- Engineering Dependable AI Systems
- Unpacking Human-AI Interaction in Safety-Critical Industries: A Systematic Literature Review
- Requirements Engineering Challenges in Building AI-Based Complex Systems
- Quantification of the Impact of Random Hardware Faults on Safety-Critical AI Applications: CNN-Based Traffic Sign Recognition Case Study
- Resilience of Deep Learning applications: a systematic literature review of analysis and hardening techniques
- Where AI Assurance Might Go Wrong: Initial lessons from engineering of critical systems
- What's your ML test score? A rubric for ML production systems
- Addressing uncertainty in the safety assurance of machine-learning
- Rethinking the Maturity of Artificial Intelligence in Safety-Critical Settings
- Towards Guaranteed Safe AI: A Framework for Ensuring Robust and Reliable AI Systems
- Output range analysis for deep feedforward neural networks
- AI Safety for Physical Infrastructures: A Collaborative and Interdisciplinary Approach
- Machine learning safety: An overview
- The role of AI in detecting and mitigating human errors in safety-critical industries: A review
- Artificial intelligence in health care: accountability and safety
- Trustworthy Artificial Intelligence in Medical Imaging
- A Goal-Directed Dialogue System for Assistance in Safety-Critical Application
- The Increasing Risks of Risk Assessment: On the Rise of Artificial Intelligence and Non-Determinism in Safety-Critical Systems
- AI-supported estimation of safety critical wind shear-induced aircraft go-around events utilizing pilot reports - 0)
- Automated Verification of Neural Networks: Advances, Challenges and Perspectives
- Trustworthy AI: From Principles to Practices
- Understanding and Identifying Challenges in Design of Safety-Critical AI Systems
- Formal Specification and Verification of Autonomous Robotic Systems: A Survey
- Large-scale machine learning systems in real-world industrial settings: A review of challenges and solutions
- SoK: Security and Privacy in Machine Learning
- Challenges of Machine Learning Applied to Safety-Critical Cyber-Physical Systems
- Assurance Argument Patterns and Processes for Machine Learning in Safety-Related Systems
- Collaborative Intelligence for Safety-Critical Industries: A Literature Review
- Hidden Technical Debt in Machine Learning Systems
- Towards Verified Artificial Intelligence
- Neural Bridge Sampling for Evaluating Safety-Critical Autonomous Systems
- Expert-in-the-loop Systems Towards Safety-critical Machine Learning Technology in Wildfire Intelligence
- How to Certify Machine Learning Based Safety-critical Systems? A Systematic Literature Review
- Effective Mitigations for Systemic Risks from General-Purpose AI
- A corroborative approach to verification and validation of human-robot teams
- Holistic Safety and Responsibility Evaluations of Advanced AI Models
- A Survey on Failure Analysis and Fault Injection in AI Systems
- Testing and verification of neural-network-based safety-critical control software: A systematic literature review
- Machine Learning Testing: Survey, Landscapes and Horizons
- The Brittleness of AI-Generated Image Watermarking Techniques: Examining Their Robustness Against Visual Paraphrasing Attacks
- AI at work – Mitigating safety and discriminatory risk with technical standards
- Can Large Language Models Transform Natural Language Intent into Formal Method Postconditions?
- Assurance for Autonomy – JPL's past research, lessons learned, and future directions
- Formal-LLM: Integrating Formal Language and Natural Language for Controllable LLM-based Agents
- Understanding and Avoiding AI Failures: A Practical Guide
- Machine Learning and Software Product Assurance: Bridging the Gap
- The Fusion of Large Language Models and Formal Methods for Trustworthy AI Agents: A Roadmap
- Ukraine's Future Vision and Current Capabilities for Waging AI-Enabled Autonomous Warfare
- Architectural Patterns for Integrating AI Technology into Safety-Critical System
- Artificial Intelligence for Safety-Critical Systems in Industrial and Transportation Domains: A Survey
- Ignore This Title and HackAPrompt: Exposing Systemic Vulnerabilities of LLMs through a Global Scale Prompt Hacking Competition
- The Prompt Report: A Systematic Survey of Prompt Engineering Techniques
- White Paper Machine Learning in Certified Systems
- Datasheets for Datasets
- Model cards for model reporting
- Data Cards: Purposeful and Transparent Dataset Documentation for Responsible AI
- A Review of Formal Methods applied to Machine Learning
- NAPER: Fault Protection for Real-Time Resource-Constrained Deep Neural Networks
- DeepHunter: a coverage-guided fuzz testing framework for deep neural networks
- Safety Case Templates for Autonomous Systems
- The world and the machine
- The EU AI Act, Stakeholder Needs, and Explainable AI: Aligning Regulatory Compliance in a Clinical Decision Support System
- Making data science systems work
- Machine Learning that Matters
- Software Engineering for Machine Learning: A Case Study
- Software Engineering Challenges of Deep Learning
- 150 Successful Machine Learning Models: 6 Lessons Learned at Booking.com
- Towards Verifiable Text Generation with Symbolic References
- Machine Learning Practices Outside Big Tech: How Resource Constraints Challenge Responsible Development
- Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety
- Engineering problems in machine learning systems
- A Conceptual Framework for AI-based Decision Systems in Critical Infrastructures
- N-Version Machine Learning Models for Safety Critical Systems
- Safety-critical computer vision: an empirical survey of adversarial evasion attacks and defenses on computer vision systems
- Collaboration challenges in building ML-enabled systems: communication, documentation, engineering, and process
- What Is Really Different in Engineering AI-Enabled Systems?
- Towards Secure MLOps: Surveying Attacks, Mitigation Strategies, and Research Challenges
- Error Resilient Machine Learning for Safety-Critical Systems: Position Paper
- Of Models and Tin Men: A Behavioural Economics Study of Principal-Agent Problems in AI Alignment using Large-Language Models
- Model-Based Approaches in Safety-Critical Embedded System Design
- Open Problems in Technical AI Governance
- "Everyone wants to do the model work, not the data work": Data Cascades in High-Stakes AI
- Detecting adversarial advertisements in the wild
- Real-World Integration of a Sepsis Deep Learning Technology Into Routine Clinical Care: Implementation Study
- Accessorize to a Crime: Real and Stealthy Attacks on State-of-the-Art Face Recognition
- Can Software Engineering Harness the Benefits of Advanced AI?
- Artificial intelligence in safety-critical systems: a systematic review
- Explainable Machine Learning in Critical Decision Systems: Ensuring Safe Application and Correctness
- The Role of Design in Creating Machine-Learning-Enhanced User Experience
- Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety
- Engineering problems in machine learning systems
- Safety-critical computer vision: an empirical survey of adversarial evasion attacks and defenses on computer vision systems
- Neural Network Based Runway Landing Guidance for General Aviation Autoland
- End to End Learning for Self-Driving Cars
- Formal Methods for AI: Lessons from the past, promisses of the future
- Toward Certification of Machine-Learning Systems for Low Criticality Airborne Applications
- Runway Sign Classifier: A DAL C Certifiable Machine Learning System
- A policy-blending formalism for shared control
- Structuring validation targets of a machine learning function applied to automated driving
- AI2: Safety and Robustness Certification of Neural Networks with Abstract Interpretation
- COMPL-AI Framework: A Technical Interpretation and LLM Benchmarking Suite for the EU Artificial Intelligence Act
- What do we need to build explainable AI systems for the medical domain?
- Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety
- Private and Reliable Neural Network Inference
- Engineering problems in machine learning systems
- AI Safety Gridworlds
- Safety-critical computer vision: an empirical survey of adversarial evasion attacks and defenses on computer vision systems
- "Why Should I Trust You?": Explaining the Predictions of Any Classifier
- On a Formal Model of Safe and Scalable Self-driving Cars
- Engineering Safety in Machine Learning
- On the Safety of Interpretable Machine Learning: A Maximum Deviation Approach
- Structuring validation targets of a machine learning function applied to automated driving
- Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety
- Engineering problems in machine learning systems
- Safety-critical computer vision: an empirical survey of adversarial evasion attacks and defenses on computer vision systems
- Structuring validation targets of a machine learning function applied to automated driving
- Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety
- Engineering problems in machine learning systems
- Safety-critical computer vision: an empirical survey of adversarial evasion attacks and defenses on computer vision systems
- Structuring validation targets of a machine learning function applied to automated driving
- Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety
- Engineering problems in machine learning systems
- Safety-critical computer vision: an empirical survey of adversarial evasion attacks and defenses on computer vision systems
- Structuring validation targets of a machine learning function applied to automated driving
- Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety
- Engineering problems in machine learning systems
- Safety-critical computer vision: an empirical survey of adversarial evasion attacks and defenses on computer vision systems
- Statistical Modeling: The Two Cultures
- Structuring validation targets of a machine learning function applied to automated driving
- Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety
- Leakage and the Reproducibility Crisis in ML-based Science
- Engineering problems in machine learning systems
- Safety-critical computer vision: an empirical survey of adversarial evasion attacks and defenses on computer vision systems
- Structuring validation targets of a machine learning function applied to automated driving
- Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety
- Engineering problems in machine learning systems
- Safety-critical computer vision: an empirical survey of adversarial evasion attacks and defenses on computer vision systems
- Reliability modeling for three-version machine learning systems through Bayesian networks
- Structuring validation targets of a machine learning function applied to automated driving
- Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety
- Engineering problems in machine learning systems
- Safety-critical computer vision: an empirical survey of adversarial evasion attacks and defenses on computer vision systems
- Structuring validation targets of a machine learning function applied to automated driving
- Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety
- Engineering problems in machine learning systems
- Safety-critical computer vision: an empirical survey of adversarial evasion attacks and defenses on computer vision systems
- Structuring validation targets of a machine learning function applied to automated driving
- Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety
- Engineering problems in machine learning systems
- Safety-critical computer vision: an empirical survey of adversarial evasion attacks and defenses on computer vision systems
- Structuring validation targets of a machine learning function applied to automated driving
- Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety
- Engineering problems in machine learning systems
- Safety-critical computer vision: an empirical survey of adversarial evasion attacks and defenses on computer vision systems
- Structuring validation targets of a machine learning function applied to automated driving
- Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety
- Engineering problems in machine learning systems
- Safety-critical computer vision: an empirical survey of adversarial evasion attacks and defenses on computer vision systems
- Structuring validation targets of a machine learning function applied to automated driving
- Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety
- Engineering problems in machine learning systems
- Safety-critical computer vision: an empirical survey of adversarial evasion attacks and defenses on computer vision systems
- Structuring validation targets of a machine learning function applied to automated driving
- Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety
- Engineering problems in machine learning systems
- Safety-critical computer vision: an empirical survey of adversarial evasion attacks and defenses on computer vision systems
- Structuring validation targets of a machine learning function applied to automated driving
- Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety
- Engineering problems in machine learning systems
- Safety-critical computer vision: an empirical survey of adversarial evasion attacks and defenses on computer vision systems
- Structuring validation targets of a machine learning function applied to automated driving
- Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety
- Engineering problems in machine learning systems
- Safety-critical computer vision: an empirical survey of adversarial evasion attacks and defenses on computer vision systems
- Structuring validation targets of a machine learning function applied to automated driving
- Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety
- Engineering problems in machine learning systems
- Safety-critical computer vision: an empirical survey of adversarial evasion attacks and defenses on computer vision systems
- How to Verify Generalization Capability of a Neural Network with Formal Methods
- Structuring validation targets of a machine learning function applied to automated driving
- Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety
- Engineering problems in machine learning systems
- Safety-critical computer vision: an empirical survey of adversarial evasion attacks and defenses on computer vision systems
- Efficient Neural Network Robustness Certification with General Activation Functions
- Structuring validation targets of a machine learning function applied to automated driving
- Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety
- Engineering problems in machine learning systems
- Safety-critical computer vision: an empirical survey of adversarial evasion attacks and defenses on computer vision systems
- Machine learning for healthcare that matters: Reorienting from technical novelty to equitable impact
- Assuring autonomous operations in aviation: is use of AI a good idea?
- Assessing Trustworthiness of Autonomous Systems
- Safety verification of neural network based systems using formal methods
- Process Assurance for Object Detection Through Deep Neural Networks to Accomplish the Autonomous Aerial Refueling Task
- Artificial Intelligence in Aviation Safety: Systematic Review and Biometric Analysis
- Safety assessment of a machine learning-based aircraft emergency braking system: A case study
- Structuring validation targets of a machine learning function applied to automated driving
-
Programming Languages
Categories
<a id="articles"></a>📝 Articles
358
<a id="books"></a>📚 Books
125
<a id="tools"></a>🛠️ Tools
71
<a id="blogs"></a>✍️ Blogs / News
36
<a id="miscellaneous"></a>👾 Miscellaneous
29
<a id="courses"></a>👩🏫 Courses
28
<a id="introduction"></a>🐇 Introduction
26
<a id="standards"></a>📐 Standards
25
<a id="videos"></a>📺 Videos
23
<a id="initiatives"></a>🤝 Initiatives
20
<a id="guidelines"></a>📙 Guidelines
20
<a id="conferences"></a>🎤 Conferences
20
<a id="reports"></a>📋 Reports
17
<a id="meta"></a>🏁 Meta
10
<a id="working-groups"></a>👷🏼 Working Groups
7
<a id="whitepapers"></a>📄 Whitepapers
7
<a id="top-picks"></a>🌟 Editor's Choice
5
<a id="roadmaps"></a>🛣️ Roadmaps
4
<a id="patents"></a>💡 Patents
3
About Us
2
<a id="certifications"></a>📜 Certifications
2
<a id="blogs"></a>✍️ Blogs
2
<a id="tldr"></a>🏃 TLDR
1
Contributions
1
Sub Categories
AI in Critical Systems
557
Bleeding Edge ⚗️
115
Coding
15
Always have a safety net
13
<a id="what-is-a-critical-system"></a>What is a critical system?
10
Generic
10
Data Management
7
Model Lifecycle
6
Model Fairness & Privacy
5
Model Security
4
Model Testing & Validation
3
Model Intepretability
3
Model Evaluation
3
Adversarial Attacks
2
Oldies 🕰️
1
Miscellaneous
1
Keywords
machine-learning
21
llm
21
python
19
ai
16
llmops
12
rag
11
llm-evaluation
9
data-science
9
openai
9
mlops
9
prompt-engineering
8
llms
8
pytorch
7
llama3
6
agents
6
langchain
6
generative-ai
6
chatgpt
6
deep-learning
6
artificial-intelligence
5
ai-agents
5
interpretability
5
evaluation
5
responsible-ai
5
adversarial-machine-learning
5
privacy
5
open-source
5
large-language-models
5
data-validation
4
fine-tuning
4
tensorflow
4
ml
4
llama
4
chatbot
4
trustworthy-ai
4
awesome-list
4
llm-eval
3
agent
3
validation
3
data-quality
3
anthropic
3
gpt
3
llm-security
3
language-model
3
transformers
3
awesome
3
fairness-ai
3
deepseek
3
mistral
3
qwen
3