Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/IS2Lab/awesome-ai-testing


https://github.com/IS2Lab/awesome-ai-testing

List: awesome-ai-testing

Last synced: 2 months ago
JSON representation

Awesome Lists containing this project

README 

        
# Awesome-AI-Testing

A curated list of awesome publications and researchers on AI testing updated and maintained by [***The Intelligent System Security (IS2) Lab***](https://is2lab.github.io/).

![IS2Lab](https://github.com/IS2Lab/awesome-ai-testing/blob/main/picture/is2lab.png)



## AI Testing



***๐Ÿ‘ What is AI Testing?*** 

AI Testing refers to the process of evaluating and verifying the performance of artificial intelligence (AI) systems. It involves testing the AI models, algorithms, and systems to ensure that they function correctly, produce accurate results, and are reliable in their decision-making processes.



***๐Ÿ‘ How AI Testing works๏ผŸ*** AI Testing can be done in several ways, such as functional testing, performance testing, security testing, usability testing, and more. It also involves the creation of test cases and data sets, evaluating the accuracy of training data, validating models against real-world scenarios, and monitoring the performance of AI systems in production.



***๐Ÿ‘ What is the goal of AI Testing๏ผŸ*** The goal of AI testing is to identify and fix any errors, biases, or vulnerabilities in AI systems, ensuring that they meet the required standards and perform optimally in different environments. This is crucial for applications such as autonomous vehicles, medical diagnosis, and financial forecasting, where accuracy and reliability are essential.



## [Conferences and Journals](./files/conferences.md)

### TOP Journal 



	

		

		

    			

			


			TDSC 
 IEEE Transactions on Dependable and Secure Computing

            	

		

		

		

    			

			


			TIFS 
 IEEE Transactions on Information Forensics and Security

            	

		

		

		

    			

			


			TOSEM 
 ACM Transactions on Software Engineering and Methodology

            	

		

		

		

    			

			


			TSE 
 IEEE Transactions on Software Engineering

            	

		

	



### TOP Conferences



- **Information Securrity**  
 CCS **|** S&P **|** USENIX **|** NDSS

  



	

		

		

    			

			


			CCS 
 The ACM Conference on 
 Computer and 
 Communications Security

            	

		  

		

		

    			

			


			S&P 
 IEEE Symposium on 
 Security and Privacy

            	

		   

		

		

    			

			


			USENIX 
 The Advanced Computing 
 Systems Association

            	

		   

		

		

    			

			


			NDSS 
 Network and Distributed 
 System Security Symposium

            	

		   

	



- **Software Engineering** 
 ICSE **|** ASE **|** ISSTA **|** FSE



	

		

		

    			

			


			ICSE 
 International Conference on 
 Software Engineering

            	

		 

		

		

    			

			


			ASE 
 International Conference 
 on Automated Software 
 Engineering



            	

		 

		

		

    			

			


			ISSTA 
 International Symposium 
 on Software Testing 
 and Analysis

            	

		 

		

		

    			

			


			FSE 
 Foundations of Software 
 Engineering

            	

		 

	



 

## [Papers](./files/papers.md)  

#### โš ๏ธ Only the latest work here, please click the title for more.



### [๐ŸŒธ Survey](./files/papers.md) 



###### [***IEEE Access 2023***] [*Toward Deep-Learning-Based Methods in Image Forgery Detection: A Survey*](https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=10035377)



###### [***ACM Computing Surveys 2023***] [*A Comprehensive Survey on Poisoning Attacks and Countermeasures in Machine Learning*](https://scholar.google.co.uk/scholar?q=A+Comprehensive+Survey+on+Poisoning+Attacks+and+Countermeasures+in+Machine+Learning.&hl=zh-CN&as_sdt=0&as_vis=1&oi=scholart)



###### [***Neurocomputing 2023***] [*Adversarial examples based on object detection tasks: A survey*](https://scholar.google.co.uk/scholar?hl=zh-CN&as_sdt=0%2C5&as_vis=1&q=Adversarial+examples+based+on+object+detection+tasks%3A+A+survey.&btnG=)



### [๐ŸŒธ Evaluation of testing methods](./files/papers.md) 



###### [***CVPR 2022***] [*Is Neuron Coverage Needed to Make Person Detection More Robust?*](https://openaccess.thecvf.com/content/CVPR2022W/FaDE-TCV/papers/Pavlitskaya_Is_Neuron_Coverage_Needed_To_Make_Person_Detection_More_Robust_CVPRW_2022_paper.pdf)

    

###### [***ICSE 2021***] [*What Are We Really Testing in Mutation Testing for Machine Learning? A Critical Reflection.*](https://arxiv.org/abs/2103.01341)



### [๐ŸŒธ Causality-Based Testing](./files/papers.md) 



###### [***ICSE 2022***] [*CARE: Causality-based Neural Network Repair*](https://arxiv.org/pdf/2204.09274.pdf)



###### [***ICML 2022***] [*Inducing Causal Structure for Interpretable Neural Networks.*](https://arxiv.org/abs/2112.00826)

    

### [๐ŸŒธ Coverage-Guided Testing](./files/papers.md) 



###### [***CVPR  2022***] [*Is Neuron Coverage Needed to Make Person Detection More Robust?*](https://openaccess.thecvf.com/content/CVPR2022W/FaDE-TCV/papers/Pavlitskaya_Is_Neuron_Coverage_Needed_To_Make_Person_Detection_More_Robust_CVPRW_2022_paper.pdf)



###### [***NAACL 2022***] [*White-box Testing of NLP models with Mask Neuron Coverage.*](https://arxiv.org/abs/2205.05050)



###### [***SANER 2022***] [*Revisiting Neuron Coverage Metrics and Quality of Deep Neural Networks.*](https://arxiv.org/pdf/2201.00191.pdf)

    

### [๐ŸŒธ Mutation Testing](./files/papers.md) 



###### [***ISSTA 2022***] [*BET: Black-Box Efficient Testing for Convolutional Neural Networks.*](https://dl.acm.org/doi/pdf/10.1145/3533767.3534386)



###### [***Information and Software Technology 2023***] [*A probabilistic framework for mutation testing in deep neural networks.*](https://scholar.google.com/scholar?hl=zh-CN&as_sdt=0%2C5&q=A+probabilistic+framework+for+mutation+testing+in+deep+neural+networks.&btnG=)



###### [***Journal of Systems and Software 2023***] [*The language mutation problem: Leveraging language product lines for mutation testing of interpreters.*](https://scholar.google.com/scholar?hl=zh-CN&as_sdt=0%2C5&q=The+language+mutation+problem%3A+Leveraging+language+product+lines+for+mutation+testing+of+interpreters.+&btnG=)



## Latest Updates



#### [***ISSTA '22***](https://dblp.org/db/conf/issta/issta2022.html) _ACM SIGSOFT International Symposium on Software Testing and Analysis_

#### [***ICSE '22***](https://dblp.org/db/conf/icse/icse2022.html#DanilovaH0N22) _International Conference on Software Engineering_

#### [***S&P '22***](https://dblp.org/db/conf/sp/sp2022.html) _IEEE Symposium on Security and Privacy_

#### [***USENIX '22***](https://dblp.org/db/conf/uss/uss2022.html) _USENIX Security Symposium_

#### [***CCS '22***](https://dblp.org/db/conf/ccs/ccs2022.html) _ACM SIGSAC Conference on Computer and Communications Security_

#### [***NDSS '22***](https://dblp.org/db/conf/ndss/ndss2022.html) _Annual Network and Distributed System Security Symposium_

#### [***ASE '22***](https://dblp.org/db/conf/kbse/ase2022.html) _IEEE/ACM International Conference on Automated Software Engineering_



## Latest Achievements

This section provides links to the latest achievements for researchers to use and study.



### [ChatGPT](https://openai.com/blog/chatgpt/)   

***OpenAI November 30, 2022***

![chatgpt](https://github.com/IS2Lab/awesome-ai-testing/blob/main/picture/chatgpt.png)

ChatGPT is a model which interacts in a conversational way. The dialogue format makes it possible for ChatGPT to answer followup questions, admit its mistakes, challenge incorrect premises, and reject inappropriate requests. ChatGPT is a sibling model to InstructGPT, which is trained to follow an instruction in a prompt and provide a detailed response.