Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/Redaimao/awesome-multimodal-deception-detection

A reading list for research topics in multimodal deception detection.
https://github.com/Redaimao/awesome-multimodal-deception-detection

List: awesome-multimodal-deception-detection

Last synced: 3 months ago
JSON representation

A reading list for research topics in multimodal deception detection.

Awesome Lists containing this project

README 

        
# Awesome-Multimodal-Deception-Detection



## Table of Contents



* [Survey Papers](#survey-papers)

* [Research Papers](#research-papers)

* [Related Research Papers](#related-research-papers)  

* [Datasets](#datasets)  

* [Tutorials](#tutorials)



## Survey Papers



[Multimodal Machine Learning: A Survey and Taxonomy](https://arxiv.org/abs/1705.09406), TPAMI 2019



[Multimodal Intelligence: Representation Learning, Information Fusion, and Applications](https://arxiv.org/abs/1911.03977), arXiv 2019



[Deep Multimodal Representation Learning: A Survey](https://ieeexplore.ieee.org/abstract/document/8715409), arXiv 2019



[Representation Learning: A Review and New Perspectives](https://arxiv.org/abs/1206.5538), TPAMI 2013



## Research Papers

[Affect-Aware Machine Learning Models for Deception Detection](https://www.aaai.org/AAAI21Papers/UC-47.MathurL.pdf), AAAI 2021



[Deception Detection in Group Video Conversations using Dynamic Interaction Networks](https://cc.gatech.edu/~srijan/pubs/deceptionrank_kumar_icwsm21.pdf), ICWSM 2021



[Deception Detection in Videos using the Facial Action Coding System](https://arxiv.org/abs/2105.13659), 2021, ArXiv



[Deception Detection in Videos Using Robust Facial Features](https://link.springer.com/chapter/10.1007/978-3-030-63092-8_45)



[comment]: <> ([Multimodal Deception Detection in Videos via Analyzing Emotional State-based Feature](https://arxiv.org/abs/2104.08373), 2021, ArXiv)



[Event correlation for deception detection in long video](https://www.spiedigitallibrary.org/conference-proceedings-of-spie/11605/116050E/Event-correlation-for-deception-detection-in-long-video/10.1117/12.2586819.short) ICMV 2021



[Vid2Int: Detecting Implicit Intention from Long Dialog Videos](https://openaccess.thecvf.com/content/WACV2021/papers/Xu_Vid2Int_Detecting_Implicit_Intention_From_Long_Dialog_Videos_WACV_2021_paper.pdf), WACV 2021



[The Liar's Walk: Detecting Deception with Gait and Gesture](https://arxiv.org/pdf/1912.06874.pdf), 2021, ArXiv



[Multimodal Deception Detection in Videos via Analyzing Emotional State-based Feature](https://arxiv.org/pdf/2104.08373.pdf), 2021, ArXiv



[Multimodal Political Deception Detection](https://ieeexplore.ieee.org/abstract/document/9310253/), MM 2021



[LieNet: A deep convolution neural network framework for detecting deception](https://ieeexplore.ieee.org/abstract/document/9446553), TCDS 2021



[Introducing representations of facial affect in automated multimodal deception detection](https://arxiv.org/pdf/2008.13369), ICMI 2020



[Emotion Transformation Feature: Novel Feature For Deception Detection In Videos](https://ieeexplore.ieee.org/abstract/document/9190846/), ICIP 2020



[LieToMe: Preliminary study on hand gestures for deception detection via Fisher-LSTM](https://www.sciencedirect.com/science/article/abs/pii/S0167865520303123), PR 2020



[Facial Expression Based Imagination Index and a Transfer Learning Approach to Detect Deception](http://www.chasseauxp.hoques.com/Publications/2019/2019-Mental-face-Hasan-etal-ACII.pdf), ACII 2019



[Face-Focused Cross-Stream Network for Deception Detection in Videos](https://openaccess.thecvf.com/content_CVPR_2019/html/Ding_Face-Focused_Cross-Stream_Network_for_Deception_Detection_in_Videos_CVPR_2019_paper.html), CVPR 2019



[High-Level Features for Multimodal Deception Detection in Videos](https://openaccess.thecvf.com/content_CVPRW_2019/papers/CFS/Rill-Garcia_High-Level_Features_for_Multimodal_Deception_Detection_in_Videos_CVPRW_2019_paper.pdf), CVPRW 2019



[Automatic Deception Detection in RGB videos using Facial Action Units](https://dl.acm.org/doi/abs/10.1145/3349801.3349806?casa_token=TMB1PMcnM6QAAAAA:hDBcTW7LdVp0dcFddaugSppeatnshIP8GzDEqrpjTD-ijfio3z_1MoJOaA2GlY80ZeAjbOz_dLHRAg), ICDSC 2019



[Multimodal information fusion for deception detection in videos](https://inaoe.repositorioinstitucional.mx/jspui/bitstream/1009/1966/1/RillGR.pdf)



[Automatic Long-Term Deception Detection in Group Interaction Videos](https://arxiv.org/pdf/1905.08617.pdf), ICME 2019



[Your Eyes Say You’re Lying: An Eye Movement Pattern Analysis for Face Familiarity and Deceptive Cognition](https://arxiv.org/pdf/1811.03401.pdf), IJCNN 2019



[From Text to Speech: A Multimodal Cross-Domain Approach for Deception Detection](https://www.google.com/books?hl=zh-CN&lr=&id=0gyCDwAAQBAJ&oi=fnd&pg=PA164&ots=EHAhzd0Fyk&sig=afw45LrPpXz-z2J_2B-5wXkODg0), ICPRW 2018



[Deception Detection in Videos](http://expertize-journal.org.ua/attachments/article/780/1712.04415.pdf), AAAI 2018



[A Deep Learning Approach for Multimodal Deception Detection](https://arxiv.org/pdf/1803.00344.pdf)



[Toward End-to-End Deception Detection in Videos](https://www.researchgate.net/profile/Hamid-Karimi-21/publication/330622486_Toward_End-to-End_Deception_Detection_in_Videos/links/5cf3d8e892851c4dd02272fe/Toward-End-to-End-Deception-Detection-in-Videos.pdf), ICBD 2018



[Multimodal deception detection](http://www-personal.umich.edu/~zmohamed/PDFs/Bookch13.pdf),2018 The Handbook of Multimodal-Multisensor Interfaces



[Interpretable Multimodal Deception Detection in Videos](http://www.cse.msu.edu/~karimiha/publications/DEV-ICMI-Doctral-Consortium2018.pdf), ICMI 2018



[Deep learning driven multimodal fusion for automated deception detection](https://mandargogate.github.io/papers/SSCI2017-Deep-Learning-Multimodal-Deception.pdf), SSCI 2017



[Detecting Deceptive Behavior via Integration of Discriminative Features From Multiple Modalities](https://www.wikidata.org/entity/Q56524974), TIFS 2017



[Gender-based multimodal deception detection](http://web.eecs.umich.edu/~mihalcea/papers/abouelenien17.acmsac.pdf), SAC 2017



[The Truth and Nothing But the Truth: Multimodal Analysis for Deception Detection](https://arxiv.org/pdf/1903.04484.pdf), ICDMW 2016



[Verbal and Nonverbal Clues for Real-life Deception Detection](http://aclweb.org/anthology/D15-1281), EMNLP 2015



[Deception Detection using Real-life Trial Data](http://web.eecs.umich.edu/~zmohamed/PDFs/Trial.ICMI.pdf), ICMI 2015



[Deception detection using a multimodal approach](http://web.eecs.umich.edu/~mihalcea/papers/abouelenien.icmi14.pdf), ICMI 2014



[Automatic deception detection in italian court cases](https://www.researchgate.net/publication/257520637_Automatic_deception_detection_in_Italian_court_cases), Artificial intelligence

and law 2013



[Motion Profiles for Deception Detection Using Visual Cues](https://www.researchgate.net/profile/Judee_Burgoon/publication/221305092_Motion_Profiles_for_Deception_Detection_Using_Visual_Cues/links/53d659ff0cf220632f3d9a31.pdf), ECCV 2010



[Combining prosodic lexical and cepstral systems for deceptive speech detection](https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=2c1e0c1356972303ba5bcd1f0173efd9c099fbb2), ICASSP 2006



[Distinguishing deceptive from non-deceptive speech](https://academiccommons.columbia.edu/doi/10.7916/D81G0VMK/download) 2005



## Related Research Papers



[Openface: A general-purpose face recognition library with mobile applications](http://reports-archive.adm.cs.cmu.edu/anon/2016/CMU-CS-16-118.pdf?ref=https://githubhelp.com), Openface Report



[wav2vec 2.0: A framework for self-supervised learning of speech representations](https://proceedings.neurips.cc/paper/2020/file/92d1e1eb1cd6f9fba3227870bb6d7f07-Paper.pdf), NeurIPS 2020



[Opensmile: the munich versatile and fast open-source audio feature extractor](https://www.researchgate.net/publication/224929655_openSMILE_--_The_Munich_Versatile_and_Fast_Open-Source_Audio_Feature_Extractor), ACM MM 2010



[Cues to Deception](https://dukespace.lib.duke.edu/dspace/bitstream/handle/10161/14941/DePauloEtAl.Cues%20to%20Deception.pdf?sequence=1), Psychological bulletin 2003



[Detecting deception from emotional and unemotional cues](http://academic.kellogg.edu/talbots/Course%20docs/Psych%20201%2006/detecting%20deception%20from%20emotional%20cues.pdf), Journal of Nonverbal Behavior 2009



[comment]: <> (### Action Recognition)



[comment]: <> ([TEINet: Towards an Efficient Architecture for Video Recognition](https://arxiv.org/pdf/1911.09435.pdf), AAAI 2020)



[comment]: <> ([STH: Spatio-Temporal Hybrid Convolution for Efficient Action Recognition](https://arxiv.org/pdf/2003.08042.pdf), 2020 ArXiv)



[comment]: <> ([Split and Attentive-Aggregated Learnable Shift Module for Video Action Recognition](https://dl.acm.org/doi/abs/10.1145/3426826.3426836?casa_token=osYSvyWvYFgAAAAA:xTwl-phSC82_rbX6vztvAQSAiZOlH_4ZB30PYaMtusergdqOH9-oAZWd-VeiTeUlRSsGaB0z56ACJw), ICML 2020)



## Datasets

BOL: [Bag-of-Lies: A Multimodal Dataset for Deception Detection](https://openaccess.thecvf.com/content_CVPRW_2019/papers/CV-COPS/Gupta_Bag-Of-Lies_A_Multimodal_Dataset_for_Deception_Detection_CVPRW_2019_paper.pdf), CVPRW 2019 [data](http://iab-rubric.org/resources/BagLies.html), [[download]](https://web.eecs.umich.edu/~mihalcea/downloads.html#multimodalDialogDeception)



Real Life Trial Dataset: [Deception Detection using Real-life Trial Data](http://web.eecs.umich.edu/~mihalcea/papers/perezrosas.icmi15.pdf), [[download]](https://web.eecs.umich.edu/~mihalcea/downloads.html#RealLifeDeception)



MU3D: [Miami university deception detection database](https://link.springer.com/article/10.3758/s13428-018-1061-4), [[download]](https://sc.lib.miamioh.edu/handle/2374.MIA/6067)



DDPM:[Deception detection and remote physiological monitoring: A dataset and baseline experimental results](https://arxiv.org/pdf/2106.06583), IJCB 2021, [[download]](https://cvrl.nd.edu/projects/data/#deception-detection-and-physiological-monitoringddpm)



## Tutorials