{"id":13408791,"url":"https://github.com/AlaaLab/deep-learning-uncertainty","last_synced_at":"2025-03-14T13:32:10.065Z","repository":{"id":47308481,"uuid":"194361525","full_name":"AlaaLab/deep-learning-uncertainty","owner":"AlaaLab","description":"Literature survey, paper reviews, experimental setups and a collection of implementations for baselines methods for predictive uncertainty estimation in deep learning models.","archived":false,"fork":false,"pushed_at":"2022-08-01T23:37:51.000Z","size":8940,"stargazers_count":583,"open_issues_count":1,"forks_count":77,"subscribers_count":32,"default_branch":"master","last_synced_at":"2024-05-23T10:02:38.341Z","etag":null,"topics":["deep-learning","deep-neural-networks","prediction-uncertainty","uncertainty-estimation","uncertainty-quantification"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/AlaaLab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-06-29T03:49:18.000Z","updated_at":"2024-05-19T15:43:21.000Z","dependencies_parsed_at":"2022-09-16T23:50:56.224Z","dependency_job_id":null,"html_url":"https://github.com/AlaaLab/deep-learning-uncertainty","commit_stats":null,"previous_names":["ahmedmalaa/deep-learning-uncertainty"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AlaaLab%2Fdeep-learning-uncertainty","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AlaaLab%2Fdeep-learning-uncertainty/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AlaaLab%2Fdeep-learning-uncertainty/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AlaaLab%2Fdeep-learning-uncertainty/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/AlaaLab","download_url":"https://codeload.github.com/AlaaLab/deep-learning-uncertainty/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243584480,"owners_count":20314771,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","deep-neural-networks","prediction-uncertainty","uncertainty-estimation","uncertainty-quantification"],"created_at":"2024-07-30T20:00:55.278Z","updated_at":"2025-03-14T13:32:10.053Z","avatar_url":"https://github.com/AlaaLab.png","language":"Jupyter Notebook","funding_links":[],"categories":["Machine Learning (ML) and Data Mining (DM)"],"sub_categories":[],"readme":"# Uncertainty Quantification in Deep Learning\n\n[![Python 3.6+](https://img.shields.io/badge/Platform-Python%203.6-blue.svg)](https://www.python.org/)\n[![PyTorch 1.1.0](https://img.shields.io/badge/Implementation-Pytorch-brightgreen.svg)](https://pytorch.org/)\n\nThis repo contains literature survey and implementation of baselines for predictive uncertainty estimation in deep learning.  \n\n## Literature survey\n\n#### Basic background for uncertainty estimation \n\n- B. Efron and R. Tibshirani. \"Bootstrap methods for standard errors, confidence intervals, and other measures of statistical accuracy.\" Statistical science, 1986. [[Link]](https://www.jstor.org/stable/pdf/2245500.pdf)\n\n- R. Barber, E. J. Candes, A. Ramdas, and R. J. Tibshirani. \"Predictive inference with the jackknife+.\" arXiv, 2019. [[Link]](https://arxiv.org/abs/1905.02928)\n\n- B. Efron. \"Jackknife‐after‐bootstrap standard errors and influence functions.\" Journal of the Royal Statistical Society: Series B (Methodological), 1992. [[Link]](https://rss.onlinelibrary.wiley.com/doi/abs/10.1111/j.2517-6161.1992.tb01866.x)\n\n- J. Robins and A. Van Der Vaart. \"Adaptive nonparametric confidence sets.\" The Annals of Statistics, 2006. [[Link]](https://projecteuclid.org/download/pdfview_1/euclid.aos/1146576262)\n\n- V. Vovk, et al., \"Cross-conformal predictive distributions.\" JMLR, 2018. [[Link]](http://proceedings.mlr.press/v91/vovk18a/vovk18a.pdf) \n\n- M. H Quenouille., \"Approximate tests of correlation in time-series.\" Journal of the Royal Statistical Society, 1949. [[Link]](https://www.jstor.org/stable/2983696?seq=1#metadata_info_tab_contents) \n\n- M. H Quenouille. \"Notes on bias in estimation.\" Biometrika, 1956. [[Link]](https://www.jstor.org/stable/2332914?seq=1#metadata_info_tab_contents) \n\n- J. Tukey. \"Bias and confidence in not quite large samples.\" Ann. Math. Statist, 1958. \n\n- R. G. Miller. \"The jackknife–a review.\" Biometrika, 1974. [[Link]](https://www.jstor.org/stable/2334280?seq=1#metadata_info_tab_contents) \n\n- B. Efron. \"Bootstrap methods: Another look at the jackknife.\" Ann. Statist., 1979. [[Link]](https://projecteuclid.org/euclid.aos/1176344552) \n\n- R. A Stine. \"Bootstrap prediction intervals for regression.\" Journal of the American Statistical Association, 1985. [[Link]](https://amstat.tandfonline.com/doi/abs/10.1080/01621459.1985.10478220) \n\n- R. F. Barber, E. J. Candes, A. Ramdas, and R. J. Tibshirani. \"Conformal prediction under covariate shift.\" arXiv preprint arXiv:1904.06019, 2019. [[Link]](https://arxiv.org/pdf/1904.06019.pdf) \n\n- R. F. Barber, E. J. Candes, A. Ramdas, and R. J. Tibshirani. \"The limits of distribution-free conditional predictive inference.\" arXiv preprint arXiv:1903.04684, 2019b. [[Link]](https://arxiv.org/pdf/1903.04684.pdf) \n\n- J. Lei, M. G'Sell, A. Rinaldo, R. J. Tibshirani, and L. Wasserman. \"Distribution-free predictive inference for regression.\" Journal of the American Statistical Association, 2018. [[Link]](https://www.tandfonline.com/doi/pdf/10.1080/01621459.2017.1307116) \n\n- R. Giordano, M. I. Jordan, and T. Broderick. \"A Higher-Order Swiss Army Infinitesimal Jackknife.\" arXiv, 2019. [[Link]](https://arxiv.org/pdf/1907.12116.pdf) \n\n- P. W. Koh, K. Ang, H. H. K. Teo, and P. Liang. \"On the Accuracy of Influence Functions for Measuring Group Effects.\" arXiv, 2019. [[Link]](https://arxiv.org/pdf/1905.13289.pdf)  \n\n- D. H. Wolpert. \"Stacked generalization.\" Neural networks, 1992. [[Link]](https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.133.8090\u0026rep=rep1\u0026type=pdf)  \n\n- R. D. Cook, and S. Weisberg. \"Residuals and influence in regression.\" New York: Chapman and Hall, 1982. [[Link]](https://conservancy.umn.edu/handle/11299/37076)  \n\n- R. Giordano, W. Stephenson, R. Liu, M. I. Jordan, and T. Broderick. \"A Swiss Army Infinitesimal Jackknife.\" arXiv preprint arXiv:1806.00550, 2018. [[Link]](https://arxiv.org/pdf/1806.00550.pdf)  \n\n- P. W. Koh, and P. Liang. \"Understanding black-box predictions via influence functions.\" ICML, 2017. [[Link]](https://dl.acm.org/citation.cfm?id=3305576) \n\n- S. Wager and S. Athey. \"Estimation and inference of heterogeneous treatment effects using random forests.\" Journal of the American Statistical Association, 2018. [[Link]](https://www.tandfonline.com/doi/full/10.1080/01621459.2017.1319839) \n\n- J. F. Lawless, and M. Fredette. \"Frequentist prediction intervals and predictive distributions.\" Biometrika, 2005. [[Link]](https://ideas.repec.org/a/oup/biomet/v92y2005i3p529-542.html) \n\n- F. R. Hampel, E. M. Ronchetti, P. J. Rousseeuw, and W. A. Stahel. \"Robust statistics: the approach based on influence functions.\" John Wiley and Sons, 2011. [[Link]](https://www.wiley.com/en-us/Robust+Statistics%3A+The+Approach+Based+on+Influence+Functions-p-9781118150689)\n\n- P. J. Huber and E. M. Ronchetti. \"Robust Statistics.\" John Wiley and Sons, 1981.\n\n- Y. Romano, R. F. Barber, C. Sabatti, E. J. Candès. \"With Malice Towards None: Assessing Uncertainty via Equalized Coverage.\" arXiv, 2019. [[Link]](https://arxiv.org/pdf/1908.05428.pdf)\n\n- H. R. Kunsch. \"The Jackknife and the Bootstrap for General Stationary Observations.\" The annals of Statistics, 1989. [[Link]](https://www.jstor.org/stable/pdf/2241719.pdf)\n\n\n#### Predictive uncertainty for general machine learning models\n\n- A. Malinin, L. Prokhorenkova, A. Ustimenko. \"Uncertainty in Gradient Boosting via Ensembles.\" ICLR, 2021. [[Link]](https://openreview.net/pdf?id=1Jv6b0Zq3qi)  \n\n- S. Feldman, S. Bates, Y. Romano. \"Improving Conditional Coverage via Orthogonal Quantile Regression.\" arXiv preprint, 2021. [[Link]](https://arxiv.org/pdf/2101.02703.pdf)  \n\n- S. Bates, A. Angelopoulos , L. Lei, J. Malik, and M. I. Jordan. \"Distribution-Free, Risk-Controlling Prediction Sets.\" arXiv preprint, 2021. [[Link]](https://arxiv.org/pdf/2101.02703.pdf) \n\n- S. Wager, T. Hastie, and B. Efron. \"Confidence intervals for random forests: The jackknife and the infinitesimal jackknife.\" The Journal of Machine Learning Research, 2014. [[Link]](http://jmlr.org/papers/volume15/wager14a/wager14a.pdf)\n\n- L. Mentch and G. Hooker. \"Quantifying uncertainty in random forests via confidence intervals and hypothesis tests.\" The Journal of Machine Learning Research, 2016. [[Link]](http://jmlr.org/papers/volume17/14-168/14-168.pdf)\n\n- J. Platt. \"Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods.\" Advances in large margin classifiers, 1999. [[Link]](https://www.researchgate.net/profile/John_Platt/publication/2594015_Probabilistic_Outputs_for_Support_Vector_Machines_and_Comparisons_to_Regularized_Likelihood_Methods/links/004635154cff5262d6000000.pdf)\n\n- A. Abadie, S. Athey, G. Imbens. \"Sampling-based vs. design-based uncertainty in regression analysis.\" arXiv preprint (arXiv:1706.01778), 2017. [[Link]](https://arxiv.org/pdf/1706.01778.pdf) \n\n- T. Duan, A. Avati, D. Y. Ding, S. Basu, Andrew Y. Ng, and A. Schuler. \"NGBoost: Natural Gradient Boosting for Probabilistic Prediction.\" arXiv preprint, 2019. [[Link]](https://arxiv.org/pdf/1910.03225.pdf) \n\n- V. Franc, and D. Prusa. \"On Discriminative Learning of Prediction Uncertainty.\" ICML, 2019. [[Link]](http://proceedings.mlr.press/v97/franc19a/franc19a.pdf)\n\n- Y. Romano, M. Sesia, and E. J. Candès. \"Classification with Valid and Adaptive Coverage.\" arXiv preprint, 2020. [[Link]](https://arxiv.org/pdf/2006.02544.pdf) \n\n\n#### Predictive uncertainty for deep learning\n\n- I. Osband, Z. Wen, M. Asghari, M. Ibrahimi, X. Lu, and B. Van Roy \"Epistemic Neural Networks.\" arXiv, 2021. [[Link]](https://arxiv.org/pdf/2107.08924.pdf)\n\n- Abdar, Moloud, et al. \"A review of uncertainty quantification in deep learning: Techniques, applications and challenges.\" Information Fusion, 2021. [[Link]](https://www.sciencedirect.com/science/article/pii/S1566253521001081?casa_token=J1xs2tpO6Q4AAAAA:zjT_-8Un4Sdw2x4Q5zeqmn_mZg40As5_El-dl70FahcALgWs785fEmmvOfHf1msvFe88bejk) \n\n- Gawlikowski, Jakob, et al. \"A Survey of Uncertainty in Deep Neural Networks.\" arXiv preprint, 2021. [[Link]](https://arxiv.org/pdf/2107.03342v1.pdf) \n\n- P. Morales-Alvarez, D. Hernández-Lobato, R. Molina, J. M. Hernández-Lobato. \"Activation-level uncertainty in deep neural networks.\" ICLR, 2021. [[Link]](https://openreview.net/pdf?id=UvBPbpvHRj-)  \n\n- A. Angelopoulos, S. Bates, J. Malik, and M. I. Jordan. \"Uncertainty Sets for Image Classifiers using Conformal Prediction.\" ICLR, 2021. [[Link]](https://arxiv.org/abs/2009.14193) \n\n- K. Patel, W. H. Beluch, B. Yang, M. Pfeiffer, D. Zhang. \"Multi-Class Uncertainty Calibration via Mutual Information Maximization-based Binning.\" ICLR 2021. [[Link]](https://openreview.net/pdf?id=AICNpd8ke-m)   \n\n- B. Adlam, J. Lee, L. Xiao, J. Pennington, J. Snoek. \"Exploring the Uncertainty Properties of Neural Networks’ Implicit Priors in the Infinite-Width Limit.\" ICLR 2021. [[Link]](https://openreview.net/pdf?id=MjvduJCsE4)  \n\n- A. Harakeh, S. L. Waslander. \"Estimating and Evaluating Regression Predictive Uncertainty in Deep Object Detectors.\" ICLR 2021. [[Link]](https://openreview.net/pdf?id=YLewtnvKgR7)\n\n- J. Antoran, U. Bhatt, T. Adel, A. Weller, J. M. Hernández-Lobato. \"Getting a CLUE: A Method for Explaining Uncertainty Estimates.\" ICLR 2021. [[Link]](https://openreview.net/pdf?id=XSLF1XFq5h)\n\n- A.-K. Kopetzki, B. Charpentier, D. Zügner, S. Giri, S. Günnemann. \"Evaluating Robustness of Predictive Uncertainty Estimation: Are Dirichlet-based Models Reliable?\" ICML 2021. [[Link]](https://arxiv.org/abs/2010.14986) \n\n- A. Zhou and S. Levine. \"Amortized Conditional Normalized Maximum Likelihood: Reliable Out of Distribution Uncertainty Estimation.\" ICML, 2021. [[Link]](https://arxiv.org/pdf/2011.02696.pdf)\n\n- M. Havasi, R. Jenatton, S. Fort, J. Z. Liu, J. Snoek, B. Lakshminarayanan, A. M. Dai, and D. Tran. \"Training independent subnetworks for robust prediction.\" ICLR, 2021. [[Link]](https://arxiv.org/pdf/2010.06610.pdf) \n\n- B. Adlam, J. Lee, L. Xiao, J. Pennington, J. Snoek. \"Exploring the Uncertainty Properties of Neural Networks’ Implicit Priors in the Infinite-Width Limit\". ICLR, 2021. [[Link]](https://openreview.net/forum?id=MjvduJCsE4)\n\n- A. N. Angelopoulos, S. Bates, T. Zrnic, M. I. Jordan. \"Private Prediction Sets.\" arXiv, 2021. [[Link]](https://arxiv.org/abs/2102.06202)\n\n- B. Charpentier, D. Zügner, S. Günnemann. \"Posterior Network: Uncertainty Estimation without OOD Samples via Density-Based Pseudo-Counts.\" NeurIPS, 2020. [[Link]](https://proceedings.neurips.cc/paper/2020/hash/0eac690d7059a8de4b48e90f14510391-Abstract.html) \n\n- L. Meronen, C. Irwanto, A. Solin. \"Stationary Activations for Uncertainty Calibration in Deep Learning.\" NeurIPS, 2020. [[Link]](https://proceedings.neurips.cc/paper/2020/hash/18a411989b47ed75a60ac69d9da05aa5-Abstract.html)  \n\n- F. Wenzel, J. Snoek, D. Tran, R. Jenatton. \"Hyperparameter Ensembles for Robustness and Uncertainty Quantification.\" NeurIPS, 2020. [[Link]](https://proceedings.neurips.cc/paper/2020/hash/481fbfa59da2581098e841b7afc122f1-Abstract.html)   \n\n- J. Liu, Z. Lin, S. Padhy, D. Tran, T. Bedrax Weiss, B. Lakshminarayanan. \"Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awareness.\" NeurIPS, 2020. [[Link]](https://proceedings.neurips.cc/paper/2020/hash/543e83748234f7cbab21aa0ade66565f-Abstract.html)   \n\n- J. Lindinger, D. Reeb, C. Lippert, B. Rakitsch. \"Beyond the Mean-Field: Structured Deep Gaussian Processes Improve the Predictive Uncertainties.\" NeurIPS, 2020. [[Link]](https://proceedings.neurips.cc/paper/2020/hash/60a70bb05b08d6cd95deb3bdb750dce8-Abstract.html)   \n\n- J. Antoran, J. Allingham, J. M. Hernández-Lobato. \"Depth Uncertainty in Neural Networks.\" NeurIPS, 2020. [[Link]](https://proceedings.neurips.cc/paper/2020/hash/781877bda0783aac5f1cf765c128b437-Abstract.html)    \n\n- M. Monteiro, L. Le Folgoc, D. C. de Castro, N. Pawlowski, B. Marques, K. Kamnitsas, M. van der Wilk, B. Glocker. \"Stochastic Segmentation Networks: Modelling Spatially Correlated Aleatoric Uncertainty.\" NeurIPS, 2020. [[Link]](https://proceedings.neurips.cc/paper/2020/hash/95f8d9901ca8878e291552f001f67692-Abstract.html) \n\n- W. Shi, X. Zhao, F. Chen, Q. Yu. \"Multifaceted Uncertainty Estimation for Label-Efficient Deep Learning.\" NeurIPS, 2020. [[Link]](https://proceedings.neurips.cc/paper/2020/hash/c80d9ba4852b67046bee487bcd9802c0-Abstract.html) \n\n- R. Krishnan, O. Tickoo. \"Improving model calibration with accuracy versus uncertainty optimization.\" NeurIPS, 2020. [[Link]](https://proceedings.neurips.cc/paper/2020/hash/d3d9446802a44259755d38e6d163e820-Abstract.html) \n\n- J. A. Leonard, M. A. Kramer, and L. H. Ungar. \"A neural network architecture that computes its own reliability.\" Computers \u0026 chemical engineering, 1992. [[Link]](https://www.sciencedirect.com/science/article/pii/0098135492800358)\n\n- C. Blundell, J. Cornebise, K. Kavukcuoglu, and D. Wierstra. \"Weight uncertainty in neural networks.\" ICML, 2015. [[Link]](https://arxiv.org/pdf/1505.05424.pdf) \n\n- B. Lakshminarayanan, A. Pritzel, and C. Blundell. \"Simple and scalable predictive uncertainty estimation using deep ensembles.\" NeurIPS, 2017. [[Link]](http://papers.nips.cc/paper/7219-simple-and-scalable-predictive-uncertainty-estimation-using-deep-ensembles.pdf)\n\n- Y. Gal and Z. Ghahramani. \"Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning.\" ICML, 2016. [[Link]](https://arxiv.org/pdf/1506.02142.pdf)\n\n- V. Kuleshov, N. Fenner, and S. Ermon. \"Accurate Uncertainties for Deep Learning Using Calibrated Regression.\" ICML, 2018. [[Link]](http://proceedings.mlr.press/v80/kuleshov18a/kuleshov18a.pdf)\n\n- J. Hernández-Lobato and R. Adams. \"Probabilistic backpropagation for scalable learning of bayesian neural networks.\" ICML, 2015. [[Link]](http://proceedings.mlr.press/v37/hernandez-lobatoc15.pdf)\n\n- S. Liang, Y. Li, and R. Srikant. \"Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks.\" ICLR, 2018. [[Link]](https://openreview.net/forum?id=H1VGkIxRZ)\n\n- K. Lee, H. Lee, K. Lee, and J. Shin. \"Training Confidence-calibrated classifiers for detecting out-of-distribution samples.\" ICLR, 2018. [[Link]](https://openreview.net/forum?id=ryiAv2xAZ)\n\n- P. Schulam and S. Saria \"Can You Trust This Prediction? Auditing Pointwise Reliability After Learning.\" AISTATS, 2019. [[Link]](http://proceedings.mlr.press/v89/schulam19a/schulam19a.pdf) \n\n- A. Malinin and M. Gales. \"Predictive uncertainty estimation via prior networks.\" NeurIPS, 2018. [[Link]](http://papers.nips.cc/paper/7936-predictive-uncertainty-estimation-via-prior-networks.pdf) \n\n- D. Hendrycks, M. Mazeika, and T. G. Dietterich. \"Deep anomaly detection with outlier exposure.\" arXiv preprint arXiv:1812.04606, 2018. [[Link]](https://arxiv.org/pdf/1812.04606.pdf)\n\n- A-A. Papadopoulos, M. R. Rajati, N. Shaikh, and J. Wang. \"Outlier exposure with confidence control for out-of-distribution detection.\" arXiv preprint arXiv:1906.03509, 2019. [[Link]](https://arxiv.org/pdf/1906.03509.pdf)\n\n- D. Madras, J. Atwood, A. D'Amour, \"Detecting Extrapolation with Influence Functions.\" ICML Workshop on Uncertainty and Robustness in Deep Learning, 2019. [[Link]](http://www.gatsby.ucl.ac.uk/~balaji/udl2019/accepted-papers/UDL2019-paper-05.pdf)  \n\n- M. Sensoy, L. Kaplan, and M. Kandemir. \"Evidential deep learning to quantify classification uncertainty.\" NeurIPS, 2018. [[Link]](https://papers.nips.cc/paper/7580-evidential-deep-learning-to-quantify-classification-uncertainty.pdf)\n\n- W. Maddox, T. Garipov, P. Izmailov, D. Vetrov, and A. G. Wilson. \"A simple baseline for bayesian uncertainty in deep learning.\" arXiv preprint arXiv:1902.02476, 2019. [[Link]](https://arxiv.org/pdf/1902.02476.pdf)\n\n- Y. Ovadia, et al. \"Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift.\" arXiv preprint arXiv:1906.02530, 2019. [[Link]](https://arxiv.org/pdf/1906.02530.pdf)\n\n- D. Hendrycks, et al. \"Using Self-Supervised Learning Can Improve Model Robustness and Uncertainty.\" arXiv preprint arXiv:1906.12340, 2019. [[Link]](https://arxiv.org/pdf/1906.12340.pdf)\n\n- A. Kumar, P. Liang, T. Ma. \"Verified Uncertainty Calibration.\" arXiv preprint, 2019. [[Link]](https://arxiv.org/abs/1909.10155) \n\n- I. Osband, C. Blundell, A. Pritzel, and B. Van Roy. \"Deep Exploration via Bootstrapped DQN.\" NeurIPS, 2016. [[Link]](https://papers.nips.cc/paper/6501-deep-exploration-via-bootstrapped-dqn.pdf) \n\n- I. Osband. \"Risk versus Uncertainty in Deep Learning: Bayes, Bootstrap and the Dangers of Dropout.\" NeurIPS Workshop, 2016. [[Link]](http://bayesiandeeplearning.org/2016/papers/BDL_4.pdf) \n\n- J. Postels et al. \"Sampling-free Epistemic Uncertainty Estimation Using Approximated Variance Propagation.\" ICCV, 2019. [[Link]](http://openaccess.thecvf.com/content_ICCV_2019/papers/Postels_Sampling-Free_Epistemic_Uncertainty_Estimation_Using_Approximated_Variance_Propagation_ICCV_2019_paper.pdf)  \n\n- A. Kendall and Y. Gal. \"What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?\" NeurIPS, 2017. [[Link]](https://arxiv.org/pdf/1703.04977.pdf) \n\n- N. Tagasovska and D. Lopez-Paz. \"Single-Model Uncertainties for Deep Learning.\" NeurIPS, 2019. [[Link]](https://papers.nips.cc/paper/8870-single-model-uncertainties-for-deep-learning.pdf) \n\n- A. Der Kiureghian and O. Ditlevsen. \"Aleatory or Epistemic? Does it Matter?.\" Structural Safety, 2009. [[Link]](https://www.sciencedirect.com/science/article/pii/S0167473008000556) \n\n- D. Hafner, D. Tran, A. Irpan, T. Lillicrap, and J. Davidson. \"Reliable uncertainty estimates in deep neural networks using noise contrastive priors.\" arXiv, 2018. [[Link]](https://arxiv.org/pdf/1807.09289.pdf)\n\n- S. Depeweg, J. M. Hernández-Lobato, F. Doshi-Velez, and S. Udluft. \"Decomposition of uncertainty in Bayesian deep learning for efficient and risk-sensitive learning.\" ICML, 2018. [[Link]](http://publications.eng.cam.ac.uk/945907/)\n\n- L. Smith and Y. Gal, \"Understanding Measures of Uncertainty for Adversarial Example Detection.\" UAI, 2018. [[Link]](https://arxiv.org/pdf/1803.08533.pdf)\n\n- L. Zhu and N. Laptev. \"Deep and Confident Prediction for Time series at Uber.\" IEEE International Conference on Data Mining Workshops, 2017. [[Link]](https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=8215650)\n\n- M. W. Dusenberry, G. Jerfel, Y. Wen, Yi-an Ma, J. Snoek, K. Heller, B. Lakshminarayanan, D. Tran. \"Efficient and Scalable Bayesian Neural Nets with Rank-1 Factors.\" arXiv, 2020. [[Link]](https://arxiv.org/abs/2005.07186)\n\n- J. van Amersfoort, L. Smith, Y. W. Teh, and Y. Gal. \"Uncertainty Estimation Using a Single Deep Deterministic Neural Network.\" ICML, 2020. [[Link]](https://arxiv.org/abs/2003.02037)\n\n- E. Begoli, T. Bhattacharya and D. Kusnezov. \"The need for uncertainty quantification in machine-assisted medical decision making.\" Nature Machine Intelligence, 2019. [[Link]](https://www.nature.com/articles/s42256-018-0004-1)  \n\n- T. S. Salem, H. Langseth, and H. Ramampiaro. \"Prediction Intervals: Split Normal Mixture from Quality-Driven Deep Ensembles.\" UAI, 2020. [[Link]](http://proceedings.mlr.press/v124/saleh-salem20a/saleh-salem20a.pdf)  \n\n- K. Posch, and J. Pilz, \"Correlated Parameters to Accurately Measure Uncertainty in Deep Neural Networks.\" IEEE Transactions on Neural Networks and Learning Systems, 2020. [[Link]](https://ieeexplore.ieee.org/document/9070148)\n\n- B. Kompa, J. Snoek, and A. Beam. \"Empirical Frequentist Coverage of Deep Learning Uncertainty Quantification Procedures.\" arXiv, 2020. [[Link]](https://arxiv.org/pdf/2010.03039.pdf)\n\n\n#### Predictive uncertainty in sequential models\n\n- A. Malinin, M. Gales. \"Uncertainty Estimation in Autoregressive Structured Prediction.\" ICLR, 2021. [[Link]](https://openreview.net/pdf?id=jN5y-zb5Q7m)\n\n- C. Wang, C. Lawrence, M. Niepert. \"Uncertainty Estimation and Calibration with Finite-State Probabilistic RNNs.\" ICLR, 2021. [[Link]](https://openreview.net/pdf?id=9EKHN1jOlA)\n\n- S. Sankaranarayanan, Y. Chou, E. Goubault, S. Putot. \"Reasoning about Uncertainties in Discrete-Time Dynamical Systems using Polynomial Forms.\" NeurIPS, 2020. [[Link]](https://proceedings.neurips.cc/paper/2020/hash/ca886eb9edb61a42256192745c72cd79-Abstract.html)\n\n- R. Wen, K. Torkkola, B. Narayanaswamy, and D. Madeka. \"A Multi-horizon Quantile Recurrent Forecaster.\" arXiv, 2017. [[Link]](https://arxiv.org/abs/1711.11053)\n\n- D. T. Mirikitani and N. Nikolaev. \"Recursive bayesian recurrent neural networks for time-series modeling.\" IEEE Transactions on Neural Networks, 2009. [[Link]](https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=5361332)\n\n- M. Fortunato, C. Blundell and O. Vinyals. \"Bayesian Recurrent Neural Networks.\" arXiv, 2019. [[Link]](https://arxiv.org/pdf/1704.02798.pdf)\n\n- P. L. McDermott, C. K. Wikle. \"Bayesian Recurrent Neural Network Models for Forecasting and Quantifying Uncertainty in Spatial-Temporal Data.\" Entropy, 2019. [[Link]](https://www.mdpi.com/1099-4300/21/2/184)\n\n- Y. Gal, Z. Ghahramani. \"A theoretically grounded application of dropout in recurrent neural networks.\" NeurIPS, 2016. [[Link]](https://papers.nips.cc/paper/6241-a-theoretically-grounded-application-of-dropout-in-recurrent-neural-networks.pdf) \n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FAlaaLab%2Fdeep-learning-uncertainty","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FAlaaLab%2Fdeep-learning-uncertainty","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FAlaaLab%2Fdeep-learning-uncertainty/lists"}