Content
Metrology in data science and machine learning
Fundamental topics in stochastics, probability theory, and statistics
- Random variables, probability spaces, probability density functions
- Independence, marginalization
- Law of large numbers
- Central limit theorem
- Bayesian inference, priors and posteriors
- Confidence and credibility intervals
- Bootstrap, Jackknife, method of surrogate data
Uncertainty propagation through fixed measurement equations
- Guide to the Expression of Uncertainty in Measurement (GUM)
- Polynomial chaos
- Monte Carlo methods
Uncertainty estimation in Machine Learning and Deep Learning
- Aleatoric and epistemic uncertainty
- Hierarchical and empirical Bayesian models, predictive distribution
- Gaussian processes
- Errors-in-variables models
- Robust regression
- Bayesian/Probabilistic Neural Networks
- Dropout
- Ensemble Methods
- Monte Carlo methods
- Conformal prediction
- Variational inference
- Normalizing flows, invertible neural networks, diffusion models
Metrics of Uncertainty Calibration
- Proper Scoring Rules
- Calibration Curves
Advanced Topics
- Out-of-distribution detection
- Active learning for uncertainty reduction
Applications of uncertainty estimation in machine learning