Reliable ML and Reproducible Research

About the course

This course teaches you to build and evaluate ML systems you can trust. It covers sources of unrepresentative data, methods for detecting and mitigating bias, probability calibration, uncertainty quantification, and how to run a research cycle correctly without HARKing or p-hacking. The course prepares you to write rigorous research papers and build production systems where mistakes carry a real cost.

What you'll learn

✓Identify and document sources of bias in data and models

✓Calibrate probabilistic predictions and estimate uncertainty using conformal prediction

✓Design experiments without label leakage, correcting for multiple comparisons

✓Apply SHAP and probing classifiers to interpret model behavior

Key topics

•Dataset shift: covariate, label, concept drift

•Bias and fairness: definitions, metrics, mitigation methods

•Probability calibration: Platt scaling, isotonic regression, temperature scaling

•Uncertainty quantification: epistemic vs aleatoric, conformal prediction

•Reproducibility: HARKing, p-hacking, multiple comparisons

•Ablation studies and sound experimental design

•Evaluation leakage: data contamination, benchmark overfitting

•Interpretability: SHAP, LIME, attention, probing classifiers

•Adversarial robustness: basic attacks and defenses

This description was generated automatically and has not yet been reviewed by an instructor — it's a draft for discussion.

About the course

What you'll learn

Key topics

Feedback