Research theme
Uncertainty, Evaluation & Reliable Decision-Making
When should a model prediction be trusted—and what evidence supports that decision?
My research studies how intelligent systems can recognize uncertainty, identify potential failures, and support decisions about when to act, reject a prediction, or seek additional evidence.
This work connects statistical measures of uncertainty with practical evaluation methods. It spans classification and regression, generated-content assessment, and the privacy risks associated with trained models.
Our work on prediction errors leads to a practical question for auditors: what evidence should they collect next under a limited testing budget? Our position paper on language-model auditing examines this question in changing deployment contexts.
Key questions
- Can a model appear confident while being wrong?
- Which uncertainty measures help identify unreliable predictions?
- How should we evaluate generated content and information leakage?
Selected contributions
NeurIPS · 2021
DOCTOR: A Simple Method for Detecting Misclassification Errors
Identifying potentially incorrect classifier predictions from the probabilities produced by a trained model.
ICLR · 2024
A Data-Driven Measure of Relative Uncertainty for Misclassification Detection
Learning uncertainty measures that can distinguish errors even when conventional confidence measures are misleading.
ICML · 2024
Beyond the Norms: Detecting Prediction Errors in Regression Models
Extending the study of prediction reliability to regression.
ACL · 2024
COSMIC: Mutual Information for Task-Agnostic Summarization Evaluation
Using an information-theoretic perspective to assess summaries across potential downstream uses.
JMLR · 2025
Fundamental Limits of Membership Inference Attacks on Machine Learning Models
Characterizing statistical limits on attempts to infer whether a data point was used in training.