Research

Severity-Aware Conformal Risk Control for Medical Claim Filtering

Under review · ML4H 2026

Sashuv Kafle, Binod Bhattarai, Prashnna Kumar Gyawali

• Trustworthy ML
• Uncertainty quantification
• AI for healthcare

Realized risk against the confidence threshold for benign and dangerous claim tiers. One global threshold leaves the dangerous tier above its budget; tier-specific thresholds meet both budgets.

The problem

When a language model answers a medical question, some of its claims are harmless if wrong and some are dangerous. Standard conformal risk control filters claims with a single threshold that bounds the average error rate. One flat bound for everything means dangerous claims can slip through as long as the benign ones pull the average down.

Approach

I extend conformal risk control to tier-specific risk budgets. Each claim is assigned a severity tier, and each tier gets its own calibrated threshold, so claims are held to a stricter bound the more clinically dangerous they are. The guarantee is still distribution-free: it needs no assumptions about the model or the data beyond exchangeability.

The figure shows why this matters. A single global threshold keeps the benign tier well under its budget, but leaves the dangerous tier above its own. Tier-specific thresholds meet both.

Results

• The guarantee holds regardless of which scoring function is used. I tested four very different ones: model self-confidence, sequence log-probability, NLI entailment, and a random baseline.
• Evaluated across Llama-3-8B-Instruct, Mistral-7B-Instruct and Qwen2.5-3B-Instruct.

← All research