FUURAA AI Knowledge Library · Calibration and uncertainty evaluation

How to evaluate AI calibration and uncertainty claims

Use six evidence gates to turn “the model is confident” into a defined decision, target population, separate calibration data, reproducible score, accuracy and calibration, risk-coverage, shifts, complete failures and a dated conclusion. Apply it to classification, generative AI, question answering, prediction sets, abstention, model reports, procurement and engineering review.

Enter the six evidence gates ↓Open the failure matrix

Published6 October 2026Evidence statusMethod synthesis grounded in primary calibration, selective-prediction and uncertainty researchScopeClassification, generative AI, QA, prediction sets, abstention, procurement and engineering review

A confidence score is not reliable uncertainty

An uncertainty claim must keep the score, outcomes, population, calibration protocol, coverage and real decision together.

Accuracy asks how often a system is right. Calibration asks whether outcomes among similarly scored predictions match their stated confidence. Selective risk asks what errors remain at a given coverage after abstention. They are related, but not interchangeable.

Applicability boundaryThis is a public research method, not a FUURAA or FUUVO product-capability claim or an assessment of any model, provider or deployment. It is not a capability guarantee, certification, audit, procurement, investment, legal, medical or compliance opinion.

Six rejectable evidence gates

Each gate answers one decision question, requires minimum evidence and stops transfer or narrows the conclusion when material unknowns remain.

Freeze the uncertainty claim and decision

Decision question: What exact confidence, probability, uncertainty, prediction-set or abstention claim is being made, for which system, output and decision?
Minimum evidence: Verbatim claim and date; exact model, prompt, decoding and serving versions; target output and eligible population; decision, loss, error costs, threshold, owner and expiry.
Stop condition: Stop when a raw model score, verbal hedge or polished confidence statement is treated as a probability of correctness without an operational definition.

Version outcomes, populations and calibration data

Decision question: Which predictions and outcomes define correctness, how were they sampled and labelled, and which separate data calibrated and tested the claim?
Minimum evidence: Source, collection dates and sampling frame; outcome timing and label rules; exclusions, missingness, ambiguity and adjudication; train, calibration and test separation; subgroup, language, task and time coverage.
Stop condition: Stop when calibration and evaluation reuse the same outcomes, labels are unresolved, or the target population and sampling process are unknown.

Reconstruct the score, calibration and abstention pipeline

Decision question: How does raw model behaviour become the reported score, interval, set, confidence label or refusal?
Minimum evidence: Raw logits, likelihoods or sampled outputs where available; semantic grouping; calibration method and fit; score normalisation; set construction; thresholds; prompt and sampling settings; retrieval or tool state; fallback and routing.
Stop condition: Stop when only a final percentage remains and the transformation, calibration sample, threshold or abstention path cannot be reconstructed.

Separate accuracy, calibration, coverage and usefulness

Decision question: Is the system correct, are its stated probabilities reliable, what does it cover or reject, and does uncertainty improve the target decision?
Minimum evidence: Accuracy and task metrics; reliability diagrams with declared bins; NLL and Brier score; ECE with variants and confidence intervals; risk-coverage curves; prediction-set coverage and size; abstention precision; decision utility at declared costs.
Stop condition: Stop when one average calibration number hides poor accuracy, local or subgroup miscalibration, low coverage, oversized sets or harmful routing.

Count shifts, repeated runs, failures and cost

Decision question: What happens across every eligible case, subgroup, repeated sample, prompt variant, low-support input and changed distribution?
Minimum evidence: Complete denominators; wrong high-confidence and right low-confidence cases; unsupported and out-of-domain inputs; prompt, seed and sampling variance; subgroup and temporal slices; drift tests; retries, latency, compute, human review and downstream burden.
Stop condition: Stop when in-distribution averages or selected examples hide high-confidence errors, unstable scores, abstention burden, slow tails or total cost.

Validate the workflow and expire the claim

Decision question: Do users and downstream controls interpret the signal correctly under current data, costs and consequences, and when must the conclusion be revalidated?
Minimum evidence: Shadow or staged workflow tests; user comprehension and action studies; escalation capacity; threshold and routing review; monitoring for calibration, coverage and selective risk; rollback; model, prompt, data, policy and population change triggers; owner and expiry.
Stop condition: Stop when a benchmark score becomes a permanent trust label or autonomous authority without target-workflow validation, monitoring and rollback.

Minimum calibration and uncertainty claim failure matrix

Check these conditions deliberately to distinguish attractive confidence numbers from uncertainty evidence that can support a decision.

01
A low average ECE hides severe subgroup or local miscalibration
Record affected populations, score ranges, tasks, groups and decisions; preserve the narrowest conclusion that remains and specify whether to recalibrate, segment results, change thresholds, route to people or withdraw the claim.
02
Token likelihood is presented as the probability that an answer is true
Record affected populations, score ranges, tasks, groups and decisions; preserve the narrowest conclusion that remains and specify whether to recalibrate, segment results, change thresholds, route to people or withdraw the claim.
03
Calibration and test outcomes come from the same examples
Record affected populations, score ranges, tasks, groups and decisions; preserve the narrowest conclusion that remains and specify whether to recalibrate, segment results, change thresholds, route to people or withdraw the claim.
04
Different wordings of the same answer inflate apparent uncertainty
Record affected populations, score ranges, tasks, groups and decisions; preserve the narrowest conclusion that remains and specify whether to recalibrate, segment results, change thresholds, route to people or withdraw the claim.
05
Coverage holds in-distribution but fails after task or population shift
Record affected populations, score ranges, tasks, groups and decisions; preserve the narrowest conclusion that remains and specify whether to recalibrate, segment results, change thresholds, route to people or withdraw the claim.
06
Abstention reduces model risk but overloads users or reviewers
Record affected populations, score ranges, tasks, groups and decisions; preserve the narrowest conclusion that remains and specify whether to recalibrate, segment results, change thresholds, route to people or withdraw the claim.
07
A calibrated score is mistaken for accuracy, fairness or safety
Record affected populations, score ranges, tasks, groups and decisions; preserve the narrowest conclusion that remains and specify whether to recalibrate, segment results, change thresholds, route to people or withdraw the claim.
08
Self-reported confidence is accepted without matched outcome evidence
Record affected populations, score ranges, tasks, groups and decisions; preserve the narrowest conclusion that remains and specify whether to recalibrate, segment results, change thresholds, route to people or withdraw the claim.

Minimum calibration and uncertainty evaluation record

Let the next reviewer reconstruct the conclusion under the same population, outcomes, scores, calibration protocol, thresholds and decision boundaries.

01Exact claim, date, owner, model, output, user, decision and expiry
02Target population, source, time window, sampling, exclusions and missingness
03Outcome, correctness and ambiguity rules, annotators, disagreement and adjudication
04Train, calibration and test separation, versions and contamination checks
05Raw score, prompts, sampling, semantic grouping, calibration and thresholds
06Accuracy, NLL, Brier, ECE definition, reliability plot and uncertainty intervals
07Risk-coverage, prediction-set coverage and size, rejection and routing results
08Wrong high-confidence cases, right low-confidence cases and subgroup results
09Shift, repeated-run variance, latency, cost, human work and downstream burden
10Narrowest supported conclusion, monitoring, rollback and revalidation trigger

Common evidence states

Bind conclusions to the exact system, population, calibration sample, metrics, thresholds, decision and date.

Supported

The exact signal meets declared calibration, coverage, selective-risk and decision thresholds on a reproducible target distribution.

Conditional

Evidence supports only named tasks, populations, calibration data, thresholds, costs and workflow controls.

Mixed

Calibration, coverage or utility differs across tasks, groups, languages, shifts or confidence ranges and requires segmented reporting.

Insufficient

A raw score, confidence phrase, selected example or single aggregate metric cannot establish reliable uncertainty.

Sources, evidentiary role and non-transfer boundaries

Each source supports only what it actually measures; publication dates and boundaries must travel with the conclusion.

First submitted 14 June 2017

On Calibration of Modern Neural Networks

Evidentiary roleShows that modern neural networks can be miscalibrated and evaluates temperature scaling and other post-processing methods on image and document classification.

Do not transferResults on selected classifiers and datasets do not establish calibration under a new task, subgroup, time period or distribution shift.

Open primary source ↗

First submitted 26 January 2019

SelectiveNet — A Deep Neural Network with an Integrated Reject Option

Evidentiary roleFrames selective prediction through the risk-coverage trade-off and trains prediction and rejection together.

Do not transferBetter selective risk on benchmark data does not prove that abstentions are correctly routed, affordable or safe in a target workflow.

Open primary source ↗

First submitted 7 January 2021

Distribution-Free, Risk-Controlling Prediction Sets

Evidentiary roleDevelops set-valued predictions with finite-sample control of expected loss under a stated calibration protocol.

Do not transferThe guarantee depends on its stated loss, calibration sample and assumptions; it is not a universal guarantee for every user, shift or downstream decision.

Open primary source ↗

First submitted 19 February 2023

Semantic Uncertainty — Linguistic Invariances for Uncertainty Estimation in Natural Language Generation

Evidentiary roleIntroduces semantic entropy so differently worded answers with the same meaning are grouped when estimating uncertainty in open-ended generation.

Do not transferPerformance on selected question-answering datasets does not establish calibrated confidence for every generation task, language, model or factual domain.

Open primary source ↗

Published December 2023

On the Calibration of Large Language Models and Alignment

Evidentiary roleA Chinese-led study examines how pretraining and alignment settings affect LLM calibration across generation, factuality and understanding.

Do not transferObserved effects for studied models and evaluation choices do not transfer automatically to another model release, prompting method, language or operating context.

Open primary source ↗

Published November 2024

Calibrating the Confidence of Large Language Models by Eliciting Fidelity

Evidentiary roleA Fudan University and Meituan collaboration separates uncertainty about a question from fidelity to a generated answer and evaluates a plug-and-play calibration method.

Do not transferResults on six aligned models and four multiple-choice datasets do not establish calibration for open-ended production outputs or changed domains.

Open primary source ↗

First submitted 21 June 2024; revised 30 June 2025

LM-Polygraph — Benchmarking Uncertainty Quantification Methods for Large Language Models

Evidentiary roleBenchmarks uncertainty and score-normalisation methods across selective question answering, generation and multilingual claim-level fact checking, including Chinese.

Do not transferA benchmark comparison does not make one score universally interpretable or guarantee identical ranking under new prompts, languages, datasets or decision costs.

Open primary source ↗

FUURAA analysis

Useful uncertainty is not making a model sound cautious; it is giving a specific signal a verifiable meaning for a specific population and decision.

FUURAA’s methodological conclusion is that reliability cannot be inferred directly from one confidence number. The minimum evidence chain connects the exact system version, target population and outcomes, separate calibration data, score construction, accuracy and calibration metrics, coverage and abstention behaviour, high-confidence errors, distribution shift, human and system burden, and a dated target-workflow decision.