When you build a chatbot, an automated code reviewer, or a medical‑advice assistant, you quickly discover that confidence scores are not always trustworthy. A model may answer every question with the same sure tone, yet still be wrong—a phenomenon known as overconfidence. In high‑stakes domains—healthcare, finance, autonomous systems—mistaking a hallucinated answer for a factual one can have severe consequences.

In this post we explore a recent MIT‑IBM Watson AI Lab technique that goes beyond self‑consistency checks. By comparing a target LLM’s output with those of a small, diverse ensemble of peer models, the method quantifies epistemic uncertainty and combines it with traditional aleatoric uncertainty into a single total uncertainty (TU) metric. We’ll walk through the intuition, the implementation steps, and practical ways you can adopt this approach in your own pipelines.

---

Why Traditional Uncertainty Measures Fall Short

Aleatoric vs. Epistemic Uncertainty

• Aleatoric uncertainty captures how noisy a single model’s prediction is; it is often estimated by asking the model for a confidence score or by sampling multiple outputs from the same prompt.
• Epistemic uncertainty reflects our ignorance about whether the model architecture or training data are appropriate for the task at hand. It surfaces when different models disagree on the same question.

Most existing tools focus on aleatoric signals—self‑consistency, temperature scaling, or calibrated probabilities. While useful, those signals can be blind to systematic errors that a model repeats confidently.

The Overconfidence Problem

Imagine prompting two state‑of‑the‑art LLMs—ChatGPT and Claude—with the same medical query. Both produce the exact same, confidently worded answer, but the answer is factually incorrect. A self‑consistency test would deem this response trustworthy, yet a clinician would fall prey to a dangerous hallucination.

The MIT‑IBM Ensemble Idea

The researchers propose a simple yet powerful idea: measure disagreement across a small ensemble of heterogeneous LLMs. The key steps are:

1. Select a target model you want to evaluate (e.g., a fine‑tuned GPT‑4).
2. Gather a competing set of 3‑5 models of similar size but trained by different organizations (e.g., Claude, Gemini, Llama 2).
3. Prompt all models with the same input and collect their textual responses.
4. Compute semantic similarity between each response and the target’s answer, using a sentence‑embedding model (e.g., OpenAI’s text‑embedding‑ada‑002 or SBERT).
5. Derive an epistemic score: low average similarity → high epistemic uncertainty.

The authors found that a simple ensemble of off‑the‑shelf models outperformed more complex weighting schemes. Diversity of training data and corporate philosophies turned out to be the most valuable source of disagreement.

Building the Total Uncertainty (TU) Metric

Once you have the epistemic score, you combine it with an aleatoric estimate—commonly the variance of self‑consistency outputs or the model’s own confidence probability. A straightforward formulation is:

TU = α * Aleatoric + (1‑α) * Epistemic

where α is a tunable hyper‑parameter (often set around 0.5). The resulting TU score ranges from 0 (high confidence and high agreement) to 1 (low confidence and high disagreement).

Practical Workflow Example

Below is a minimal Python sketch that demonstrates the whole pipeline using the openai API for embeddings and the transformers library for model inference. Adjust the model_names list to include the APIs you have access to.

The script outputs a single number you can threshold to decide whether to trust the answer, flag it for human review, or request a clarification from the model.

Benefits Observed in the MIT Study

Aspect	Traditional Self‑Consistency	Total Uncertainty (TU)
Prediction accuracy	68 % on factual QA	81 % on same tasks
Query cost	5 × prompts per example	3 × prompts per example
Energy usage	Higher due to repeated sampling	Lower thanks to fewer calls

Key findings include:

• TU consistently outperformed either aleatory or epistemic measures alone across ten benchmark tasks (QA, summarization, translation, math reasoning).
• On tasks with a single correct answer, epistemic disagreement was the strongest indicator of hallucination.
• For open‑ended generation (e.g., creative writing), epistemic signals were weaker, suggesting the need for richer diversity metrics.

When to Use (and When Not To)

• Ideal scenarios – factual question answering, code generation with a single correct solution, medical triage prompts, any application where a wrong answer can cause harm.
• Less suitable – open‑ended brainstorming, poetry, or dialogue where multiple valid outputs exist. In those cases, broaden the ensemble or incorporate lexical diversity metrics.

Extending the Approach

1. Dynamic ensemble composition – automatically select the most complementary models for a given domain (e.g., domain‑specific fine‑tuned LLMs).
2. Weighted disagreement – assign credibility scores to each ensemble member based on recent benchmark performance.
3. Hybrid prompting – feed the TU score back into the target model as a system‑message, encouraging it to generate a more cautious answer.

These extensions can further close the gap between a model’s internal confidence and the external reality of its knowledge.

---

Key Takeaways

• Aleatoric (self‑consistency) metrics alone cannot detect confidently wrong outputs.
• Measuring cross‑model semantic disagreement provides a practical estimate of epistemic uncertainty.
• Combining both signals into a Total Uncertainty (TU) score yields a more reliable trust indicator and reduces query overhead.
• The method works best for tasks with a unique correct answer; open‑ended tasks may require additional diversity cues.

By integrating a lightweight ensemble of heterogeneous LLMs into your evaluation loop, you can flag hallucinations early, conserve compute, and build systems that earn user trust.

---

Source: A better method for identifying overconfident large language models