Testing LLM Statistical Self-Consistency

A study tests whether LLM probability estimates obey the law of total probability when population questions are decomposed into subgroups.

TL;DR — The study tests whether LLM estimates satisfy the law of total probability across population partitions. Using binary-tree subpopulation prompts, it reports widespread consistency violations and a “macro fallacy”: fine-grained subgroup aggregation often aligns better with human reference data than direct population-level estimates.

Problem: LLM estimates should add up

The paper examines a common interpretation of in-context learning: the prompt specifies a context, and the model's output is treated as an estimate of a conditional distribution. If that interpretation is right, then LLM estimates should obey basic probability rules. The focal rule is the law of total probability, which says that estimates over a valid population partition should aggregate into the population-level marginal when weighted by subgroup priors.

Full analysis, extracted claims, numerical results, entity graph, FAQ, related work, applications, and BibTeX are available via x402 micropayment.