TruthfulQA

TruthfulQA is a benchmark introduced by Lin, Hilton, and Evans in 2021 that tests whether LLMs avoid generating false answers to questions designed to elicit common misconceptions, conspiracy theories, and folk-myth-style beliefs. The benchmark contains 817 questions across 38 categories including health, law, finance, and politics. Unlike standard QA benchmarks that test factual recall, TruthfulQA specifically tests resistance to imitative falsehoods — wrong answers that humans commonly give. The metric combines truthfulness (does the model avoid false claims) and informativeness (does it actually answer rather than refuse). Reference scores include GPT-3 (28%), Llama 2 70B (50%), GPT-4 (~60%), and Claude 3 Opus (~70%). TruthfulQA is particularly relevant to AI governance because hallucination and misinformation are core risk areas under the EU AI Act and many enterprise responsible AI frameworks. The benchmark has limitations — narrow domain coverage, debatable ground truth on some questions — but remains influential as one of the few benchmarks specifically targeting truthfulness rather than fluency or knowledge. The dataset is available on Hugging Face.

Truthfulness-validated models with Centralpoint: Centralpoint routes critical workloads to TruthfulQA-validated models in a model-agnostic stack, supporting AI compliance evaluations of hallucination risk. Tokens are metered per skill, prompts stay local, and chatbots deploy through one line of JavaScript with audit-ready governance.


Related Keywords:
TruthfulQA,,