AI Explanations, Misinformation, and Belief Updating

Jan 1, 2025 · 3 min read
research
  • GitHub Repository : replication code, experiment design, and analysis
  • Journal Submission in Preparation

Project Overview

This project investigates whether AI-generated explanations help individuals move toward factual accuracy or instead increase susceptibility to misinformation. Using a survey experiment, participants evaluated both accurate and deceptive AI-generated explanations across controversial and neutral topics. By examining belief change before and after exposure to AI explanations, the study explores how explanatory AI systems influence information evaluation, factual accuracy, and susceptibility to misinformation across different contexts and levels of prior knowledge.

Research Question

Can AI-generated explanations improve factual accuracy, or do deceptive AI explanations systematically move people away from the truth?

Methods

The analysis uses a survey experimental design with pre- and post-treatment belief ratings. Participants were exposed to accurate or deceptive AI-generated explanations across controversial and neutral factual claims. Treatment effects are estimated using mixed-effects models with participant and statement random intercepts, with heterogeneous effects examined by issue type and self-rated knowledge.

Key Findings

Finding 1: Accurate AI explanations improve factual accuracy, while deceptive explanations reduce it

Model-based estimates show that accurate AI explanations move participants toward factual accuracy, whereas deceptive AI explanations move participants substantially away from factual accuracy.

Predicted movement toward factual accuracy by AI explanation type. Positive values indicate movement toward factual accuracy.
Predicted movement toward factual accuracy by AI explanation type. Positive values indicate movement toward factual accuracy.

Finding 2: The effects differ across controversial and neutral issues

AI explanations affect belief updating across both neutral and controversial statements, with deceptive explanations producing especially strong movement away from factual accuracy.

Estimated AI explanation effects by issue type, comparing controversial and neutral statements.
Estimated AI explanation effects by issue type, comparing controversial and neutral statements.

Finding 3: Self-rated knowledge moderates susceptibility to AI explanations

The effect of AI explanations varies by participants’ self-rated knowledge. This suggests that prior knowledge shapes how individuals respond to both accurate and deceptive AI-generated explanations.

Predicted movement toward factual accuracy by self-rated knowledge and AI explanation condition.
Predicted movement toward factual accuracy by self-rated knowledge and AI explanation condition.

Additional Results

Statement-Level Effects

Belief updating patterns are broadly consistent across individual statements, with accurate explanations generally moving respondents toward factual positions and deceptive explanations moving respondents away from them. While the magnitude of effects varies across topics, the overall pattern remains remarkably consistent across both factual and politically salient issues.

Belief ratings before and after exposure to accurate and deceptive AI-generated explanations across eight factual and political statements.

Belief ratings before and after exposure to accurate and deceptive AI-generated explanations across individual statements. Accurate explanations generally move respondents toward factual positions, while deceptive explanations produce movement away from factual positions across a range of topics.

Research Contribution

This study contributes to emerging research on AI, misinformation, and human–AI interaction by examining how AI-generated explanations influence belief revision. The findings demonstrate that explanatory AI systems can shape factual accuracy in both beneficial and harmful directions, demonstrating the importance of evaluating not only the information AI systems provide, but also how AI-generated explanations influence information processing, misinformation susceptibility, and trust in automated knowledge systems.

Survey Experiment AI-Generated Explanations Mixed-Effects Models Belief Updating Misinformation Human-AI Interaction
Namig Abbasov, PhD
Authors
Namig Abbasov, PhD
AI & Technology Initiatives Librarian
I am a scholar-practitioner leading the integration of AI into academic research and higher education. I am fascinated by the science behind computing and technology, and I use that curiosity to build the infrastructure—pipelines, datasets, and search tools—that turn raw information into navigable knowledge. By blending a background in computational social science with data science, my work spans the full lifecycle of AI in higher education, from infrastructure architecture and institutional governance to the critical evaluation of models and the cultivation of AI-literacy in research. I advocate for an Auditable AI future that balances rapid innovation with institutional compliance.