AI Explanations, Misinformation, and Belief Updating

Wed, 01 Jan 2025 00:00:00 +0000

: replication code, experiment design, and analysis
Journal Submission in Preparation

Project Overview

This project investigates whether AI-generated explanations help individuals move toward factual accuracy or instead increase susceptibility to misinformation. Using a survey experiment, participants evaluated both accurate and deceptive AI-generated explanations across controversial and neutral topics. By examining belief change before and after exposure to AI explanations, the study explores how explanatory AI systems influence information evaluation, factual accuracy, and susceptibility to misinformation across different contexts and levels of prior knowledge.

Research Question

Can AI-generated explanations improve factual accuracy, or do deceptive AI explanations systematically move people away from the truth?

Methods

The analysis uses a survey experimental design with pre- and post-treatment belief ratings. Participants were exposed to accurate or deceptive AI-generated explanations across controversial and neutral factual claims. Treatment effects are estimated using mixed-effects models with participant and statement random intercepts, with heterogeneous effects examined by issue type and self-rated knowledge.

Key Findings

Finding 1: Accurate AI explanations improve factual accuracy, while deceptive explanations reduce it

Model-based estimates show that accurate AI explanations move participants toward factual accuracy, whereas deceptive AI explanations move participants substantially away from factual accuracy.

Predicted movement toward factual accuracy by AI explanation type. Positive values indicate movement toward factual accuracy.

Finding 2: The effects differ across controversial and neutral issues

AI explanations affect belief updating across both neutral and controversial statements, with deceptive explanations producing especially strong movement away from factual accuracy.

Estimated AI explanation effects by issue type, comparing controversial and neutral statements.

Finding 3: Self-rated knowledge moderates susceptibility to AI explanations

The effect of AI explanations varies by participants’ self-rated knowledge. This suggests that prior knowledge shapes how individuals respond to both accurate and deceptive AI-generated explanations.

Predicted movement toward factual accuracy by self-rated knowledge and AI explanation condition.

Additional Results

Statement-Level Effects

Belief updating patterns are broadly consistent across individual statements, with accurate explanations generally moving respondents toward factual positions and deceptive explanations moving respondents away from them. While the magnitude of effects varies across topics, the overall pattern remains remarkably consistent across both factual and politically salient issues.

Belief ratings before and after exposure to accurate and deceptive AI-generated explanations across individual statements. Accurate explanations generally move respondents toward factual positions, while deceptive explanations produce movement away from factual positions across a range of topics.

Research Contribution

This study contributes to emerging research on AI, misinformation, and human–AI interaction by examining how AI-generated explanations influence belief revision. The findings demonstrate that explanatory AI systems can shape factual accuracy in both beneficial and harmful directions, underscoring the importance of evaluating not only the information AI systems provide, but also how AI-generated explanations influence information processing, misinformation susceptibility, and trust in automated knowledge systems.

Mixed-Effects Models |