Researches |

AI Explanations, Misinformation, and Belief Updating

Wed, 01 Jan 2025 00:00:00 +0000

: replication code, experiment design, and analysis
Journal Submission in Preparation

Project Overview

This project investigates whether AI-generated explanations help individuals move toward factual accuracy or instead increase susceptibility to misinformation. Using a survey experiment, participants evaluated both accurate and deceptive AI-generated explanations across controversial and neutral topics. By examining belief change before and after exposure to AI explanations, the study explores how explanatory AI systems influence information evaluation, factual accuracy, and susceptibility to misinformation across different contexts and levels of prior knowledge.

Research Question

Can AI-generated explanations improve factual accuracy, or do deceptive AI explanations systematically move people away from the truth?

Methods

The analysis uses a survey experimental design with pre- and post-treatment belief ratings. Participants were exposed to accurate or deceptive AI-generated explanations across controversial and neutral factual claims. Treatment effects are estimated using mixed-effects models with participant and statement random intercepts, with heterogeneous effects examined by issue type and self-rated knowledge.

Key Findings

Finding 1: Accurate AI explanations improve factual accuracy, while deceptive explanations reduce it

Model-based estimates show that accurate AI explanations move participants toward factual accuracy, whereas deceptive AI explanations move participants substantially away from factual accuracy.

Predicted movement toward factual accuracy by AI explanation type. Positive values indicate movement toward factual accuracy.

Finding 2: The effects differ across controversial and neutral issues

AI explanations affect belief updating across both neutral and controversial statements, with deceptive explanations producing especially strong movement away from factual accuracy.

Estimated AI explanation effects by issue type, comparing controversial and neutral statements.

Finding 3: Self-rated knowledge moderates susceptibility to AI explanations

The effect of AI explanations varies by participants’ self-rated knowledge. This suggests that prior knowledge shapes how individuals respond to both accurate and deceptive AI-generated explanations.

Predicted movement toward factual accuracy by self-rated knowledge and AI explanation condition.

Additional Results

Statement-Level Effects

Belief updating patterns are broadly consistent across individual statements, with accurate explanations generally moving respondents toward factual positions and deceptive explanations moving respondents away from them. While the magnitude of effects varies across topics, the overall pattern remains remarkably consistent across both factual and politically salient issues.

Belief ratings before and after exposure to accurate and deceptive AI-generated explanations across individual statements. Accurate explanations generally move respondents toward factual positions, while deceptive explanations produce movement away from factual positions across a range of topics.

Research Contribution

This study contributes to emerging research on AI, misinformation, and human–AI interaction by examining how AI-generated explanations influence belief revision. The findings demonstrate that explanatory AI systems can shape factual accuracy in both beneficial and harmful directions, demonstrating the importance of evaluating not only the information AI systems provide, but also how AI-generated explanations influence information processing, misinformation susceptibility, and trust in automated knowledge systems.

AI in Higher Education: Student Perceptions of Course-Level AI Policies

Thu, 01 Jan 2026 00:00:00 +0000

— project materials, conference paper, analysis code, and figures
— Integrating Artificial Intelligence into the Classroom: Evidence from a Conjoint Experiment, 25th IEEE International Conference on Advanced Learning Technologies (ICALT 2026)
Journal Manuscript in Preparation

Project Overview

This project examines how students evaluate different forms of artificial intelligence (AI) integration in university courses.

Rather than asking whether AI should be used in higher education, the study asks:

How should AI be used in higher education?

Using a conjoint experiment, students evaluated hypothetical course policies that varied in:

AI use in instructional materials
AI use in assessments and grading
AI-powered personalized support

The study measures perceived usefulness, support for adoption, and perceived autonomy.

Research Design

Conjoint Experiment

Participants evaluated randomly generated course policies composed of three dimensions:

Attribute	Levels
Instructional Materials	No AI, AI reviewed by instructor, AI unreviewed
Assessments & Grading	Human grading, AI reviewed by instructor, AI unreviewed
Personalized Support	No chatbot, AI chatbot

Each participant evaluated six randomly assigned profiles.

Sample

Undergraduate students
N = 358 respondents
2,148 profile evaluations
Online Qualtrics survey
Fall 2025

Main Finding

Students are not opposed to AI itself.

Students are opposed to AI operating without human oversight.

AI-assisted systems that remain under instructor review receive evaluations similar to traditional human-only approaches. By contrast, AI systems operating without instructor review produce substantial declines in support, perceived usefulness, and autonomy.

Human oversight is the strongest predictor of student evaluations of AI integration in higher education. Students evaluate AI-assisted systems far more positively when instructors remain involved in reviewing AI-generated content and decisions.

Evidence Across Student Evaluations

Perceived Usefulness

Students strongly penalize unreviewed AI systems, especially when used for grading and instructional materials.

Effects of AI policy design on perceived usefulness. AI systems operating without instructor review substantially reduce students’ evaluations of educational value.

Support for Adoption

Support for AI adoption depends heavily on whether instructors remain involved in reviewing AI outputs.

Effects of AI policy design on support for AI integration. Students generally support AI-assisted systems when instructors retain oversight.

Perceived Autonomy

Students report lower autonomy when AI assumes responsibility for instructional or evaluative decisions without human oversight.

Effects of AI policy design on perceived autonomy. Unreviewed AI systems reduce students’ perceived control over their learning experience.

Additional Results

Consistency Across Outcomes

Coefficient estimates across perceived usefulness, support for adoption, and autonomy reveal a highly consistent pattern.

AI without instructor review produces the largest negative effects.
AI with instructor review produces substantially smaller effects.
AI chatbot support has little effect on student evaluations.

Comparison of estimated effects across perceived usefulness, support for adoption, and perceived autonomy. The pattern is highly consistent across outcomes, demonstrating that student acceptance of AI depends primarily on the presence of human oversight.

Research Contribution

This project contributes to research on artificial intelligence in higher education by providing causal evidence on how students evaluate alternative AI governance designs. The findings demonstrate that student acceptance of AI depends less on AI itself and more on how authority, accountability, and human oversight are structured within educational settings. The results have implications for universities, instructors, and policymakers seeking to integrate AI into teaching and learning while maintaining student trust and legitimacy.

Predicting Peace: Machine Learning and NLP for Peace Agreement Success

Sun, 01 Jun 2025 00:00:00 +0000

Note: This is a very early prototype — work in progress. All local work will be pushed to GitHub once it is complete.

— data, scripts, trained models, and Streamlit app
— explore predictions and SHAP explanations for any agreement configuration

Project Overview

This project develops a machine learning and natural language processing framework to forecast whether a peace agreement will successfully end active armed conflict. Grounded in computational social science, it integrates structured metadata, contextual conflict characteristics, and the full text of peace agreements to build interpretable predictive models, bridging empirical conflict research with state-of-the-art AI methods.

The project uses the (University of Edinburgh), covering over 2,000 peace agreement texts with extensive metadata on conflict type, negotiation stage, institutional provisions, and implementation outcomes.

Beyond academic contribution, the project is deployed as a live, interactive web application that enables researchers and practitioners to input agreement characteristics and receive real-time success predictions with AI-generated explanations.

Research Questions

Can AI systems reliably forecast whether a peace agreement will end armed conflict — and explain why?

Which features of a peace agreement — structural, institutional, or textual — are most predictive of durability?

Methods

Feature Engineering

Metadata encoded into binary and categorical indicators (conflict type, negotiation stage, security arrangements, human rights provisions, inclusion of social groups)
Peace agreement texts preprocessed via tokenization, lowercasing, stopword removal, and TF-IDF representation
Hybrid feature matrix combining structured metadata with textual information

Classic Machine Learning Models

Trained across metadata and text combinations:

Model	Notes
Logistic Regression	Baseline linear classifier
Support Vector Machines	Margin-based classification
Random Forest	Best stable performance on metadata
Gradient Boosting / XGBoost	Ensemble boosting methods
AdaBoost	Adaptive ensemble

Key finding: text-only classic ML performed poorly; adding metadata and contextual variables substantially improved performance.

Transformer-Based Models

Fine-tuned using Hugging Face:

Model	Accuracy	Precision	Recall	F1
DistilBERT-base-uncased	—	—	—	—
DeBERTa-v3-Large	0.888	0.824	0.893	0.824

DeBERTa-v3-Large substantially outperforms classic ML for text classification, particularly on the class imbalance present in peace agreement outcomes.

Best-performing model performance summary and top SHAP-identified predictive signals across 1,508 PA-X peace agreements.

Explainable AI (SHAP)

The deployed Random Forest integrates SHAP-based interpretability, enabling users to understand why a specific agreement is predicted to succeed or fail.

Most influential features identified by SHAP:

Negotiation stage
Conflict type
International missions and enforcement mechanisms
Human rights provisions
Security guarantees

This explainability layer allows practitioners and researchers to interpret model predictions in a theory-informed way — connecting computational output to established conflict studies literature.

Interactive Application

The is a deployed Streamlit app that enables interactive exploration of model predictions.

Users can:

Input metadata features describing any peace agreement
Receive a predicted outcome (SUCCESS / FAILURE) with probability estimates
Explore SHAP feature contributions explaining the specific prediction

Research Contribution

This project contributes to computational social science and AI-for-policy research by demonstrating that transformer-based NLP, combined with structured conflict metadata, can produce accurate and interpretable forecasts of peace agreement outcomes. Unlike prior work relying on static quantitative models, this framework integrates textual content, provides local explainability, and is deployed as a live tool accessible to non-technical users.

Work in Progress

Survival analysis extending the framework to model peace duration rather than binary success (Cox proportional hazards, Random Survival Forests, DeepHit)
Natural-language explanation layer via OpenAI API to describe not only what is likely to fail, but what provisions could strengthen a treaty before signing
Full interactive web application integrating the OpenAI API for real-time, human-readable prediction explanations accessible to researchers and policymakers
Research manuscript in preparation for journal submission