Namig Abbasov
← Research

Predicting Peace: Machine Learning and NLP for Peace Agreement Success

A computational social science framework combining ML, transformer-based NLP, and SHAP explainability to forecast whether peace agreements will successfully end armed conflict — deployed as an interactive prediction tool.

Machine Learning · NLP · Transformers · Computational Social Science · Conflict Forecasting · Explainable AI · SHAP · Peace Studies
Key figure for Predicting Peace: Machine Learning and NLP for Peace Agreement Success

Note: This is a very early prototype — work in progress. All local work will be pushed to GitHub once it is complete.

Project Overview

This project develops a machine learning and natural language processing framework to forecast whether a peace agreement will successfully end active armed conflict. Grounded in computational social science, it integrates structured metadata, contextual conflict characteristics, and the full text of peace agreements to build interpretable predictive models, bridging empirical conflict research with state-of-the-art AI methods.

The project uses the PA-X Peace Agreements Database (University of Edinburgh), covering over 2,000 peace agreement texts with extensive metadata on conflict type, negotiation stage, institutional provisions, and implementation outcomes.

Beyond academic contribution, the project is deployed as a live, interactive web application that enables researchers and practitioners to input agreement characteristics and receive real-time success predictions with AI-generated explanations.

Research Questions

Can AI systems reliably forecast whether a peace agreement will end armed conflict — and explain why?

Which features of a peace agreement — structural, institutional, or textual — are most predictive of durability?

Methods

Feature Engineering

Classic Machine Learning Models

Trained across metadata and text combinations:

ModelNotes
Logistic RegressionBaseline linear classifier
Support Vector MachinesMargin-based classification
Random ForestBest stable performance on metadata
Gradient Boosting / XGBoostEnsemble boosting methods
AdaBoostAdaptive ensemble

Key finding: text-only classic ML performed poorly; adding metadata and contextual variables substantially improved performance.

Transformer-Based Models

Fine-tuned using Hugging Face:

ModelAccuracyPrecisionRecallF1
DistilBERT-base-uncased————
DeBERTa-v3-Large0.8880.8240.8930.824

DeBERTa-v3-Large substantially outperforms classic ML for text classification, particularly on the class imbalance present in peace agreement outcomes.

DeBERTa-v3-Large achieves 0.888 accuracy on 1,508 PA-X agreements. Most predictive signals: security guarantees, human rights provisions, negotiation stage, conflict type, and international missions.
Best-performing model performance summary and top SHAP-identified predictive signals across 1,508 PA-X peace agreements.

Explainable AI (SHAP)

The deployed Random Forest integrates SHAP-based interpretability, enabling users to understand why a specific agreement is predicted to succeed or fail.

Most influential features identified by SHAP:

This explainability layer allows practitioners and researchers to interpret model predictions in a theory-informed way — connecting computational output to established conflict studies literature.

Interactive Application

The Peace Agreement Success Predictor is a deployed Streamlit app that enables interactive exploration of model predictions.

Users can:

Research Contribution

This project contributes to computational social science and AI-for-policy research by demonstrating that transformer-based NLP, combined with structured conflict metadata, can produce accurate and interpretable forecasts of peace agreement outcomes. Unlike prior work relying on static quantitative models, this framework integrates textual content, provides local explainability, and is deployed as a live tool accessible to non-technical users.

Work in Progress