Predicting Peace: Machine Learning and NLP for Peace Agreement Success
Note: This is a very early prototype — work in progress. All local work will be pushed to GitHub once it is complete.
- GitHub Repository — data, scripts, trained models, and Streamlit app
- Interactive Prediction App — explore predictions and SHAP explanations for any agreement configuration
Project Overview
This project develops a machine learning and natural language processing framework to forecast whether a peace agreement will successfully end active armed conflict. Grounded in computational social science, it integrates structured metadata, contextual conflict characteristics, and the full text of peace agreements to build interpretable predictive models, bridging empirical conflict research with state-of-the-art AI methods.
The project uses the PA-X Peace Agreements Database (University of Edinburgh), covering over 2,000 peace agreement texts with extensive metadata on conflict type, negotiation stage, institutional provisions, and implementation outcomes.
Beyond academic contribution, the project is deployed as a live, interactive web application that enables researchers and practitioners to input agreement characteristics and receive real-time success predictions with AI-generated explanations.
Research Questions
Can AI systems reliably forecast whether a peace agreement will end armed conflict — and explain why?
Which features of a peace agreement — structural, institutional, or textual — are most predictive of durability?
Methods
Feature Engineering
- Metadata encoded into binary and categorical indicators (conflict type, negotiation stage, security arrangements, human rights provisions, inclusion of social groups)
- Peace agreement texts preprocessed via tokenization, lowercasing, stopword removal, and TF-IDF representation
- Hybrid feature matrix combining structured metadata with textual information
Classic Machine Learning Models
Trained across metadata and text combinations:
| Model | Notes |
|---|---|
| Logistic Regression | Baseline linear classifier |
| Support Vector Machines | Margin-based classification |
| Random Forest | Best stable performance on metadata |
| Gradient Boosting / XGBoost | Ensemble boosting methods |
| AdaBoost | Adaptive ensemble |
Key finding: text-only classic ML performed poorly; adding metadata and contextual variables substantially improved performance.
Transformer-Based Models
Fine-tuned using Hugging Face:
| Model | Accuracy | Precision | Recall | F1 |
|---|---|---|---|---|
| DistilBERT-base-uncased | — | — | — | — |
| DeBERTa-v3-Large | 0.888 | 0.824 | 0.893 | 0.824 |
DeBERTa-v3-Large substantially outperforms classic ML for text classification, particularly on the class imbalance present in peace agreement outcomes.
Explainable AI (SHAP)
The deployed Random Forest integrates SHAP-based interpretability, enabling users to understand why a specific agreement is predicted to succeed or fail.
Most influential features identified by SHAP:
- Negotiation stage
- Conflict type
- International missions and enforcement mechanisms
- Human rights provisions
- Security guarantees
This explainability layer allows practitioners and researchers to interpret model predictions in a theory-informed way — connecting computational output to established conflict studies literature.
Interactive Application
The Peace Agreement Success Predictor is a deployed Streamlit app that enables interactive exploration of model predictions.
Users can:
- Input metadata features describing any peace agreement
- Receive a predicted outcome (SUCCESS / FAILURE) with probability estimates
- Explore SHAP feature contributions explaining the specific prediction
Research Contribution
This project contributes to computational social science and AI-for-policy research by demonstrating that transformer-based NLP, combined with structured conflict metadata, can produce accurate and interpretable forecasts of peace agreement outcomes. Unlike prior work relying on static quantitative models, this framework integrates textual content, provides local explainability, and is deployed as a live tool accessible to non-technical users.
Work in Progress
- Survival analysis extending the framework to model peace duration rather than binary success (Cox proportional hazards, Random Survival Forests, DeepHit)
- Natural-language explanation layer via OpenAI API to describe not only what is likely to fail, but what provisions could strengthen a treaty before signing
- Full interactive web application integrating the OpenAI API for real-time, human-readable prediction explanations accessible to researchers and policymakers
- Research manuscript in preparation for journal submission
