EXPLAINABLE MACHINE LEARNING FOR RELIABLE PHISHING WEBSITE DETECTION: A TWO-DATASET EXPERIMENTAL STUDY

Abstract

Phishing website detection is frequently presented as a classification task in which high predictive performance is treated as sufficient evidence of operational usefulness. This approach is incomplete in law-enforcement and cybersecurity settings, where a model output must be interpretable, independently verifiable and separated from conclusions about authorship, intent or legal qualification. The study develops and evaluates a human-centred explainable artificial intelligence framework combining supervised classification with structured analyst-oriented explanations. Five classifiers were assessed on the UCI Phishing Websites benchmark, and a second experiment used the larger PhiUSIIL dataset. Phishing was consistently designated as the positive class. Extra Trees achieved test accuracy of 0.9774, precision of 0.9784, recall of 0.9704, F1-score of 0.9744 and ROC-AUC of 0.9948. Stratified bootstrap intervals quantified test uncertainty: the 95% interval was 0.9670–0.9812 for F1 and 0.9915–0.9976 for ROC-AUC. The Brier score was 0.0187 and ten-bin expected calibration error was 0.0089. TreeSHAP analysis of 400 hold-out observations identified certificate state and anchor behaviour as the dominant global contributors. Across five fitted folds, the mean pairwise Jaccard overlap of the ten leading features was 0.891 (SD 0.094). In a mode-replacement fidelity test, replacing the three leading SHAP features by their training modes reduced F1 from 0.9744 to 0.3543, whereas random three-feature replacement yielded mean F1 of 0.9478. Near-perfect PhiUSIIL performance was not interpreted as deployment readiness: grouped validation, an explicitly defined 18-feature lexical ablation and duplicate analysis exposed collection-specific separability; the lexical-only grouped Extra Trees model retained mean F1 of 0.9970. The proposed Phishing Threat Explanation Card converts a prediction into an auditable record of the score, threshold, leading indicators, verification actions, uncertainty and analyst disposition. The contribution is empirical and procedural: it reports a documented and auditable comparison, quantifies predictive uncertainty, calibration and explanation fidelity, diagnoses inflated benchmark performance and defines safeguards for human oversight.