Predicting student dropout, and showing when the prediction is useful
25,059 international students, with data arriving in three stages: applicant details, then attendance, then module results. I compared XGBoost with a neural network at each stage, using a split grouped by student and a held-out test set of 5,013.
The late-stage score of 0.993 was too good: imputed academic records were giving the answer away. Re-evaluated on the 4,339 students still on course, the model finds 105 of 136 future dropouts. A temporal split showed the decision threshold would need recalibrating for each new intake.
Python · XGBoost · Keras · scikit-learn · pandas