Solar flare prediction is an increasingly important concern in spaceweather prediction. Major solar flares have potentially catastrophic consequences for human life and infrastructure, both in space and on earth. The current lack of highly predictive models for these events saw the heliophysics community turn to data driven approaches. In this paper, we describe a novel two-step regularised gradient boosted classification tree model approach to the analysis of large multivariate time series. Applied to the prediction of major flaring events, we demonstrate that along with high performance, the critical feature selection steps increase interpretability of otherwise complex models to offer insights that could help identify physical mechanisms giving rise to solar flares. This method was developed for the IEEE BigData Cup 2019 “Solar Flare Prediction from Time Series of Solar Magnetic Field Parameters”