Time-Series Modelling

I’ve recently gotten quite into the interpretability problem of Time-Series Modelling, i.e. how might I develop a system to interpret the output for a blackbox model $\mathcal{M}:x\to y$ that is able to classify an input time series $x$?

I recently found LASTS, a similar endeavour focused on looking into single-sample based counterfactual analysis. In essence, LASTS provides 3 values:

1. The saliency map, $\Phi =\left|\hat{x}-{\hat{x}}_{\ne }\right|$
2. The exemplar values ${\hat{X}}_{=}$ and the counterexamplar values ${\hat{X}}_{\ne }$
3. The Shapelet Tree Classifier