Outliers detection in X13

Detection of a single outlier

Suppose that the current ARIMA model already contains some regression variables (calendar effects, other outliers…)

The model is estimated by applying the filter generated by the ARIMA model on the differenced endogeneous variable and on the differenced regression variables and by solving the corresponding ols problem.

We write $y_l$ and $X_l$ the variables after transformation.

The integration of a new outlier $o$ is handled in the same way. Suppose that $o_l$ is the differenced and filtered outlier. The coefficients of the augmented ols problem are given by:

\[\begin{pmatrix} \beta_X \\ \beta_o \end{pmatrix} = \begin{pmatrix} X_l' X_l && X_l' o_l\\ o_l' X_l && o_l' o_l \end{pmatrix}^{-1}\begin{pmatrix} X_l' y_l \\ o_l' y_l \end{pmatrix} = \Omega z\] \[T_o = \beta_o / \left(\sigma_{r} \sqrt{\Omega_{o,o}}\right)\]

Suppose that $U$ is the upper triangular Cholesky factor of $X’_l X_l$. The previous equation becomes

\[\begin{pmatrix} X_l' X_l && X_l' o_l\\ o_l' X_l && o_l' o_l \end{pmatrix} = \begin{pmatrix} X_l' X_l && w\\ w' && o_l' o_l \end{pmatrix} = \begin{pmatrix} U'U && U'a\\ a'U && b^2 \end{pmatrix}\]

where $w=X_l’o_l$ and $a’=w’U^{-1}$

Using inversion formulae of partitioned symmetric matrices, we get:

\[\Omega_z = \begin{pmatrix} ... & ...\\- (U^{-1}a)'\left(b^2 -a'a\right)^{-1} & \left(b^2 -a'a\right)^{-1} \end{pmatrix}\]

The variance $var_o$ and the coefficient $\beta_o$ corresponding to the new variable are given by:

\[var_o= \frac{mad^2}{b^2-a'a}\] \[\beta_o = \frac{o_l' y_l - y_l'X_l U^{-1}a}{b^2-a'a}\]

so that

\[T_o = \frac{o_l' y_l - y_l'X_l U^{-1}a}{mad \sqrt{b^2-a'a}}\]

It should be noted that the expression $y_l’X_lU^{-1}$ doesn’t depend on the new regression variable and must be computed only once.

Implementation in JD+

For computing the T-stats of all possible outliers for a given model, we proceed in JD+ as follows:

  • Compute once
    • $y_l, X_l$, by means of the Kalman filter
    • $mad(y_l)$
    • $U$, the upper Cholesky factor of $X_l’X_l$
    • $z=y’_lX_lU^{-1}$
  • For each possible outlier, compute
    • $o_l$
    • $q=y_l’ o_l$
    • $w=X’_lo_l$
    • $a’=w’U^{-1}$
    • $T_o$ and $\beta_o$ as defined above

The selected outlier is - as usual - the outlier with the highest T-stat.

The computation of $o_l$, which is the most expensive part of the algorithm, can be done in a more efficient way by means of the algorithm developed by Ansley (Cholesky decomposition of a banded matrix). See the corresponding paper for detailed explanations. That is the solution used in JD+.