Time Series Analysis
This is unfinised time series analysis note, which is wished to be done around 2023 March
ARMA
Definition[White Noise] We call a sequence of r.v \(\{e_t\}\) as white noise, if \(\mathrm{E}[e_t]=0, \mathrm{E}[e_t^2] = \sigma^2, \mathrm{E}[e_te_s] = 0, \forall t \neq s\)
Definition[strictly stationary] A process \(\{y_t\}\) is called strictly stationary if for each \(t,k,n\) \(y_t, \cdots, y_{t+k}\}\) has the same distribution as \(\{y_{t+n}, \cdots, y_{t+n+k}\}\)
Definition[weakly stationary] A process \(\{y_t\}\) is called weakly stationary if for each \(k,t\) \(\mathrm{E}[y_t], \mathrm{E}[y_t^2], \mathrm{E}[y_ty_{t+k}]\) is independent of \(t\)
Definition[Lag Operator] Lag operator is defined as \(Ly_t = y_{t-1}\)
Corollary - Lag polynomial defined as \(a(L;p):=\sum_{i=0}^{p}a_i L^i\) has commuted multiplication, i.e. \(a(L; p)b(L; q) = b(L; q)a(L; p)\) - If the roots of \(a(L)\) are out of unit circle, then it is invertible.
Here, we demonstrate a simple example of the second point. For a polynomial $a(L; 1) = 1 - L $, whose root is out of unit circle when \(|\lambda| < 1\), its inverse can be represented as, \[ a^{-1}(L,1) = \sum_{i=0}^{+\infty}\lambda^i L^i \]
Simple Processes
Definition[AR] Auto regressive process with order \(p\) is defined as \[ a(L;p)y_t = e_t \]
Definition[MA] Moving Averaging Process with order \(q\) is defined as \[ y_t = b(L;q)e_t \]
Definition[ARMA] The \((p,q)\) order of ARMA is, \[ a(L;q)y_t = b(L;q)e_t \]
AR is regression, since we can expand the definition as \(a_0 y_t = \sum_{i=1}^p(-a_i)y_{t-i} + e_t\), which is just we do regression of \(y_t\) in terms of \(y_{t-1}, \cdots, y_{t-i}\)
The intution of MA is more direct, that is, \[ y_t = \sum_{i=0}^q b_i e_{t-i} \] we use weights \(\{b_i\}\) to average another stochastic process.
Hence, ARMA just tries to find a linear combination of signal sequences \(y_t\) to fit the linear of combination of noise sequence \(e_t\).
Corollary Combing with the above invertible conditions, we claim that there exists a bridge between AR and MA.
For example, AR\((p)\) has an equivalent representation of MA\((\infty)\) if \(a(L;p)\) is invertible. The reversed claim for MA\((q)\) and AR\((\infty)\) is also correct.
Covariance
Definition[auto-covariance] \(\gamma_k := \mathrm{Cov}(y_t, y_{t+k})\) for weakly stationary process
Definition[auto-correlation] \(\rho_k : = \frac{\gamma_k}{\gamma_0}\) for weakly stationary process
Definition[covariance function] \(\tilde{\gamma}(\xi) := \sum_{i=-\infty}^{+\infty}\gamma_i \xi^i\)
Corollary[ARMA covariance function] The covariance function of ARMA is \[ \tilde{\gamma}(\xi) = \frac{b(\xi)b(\xi^{-1})}{a(\xi)a(\xi^{-1})} \sigma^2 \]