Time Series Features

Features:

 Time domain：
均值，方差，標準差，最大值，最小值，過零點個數，最大值與最小值之差，眾數
frequency domain：
直流分量，圖形的均值、方差、標準差、斜度、峭度，幅度的均值、方差、標準差、斜度、峭度

 R

 1. Spectral entropy of a time series
Computes feature of a time series based on tiled (non-overlapping) windows.
2. lumpiness is the variance of the variances
3. stability is the variance of the means
Computes feature of a time series based on sliding (overlapping) windows
4. max_level_shift finds the largest mean shift between two consecutive windows.
5. max_var_shift finds the largest var shift between two consecutive windows
6. max_kl_shift finds the largest shift in Kulback-Leibler divergence between two consecutive windows
7. Number of crossing points: the number of times a time series crosses the median
8. Number of flat spots: Number of flat spots in a time series(rel)
9. Hurst coefficent: Computes the Hurst coefficient indicating the level of fractional differencing of a time series
10. Autocorrelation-based features: Computes various measures based on autocorrelation coefficients of the original series, first-differenced series and second-differenced series
x_acf1
x_acf10
diff1_acf1
diff1_acf10
diff2_acf1
diff2_acf10
11. Partial autocorrelation-based features: Computes various measures based on partial autocorrelation coefficients of the original series, first-differenced series and second-differenced series
x_pacf5
diff1_pacf5
diff2_pacf5

12. Parameter estimates of Holt's linear trend method: Estimate the smoothing parameter for the level-alpha and the smoothing parameter for the trend-beta
13. Autocorrelation coefficient at lag 1 of the residual: Computes the first order autocorrelation of the residual series of the deterministic trend model

**. Strength of trend and seasonality of a time series:
Computes various measures of trend and seasonality of a time series based on an STL decomposition
- Summary Stat:
14. the length of a time series
15. the variance of a time series
16. the variance of the residules
17. the variance of the detrend series
18. the variance of the deseasonal series
19. the number of the seasonal
20. Measure of trend strength
21. Measure of seasonal strength
22. Find time of peak and trough for each component
23. Compute measure of spikiness
24. Compute measures of linearity and curvature
25. ACF of remainder
**. Heterogeneity coefficients
Computes various measures of heterogeneity of a time series. First the series is pre-whitened using an AR model to give a new series y.
We fit a GARCH(1,1) model to y and obtain the residuals, e. Then the four measures of heterogeneity are:
26. the sum of squares of the first 12 autocorrelations of y^2 27. the sum of squares of the first 12 autocorrelations of e^2 28. the R^2 value of an AR model applied to y^2 29. the R^2} value of an AR model applied to e^2 The statistics obtained from y^2 are the ARCH effects, while those from e^2 are the GARCH effects.

python
1. the absolute energy of the time series which is the sum over the squared values
E = \sum_{i=1,\ldots, n} x_i^2
2. the sum over the absolute value of consecutive changes in the series x
\sum_{i=1, \ldots, n-1} \mid x_{i+1}- x_i \mid
3. Calculates the value of an aggregation function (e.g. var or mean) of the autocorrelation taken over different all possible lags (1 to length of x)
\frac{1}{n-1} \sum_{l=1,\ldots, n} \frac{1}{(n-l)\sigma^{2}} \sum_{t=1}^{n-l}(X_{t}-\mu )(X_{t+l}-\mu)
where n is the length of the time series X_i, \sigma^2 its variance and \mu its mean.
4. Calculates a linear least-squares regression for values of the time series that were aggregated over chunks versus the sequence from 0 up to the number of chunks minus one. The chunk specifies how many time series values are in each chunk. Further, the aggregation function could be “max”, “min” or , “mean”, “median”
5. Implements a vectorized Approximate entropy algorithm.
For short time-series this method is highly dependent on the parameters, but should be stable for N > 2000, see:
Yentes et al. (2012) - The Appropriate Use of Approximate Entropy and Sample Entropy with Short Data Sets
Other shortcomings and alternatives discussed in:
Richman & Moorman (2000) - Physiological time-series analysis using approximate entropy and sample entropy
6. This feature fits the unconditional maximum likelihood of an autoregressive AR(k) process. The k parameter is the maximum lag of the process
X_{t}=\varphi_0 +\sum _{{i=1}}^{k}\varphi_{i}X_{{t-i}}+\varepsilon_{t}
For the configurations from param which should contain the maxlag “k” and such an AR process is calculated. Then the coefficients \varphi_{i} whose index i contained from “coeff” are returned.
7. The Augmented Dickey-Fuller test is a hypothesis test which checks whether a unit root is present in a time series sample. 
8.  Calculates the autocorrelation of the specified lag, according to the formula [1]
\frac{1}{(n-l)\sigma^{2}} \sum_{t=1}^{n-l}(X_{t}-\mu )(X_{t+l}-\mu)
where n is the length of the time series X_i, \sigma^2 its variance and \mu its mean. l denotes the lag.
References
compare with 3.

9. First bins the values of x into max_bins equidistant bins. Then calculates the value of
- \sum_{k=0}^{min(max\_bins, len(x))} p_k log(p_k) \cdot \mathbf{1}_{(p_k > 0)}
where p_k is the percentage of samples in bin k.

10. This function calculates the value of
\frac{1}{n-2lag} \sum_{i=0}^{n-2lag} x_{i + 2 \cdot lag}^2 \cdot x_{i + lag} \cdot x_{i}
which is
\mathbb{E}[L^2(X)^2 \cdot L(X) \cdot X]
where \mathbb{E} is the mean and L is the lag operator. It was proposed in [1] as a measure of non linearity in the time series.
References
[1] Schreiber, T. and Schmitz, A. (1997).
Discrimination power of measures for nonlinearity in a time series
PHYSICAL REVIEW E, VOLUME 55, NUMBER 5
11. First fixes a corridor given by the quantiles ql and qh of the distribution of x. Then calculates the average, absolute value of consecutive changes of the series x inside this corridor.
Think about selecting a corridor on the y-Axis and only calculating the mean of the absolute change of the time series inside this corridor.
12.This is an estimate for a time series complexity [1] (A more complex time series has more peaks, valleys etc.). It calculates the value of
\sqrt{ \sum_{i=0}^{n-2lag} ( x_{i} - x_{i+1})^2 }
References
[1] Batista, Gustavo EAPA, et al (2014).
CID: an efficient complexity-invariant distance for time series.
Data Mining and Knowledge Difscovery 28.3 (2014): 634-669.
13. the number of values in x that are higher than the mean of x

14. the number of values in x that are lower than the mean of x

15. Calculates a Continuous wavelet transform for the Ricker wavelet, also known as the “Mexican hat wavelet” which is defined by
\frac{2}{\sqrt{3a} \pi^{\frac{1}{4}}} (1 - \frac{x^2}{a^2}) exp(-\frac{x^2}{2a^2})
where a is the width parameter of the wavelet function.
This feature calculator takes three different parameter: widths, coeff and w. The feature calculater takes all the different widths arrays and then calculates the cwt one time for each different width array. Then the values for the different coefficient for coeff and width w are returned. (For each dic in param one feature is returned)
16. Calculates the sum of squares of chunk i out of N chunks expressed as a ratio with the sum of squares over the whole series

17. the spectral centroid (mean), variance, skew, and kurtosis of the absolute fourier transform spectrum.

18. Calculates the fourier coefficients of the one-dimensional discrete Fourier Transform for real input by fast fourier transformation algorithm
A_k = \sum_{m=0}^{n-1} a_m \exp \left \{ -2 \pi i \frac{m k}{n} \right \}, \qquad k = 0, \ldots , n-1.
The resulting coefficients will be complex, this feature calculator can return the real part (attr==”real”), the imaginary part (attr==”imag), the absolute value (attr=”“abs) and the angle in degrees (attr==”angle).

19. Returns the first location of the maximum value of x. The position is calculated relatively to the length of x.

20. the first location of the minimal value of x. The position is calculated relatively to the length of x.

21.Coefficients of polynomial h(x), which has been fitted to the deterministic dynamics of Langevin model
\dot{x}(t) = h(x(t)) + \mathcal{N}(0,R)
as described by [1].
For short time-series this method is highly dependent on the parameters.
References
[1] Friedrich et al. (2000): Physics Letters A 271, p. 217-222
Extracting model equations from experimental data
22. Checks if any value in x occurs more than once

23. Checks if the maximum value of x is observed more than once

24.Checks if the minimal value of x is observed more than once
25.Those apply features calculate the relative index i where q% of the mass of the time series x lie left of i. For example for q = 50% this feature calculator will return the mass center of the time series

26.  the kurtosis of x (calculated with the adjusted Fisher-Pearson standardized moment coefficient G2).

27.Boolean variable denoting if the standard dev of x is higher than ‘r’ times the range = difference between max and min of x. Hence it checks if
std(x) > r * (max(X)-min(X))
According to a rule of the thumb, the standard deviation should be a forth of the range of the values.

28. the relative last location of the maximum value of x. The position is calculated relatively to the length of x.

29. the last location of the minimal value of x. The position is calculated relatively to the length of x.
30. the length of x
31.Calculate a linear least-squares regression for the values of the time series versus the sequence from 0 to length of the time series minus one. This feature assumes the signal to be uniformly sampled. It will not use the time stamps to fit the model. The parameters control which of the characteristics are returned.
Possible extracted attributes are “pvalue”, “rvalue”, “intercept”, “slope”, “stderr”, see the documentation of linregress for more information.
32. the length of the longest consecutive subsequence in x that is bigger than the mean of x

33. the length of the longest consecutive subsequence in x that is smaller than the mean of x

34. Largest fixed point of dynamics :math:argmax_x {h(x)=0}` estimated from polynomial h(x), which has been fitted to the deterministic dynamics of Langevin model
\dot(x)(t) = h(x(t)) + R \mathcal(N)(0,1)
as described by
Friedrich et al. (2000): Physics Letters A 271, p. 217-222 Extracting model equations from experimental data
For short time-series this method is highly dependent on the parameters.

35. Calculates the highest value of the time series x.
36. the mean of x
37. the mean over the absolute differences between subsequent time series values which is
\frac{1}{n} \sum_{i=1,\ldots, n-1} | x_{i+1} - x_{i}|
38.  the mean over the absolute differences between subsequent time series values which is
\frac{1}{n} \sum_{i=1,\ldots, n-1} x_{i+1} - x_{i}

39.  the mean value of a central approximation of the second derivative
\frac{1}{n} \sum_{i=1,\ldots, n-1} \frac{1}{2} (x_{i+2} - 2 \cdot x_{i+1} + x_i)
40. the median of x

41. Calculates the lowest value of the time series x.
42. Calculates the number of crossings of x on m. A crossing is defined as two sequential values where the first value is lower than m and the next is greater, or vice-versa. If you set m to zero, you will get the number of zero crossings.
43. This feature calculator searches for different peaks in x. To do so, x is smoothed by a ricker wavelet and for widths ranging from 1 to n. This feature calculator returns the number of peaks that occur at enough width scales and with sufficiently high Signal-to-Noise-Ratio (SNR)
44. Calculates the number of peaks of at least support n in the time series x. A peak of support n is defined as a subsequence of x where a value occurs, which is bigger than its n neighbours to the left and to the right.
Hence in the sequence
>>> x = [3, 0, 0, 4, 0, 0, 13]
4 is a peak of support 1 and 2 because in the subsequences
>>> [0, 4, 0]
>>> [0, 0, 4, 0, 0]
4 is still the highest value. Here, 4 is not a peak of support 3 because 13 is the 3th neighbour to the right of 4 and its bigger than 4.
45. Calculates the value of the partial autocorrelation function at the given lag. The lag k partial autocorrelation of a time series \lbrace x_t, t = 1 \ldots T \rbrace equals the partial correlation of x_t and x_{t-k}, adjusted for the intermediate variables \lbrace x_{t-1}, \ldots, x_{t-k+1} \rbrace ([1]). Following [2], it can be defined as
\alpha_k = \frac{ Cov(x_t, x_{t-k} | x_{t-1}, \ldots, x_{t-k+1})} {\sqrt{ Var(x_t | x_{t-1}, \ldots, x_{t-k+1}) Var(x_{t-k} | x_{t-1}, \ldots, x_{t-k+1} )}}
with (a) x_t = f(x_{t-1}, \ldots, x_{t-k+1}) and (b) x_{t-k} = f(x_{t-1}, \ldots, x_{t-k+1}) being AR(k-1) models that can be fitted by OLS. Be aware that in (a), the regression is done on past values to predict x_twhereas in (b), future values are used to calculate the past value x_{t-k}. It is said in [1] that “for an AR(p), the partial autocorrelations [ \alpha_k ] will be nonzero for k<=p and zero for k>p.” With this property, it is used to determine the lag of an AR-Process.
References
[1] Box, G. E., Jenkins, G. M., Reinsel, G. C., & Ljung, G. M. (2015).
Time series analysis: forecasting and control. John Wiley & Sons.
[2] https://onlinecourses.science.psu.edu/stat510/node/62
 
46. the percentage of unique values, that are present in the time series more than once.
len(different values occurring more than once) / len(different values)
This means the percentage is normalized to the number of unique values, in contrast to the percentage_of_reoccurring_values_to_all_values.

47. the ratio of unique values, that are present in the time series more than once.
# of data points occurring more than once / # of all data points
This means the ratio is normalized to the number of data points in the time series, in contrast to the percentage_of_reoccurring_datapoints_to_all_datapoints.
48. Calculates the q quantile of x. This is the value of x greater than q% of the ordered values from x.

49. Count observed values within the interval [min, max)
50. Ratio of values that are more than r*std(x) (so r sigma) away from the mean of x. 
51. Returns a factor which is 1 if all values in the time series occur only once, and below one if this is not the case. In principle, it just returns
# unique values / # values
52.Calculate and return sample entropy of x.
References
[1] http://en.wikipedia.org/wiki/Sample_Entropy
[2] https://www.ncbi.nlm.nih.gov/pubmed/10843903?dopt=Abstract
53. the sample skewness of x (calculated with the adjusted Fisher-Pearson standardized moment coefficient G1).
54. This feature calculator estimates the cross power spectral density of the time series x at different frequencies. To do so, the time series is first shifted from the time domain to the frequency domain.
The feature calculators returns the power spectrum of the different frequencies.
55. the standard deviation of x
56. the sum of all data points, that are present in the time series more than once.
57. Returns the sum of all values, that are present in the time series more than once.
58. Calculates the sum over the time series values
59. Boolean variable denoting if the distribution of x looks symmetric. This is the case if
| mean(X)-median(X)| < r * (max(X)-min(X))

60. This function calculates the value of
\frac{1}{n-2lag} \sum_{i=0}^{n-2lag} x_{i + 2 \cdot lag}^2 \cdot x_{i + lag} - x_{i + lag} \cdot x_{i}^2
which is
\mathbb{E}[L^2(X)^2 \cdot L(X) - L(X) \cdot X^2]
where \mathbb{E} is the mean and L is the lag operator. It was proposed in [1] as a promising feature to extract from time series.
References
[1] Fulcher, B.D., Jones, N.S. (2014).
Highly comparative feature-based time-series classification.
Knowledge and Data Engineering, IEEE Transactions on 26, 3026–3037.

61.Count occurrences of value in time series x.
62. the variance of x 
63. Boolean variable denoting if the variance of x is greater than its standard deviation. Is equal to variance of x being larger than 1

留言

張貼留言

這個網誌中的熱門文章

標準差與 Wald 統計量

圖片
可能性或信賴區間對 MLE 是有用的添補，認可了在一個參數 theta 的不確定性; 它比 likelihood function 更簡單。前面我們也提過 observed Fisher Information : I(theta) 也是 MLE 的添補。那麼它與基於可能性的區間的關係為何呢? 在 regular 的情形下，log-likelihood 的二次趨近運作良好， I(theta) 是有其意義的。這時我們有: Log(L(theta)/L(theta-hat)) ~ -1/2*I(theta-hat)(theta-theta-hat)^2 如此，可能性區間 { theta | L(theta)/L(theta-hat) > c } 就可趨近於: theta-hat  ±  - (-2*log(c))^(1/2)(I(theta-hat))^(-1/2)  在 normal mean 模型，就有一個確切的有信心水準的信賴區間 Prob(chi-squared < -2log(c)) 例如:  theta-hat ± - 1.96 * I(theta-hat)^(1/2) 是一個確切的有 95% 信心水準的信賴區間。  在非 normal 的情況下，如上篇提到的有一 95%信心水準的信賴區間。注意: 設置此區間涉及兩層的趨近: log-likelihood 的二次趨近(I(theta-hat)與信心水準(Willk's) 的趨近。 也可與 normal mean 模型類比，一般 (I(theta-hat))^(-1/2) 提供了 theta-hat 的標準差訊息。常例以 'MLE{標準差}'一對的形式來報告。它主要用途是用 Wald 統計量來檢定 H0: theta = theta0。 z = (theta-hat - theta0)/se(theta-hat) 或去計算 Wald 信賴區間。例如 Wald 對 theta  95% 信心水準的信賴區間就是:  theta-hat ± 1.96*se(theta-hat) 在 normal mean 模型，在 H0 條件下，這個 Wald z 統計量...
閱讀完整內容

可能性比檢定(Likelihood ratio test)

直接使用 likelihood  來做假設檢定，例如去檢定一個 Null hypothesis: H0: theta = theta0，我們可以以報告 H0 的可能性 likelihood 作為 theta0 標準化過的可能性 likelihood : L(theta0)/L(theta-hat) 我們能夠去 reject H0，宣告數據不支持它，假如它的可能性是太小，意味著有其它被數據有多很多支持的假設。多小才算太小就留給主觀判斷，它可依應用情況或其它可能包含先驗知識的來考量。 在法庭的許多情況，對 "被告是無辜的" 的假設，在我們可以否決它之前，可能設定它在低可能性的水準，與此同時 "經基因工程的食物沒有副作用" 的假設可以更輕而易舉的被否決。 無論如何，可能性測定的議題在這裡也是有關係的，如先說一個 5% 的可能性，這可能沒有一個固定的意義。 在正常的一個參數的問題中，可以拿到一個藉著 Wilk's statistic 的分布，基於機率測定所產生的 P-值。由之前的例子，察看到 L(theta0)/L(theta-hat) = c，P值的趨近值是 P( chi-squared > -2 log(c)  這展示出在可能性比(likelihood ratio) 與 P-值間通常有些關係，如此 P-值用到的時機就是用它來做為支持度的量測。無論如何，因為 P-值會依賴實驗的樣本空間，故這關係會視實驗而定。雖然 P-值一直以來受到一些統計學家的批評，但它在實務上還是受到廣泛的使用，它似乎不太可能消失。 在 non-regular 的問題中，我們不知道如何去測定可能性，我們可能必須使用純可能性比(likelihood ratio)作為支持度的量測。當沒有基於機率推論的正當理由時，這個與純可能性區間(likelihood intervals)的使用一致。例如說，去檢定 theta = 0.5， 而如何去定義此檢定的一 P-值不是很清楚時。另外的例子是一般在檢定統計量的分布為非對稱的情況下: 確切兩邊的 P-值是模擬兩可，此時就可能有好幾個角逐的定義。
閱讀完整內容

Wold Decomposition Theorem

圖片
在一些介紹 ARIMA 的文章或書中，你偶爾會讀到 Wold Decomposition Theorem。去查文獻你可能會看到一堆公式，在不知所云之下就放棄了。 事實上與 Wold Decomposition Theorem 最有關係的是 MA process。 Wold Decomposition Theorem 以簡化的說，這是說任何 stationary process 可以表示成兩類型的 processes 的和: 一個是 non-deterministic 另一個是 deterministic。一個 process 稱為 deterministic (或 singular) 是當此 process 可以被一個線性回歸依據過去的值(甚至無法取得最近的值)所正確的預測。一個 process 稱為 non-deterministic( regular 或 stochastic) 是當一個線性回歸依據過去的值對預測是無用的。 這理論與 MA processes 連結如下: 這 Wold decomposition 的 non-deterministic 部份可以表示成一個 MA process 有可能有無限項並需要連續 Z-t 序列的是 uncorrelated 而不是 independent (如一些作者定義 MA processes 需要)。 任何 stationary non-deterministic 時間序列正式地可以表示這種形式， 上式 psi0 = 1 同時 psi1*psi1 + psi2*psi2+ ... <  Inf  {Z-t}  表示一個純的 random process( 或 uncorrelated white noise) 有 mean 0 與常數 variance (它與 process 的 deterministic 部分無關連)。  所謂 {Zt} 是用最佳的線性預測子來做預測的時候所產生的 one-step-ahead 預測誤差， {Zt} 有時稱作 innovations。上式是一個 MA(∞) process，它通常稱作 process 的 Wold representation。它有時也叫做一個 linear process 或 general linea...
閱讀完整內容