2.1 Moving Average Models (MA models) Time series models known as ARIMA models may include autoregressive terms andor moving average terms. In Week 1, we learned an autoregressive term in a time series model for the variable x t is a lagged value of x t . For instance, a lag 1 autoregressive term is x t-1 (multiplied by a coefficient). This lesson defines moving average terms. A moving average term in a time series model is a past error (multiplied by a coefficient). Let (wt overset N(0, sigma2w)), meaning that the w t are identically, independently distributed, each with a normal distribution having mean 0 and the same variance. The 1 st order moving average model, denoted by MA(1) is (xt mu wt theta1w ) The 2 nd order moving average model, denoted by MA(2) is (xt mu wt theta1w theta2w ) The q th order moving average model, denoted by MA(q) is (xt mu wt theta1w theta2w dots thetaqw ) Note . Many textbooks and software programs define the model with negative signs before the terms. This doesnt change the general theoretical properties of the model, although it does flip the algebraic signs of estimated coefficient values and (unsquared) terms in formulas for ACFs and variances. You need to check your software to verify whether negative or positive signs have been used in order to correctly write the estimated model. R uses positive signs in its underlying model, as we do here. Theoretical Properties of a Time Series with an MA(1) Model Note that the only nonzero value in the theoretical ACF is for lag 1 . All other autocorrelations are 0. Thus a sample ACF with a significant autocorrelation only at lag 1 is an indicator of a possible MA(1) model. For interested students, proofs of these properties are an appendix to this handout. Example 1 Suppose that an MA(1) model is x t 10 w t .7 w t-1 . where (wt overset N(0,1)). Thus the coefficient 1 0.7. The theoretical ACF is given by A plot of this ACF follows. The plot just shown is the theoretical ACF for an MA(1) with 1 0.7. In practice, a sample wont usually provide such a clear pattern. Using R, we simulated n 100 sample values using the model x t 10 w t .7 w t-1 where w t iid N(0,1). For this simulation, a time series plot of the sample data follows. We cant tell much from this plot. The sample ACF for the simulated data follows. We see a spike at lag 1 followed by generally non-significant values for lags past 1. Note that the sample ACF does not match the theoretical pattern of the underlying MA(1), which is that all autocorrelations for lags past 1 will be 0. A different sample would have a slightly different sample ACF shown below, but would likely have the same broad features. Theroretical Properties of a Time Series with an MA(2) Model For the MA(2) model, theoretical properties are the following: Note that the only nonzero values in the theoretical ACF are for lags 1 and 2. Autocorrelations for higher lags are 0. So, a sample ACF with significant autocorrelations at lags 1 and 2, but non-significant autocorrelations for higher lags indicates a possible MA(2) model. iid N(0,1). The coefficients are 1 0.5 and 2 0.3. Because this is an MA(2), the theoretical ACF will have nonzero values only at lags 1 and 2. Values of the two nonzero autocorrelations are A plot of the theoretical ACF follows. As nearly always is the case, sample data wont behave quite so perfectly as theory. We simulated n 150 sample values for the model x t 10 w t .5 w t-1 .3 w t-2 . where w t iid N(0,1). The time series plot of the data follows. As with the time series plot for the MA(1) sample data, you cant tell much from it. The sample ACF for the simulated data follows. The pattern is typical for situations where an MA(2) model may be useful. There are two statistically significant spikes at lags 1 and 2 followed by non-significant values for other lags. Note that due to sampling error, the sample ACF did not match the theoretical pattern exactly. ACF for General MA(q) Models A property of MA(q) models in general is that there are nonzero autocorrelations for the first q lags and autocorrelations 0 for all lags gt q. Non-uniqueness of connection between values of 1 and (rho1) in MA(1) Model. In the MA(1) model, for any value of 1 . the reciprocal 1 1 gives the same value for As an example, use 0.5 for 1 . and then use 1(0.5) 2 for 1 . Youll get (rho1) 0.4 in both instances. To satisfy a theoretical restriction called invertibility . we restrict MA(1) models to have values with absolute value less than 1. In the example just given, 1 0.5 will be an allowable parameter value, whereas 1 10.5 2 will not. Invertibility of MA models An MA model is said to be invertible if it is algebraically equivalent to a converging infinite order AR model. By converging, we mean that the AR coefficients decrease to 0 as we move back in time. Invertibility is a restriction programmed into time series software used to estimate the coefficients of models with MA terms. Its not something that we check for in the data analysis. Additional information about the invertibility restriction for MA(1) models is given in the appendix. Advanced Theory Note . For a MA(q) model with a specified ACF, there is only one invertible model. The necessary condition for invertibility is that the coefficients have values such that the equation 1- 1 y-. - q y q 0 has solutions for y that fall outside the unit circle. R Code for the Examples In Example 1, we plotted the theoretical ACF of the model x t 10 w t . 7w t-1 . and then simulated n 150 values from this model and plotted the sample time series and the sample ACF for the simulated data. The R commands used to plot the theoretical ACF were: acfma1ARMAacf(mac(0.7), lag. max10) 10 lags of ACF for MA(1) with theta1 0.7 lags0:10 creates a variable named lags that ranges from 0 to 10. plot(lags, acfma1,xlimc(1,10), ylabr, typeh, main ACF for MA(1) with theta1 0.7) abline (h0) adds a horizontal axis to the plot The first command determines the ACF and stores it in an object named acfma1 (our choice of name). The plot command (the 3rd command) plots lags versus the ACF values for lags 1 to 10. The ylab parameter labels the y-axis and the main parameter puts a title on the plot. To see the numerical values of the ACF simply use the command acfma1. The simulation and plots were done with the following commands. xcarima. sim(n150, list(mac(0.7))) Simulates n 150 values from MA(1) xxc10 adds 10 to make mean 10. Simulation defaults to mean 0. plot(x, typeb, mainSimulated MA(1) data) acf(x, xlimc(1,10), mainACF for simulated sample data) In Example 2, we plotted the theoretical ACF of the model x t 10 w t .5 w t-1 .3 w t-2 . and then simulated n 150 values from this model and plotted the sample time series and the sample ACF for the simulated data. The R commands used were acfma2ARMAacf(mac(0.5,0.3), lag. max10) acfma2 lags0:10 plot(lags, acfma2,xlimc(1,10), ylabr, typeh, main ACF for MA(2) with theta1 0.5,theta20.3) abline (h0) xcarima. sim(n150, list(mac(0.5, 0.3))) xxc10 plot (x, typeb, main Simulated MA(2) Series) acf(x, xlimc(1,10), mainACF for simulated MA(2) Data) Appendix: Proof of Properties of MA(1) For interested students, here are proofs for theoretical properties of the MA(1) model. Variance: (text (xt) text (mu wt theta1 w ) 0 text (wt) text (theta1w ) sigma2w theta21sigma2w (1theta21)sigma2w) When h 1, the previous expression 1 w 2. For any h 2, the previous expression 0. The reason is that, by definition of independence of the w t . E( w k w j ) 0 for any k j. Further, because the w t have mean 0, E( w j w j ) E( w j 2 ) w 2 . For a time series, Apply this result to get the ACF given above. An invertible MA model is one that can be written as an infinite order AR model that converges so that the AR coefficients converge to 0 as we move infinitely back in time. Well demonstrate invertibility for the MA(1) model. We then substitute relationship (2) for w t-1 in equation (1) (3) (zt wt theta1(z - theta1w ) wt theta1z - theta2w ) At time t-2 . equation (2) becomes We then substitute relationship (4) for w t-2 in equation (3) (zt wt theta1 z - theta21w wt theta1z - theta21(z - theta1w ) wt theta1z - theta12z theta31w ) If we were to continue (infinitely), we would get the infinite order AR model (zt wt theta1 z - theta21z theta31z - theta41z dots ) Note however, that if 1 1, the coefficients multiplying the lags of z will increase (infinitely) in size as we move back in time. To prevent this, we need 1 lt1. This is the condition for an invertible MA(1) model. Infinite Order MA model In week 3, well see that an AR(1) model can be converted to an infinite order MA model: (xt - mu wt phi1w phi21w dots phik1 w dots sum phij1w ) This summation of past white noise terms is known as the causal representation of an AR(1). In other words, x t is a special type of MA with an infinite number of terms going back in time. This is called an infinite order MA or MA(). A finite order MA is an infinite order AR and any finite order AR is an infinite order MA. Recall in Week 1, we noted that a requirement for a stationary AR(1) is that 1 lt1. Lets calculate the Var( x t ) using the causal representation. This last step uses a basic fact about geometric series that requires (phi1lt1) otherwise the series diverges. NavigationAutoregressive Moving-Average Simulation (First Order) The Demonstration is set such that the same random series of points is used no matter how the constants and are varied. However, when the quotrandomizequot button is pressed, a new random series will be generated and used. Keeping the random series identical allows the user to see exactly the effects on the ARMA series of changes in the two constants. The constant is limited to (-1,1) because divergence of the ARMA series results when . The Demonstration is for a first-order process only. Additional AR terms would enable more complex series to be generated, while additional MA terms would increase the smoothing. For a detailed description of ARMA processes, see, for instance, G. Box, G. M. Jenkins, and G. Reinsel, Time Series Analysis: Forecasting and Control . 3rd ed. Englewood Cliffs, NJ: Prentice-Hall, 1994. RELATED LINKSAutoregressive moving-average error processes (ARMA errors) and other models that involve lags of error terms can be estimated by using FIT statements and simulated or forecast by using SOLVE statements. ARMA models for the error process are often used for models with autocorrelated residuals. The AR macro can be used to specify models with autoregressive error processes. The MA macro can be used to specify models with moving-average error processes. Autoregressive Errors A model with first-order autoregressive errors, AR(1), has the form while an AR(2) error process has the form and so forth for higher-order processes. Note that the s are independent and identically distributed and have an expected value of 0. An example of a model with an AR(2) component is and so forth for higher-order processes. For example, you can write a simple linear regression model with MA(2) moving-average errors as where MA1 and MA2 are the moving-average parameters. Note that RESID. Y is automatically defined by PROC MODEL as The ZLAG function must be used for MA models to truncate the recursion of the lags. This ensures that the lagged errors start at zero in the lag-priming phase and do not propagate missing values when lag-priming period variables are missing, and it ensures that the future errors are zero rather than missing during simulation or forecasting. For details about the lag functions, see the section Lag Logic. This model written using the MA macro is as follows: General Form for ARMA Models The general ARMA( p, q ) process has the following form An ARMA( p , q ) model can be specified as follows: where AR i and MA j represent the autoregressive and moving-average parameters for the various lags. You can use any names you want for these variables, and there are many equivalent ways that the specification could be written. Vector ARMA processes can also be estimated with PROC MODEL. For example, a two-variable AR(1) process for the errors of the two endogenous variables Y1 and Y2 can be specified as follows: Convergence Problems with ARMA Models ARMA models can be difficult to estimate. If the parameter estimates are not within the appropriate range, a moving-average models residual terms grow exponentially. The calculated residuals for later observations can be very large or can overflow. This can happen either because improper starting values were used or because the iterations moved away from reasonable values. Care should be used in choosing starting values for ARMA parameters. Starting values of 0.001 for ARMA parameters usually work if the model fits the data well and the problem is well-conditioned. Note that an MA model can often be approximated by a high-order AR model, and vice versa. This can result in high collinearity in mixed ARMA models, which in turn can cause serious ill-conditioning in the calculations and instability of the parameter estimates. If you have convergence problems while estimating a model with ARMA error processes, try to estimate in steps. First, use a FIT statement to estimate only the structural parameters with the ARMA parameters held to zero (or to reasonable prior estimates if available). Next, use another FIT statement to estimate the ARMA parameters only, using the structural parameter values from the first run. Since the values of the structural parameters are likely to be close to their final estimates, the ARMA parameter estimates might now converge. Finally, use another FIT statement to produce simultaneous estimates of all the parameters. Since the initial values of the parameters are now likely to be quite close to their final joint estimates, the estimates should converge quickly if the model is appropriate for the data. AR Initial Conditions The initial lags of the error terms of AR( p ) models can be modeled in different ways. The autoregressive error startup methods supported by SASETS procedures are the following: conditional least squares (ARIMA and MODEL procedures) unconditional least squares (AUTOREG, ARIMA, and MODEL procedures) maximum likelihood (AUTOREG, ARIMA, and MODEL procedures) Yule-Walker (AUTOREG procedure only) Hildreth-Lu, which deletes the first p observations (MODEL procedure only) See Chapter 8, The AUTOREG Procedure, for an explanation and discussion of the merits of various AR(p) startup methods. The CLS, ULS, ML, and HL initializations can be performed by PROC MODEL. For AR(1) errors, these initializations can be produced as shown in Table 18.2. These methods are equivalent in large samples. Table 18.2 Initializations Performed by PROC MODEL: AR(1) ERRORS The initial lags of the error terms of MA( q ) models can also be modeled in different ways. The following moving-average error start-up paradigms are supported by the ARIMA and MODEL procedures: unconditional least squares conditional least squares The conditional least squares method of estimating moving-average error terms is not optimal because it ignores the start-up problem. This reduces the efficiency of the estimates, although they remain unbiased. The initial lagged residuals, extending before the start of the data, are assumed to be 0, their unconditional expected value. This introduces a difference between these residuals and the generalized least squares residuals for the moving-average covariance, which, unlike the autoregressive model, persists through the data set. Usually this difference converges quickly to 0, but for nearly noninvertible moving-average processes the convergence is quite slow. To minimize this problem, you should have plenty of data, and the moving-average parameter estimates should be well within the invertible range. This problem can be corrected at the expense of writing a more complex program. Unconditional least squares estimates for the MA(1) process can be produced by specifying the model as follows: Moving-average errors can be difficult to estimate. You should consider using an AR( p ) approximation to the moving-average process. A moving-average process can usually be well-approximated by an autoregressive process if the data have not been smoothed or differenced. The AR Macro The SAS macro AR generates programming statements for PROC MODEL for autoregressive models. The AR macro is part of SASETS software, and no special options need to be set to use the macro. The autoregressive process can be applied to the structural equation errors or to the endogenous series themselves. The AR macro can be used for the following types of autoregression: unrestricted vector autoregression restricted vector autoregression Univariate Autoregression To model the error term of an equation as an autoregressive process, use the following statement after the equation: For example, suppose that Y is a linear function of X1, X2, and an AR(2) error. You would write this model as follows: The calls to AR must come after all of the equations that the process applies to. The preceding macro invocation, AR(y,2), produces the statements shown in the LIST output in Figure 18.58. Figure 18.58 LIST Option Output for an AR(2) Model The PRED prefixed variables are temporary program variables used so that the lags of the residuals are the correct residuals and not the ones redefined by this equation. Note that this is equivalent to the statements explicitly written in the section General Form for ARMA Models. You can also restrict the autoregressive parameters to zero at selected lags. For example, if you wanted autoregressive parameters at lags 1, 12, and 13, you can use the following statements: These statements generate the output shown in Figure 18.59. Figure 18.59 LIST Option Output for an AR Model with Lags at 1, 12, and 13 The MODEL Procedure Listing of Compiled Program Code Statement as Parsed PRED. y a b x1 c x2 RESID. y PRED. y - ACTUAL. y ERROR. y PRED. y - y OLDPRED. y PRED. y yl1 ZLAG1( y - PREDy ) yl12 ZLAG12( y - PREDy ) yl13 ZLAG13( y - PREDy ) RESID. y PRED. y - ACTUAL. y ERROR. y PRED. y - y There are variations on the conditional least squares method, depending on whether observations at the start of the series are used to warm up the AR process. By default, the AR conditional least squares method uses all the observations and assumes zeros for the initial lags of autoregressive terms. By using the M option, you can request that AR use the unconditional least squares (ULS) or maximum-likelihood (ML) method instead. For example, Discussions of these methods is provided in the section AR Initial Conditions. By using the MCLS n option, you can request that the first n observations be used to compute estimates of the initial autoregressive lags. In this case, the analysis starts with observation n 1. For example: You can use the AR macro to apply an autoregressive model to the endogenous variable, instead of to the error term, by using the TYPEV option. For example, if you want to add the five past lags of Y to the equation in the previous example, you could use AR to generate the parameters and lags by using the following statements: The preceding statements generate the output shown in Figure 18.60. Figure 18.60 LIST Option Output for an AR model of Y This model predicts Y as a linear combination of X1, X2, an intercept, and the values of Y in the most recent five periods. Unrestricted Vector Autoregression To model the error terms of a set of equations as a vector autoregressive process, use the following form of the AR macro after the equations: The processname value is any name that you supply for AR to use in making names for the autoregressive parameters. You can use the AR macro to model several different AR processes for different sets of equations by using different process names for each set. The process name ensures that the variable names used are unique. Use a short processname value for the process if parameter estimates are to be written to an output data set. The AR macro tries to construct parameter names less than or equal to eight characters, but this is limited by the length of processname . which is used as a prefix for the AR parameter names. The variablelist value is the list of endogenous variables for the equations. For example, suppose that errors for equations Y1, Y2, and Y3 are generated by a second-order vector autoregressive process. You can use the following statements: which generate the following for Y1 and similar code for Y2 and Y3: Only the conditional least squares (MCLS or MCLS n ) method can be used for vector processes. You can also use the same form with restrictions that the coefficient matrix be 0 at selected lags. For example, the following statements apply a third-order vector process to the equation errors with all the coefficients at lag 2 restricted to 0 and with the coefficients at lags 1 and 3 unrestricted: You can model the three series Y1Y3 as a vector autoregressive process in the variables instead of in the errors by using the TYPEV option. If you want to model Y1Y3 as a function of past values of Y1Y3 and some exogenous variables or constants, you can use AR to generate the statements for the lag terms. Write an equation for each variable for the nonautoregressive part of the model, and then call AR with the TYPEV option. For example, The nonautoregressive part of the model can be a function of exogenous variables, or it can be intercept parameters. If there are no exogenous components to the vector autoregression model, including no intercepts, then assign zero to each of the variables. There must be an assignment to each of the variables before AR is called. This example models the vector Y(Y1 Y2 Y3) as a linear function only of its value in the previous two periods and a white noise error vector. The model has 18(3 3 3 3) parameters. Syntax of the AR Macro There are two cases of the syntax of the AR macro. When restrictions on a vector AR process are not needed, the syntax of the AR macro has the general form specifies a prefix for AR to use in constructing names of variables needed to define the AR process. If the endolist is not specified, the endogenous list defaults to name . which must be the name of the equation to which the AR error process is to be applied. The name value cannot exceed 32 characters. is the order of the AR process. specifies the list of equations to which the AR process is to be applied. If more than one name is given, an unrestricted vector process is created with the structural residuals of all the equations included as regressors in each of the equations. If not specified, endolist defaults to name . specifies the list of lags at which the AR terms are to be added. The coefficients of the terms at lags not listed are set to 0. All of the listed lags must be less than or equal to nlag . and there must be no duplicates. If not specified, the laglist defaults to all lags 1 through nlag . specifies the estimation method to implement. Valid values of M are CLS (conditional least squares estimates), ULS (unconditional least squares estimates), and ML (maximum likelihood estimates). MCLS is the default. Only MCLS is allowed when more than one equation is specified. The ULS and ML methods are not supported for vector AR models by AR. specifies that the AR process is to be applied to the endogenous variables themselves instead of to the structural residuals of the equations. Restricted Vector Autoregression You can control which parameters are included in the process, restricting to 0 those parameters that you do not include. First, use AR with the DEFER option to declare the variable list and define the dimension of the process. Then, use additional AR calls to generate terms for selected equations with selected variables at selected lags. For example, The error equations produced are as follows: This model states that the errors for Y1 depend on the errors of both Y1 and Y2 (but not Y3) at both lags 1 and 2, and that the errors for Y2 and Y3 depend on the previous errors for all three variables, but only at lag 1. AR Macro Syntax for Restricted Vector AR An alternative use of AR is allowed to impose restrictions on a vector AR process by calling AR several times to specify different AR terms and lags for different equations. The first call has the general form specifies a prefix for AR to use in constructing names of variables needed to define the vector AR process. specifies the order of the AR process. specifies the list of equations to which the AR process is to be applied. specifies that AR is not to generate the AR process but is to wait for further information specified in later AR calls for the same name value. The subsequent calls have the general form is the same as in the first call. specifies the list of equations to which the specifications in this AR call are to be applied. Only names specified in the endolist value of the first call for the name value can appear in the list of equations in eqlist . specifies the list of equations whose lagged structural residuals are to be included as regressors in the equations in eqlist . Only names in the endolist of the first call for the name value can appear in varlist . If not specified, varlist defaults to endolist . specifies the list of lags at which the AR terms are to be added. The coefficients of the terms at lags not listed are set to 0. All of the listed lags must be less than or equal to the value of nlag . and there must be no duplicates. If not specified, laglist defaults to all lags 1 through nlag . The MA Macro The SAS macro MA generates programming statements for PROC MODEL for moving-average models. The MA macro is part of SASETS software, and no special options are needed to use the macro. The moving-average error process can be applied to the structural equation errors. The syntax of the MA macro is the same as the AR macro except there is no TYPE argument. When you are using the MA and AR macros combined, the MA macro must follow the AR macro. The following SASIML statements produce an ARMA(1, (1 3)) error process and save it in the data set MADAT2. The following PROC MODEL statements are used to estimate the parameters of this model by using maximum likelihood error structure: The estimates of the parameters produced by this run are shown in Figure 18.61. Figure 18.61 Estimates from an ARMA(1, (1 3)) Process There are two cases of the syntax for the MA macro. When restrictions on a vector MA process are not needed, the syntax of the MA macro has the general form specifies a prefix for MA to use in constructing names of variables needed to define the MA process and is the default endolist . is the order of the MA process. specifies the equations to which the MA process is to be applied. If more than one name is given, CLS estimation is used for the vector process. specifies the lags at which the MA terms are to be added. All of the listed lags must be less than or equal to nlag . and there must be no duplicates. If not specified, the laglist defaults to all lags 1 through nlag . specifies the estimation method to implement. Valid values of M are CLS (conditional least squares estimates), ULS (unconditional least squares estimates), and ML (maximum likelihood estimates). MCLS is the default. Only MCLS is allowed when more than one equation is specified in the endolist . MA Macro Syntax for Restricted Vector Moving-Average An alternative use of MA is allowed to impose restrictions on a vector MA process by calling MA several times to specify different MA terms and lags for different equations. The first call has the general form specifies a prefix for MA to use in constructing names of variables needed to define the vector MA process. specifies the order of the MA process. specifies the list of equations to which the MA process is to be applied. specifies that MA is not to generate the MA process but is to wait for further information specified in later MA calls for the same name value. The subsequent calls have the general form is the same as in the first call. specifies the list of equations to which the specifications in this MA call are to be applied. specifies the list of equations whose lagged structural residuals are to be included as regressors in the equations in eqlist . specifies the list of lags at which the MA terms are to be added.4.2 Linear Stationary Models for Time Series where the random variable is called the innovation because it represents the part of the observed variable that is unpredictable given the past values . The general model (4.4 ) assumes that is the output of a linear filter that transforms the past innovations , that is, is a linear process. This linearity assumption is based on the Wolds decomposition theorem (Wold 1938 ) that says that any discrete stationary covariance process can be expressed as the sum of two uncorrelated processes, where is purely deterministic and is a purely indeterministic process that can be written as a linear sum of the innovation process : where is a sequence of serially uncorrelated random variables with zero mean and common variance . Condition is necessary for stationarity. The formulation (4.4 ) is a finite reparametrization of the infinite representation (4.5 )-(4.6 ) with constant. It is usually written in terms of the lag operator defined by , that gives a shorter expression: where the lag operator polynomials and are called the polynomial and the polynomial, respectively. In order to avoid parameter redundancy, we assume that there are not common factors between the and the components. Next, we will study the plot of some time series generated by stationary models with the aim of determining the main patterns of their temporal evolution. Figure 4.2 includes two series generated from the following stationary processes computed by means of the genarma quantlet: Figure 4.2: Time series generated by models As expected, both time series move around a constant level without changes in variance due to the stationary property. Moreover, this level is close to the theoretical mean of the process, , and the distance of each point to this value is very rarely outside the bounds . Furthermore, the evolution of the series shows local departures from the mean of the process, which is known as the mean reversion behavior that characterizes the stationary time series. Let us study with some detail the properties of the different processes, in particular, the autocovariance function which captures the dynamic properties of a stochastic stationary process. This function depends on the units of measure, so the usual measure of the degree of linearity between variables is the correlation coefficient. In the case of stationary processes, the autocorrelation coefficient at lag , denoted by , is defined as the correlation between and : Thus, the autocorrelation function (ACF) is the autocovariance function standarized by the variance . The properties of the ACF are: Given the symmetry property (4.10 ), the ACF is usually represented by means of a bar graph at the nonnegative lags that is called the simple correlogram. Another useful tool to describe the dynamics of a stationary process is the partial autocorrelation function (PACF). The partial autocorrelation coefficient at lag measures the linear association between and adjusted for the effects of the intermediate values . Therefore, it is just the coefficient in the linear regression model: The properties of the PACF are equivalent to those of the ACF (4.8 )-(4.10 ) and it is easy to prove that (Box and Jenkins 1976 ). Like the ACF, the partial autocorrelation function does not depend on the units of measure and it is represented by means of a bar graph at the nonnegative lags that is called partial correlogram. The dynamic properties of each stationary model determine a particular shape of the correlograms. Moreover, it can be shown that, for any stationary process, both functions, ACF and PACF, approach to zero as the lag tends to infinity. The models are not always stationary processes, so it is necessary first to determine the conditions for stationarity. There are subclasses of models which have special properties so we shall study them separately. Thus, when and , it is a white noise process . when , it is a pure moving average process of order . , and when it is a pure autoregressive process of order . . 4.2.1 White Noise Process The simplest model is a white noise process, where is a sequence of uncorrelated zero mean variables with constant variance . It is denoted by . This process is stationary if its variance is finite, , since given that: verifies conditions (4.1 )-(4.3 ). Moreover, is uncorrelated over time, so its autocovariance function is: Figure 4.7 shows two simulated time series generated from processes with zero mean and parameters and -0.7, respectively. The autoregressive parameter measures the persistence of past events into the current values. For example, if , a positive (or negative) shock affects positively (or negatively) for a period of time which is longer the larger the value of . When , the series moves more roughly around the mean due to the alternation in the direction of the effect of , that is, a shock that affects positively in moment , has negative effects on , positive in . The process is always invertible and it is stationary when the parameter of the model is constrained to lie in the region . To prove the stationary condition, first we write the in the moving average form by recursive substitution of in (4.14 ): Figure 4.8: Population correlograms for processes That is, is a weighted sum of past innovations. The weights depend on the value of the parameter : when , (or ), the influence of a given innovation increases (or decreases) through time. Taking expectations to (4.15 ) in order to compute the mean of the process, we get: Given that , the result is a sum of infinite terms that converges for all value of only if , in which case . A similar problem appears when we compute the second moment. The proof can be simplified assuming that , that is, . Then, variance is: Again, the variance goes to infinity except for , in which case . It is easy to verify that both the mean and the variance explode when that condition doesnt hold. The autocovariance function of a stationary process is Therefore, the autocorrelation function for the stationary model is: That is, the correlogram shows an exponential decay with positive values always if is positive and with negative-positive oscillations if is negative (see figure 4.8 ). Furthermore, the rate of decay decreases as increases, so the greater the value of the stronger the dynamic correlation in the process. Finally, there is a cutoff in the partial autocorrelation function at the first lag. Figure 4.9: Population correlograms for processes It can be shown that the general process (Box and Jenkins 1976 ): Is stationary only if the roots of the characteristic equation of the polynomial lie outside the unit circle. The mean of a stationary model is . Is always invertible for any values of the parameters. Its ACF goes to zero exponentially when the roots of are real or with sine-cosine wave fluctuations when they are complex. Its PACF has a cutoff at the lag, that is,.Some examples of correlograms for more complex models, such as the , can be seen in figure 4.9. They are very similar to the patterns when the processes have real roots, but take a very different shape when the roots are complex (see the first pair of graphics of figure 4.9 ). 4.2.4 Autoregressive Moving Average Model The general (finite-order) autoregressive moving average model of orders , , is:
No comments:
Post a Comment