8 Dependence
Sections 6 and 7 treated the observations as independent.
This assumption may not be appropriate when observations belong to the same group or are ordered in time.
Examples include students grouped by school, repeated observations of the same state, and weekly prices.
The OLS point estimator may remain unchanged. The covariance estimator must account for the sampling structure.
We examine clustered observations, panel data and time series.
8.1 Dependence and Uncertainty
We first examine how dependence affects the variance of the sample mean.
Consider two observations with the same variance \sigma_Y^2. The variance rules from Section 3 give \begin{aligned} \mathrm{Var}\!\left(\frac{Y_1+Y_2}{2}\right) &=\frac14\left[\mathrm{Var}(Y_1)+\mathrm{Var}(Y_2) +2\mathrm{Cov}(Y_1,Y_2)\right]\\ &=\frac{\sigma_Y^2}{2}+\frac{\mathrm{Cov}(Y_1,Y_2)}{2}. \end{aligned}
Under independence, the covariance term is zero. If Y_1 and Y_2 have positive covariance, the sample mean varies more across samples than under independence.
The same issue arises for regression coefficients.
Recall the sample second-moment matrix \widehat{\boldsymbol Q}:=\frac1n\sum_{i=1}^n\boldsymbol X_i\boldsymbol X_i' from Section 6.
The OLS decomposition is \widehat{\boldsymbol\beta}-\boldsymbol\beta =\widehat{\boldsymbol Q}^{-1} \frac1n\sum_{i=1}^n\boldsymbol X_i u_i. \tag{8.1}
The estimation error depends on the sum of the vectors \boldsymbol X_i u_i.
Under i.i.d. sampling, these vectors are uncorrelated across i. Under dependence, they may be correlated across observations.
Under the sampling and moment conditions below, the OLS estimator is asymptotically normal, as in Section 6: \sqrt n(\widehat{\boldsymbol\beta}-\boldsymbol\beta) \overset{d}{\to} \mathcal N\!\left(\boldsymbol0, \boldsymbol Q^{-1}\boldsymbol\Omega\boldsymbol Q^{-1}\right). \tag{8.2}
Here, \boldsymbol Q is the probability limit of \widehat{\boldsymbol Q}. The matrix \boldsymbol\Omega is the limiting covariance matrix of n^{-1/2}\sum_{i=1}^n\boldsymbol X_i u_i.
Thus, for large samples, \widehat{\boldsymbol\beta} \overset{\mathrm{approx.}}{\sim} \mathcal N\!\left(\boldsymbol\beta, \frac1n\boldsymbol Q^{-1}\boldsymbol\Omega\boldsymbol Q^{-1}\right).
Replacing the population matrices by consistent estimates gives \widehat{\mathrm{Var}} (\widehat{\boldsymbol\beta}) =\frac1n\widehat{\boldsymbol Q}^{-1} \widehat{\boldsymbol\Omega} \widehat{\boldsymbol Q}^{-1}. \tag{8.3}
Under i.i.d. sampling, \boldsymbol\Omega=E[u_i^2\boldsymbol X_i\boldsymbol X_i']. HC0 estimates this matrix by \widehat{\boldsymbol\Omega}_{HC0} =\frac1n\sum_{i=1}^n \widehat u_i^2\boldsymbol X_i\boldsymbol X_i'.
Under dependence, \boldsymbol\Omega generally differs from E[u_i^2\boldsymbol X_i\boldsymbol X_i']. It also accounts for covariances between the vectors \boldsymbol X_i u_i from different observations.
We therefore adapt \widehat{\boldsymbol\Omega} to the sampling structure:
- HC estimators use only the products \widehat u_i^2\boldsymbol X_i\boldsymbol X_i' from individual observations.
- Cluster-robust estimators also include cross-products within each cluster.
- HAC estimators also include cross-products at different time lags.
Selecting another covariance estimator changes standard errors, confidence intervals, and tests. It does not change the OLS point estimates.
8.2 Clustered Observations
We observe G independently and identically distributed clusters. Observations within a cluster may be dependent.
For simplicity, each cluster contains the same fixed number M of observations, indexed by j=1,\ldots,M. The total sample size is n=GM.
The regression model is Y_{gj}=\boldsymbol X_{gj}'\boldsymbol\beta+u_{gj}, \qquad g=1,\ldots,G,\quad j=1,\ldots,M. \tag{8.4}
As before, E[u_{gj}\mid\boldsymbol X_{gj}]=0 specifies the linear population CEF and implies E[\boldsymbol X_{gj}u_{gj}]=\boldsymbol0.
Clustered Observations
Within cluster g, observations (Y_{gj},\boldsymbol X_{gj}) and (Y_{g\ell},\boldsymbol X_{g\ell}) may be dependent for j\neq\ell.
Different clusters are assumed to be mutually independent. In particular, (Y_{gj},\boldsymbol X_{gj}) and (Y_{h\ell},\boldsymbol X_{h\ell}) are independent whenever g\neq h.
For example, we may randomly select many schools and observe the same number of students within each selected school.
Students from the same school may share teachers, facilities, and local conditions.
We estimate the model by pooled OLS, using all observations together: \widehat{\boldsymbol\beta} =\left(\sum_{g=1}^G\sum_{j=1}^{M} \boldsymbol X_{gj}\boldsymbol X_{gj}'\right)^{-1} \sum_{g=1}^G\sum_{j=1}^{M}\boldsymbol X_{gj}Y_{gj}. \tag{8.5}
Within-Cluster Dependence
Sum the vectors \boldsymbol X_{gj}u_{gj} within cluster g: \boldsymbol S_g :=\sum_{j=1}^{M}\boldsymbol X_{gj}u_{gj}.
Using this notation, Equation 8.1 becomes \widehat{\boldsymbol\beta}-\boldsymbol\beta =\widehat{\boldsymbol Q}^{-1}\frac1n \sum_{g=1}^G\boldsymbol S_g, where \widehat{\boldsymbol Q}=\frac1n\sum_{g=1}^G\sum_{j=1}^{M}\boldsymbol X_{gj}\boldsymbol X_{gj}'.
The cluster sums \boldsymbol S_1,\ldots,\boldsymbol S_G are i.i.d. with mean zero. Observations within each cluster may still be dependent.
The population matrix in Equation 8.2 is \boldsymbol\Omega=\boldsymbol\Omega_{CR} :=\frac1M E[\boldsymbol S_g\boldsymbol S_g']. \tag{8.6}
Since n=GM, the CLT gives \frac1{\sqrt n}\sum_{g=1}^G\boldsymbol S_g \overset{d}{\to} \mathcal N(\boldsymbol0,\boldsymbol\Omega_{CR}) \qquad\text{as }G\to\infty.
With \widehat{\boldsymbol Q}\overset{p}{\to}\boldsymbol Q and \boldsymbol Q invertible, we obtain \sqrt n(\widehat{\boldsymbol\beta}-\boldsymbol\beta) \overset{d}{\to} \mathcal N\!\left(\boldsymbol0, \boldsymbol Q^{-1}\boldsymbol\Omega_{CR}\boldsymbol Q^{-1}\right).
Many Independent Clusters
Cluster-robust inference here uses G\to\infty while M remains fixed.
Simply adding more observations to a fixed number of clusters (i.e. M \to \infty and G fixed) does not provide the same justification.
For example, 100 schools with 20 students each give 2,000 rows but 100 independent clusters. Treating all students as independent would ignore their shared school environment.
Cluster-Robust Covariance Matrix
Using the estimated cluster sums \widehat{\boldsymbol S}_g:=\sum_{j=1}^{M}\boldsymbol X_{gj}\widehat u_{gj}, we estimate \boldsymbol\Omega_{CR} by \widehat{\boldsymbol\Omega}_{CR0} :=\frac1n\sum_{g=1}^G \widehat{\boldsymbol S}_g\widehat{\boldsymbol S}_g'.
The factor 1/n matches the normalization of \widehat{\boldsymbol Q} over all n observations.
Multiplying a cluster sum by its transpose includes cross-products between different observations in that cluster. HC0 omits these cross-products.
A Cluster with Two Observations
For a cluster containing two observations, \widehat{\boldsymbol S}_g =\boldsymbol X_{g1}\widehat u_{g1} +\boldsymbol X_{g2}\widehat u_{g2}.
Multiplying out gives \begin{aligned} \widehat{\boldsymbol S}_g\widehat{\boldsymbol S}_g' &=\boldsymbol X_{g1}\boldsymbol X_{g1}'\widehat u_{g1}^2 +\boldsymbol X_{g2}\boldsymbol X_{g2}'\widehat u_{g2}^2\\ &\quad+\boldsymbol X_{g1}\boldsymbol X_{g2}'\widehat u_{g1}\widehat u_{g2} +\boldsymbol X_{g2}\boldsymbol X_{g1}'\widehat u_{g2}\widehat u_{g1}. \end{aligned}
HC0 retains the two terms in the first line. CR0 also includes the cross-products in the second line.
If every cluster contains just one observation, CR0 reduces to HC0.
Covariance Estimation
Cluster-Robust Covariance Estimator
Substituting \widehat{\boldsymbol\Omega}_{CR0} into Equation 8.3 gives \widehat{\mathrm{Var}}_{CR0} (\widehat{\boldsymbol\beta}) =\frac1n\widehat{\boldsymbol Q}^{-1} \widehat{\boldsymbol\Omega}_{CR0} \widehat{\boldsymbol Q}^{-1}. \tag{8.7}
Clustered standard errors are the square roots of the diagonal entries.
Cluster at the level at which observations may be dependent. For students sharing a school, use school clusters rather than individual students.
For unequal cluster sizes, the sample estimators have the same form, using M_g in the within-cluster sums and n=\sum_{g=1}^G M_g.
8.3 Panels and Fixed Effects
In a panel, we observe the same units over several periods. We treat them as clustered data, with each unit forming one cluster.
Let i=1,\ldots,N index units and t=1,\ldots,T index periods.
For simplicity, we use a balanced panel: each unit is observed in the same T periods, so n=NT. In the cluster notation, G=N and M=T.
We assume units are i.i.d. across i while allowing dependence across t within each unit.
With a common intercept and slopes, the panel regression model is Y_{it}=\boldsymbol X_{it}'\boldsymbol\beta+u_{it}, where \boldsymbol X_{it} includes a constant. We use the same pooled OLS estimator as in Equation 8.5, with sums over units i and periods t.
Traffic Fatalities
The Fatalities data contain annual observations for 48 US states from 1982 to 1988.
Thus, there are 336 observations but 48 state clusters.
The outcome is the number of traffic fatalities per 10,000 residents. The regressor is the tax on a case of beer, measured in inflation-adjusted 1987 dollars.
We compare two models:
- Pooled model: a common intercept. The slope combines comparisons across states and changes within states.
- State fixed effects: separate state intercepts. The slope describes the relationship while holding time-invariant state characteristics fixed.
Unit Fixed Effects
We now allow a separate intercept \alpha_i for each unit. Let \boldsymbol Z_{it} contain regressors that vary over time, without a constant.
The unit fixed-effects model is Y_{it} =\alpha_i+\boldsymbol Z_{it}'\boldsymbol\beta+u_{it}. \tag{8.8}
The unit effect \alpha_i represents the combined additive contribution of observed and unobserved characteristics that remain constant over time.
The usual intercept is a population parameter shared by all units.
In our sampling framework, \alpha_i varies across randomly sampled units and is therefore a random variable. It remains constant over time within each unit and may be related to the regressors.
If \alpha_i is omitted, it becomes part of the regression error. If the unit effect is correlated with the regressors, pooled OLS generally does not consistently estimate the slope vector \boldsymbol\beta in Equation 8.8.
One sufficient population condition is strict exogeneity: E[u_{it}\mid \alpha_i, \boldsymbol Z_{i1},\ldots,\boldsymbol Z_{iT}] =0 \qquad\text{for every }t. \tag{8.9}
The error must have conditional mean zero while holding the unit effect fixed. The condition is strict because we also condition on the regressors from every period.
It implies E[Y_{it}\mid\alpha_i,\boldsymbol Z_{it}] =\alpha_i+\boldsymbol Z_{it}'\boldsymbol\beta.
The coefficients describe changes in the conditional mean while holding the unit effect and the other regressors fixed.
In the traffic example, strict exogeneity can fail if an unexpected rise in fatalities leads a state to change its future beer tax. Future taxes then contain information about the current error.
Within Estimation
Write \overline Y_i:=\frac1T\sum_{t=1}^T Y_{it}, and define \overline{\boldsymbol Z}_i and \overline u_i analogously.
Averaging Equation 8.8 over time gives \overline Y_i =\alpha_i+\overline{\boldsymbol Z}_i'\boldsymbol\beta +\overline u_i.
Subtracting this equation from Equation 8.8 gives Y_{it}-\overline Y_i = (\boldsymbol Z_{it}-\overline{\boldsymbol Z}_i)' \boldsymbol\beta +(u_{it}-\overline u_i). \tag{8.10}
The time-invariant effect \alpha_i disappears.
OLS applied to Equation 8.10 is the within estimator.
That is, we regress Y_{it}-\overline Y_i on \boldsymbol Z_{it}-\overline{\boldsymbol Z}_i, without an intercept.
Strict exogeneity implies that the demeaned error has conditional mean zero given the demeaned regressors: E\!\left[u_{it}-\overline u_i \,\middle|\, \boldsymbol Z_{it}-\overline{\boldsymbol Z}_i\right]=0.
Consequently, E\!\left[ (\boldsymbol Z_{it}-\overline{\boldsymbol Z}_i) (u_{it}-\overline u_i) \right]=\boldsymbol0.
Alternatively, OLS on the original variables with a dummy variable for each unit and no separate intercept gives the same slope estimates as the within transformation.
Here, consistency and cluster-robust inference use many independent units, N\to\infty, with a fixed number of periods T.
For unbalanced panels, the sample formulas use each unit’s observed periods and their count T_i in place of T, with n=\sum_{i=1}^N T_i.
Empirical Results
We return to the traffic data and compare the pooled and within-state fits.
The pooled regression combines differences between states with changes within states. States with higher beer taxes may also have persistently higher traffic-fatality risks.
For example, states with more dangerous driving conditions might maintain higher beer taxes partly in response to those risks. These differences could produce a positive pooled association.
State fixed effects absorb the time-invariant differences. The within-state regression compares years within the same state. In these data, years with higher beer taxes tend to have fewer fatalities.
We compare pooled OLS with a model including state fixed effects.
Here, state is stored as a factor, meaning a categorical variable. R includes the corresponding indicators as in the dummy-coding example in Section 4.
We calculate the CR0 covariance matrix for each model, clustering by state:
The options type = "HC0", cadjust = FALSE give the CR0 estimator in Equation 8.7.
The option df = Inf in coeftest uses the standard normal approximation for the p-values:
The table reports the beer-tax coefficients, with CR0 standard errors in parentheses. Significance stars follow the convention introduced in Section 7.4.
modelsummary(
list("Pooled" = pooled_test, "State FE" = state_test),
coef_map = c("beertax" = "Beer tax"),
stars = c("*" = 0.10, "**" = 0.05, "***" = 0.01),
gof_map = "nobs"
)| Pooled | State FE | |
|---|---|---|
| * p < 0.1, ** p < 0.05, *** p < 0.01 | ||
| Beer tax | 0.365*** | -0.656** |
| (0.118) | (0.288) | |
| Num.Obs. | 336 | 336 |
Adding state effects changes the point estimate because the coefficient is now estimated from within-state variation.
In the state fixed-effects model, the estimated coefficient is -0.656.
According to this model, years with a one-dollar higher inflation-adjusted beer tax have, on average, 0.656 fewer fatalities per 10,000 residents when comparing years within the same state.
The covariance estimator also matters for the inference. For comparison, calculate the covariance matrix treating rows as independent:
state_HC0 = vcovHC(state_fe_fit, type = "HC0")| Covariance estimator | Standard error | Approximate 95% confidence interval |
|---|---|---|
| HC0, treating rows as independent | 0.188 | [-1.024,-0.288] |
| CR0, clustering by state | 0.288 | [-1.221,-0.091] |
Both intervals use the same coefficient estimate and standard normal critical value. Clustering gives a wider interval. Both intervals exclude zero.
The fixed effects change the fitted conditional-mean model. Clustering changes the estimated sampling uncertainty.
Within-State Variation
The inflation-adjusted beer tax varies over time within all 48 states. It can change even when a state leaves its nominal tax unchanged.
In this single-regressor model, its coefficient can therefore be estimated from within-state comparisons.
With several regressors, each coefficient requires sufficient within-unit variation after accounting for the other regressors.
Now consider adding a state’s land area L_i as a time-invariant control. Subtracting its state average gives zero: L_i-\frac1T\sum_{t=1}^T L_i=0.
Its coefficient cannot be estimated separately from the state effect. The state effect already absorbs its additive contribution.
8.4 Time Series
A time series is a sequence of observations ordered by time.
The index t is used because observations correspond to time points. T is the number of observed periods.
Nearby observations may be strongly related. We nevertheless need stable behavior as the sample period grows.
Stationarity
Let \boldsymbol W_t be the vector of all variables observed in period t. The sequence \{\boldsymbol W_t\} is a multivariate time series.
Strict Stationarity
The time series \{\boldsymbol W_t\} is strictly stationary if the vector (\boldsymbol W_t,\ldots,\boldsymbol W_{t+m}) has the same distribution as (\boldsymbol W_{t+h},\ldots,\boldsymbol W_{t+m+h}) for every starting date t, every integer m\ge0, and every time shift h for which the indices are defined.
In words, shifting a block of observations in time does not change its joint distribution.
If the second moments are finite, strict stationarity implies that
- the mean and variance do not change over time,
- and the covariance between two observations depends on their lag, not on their dates.
The lag h tells us how far apart two observations are. At lag one, we compare neighboring periods.
For a stationary scalar series with 0<\mathrm{Var}(Y_t)<\infty, the lag-h autocorrelation is \rho(h) :=\frac{\mathrm{Cov}(Y_t,Y_{t-h})} {\mathrm{Var}(Y_t)}.
An i.i.d. sequence is stationary. A stationary sequence need not be independent.
Stationary and Nonstationary Examples
The stationary solution of a first-order autoregressive model, abbreviated AR(1), satisfies Y_t=\phi Y_{t-1}+\varepsilon_t, \qquad |\phi|<1, where the new shocks \varepsilon_t are i.i.d. with mean zero and finite positive variance.
Its autocorrelation is \rho(h)=\phi^{|h|}.
A random walk satisfies Y_t=Y_{t-1}+\varepsilon_t. \tag{8.11}
Here, the new shocks \varepsilon_t are again i.i.d. with mean zero and finite positive variance.
Starting at Y_0=0 with innovation variance \sigma_\varepsilon^2, \mathrm{Var}(Y_t)=t\sigma_\varepsilon^2.
Its distribution changes with t, so the random walk is nonstationary.
Its first difference \Delta Y_t:=Y_t-Y_{t-1}=\varepsilon_t is stationary.
Sample ACF
With \overline Y=T^{-1}\sum_{t=1}^T Y_t, the sample autocovariance is \widehat\gamma(h) :=\frac1T\sum_{t=h+1}^T (Y_t-\overline Y)(Y_{t-h}-\overline Y) for h=0,\ldots,T-1.
For a positive sample variance, the sample autocorrelation is \widehat\rho(h):=\frac{\widehat\gamma(h)}{\widehat\gamma(0)}.
At lag zero, \widehat\gamma(0) is the sample variance. A plot of \widehat\rho(h) across lags is called a sample autocorrelation function (ACF).
The ACF compares a series with its own past values. We simulate an AR(1) with \phi=0.8 and a random walk, each with T=300 observations.
- AR(1): The sample autocorrelations decline toward zero as the lag increases.
- Random walk: The sample autocorrelations remain large over many lags.
- First differences: At positive lags, the sample autocorrelations fluctuate around zero, as expected for independent shocks.
These patterns help us assess persistence. An ACF plot alone does not establish stationarity.
Spurious Correlation
We now compare two independently generated series for each model.
In this simulation, the sample correlation is 0.052 for the stationary series but 0.534 for the independent random walks.
After differencing the random walks, the sample correlation is -0.001.
Regressing unrelated nonstationary series on one another can produce a convincing but spurious regression.
Increasing the sample period does not by itself remove this problem.
8.5 Time-Series Regression
Consider Y_t=\boldsymbol X_t'\boldsymbol\beta+u_t, \qquad t=1,\ldots,T. \tag{8.12}
The regressor vector may include current explanatory variables, their lags, lags of the outcome, and a constant.
For an exact linear conditional mean, we require E[u_t\mid\boldsymbol X_t]=0. \tag{8.13}
Define \boldsymbol Q:=E[\boldsymbol X_t\boldsymbol X_t'].
The conditional-mean condition implies the population moment E[\boldsymbol X_tu_t]=\boldsymbol 0. \tag{8.14}
If \boldsymbol Q is invertible, this moment identifies \boldsymbol\beta.
Weak Dependence
A stationary process may still have strong dependence between observations far apart in time.
Intuitively, weak dependence means that \boldsymbol W_t and \boldsymbol W_{t-h} may be dependent, but approach independence as h\to\infty.
For large h, knowing \boldsymbol W_{t-h} then gives little information about the distribution of \boldsymbol W_t.
For the LLN and CLT used here, we require stationarity and weak dependence.
Consistency
The estimator is still OLS. Write \widehat{\boldsymbol Q} :=\frac1T\sum_{t=1}^T\boldsymbol X_t\boldsymbol X_t', \qquad \widehat{\boldsymbol q} :=\frac1T\sum_{t=1}^T\boldsymbol X_tY_t.
Assume stationarity, sufficiently weak dependence, suitable moment conditions and an invertible \boldsymbol Q. The LLN and the rules for probability limits give \begin{gathered} \widehat{\boldsymbol Q}\overset p\to\boldsymbol Q, \qquad \widehat{\boldsymbol q}\overset p\to E[\boldsymbol X_tY_t]=\boldsymbol Q\boldsymbol\beta,\\ \widehat{\boldsymbol\beta} =\widehat{\boldsymbol Q}^{-1}\widehat{\boldsymbol q} \overset p\to\boldsymbol\beta. \end{gathered}
The logic is the same as in Section 4. The LLN now needs conditions that allow dependent observations.
Sampling Uncertainty
First consider the sample mean \overline Y of a stationary series. Write its lag-h covariance as \gamma(h):=\mathrm{Cov}(Y_t,Y_{t-h}), with \gamma(0)=\mathrm{Var}(Y_t).
For a large T, under suitable weak dependence and moment conditions, \mathrm{Var}(\overline Y) \approx\frac1T \left[\gamma(0)+2\sum_{h=1}^{\infty}\gamma(h)\right]. \tag{8.15}
Under independence, the covariances at lags h\geq1 are zero. The variance of the sample mean is then \mathrm{Var}(Y_t)/T.
With serial dependence, the lag covariances also contribute to the uncertainty of the sample mean. The expression in brackets is the long-run variance of the series.
For regression, we apply the same idea to the vectors \boldsymbol X_tu_t that enter the estimation error.
Write \boldsymbol S_t:=\boldsymbol X_tu_t for period t.
Define the lag-h covariance matrix \boldsymbol\Gamma(h) :=E[\boldsymbol S_t\boldsymbol S_{t-h}'] =E[\boldsymbol X_t\boldsymbol X_{t-h}'u_tu_{t-h}].
Because E[\boldsymbol S_t]=\boldsymbol0, \boldsymbol\Gamma(h) is the covariance matrix between \boldsymbol S_t and \boldsymbol S_{t-h}.
For time series, the population matrix \boldsymbol\Omega is the long-run covariance matrix: \boldsymbol\Omega=\boldsymbol\Omega_{LR} :=\boldsymbol\Gamma(0) +\sum_{h=1}^{\infty} [\boldsymbol\Gamma(h)+\boldsymbol\Gamma(h)']. \tag{8.16}
The first term is the covariance within one period. The remaining terms account for dependence between periods.
Equivalently, \boldsymbol\Omega_{LR}=\sum_{h=-\infty}^{\infty}\boldsymbol\Gamma(h).
Asymptotic Normality with Time Dependence
Suppose:
- (Y_t,\boldsymbol X_t')' is stationary and sufficiently weakly dependent,
- Equation 8.14 holds,
- the required moments and \boldsymbol\Omega_{LR} are finite,
- and \boldsymbol Q is invertible.
Then \sqrt T(\widehat{\boldsymbol\beta}-\boldsymbol\beta) \overset{d}{\to} \mathcal N\left( \boldsymbol 0, \boldsymbol Q^{-1} \boldsymbol\Omega_{LR} \boldsymbol Q^{-1} \right). \tag{8.17}
HC estimators retain only the contemporaneous term \boldsymbol\Gamma(0).
They are generally invalid when the vectors \boldsymbol S_t remain correlated over time.
HAC Covariance Matrix
HAC stands for heteroskedasticity and autocorrelation consistent.
Using the OLS residuals, write \widehat{\boldsymbol S}_t=\boldsymbol X_t\widehat u_t and estimate the lag-h covariance by \widehat{\boldsymbol\Gamma}(h) :=\frac1T\sum_{t=h+1}^T \widehat{\boldsymbol S}_t\widehat{\boldsymbol S}_{t-h}'.
For a maximum lag L, use the Bartlett weights w_h=1-h/(L+1). The Newey–West estimator of \boldsymbol\Omega_{LR} is \widehat{\boldsymbol\Omega}_{HAC} :=\widehat{\boldsymbol\Gamma}(0) +\sum_{h=1}^{L}w_h \left[\widehat{\boldsymbol\Gamma}(h) +\widehat{\boldsymbol\Gamma}(h)'\right]. \tag{8.18}
The weights decrease with the lag, giving less weight to the noisier covariance estimates at longer lags.
Using Equation 8.3 with sample size T gives \widehat{\mathrm{Var}}_{HAC}(\widehat{\boldsymbol\beta}) =\frac1T\widehat{\boldsymbol Q}^{-1} \widehat{\boldsymbol\Omega}_{HAC} \widehat{\boldsymbol Q}^{-1}. \tag{8.19}
This is the sandwich formula from Section 6 with a middle matrix that also accounts for dependence across periods.
A small L may omit relevant dependence. A large L adds noisy long-lag covariance estimates.
For consistency, L grows with the sample size but remains small relative to it: L\to\infty and L/T\to0.
A common rule is L=\lfloor4(T/100)^{2/9}\rfloor, where \lfloor\cdot\rfloor means rounding down to an integer. We use this rule below.
8.6 Nonstationarity and Applications
The regression results in Section 8.5 assume stationarity. We now use unit-root tests to assess whether a stationary model is plausible, then apply the regression methods to gasoline and oil returns.
Unit Root Testing
A highly persistent stationary series can look like a random walk. The Dickey–Fuller test takes a unit root as its null hypothesis.
Dickey–Fuller Test
We compare a unit root with a stationary AR(1) alternative: \underbrace{H_0:\phi=1}_{\text{unit root}} \qquad\text{with}\qquad \underbrace{H_1:|\phi|<1}_{\text{stationarity}}.
We estimate the following regression with an intercept by OLS: \Delta Y_t=c+\underbrace{(\phi-1)}_{\psi}Y_{t-1}+\varepsilon_t. \tag{8.20}
Thus, the null hypothesis is H_0:\psi=0. Under a stationary alternative, \psi<0.
The Dickey–Fuller statistic is T_{DF} :=\frac{\widehat\psi}{\mathrm{se}_{hom}(\widehat\psi)} =\frac{\widehat\phi-1}{\mathrm{se}_{hom}(\widehat\phi)}. \tag{8.21}
Here, \mathrm{se}_{hom} is the usual homoskedastic standard error from Section 6.9.
The statistic has the familiar form of a t-ratio. Under H_0, however, the lagged regressor Y_{t-1} is nonstationary, so the normal approximation from Section 8.5 does not apply.
We instead use Dickey–Fuller critical values for the regression with an intercept.
For this intercept specification, the large-sample critical values are:
| Lower-tail probability \alpha | 0.01 | 0.05 | 0.10 |
|---|---|---|---|
| Standard normal | -2.33 | -1.64 | -1.28 |
| Dickey–Fuller | -3.43 | -2.86 | -2.57 |
More negative statistics provide stronger evidence against the unit root. At the 5% level, the rejection rule is \text{Reject }H_0\quad\text{if}\quad T_{DF}<-2.86.
For example, a statistic of -2 would lead to rejection using the normal critical value, but not using the appropriate Dickey–Fuller critical value.
Augmented Dickey–Fuller Test
The simple AR(1) model may leave dependence in the errors. The augmented Dickey–Fuller (ADF) test adds p lagged differences: \Delta Y_t=c+\psi Y_{t-1} +\sum_{\ell=1}^{p}\theta_\ell\Delta Y_{t-\ell} +\varepsilon_t. \tag{8.22}
These additional regressors account for short-run dynamics. We choose enough lags to remove substantial serial correlation from the remaining errors.
The test still concerns H_0:\psi=0. Its statistic is T_{ADF}=\widehat\psi/\mathrm{se}_{hom}(\widehat\psi), using the augmented regression.
In practice, p is often selected using an information criterion, such as AIC or BIC.
The ADF regression also includes an intercept and uses the same large-sample critical values as above. Too few lags can distort the test. Too many can reduce its power.
Gasoline Prices and Returns
We use weekly U.S. regular conventional gasoline prices from 1991 to 2023, published by the U.S. Energy Information Administration through FRED.
Let P_t denote the price and define its log return as r_t:=\Delta\log(P_t)=\log(P_t)-\log(P_{t-1}).
As in Section 5, 100r_t is approximately the percentage price change. We test the log price and its first difference separately.
Save the prepared CSV file in a data subfolder.
In ur.df, type = "drift" selects this intercept specification. We include four lagged differences, representing the previous four weeks.
The command @teststat[1] extracts the ADF statistic:
adf_price = ur.df(log_price, type = "drift", lags = 4)
adf_price@teststat[1]adf_return = ur.df(gas_return, type = "drift", lags = 4)
adf_return@teststat[1]Using the 5% critical value -2.86:
| Series | ADF statistic | Decision at 5% |
|---|---|---|
| Log gasoline price | -1.965 | Fail to reject the unit-root null |
| Gasoline log return | -15.158 | Reject the unit-root null |
The first result does not establish a unit root. A test may have difficulty distinguishing a unit root from a highly persistent stationary process.
The second result provides evidence against a unit root in returns. Together, the results support using returns rather than price levels for the stationary time-series model below.
Gasoline and Oil Returns
We add Brent crude-oil prices from EIA through FRED for the same period, 1991 to 2023.
The CSV pairs each Monday’s gasoline price with that day’s Brent price. If the Brent quotation is unavailable, it uses the latest earlier quotation.
For both prices, we use the log returns r_t=\Delta\log(P_t) defined above. Returns are recorded as decimal changes, without multiplying by 100.
A distributed-lag (DL) model relates the current outcome to earlier values of another variable. We use the first two lags of the oil return: \mathrm{gas}_t =\alpha+\delta_1\mathrm{oil}_{t-1} +\delta_2\mathrm{oil}_{t-2}+u_t.
Under Equation 8.13, \delta_h compares weeks with different oil returns h weeks earlier and the same values of the other included lags.
It is the change in the mean gasoline return per unit difference in that lagged oil return.
Adding Outcome Lags
An autoregressive distributed-lag (ADL) model additionally includes lags of the outcome: \begin{aligned} \mathrm{gas}_t &=\alpha+\delta_1\mathrm{oil}_{t-1} +\delta_2\mathrm{oil}_{t-2}\\ &\quad+\phi_1\mathrm{gas}_{t-1} +\phi_2\mathrm{gas}_{t-2}+u_t. \end{aligned}
These outcome lags can represent persistence in gasoline returns.
Estimation and Comparison
In the code, oil_1 and oil_2 contain oil returns one and two weeks earlier. The variables gas_1 and gas_2 contain the corresponding gasoline returns.
Prepare returns and lags
DL_fit = lm(gas ~ oil_1 + oil_2, data = time_data)
ADL_fit = lm(gas ~ oil_1 + oil_2 + gas_1 + gas_2,
data = time_data)
L = floor(4 * (nobs(DL_fit) / 100)^(2/9))
DL_HC0 = vcovHC(DL_fit, type = "HC0")
DL_HAC = NeweyWest(DL_fit, lag = L, prewhite = FALSE)
ADL_HC0 = vcovHC(ADL_fit, type = "HC0")
ADL_HAC = NeweyWest(ADL_fit, lag = L, prewhite = FALSE)Both models use T=1678 observations. The lag rule gives L=7 weeks.
The option prewhite = FALSE selects the estimator in Equation 8.18.
The table compares the specifications and covariance estimators. Standard errors are in parentheses. Again, df = Inf uses standard normal p-values.
modelsummary(
list("DL: HC0" = coeftest(DL_fit, vcov. = DL_HC0, df = Inf),
"DL: HAC" = coeftest(DL_fit, vcov. = DL_HAC, df = Inf),
"ADL: HC0" = coeftest(ADL_fit, vcov. = ADL_HC0, df = Inf),
"ADL: HAC" = coeftest(ADL_fit, vcov. = ADL_HAC, df = Inf)),
fmt = 4,
stars = c("*" = 0.10, "**" = 0.05, "***" = 0.01),
gof_map = "nobs"
)| DL: HC0 | DL: HAC | ADL: HC0 | ADL: HAC | |
|---|---|---|---|---|
| * p < 0.1, ** p < 0.05, *** p < 0.01 | ||||
| (Intercept) | 0.0004 | 0.0004 | 0.0002 | 0.0002 |
| (0.0004) | (0.0006) | (0.0004) | (0.0004) | |
| oil_1 | 0.1703*** | 0.1703*** | 0.1231*** | 0.1231*** |
| (0.0093) | (0.0115) | (0.0099) | (0.0092) | |
| oil_2 | 0.0846*** | 0.0846*** | 0.0081 | 0.0081 |
| (0.0097) | (0.0114) | (0.0108) | (0.0109) | |
| gas_1 | 0.3745*** | 0.3745*** | ||
| (0.0592) | (0.0446) | |||
| gas_2 | 0.1092** | 0.1092*** | ||
| (0.0431) | (0.0327) | |||
| Num.Obs. | 1678 | 1678 | 1678 | 1678 |
Within each model, HC0 and HAC give identical point estimates. For the first oil lag in the DL model, the standard error increases from 0.0093 with HC0 to 0.0115 with HAC.
In the fitted DL model, if last week’s oil return is one percentage point higher, this week’s predicted gasoline return is 0.170 percentage points higher, holding the oil return from two weeks ago fixed.
The ADL comparison also holds the previous two gasoline returns fixed. Its estimated coefficient on the first oil lag is 0.123.
For the oil coefficients, HC0 and HAC standard errors are closer in the ADL model. This does not hold for every coefficient, and HAC standard errors can be smaller than HC0 standard errors.
The second oil-lag estimate falls from 0.0846 in the DL model to 0.0081 in the ADL model. Its HAC standard error changes little, from 0.0114 to 0.0109.
The ADL comparison additionally holds the past two gasoline returns fixed. The second oil lag is no longer statistically significant at 5%. This change mainly reflects the smaller estimated coefficient, rather than greater estimated uncertainty.
The oil return from two weeks ago helps predict the current gasoline return in the DL model. After accounting for the past two gasoline returns, we find little evidence that this older oil return provides additional predictive information.
Residual Autocorrelations
If u_t remains serially correlated, \mathrm{gas}_{t-1} can be correlated with u_t because it contains earlier errors. If this correlation is present, the population moment condition for the ADL model fails.
We apply the sample ACF formula to the OLS residuals. Each bar measures their sample autocorrelation at the indicated lag.
The residual lag-one autocorrelation is 0.359 in the DL model and -0.010 in the ADL model.
The residual ACF is useful for detecting serial correlation. However, small sample autocorrelations do not prove that serial correlation is absent.
We therefore use HAC standard errors as the default for time-series regressions in this course.
Inference with Dependent Observations
The confidence intervals and hypothesis tests from Sections 6 and 7 remain applicable.
We replace the HC0 covariance matrix by the estimator appropriate for the sampling structure.
| Sampling structure | Quantity that must grow | Covariance estimator |
|---|---|---|
| Independent observations | Number of observations n | HC0 |
| Independent clusters | Number of clusters G | CR0 |
| Time series | Number of periods T | HAC |
Under H_0:\theta=\theta_0 and the corresponding large-sample conditions, \frac{\widehat\theta-\theta_0} {\mathrm{se}_{dep}(\widehat\theta)} \overset{d}{\to}\mathcal N(0,1), where \mathrm{se}_{dep} is based on CR0 or HAC.
The Wald test uses the corresponding covariance matrix in exactly the same way.
For example, how would we test whether the two oil lags jointly help predict gasoline returns, holding the past two gasoline returns fixed?
The joint null is H_0:\delta_1=\delta_2=0. We use the ADL model’s HAC covariance matrix in the Wald statistic from Section 7.5.
The regression specification determines how we model the conditional mean. The sampling structure determines how we estimate the uncertainty of the coefficient estimates.