Add Journal to My Library
Journal of Financial Econometrics
, Volume Advance Article – May 24, 2018

49 pages

/lp/ou_press/comparing-predictive-accuracy-under-long-memory-with-an-application-to-XU6aQpDDn8

- Publisher
- Oxford University Press
- ISSN
- 1479-8409
- eISSN
- 1479-8417
- D.O.I.
- 10.1093/jjfinec/nby011
- Publisher site
- See Article on Publisher Site

Abstract This article extends the popular Diebold–Mariano test for equal predictive accuracy to situations when the forecast error loss differential exhibits long memory. This situation can arise frequently since long memory can be transmitted from forecasts and the forecast objective to forecast error loss differentials. The nature of this transmission depends on the (un)biasedness of the forecasts and whether the involved series share common long memory. Further theoretical results show that the conventional Diebold–Mariano test is invalidated under these circumstances. Robust statistics based on a memory and autocorrelation consistent estimator and an extended fixed-bandwidth approach are considered. The subsequent extensive Monte Carlo study provides numerical results on various issues. As empirical applications, we consider recent extensions of the HAR model for the S&P500 realized volatility. While we find that forecasts improve significantly if jumps are considered, improvements achieved by the inclusion of an implied volatility index turn out to be insignificant. 1 Introduction If the accuracy of competing forecasts is to be evaluated in a (pseudo-)out-of-sample setup, it has become standard practice to employ the test of Diebold and Mariano (1995) (hereafter DM test). Let ŷ1t and ŷ2t denote two competing forecasts for the forecast objective series yt and let the loss function be given by g(yt,ŷit)≥0 for i = 1, 2. The forecast error loss differential is then denoted by zt=g(yt,ŷ1t)−g(yt,ŷ2t). (1) By only imposing restrictions on the loss differential zt, instead of the forecast objective and the forecasts, Diebold and Mariano (1995) test the null hypothesis of equal predictive accuracy, that is H0:E(zt)=0, by means of a simple t-statistic for the mean of the loss differentials. To account for serial correlation, a long-run variance estimator such as the heteroscedasticity and autocorrelation consistent (HAC) estimator is applied (see Newey and West (1987), Andrews (1991) and Andrews and Monahan (1992)). For weakly dependent and second-order stationary processes, this leads to an asymptotic standard normal distribution of the t-statistic. Apart from the development of other forecast comparison tests such as those of West (1996) or Giacomini and White (2006), several direct extensions and improvements of the DM test have been proposed. Harvey, Leybourne, and Newbold (1997) suggest a version that corrects for the bias of the long-run variance estimation in finite samples. A multivariate DM test is derived by Mariano and Preve (2012). To mitigate the well-known size issues of HAC-based tests in finite samples of persistent short memory processes, Choi and Kiefer (2010) construct a DM test using the so-called fixed-bandwidth (or in short, fixed-b) asymptotics, originally introduced in Kiefer and Vogelsang (2005) (see also Li and Patton (2018)). The issue of near unit root asymptotics is tackled by Rossi (2005). These studies belong to the classical I(0)/I(1) framework. Contrary to the aforementioned studies, we consider the situation in which the loss differentials follow long memory processes. Our first contribution is to show that long memory can be transmitted from the forecasts and the forecast objective to the forecast errors and subsequently to the forecast error loss differentials. We provide theoretical results for the mean squared error (MSE) loss function and Gaussian processes. We give conditions under which the transmission occurs and characterize the memory properties of the forecast error loss differential. The memory transmission for non-Gaussian processes and other loss functions is demonstrated by means of Monte Carlo simulations resembling typical forecast scenarios. As a second contribution, we show (both theoretically and via simulations) that the original DM test is invalidated under long memory and suffers from severe upward size distortions. Third, we study two simple extensions of the DM statistic that permit valid inference under long and short memory. These extensions are based on the memory and autocorrelation consistent (MAC) estimator of Robinson (2005) (see also Abadir, Distaso, and Giraitis (2009)) and the extended fixed-b asymptotics (EFB) of McElroy and Politis (2012). The performance of these modified statistics is analyzed in a Monte Carlo study that is specifically tailored to reflect the properties that are likely to occur in the loss differentials. We compare several bandwidth and kernel choices that allow recommendations for practical applications. Our fourth contribution is an empirical application in which we reconsider two recent extensions of the heterogeneous autoregressive model for realized volatility (HAR-RV) by Corsi (2009). First, we test whether forecasts obtained from HAR-RV type models can be improved by including information on model-free risk-neutral implied volatility, which is measured by the CBOE volatility index (VIX). We find that short memory approaches (classic DM test and fixed-b versions) reject the null hypothesis of equal predictive accuracy in favor of models including implied volatility. On the contrary, our long memory robust statistics do not indicate a significant improvement in forecast performance which implies that previous rejections might be spurious due to neglected long memory. The second issue we tackle in our empirical applications relates to earlier work by inter alia Andersen, Bollerslev, and Diebold (2007) and Corsi, Pirino, and Renò (2010), who consider the decomposition of the quadratic variation of the log-price process into a continuous integrated volatility component and a discrete jump component. Here, we find that the separate treatment of continuous components and jump components significantly improves forecasts of realized variance for short forecast horizons even if the memory in the loss differentials is accounted for. The rest of this article is organized as follows. Section 2 reviews the classic DM test and presents the fixed-b approach for the short memory case. Section 3 covers the case of long-range dependence and contains our theoretical results on the transmission of long memory to the loss differential series. Two distinct approaches to design a robust t-statistic are discussed in Section 4. Section 5 contains our Monte Carlo study and in Section 6 we present our empirical results. Conclusions are drawn in Section 7. All proofs are contained in the Appendix. 2 DM Test Diebold and Mariano (1995) construct a test for H0:E[g(yt,ŷ1t)−g(yt,ŷ2t)]=E(zt)=0, solely based on assumptions on the loss differential series zt. Suppose that zt follows the weakly stationary linear process zt=μz+∑j=0∞θjvt−j , (2) where it is required that |μz|<∞ and ∑j=0∞θj2<∞ hold. For simplicity of the exposition we additionally assume that vt∼iid(0,σv2). If ŷ1t and ŷ2t perform equally well according to the loss function g(·), then μz=0 holds, otherwise μz≠0. The corresponding t-statistic is based on the sample mean z¯=T−1∑t=1Tzt and an estimate (V̂) of the long-run variance V=limT→∞Var(Tτ(z¯−μz)). The DM statistic is given by tDM=Tτz¯V̂ . (3) Under stationary short memory, we have τ=1/2, while the rate changes to τ=1/2−d under stationary long memory, with 0<d<1/2 being the long memory parameter. The (asymptotic) distribution of this t-statistic hinges on the autocorrelation properties of the loss differential series zt. In the following, we shall distinguish two cases: (i) zt is a stationary short memory process with d = 0 and (ii) strong dependence in form of a long memory process (with 0<d<1/2) is present in zt as presented in Section 3. 2.1 Conventional Approach: HAC For the estimation of the long-run variance V, Diebold and Mariano (1995) suggest using the truncated long-run variance of an MA(h – 1) process for an h-step-ahead forecast. This is motivated by the fact that optimal h-step-ahead forecast errors of a linear time series process follow an MA(h – 1) process. Nevertheless, as pointed out by Diebold (2015), among others, the test is readily extendable to more general situations if, for example, HAC estimators are used (see also Clark (1999) for some early simulation evidence). The latter have become the standard class of estimators for the long-run variance. In particular, V̂HAC=∑j=−T+1T−1k(jB)γ̂z(j) , (4) where k(·) is a user-chosen kernel function, B denotes the bandwidth and γ̂z(j) =1T∑t=|j|+1T(zt−z¯)(zt−|j|−z¯) is the usual estimator for the autocovariance of process zt at lag j. The corresponding DM statistic is given by tHAC=T1/2z¯V̂HAC. (5) If zt is weakly stationary with absolutely summable autocovariances γz(j), it holds that V=∑j=−∞∞γz(j). Regularity conditions assumed1 a central limit theorem applies for z¯. Consistency of the long-run variance estimator V̂HAC requires some additional regularity conditions (see, for instance, Andrews (1991) for additional technical details), and in particular the assumption that the ratio b=B/T converges to zero as T→∞. It follows that the tHAC-statistic is asymptotically standard normal under the null hypothesis, that is tHAC⇒N(0,1). For the sake of a comparable notation to the long memory case, note that V=2πfz(0), where fz(0) is the spectral density function of zt at frequency zero. 2.2 Fixed-bandwidth Approach Even though nowadays the application of HAC estimators is standard practice, related tests are often found to be seriously size-distorted in finite samples, especially under strong persistence. Kiefer and Vogelsang (2005) develop an alternative asymptotic framework in which the ratio B / T approaches a fixed constant b∈(0,1] as T→∞. Therefore, it is called fixed-b inference as opposed to the classical small-b HAC approach where b→0. In the case of fixed-b (FB), the estimator V̂(k,b) does not converge to V any longer. Instead, V̂(k,b) converges to V multiplied by a functional of a Brownian bridge process. Hence, V̂(k,b)⇒VQ(k,b). The corresponding t-statistic tFB=T1/2z¯V̂(k,b) (6) has a nonnormal and nonstandard limiting distribution, that is tFB⇒W(1)Q(k,b) . Here, W(r) is a standard Brownian motion on r∈[0,1]. Both, the choice of the bandwidth parameter b and the (twice continuously differentiable) kernel k appear in the limit distribution. For example, for the Bartlett kernel we have Q(k,b)=2b(∫01W˜(r)2dr−∫01−bW˜(r+b)W˜(r)dr), with W˜(r)=W(r)−rW(1) denoting a standard Brownian bridge. Thus, critical values reflect the user choices on the kernel and the bandwidth even in the limit. In many settings, fixed-b inference is more accurate than the conventional HAC estimation approach. An example of its application to forecast comparisons are the aforementioned articles of Choi and Kiefer (2010) and Li and Patton (2018), who apply both techniques (HAC and fixed-b) to compare exchange rate forecasts. Our Monte Carlo simulation study sheds additional light on their relative empirical performance. 3 Long Memory in Forecast Error Loss Differentials 3.1 Preliminaries Under long-range dependence in zt, one has to expect that neither conventional HAC estimators nor the fixed-b approach can be applied without any further modification since strong dependence such as fractional integration is ruled out by assumption of a weakly stationary linear process. Given that zt has long memory, we show that HAC-based tests reject with probability one in the limit (as T→∞) even under the null. This result is stated in our Proposition 6 (at the end of this section). As our finite-sample simulations clearly demonstrate, this implies strong upward size distortions and invalidates the use of the classic DM test statistic. Before we actually state these results formally, we first show that the loss differential zt may exhibit long memory in various situations. We start with a basic definition of stationary long memory time series, c.f. Definition 1.2 of Beran et al. (2013). Definition 1 A time series at with spectral density fa(λ), for λ∈[−π,π], has long memory with memory parameter da∈(0,1/2), if fa(λ)∼Lf|λ|−2da, for da∈(0,1/2), as λ→0. The symmetric function Lf(·)is slowly varying at the origin. We then write at∼LM(da). This is the usual definition of a stationary long memory process and Theorem 1.3 of Beran et al. (2013) states that under this restriction and mild regularity conditions, Definition 1 is equivalent to γa(j)∼Lγ|j|2da−1 as j→∞, where γa(j) is the autocovariance function of at at lag j and Lγ(·) is slowly varying at infinity. If da = 0 holds, the process has short memory. Our results build on the asymptotic behavior of the autocovariances that have the long memory property from Definition 1. Whether this memory is generated by fractional integration can not be inferred. However, this does not affect the validity of the test statistics introduced in Section 4. We therefore adopt Definition 1 that covers fractional integration as a special case. A similar approach is taken by Dittmann and Granger (2002).2 Given Definition 1, we now state some assumptions regarding the long memory structure of the forecast objective and the forecasts. Assumption 1 (Long Memory). The time series yt, ŷ1t, ŷ2twith expectations E(yt)=μy, E(ŷ1t)=μ1and E(ŷ2t)=μ2are causal Gaussian long memory processes (according to Definition 1) of orders dy, d1,and d2, respectively. Similar to Dittmann and Granger (2002), we rely on the assumption of Gaussianity since no results for the memory structure of squares and cross-products of non-Gaussian long memory processes are available in the existing literature. It shall be noted that Gaussianity is only assumed for the derivation of the memory transmission from the forecasts and the forecast objective to the loss differential, but not for the subsequent results. In the following, we make use of the concept of common long memory in which a linear combination of long memory series has reduced memory. The amount of reduction is labeled as δ. Definition 2 (Common Long Memory). The time series at and bt have common long memory (CLM) if both at and bt are LM(d) and there exists a linear combination ct=at−ψ0−ψ1btwith ψ0∈Rand ψ1∈R\0such that ct∼LM(d−δ), for some d≥δ>0. We write at,bt∼CLM(d,d−δ). For simplicity and ease of exposition, we first exclude the possibility of common long memory among the series. This assumption is relaxed later on. Assumption 2 (Absence of Common Long Memory). If at,bt∼LM(d), then at−ψ0−ψ1bt∼LM(d)for all ψ0∈R,ψ1∈Rand at,bt∈{yt,ŷ1t,ŷ2t}. To derive the long memory properties of the forecast error loss differential, we make use of a result in Leschinski (2017) that characterizes the memory structure of the product series atbt for two long memory time series at and bt. Such products play an important role in the following analysis. The result is therefore shown as Proposition 1 below, for convenience. Proposition 1 (Leschinski (2017), Memory of Products). Let at and bt be long memory series according to Definition 1 with memory parameters da and db, and means μa and μb, respectively. Then atbt∼ {LM(max{da,db}),for μa,μb≠0LM(da),for μa=0,μb≠0LM(db),for μb=0,μa≠0LM(max{da+db−1/2,0}),for μa=μb=0 and Sa,b≠0LM(da+db−1/2),for μa=μb=0 and Sa,b=0, where Sa,b=∑j=−∞∞γa(j)γb(j) with γa(·) and γb(·) denoting the autocovariance functions of at and bt, respectively. Proposition 1 shows that the memory of products of long memory time series critically depends on the means μa and μb of the series at and bt. If both series are mean zero, the memory of the product is either the maximum of the sum of the memory parameters of both factor series minus one half—or it is zero—depending on the sum of autocovariances. Since da,db<1/2, this is always smaller than any of the original memory parameters. If only one of the series is mean zero, the memory of the product atbt is determined by the memory of this particular series. Finally, if both series have nonzero means, the memory of the product is equal to the maximum of the memory orders of the two series. Furthermore, Proposition 1 makes a distinction between antipersistent series and short memory series if the processes have zero means and da+db−1/2<0. Our results below, however, do not require this distinction. The reason being that a linear combination involving the square of at least one of the series appears in each case, and these cannot be antipersistent long memory processes (see the proofs of Propositions 2 and 5 for details). As discussed in Leschinski (2017), Proposition 1 is related to the results in Dittmann and Granger (2002), who consider the memory of nonlinear transformations of zero mean long memory time series that can be represented through a finite sum of Hermite polynomials. Their results include the square at2 of a time series which is also covered by Proposition 1 if at = bt. If the mean is zero ( μa=0), we have at2∼LM(max{2da−1/2,0}). Therefore, the memory is reduced to zero if d≤1/4. However, as can be seen from Proposition 1, this behavior depends critically on the expectation of the series. Since it is the most widely used loss function in practice, we focus on the MSE loss function g(yt,ŷit)=(yt−ŷit)2 for i = 1, 2. The quadratic forecast error loss differential is then given by zt=(yt−ŷ1t)2−(yt−ŷ2t)2=ŷ1t2−ŷ2t2−2yt(ŷ1t−ŷ2t). (7) As usual in DM tests, we do not need to know, or assume the form for, the forecasting models or methods used to generate the forecasts. The forecasts are taken as “primitives” in this analysis. 3.2 Transmission of Long Memory to the Loss Differential Following the introduction of the necessary definitions and a preliminary result, we now present the result for the memory order of zt defined via (7) in Proposition 2. It is based on the memory of yt, ŷ1t and ŷ2t and assumes the absence of common long memory for simplicity. Proposition 2 (Memory Transmission in the Absence of Common Long Memory). Under Assumptions 1 and 2, the forecast error loss differential in (7) is zt∼LM(dz), where dz={max{dy, d1, d2}, if μ1≠μ2≠μymax{d1, d2}, if μ1=μ2≠μymax{2d1−1/2, d2, dy}, if μ1=μy≠μ2max{2d2−1/2, d1, dy}, if μ1≠μy=μ2max{2max{d1, d2}−1/2, dy+max{d1, d2}−1/2, 0}, if μ1=μ2=μy. Proof See the Appendix. The basic idea of the proof relates to Proposition 3 of Chambers (1998). It shows that the behavior of a linear combination of long memory series is dominated by the series with the strongest memory. Since we know from Proposition 1 that μ1,μ2, and μy play an important role for the memory of a squared long memory series, we set yt=yt*+μy and ŷit=ŷit*+μi, so that the starred series denote the demeaned series and μi denotes the expected value of the respective series. Straightforward algebra yields zt=ŷ1t*2−ŷ2t*2−2[yt*(μ1−μ2)+ŷ1t*(μy−μ1)+ŷ2t*(μy−μ2)]−2[yt*(ŷ1t*−ŷ2t*)]+const. (8) From (8), it is apparent that zt is a linear combination of (i) the squared forecasts ŷ1t*2 and ŷ2t*2, (ii) the forecast objective yt, (iii) the forecast series ŷ1t*, and ŷ2t* and (iv) products of the forecast objective with the forecasts, that is yt*ŷ1t* and yt*ŷ2t*. The memory of the squared series and the product series is determined in Proposition 1, from which the zero mean product series yt*ŷit* is LM(max{dy+di−1/2, 0}) or LM(dy+di−1/2). Moreover, the memory of the squared zero mean series ŷit*2 is max{2di−1/2, 0}. By combining these results with that of Chambers (1998), the memory of the loss differential zt is the maximum of all memory parameters of the components in (8). Proposition 2 then follows from a case-by-case analysis. It demonstrates the transmission of long memory from the forecasts ŷ1t, ŷ2t and the forecast objective yt to the loss differential zt. The nature of this transmission, however, critically hinges on the (un)biasedness of the forecasts. If both forecasts are unbiased (i.e., if μ1=μ2=μy), the memory from all three input series is reduced and the memory of the loss differential zt is equal to the maximum of (i) these reduced orders and (ii) zero. Therefore, only if memory parameters are small enough such that dy+max{d1+d2}<1/2, the memory of the loss differential zt is reduced to zero. In all other cases, there is a transmission of dependence from the forecast and/or the forecast objective to the loss differential. The reason for this can immediately be seen from (8). Terms in the first bracket have larger memory than the remaining ones, because di>2di−1/2 and max{dy,di}>dy+di−1/2. Therefore, these terms dominate the memory of the products and squares whenever biasedness is present, that is μi−μy≠0 holds. Interestingly, the transmission of memory from the forecast objective yt is prevented if both forecasts have equal bias, that is μ1=μ2. On the contrary, if μ1≠μ2, dz is at least as high as dy. 3.3 Memory Transmission under Common Long Memory The results in Proposition 2 are based on Assumption 2 that precludes common long memory among the series. Of course, in practice it is likely that such an assumption is violated. In fact, it can be argued that reasonable forecasts of long memory time series should have common long memory with the forecast objective. Therefore, we relax this assumption and replace it with Assumption 3, below. Assumption 3 (Common Long Memory). The causal Gaussian process xt has long memory according to Definition 1 of order dx with expectation E(xt)=μx. If at,bt∼CLM(dx,dx−δ), then they can be represented as yt=βy+ξyxt+ηtfor at,bt=ytand ŷit=βi+ξixt+ɛit, for at,bt=ŷit, with ξy,ξi≠0. Both, ηt and ɛitare mean zero causal Gaussian long memory processes with parameters dηand dɛifulfilling 1/2>dx>dη,dɛi≥0, for i = 1, 2. Assumption 3 restricts the common long memory to be of a form so that both series at and bt can be represented as linear functions of their joint factor xt. This excludes more complicated forms of dependence that are sometimes considered in the cointegration literature such as nonlinear or time-varying cointegration. We know from Proposition 2 that the transmission of memory critically depends on the biasedness of the forecasts which leads to a complicated case-by-case analysis. If common long memory according to Assumption 3 is allowed for, we have an even more complex situation since there are several possible relationships: CLM of yt with one of the ŷit; CLM of both ŷit with each other, but not with yt; and CLM of each ŷit with yt. Each of these situations has to be considered with all possible combinations of the ξj and the μj for all j∈{y,1,2}. To deal with this complexity, we focus on three important special cases: (i) the forecasts are biased and the ξj differ from each other, (ii) the forecasts are biased, but the ξj are equal, and (iii) the forecasts are unbiased and ξa=ξb if at and bt are in a common long memory relationship. To understand the role of the coefficients ξa and ξb in the series that are subject to CLM, note that the forecast errors yt−ŷit impose a cointegrating vector of (1,−1). A different scaling of the forecast objective and the forecasts is not possible. In the case of CLM between yt and ŷit, for example, we have from Assumption 3 that yt−ŷit=βy−βi+xt(ξy−ξi)+ηt−ɛit, (9) so that xt(ξy−ξi) does not disappear from the linear combination if the scaling parameters ξy and ξi are different from each other. We refer to a situation where ξa=ξb as “balanced CLM,” whereas CLM with ξa≠ξb is referred to as “unbalanced CLM.” In the special case (i) both forecasts are biased and the presence of CLM does not lead to a cancellation of the memory of xt in the loss differential. Of course this can be seen as an extreme case, but it serves to illuminate the mechanisms at work—especially in contrast to the results in Propositions 4 and 5, below. By substituting the linear relations from Assumption 3 for those series involved in the CLM relationship in the loss differential zt=ŷ1t2−ŷ2t2−2yt(ŷ1t−ŷ2t) and again setting at=at*+μa for those series that are not involved in the CLM relationship, it is possible to find expressions that are analogous to (8). Since analogous terms to those in the first bracket of (8) appear in each case, it is possible to focus on the transmission of memory from the forecasts and the objective function to the loss differential. We obtain the following result. Proposition 3 (Memory Transmission with Biased Forecasts and Unbalanced CLM). Let ξi≠ξy, ξ1≠ξ2, μi≠μy, and μ1≠μ2, for i = 1, 2. Then under Assumptions 1 and 3, the forecast error loss differential in (7) is zt∼LM(dz), where dz={max{dy,dx}, if ŷ1t,ŷ2t∼CLM(dx,dx−δ), except if ξ1/ξ2=(μy−μ2)/(μy−μ1)max{d2,dx}, if ŷ1t,yt∼CLM(dx,dx−δ), except if ξ1/ξy=−(μ1−μ2)/(μy−μ1)max{d1,dx}, if ŷ2t,yt∼CLM(dx,dx−δ), except if ξ2/ξy=−(μ1−μ2)/(μy−μ2)dx, if ŷ1t,ŷ2t,yt∼CLM(dx,dx−δ), except if ξ1(μy−μ1)+ξy(μ1−μ2)=ξ2(μy−μ2). Proof See the Appendix. In absence of common long memory we observe in Proposition 1 that the memory is given as max{d1,d2,dy} if the means differ from each other. Now, if two of the series share common long memory, they both have memory dx. Hence, Proposition 3 shows that the transmission mechanism is essentially unchanged and the memory of the loss differential is still dominated by the largest memory parameter. The only exception to this rule is the knife-edge case where the differences in the means and the memory parameters offset each other. Similar to (i), case (ii) refers to a situation of biasedness, but now with balanced CLM, so that the underlying long memory factor xt cancels out in the forecast error loss differentials. The memory transmission can thus be characterized by the following proposition. Proposition 4 (Memory Transmission with Biased Forecasts and Balanced CLM). Let ξ1=ξ2=ξy. Then under Assumptions 1 and 3, the forecast error loss differential in (7) is zt∼LM(dz), where dz={max{dy,dx}, if ŷ1t,ŷ2t∼CLM(dx,dx−δ), and μ1≠μ2max{d2,dx}, if ŷ1t,yt∼CLM(dx,dx−δ), and μy≠μ2max{d1,dx}, if ŷ2t,yt∼CLM(dx,dx−δ), and μy≠μ1d˜, if ŷ1t,ŷ2t,yt∼CLM(dx,dx−δ),for some 0≤d˜<dx. Proof See the Appendix. We refer to the first three cases in Propositions 3 and 4 as “partial CLM” as there is always one of the ŷit or yt that is not part of the CLM relationship and the fourth case as “full CLM.” We can observe that the dominance of the memory of the most persistent series under partial CLM is preserved for both balanced and unbalanced CLM. We therefore conclude that this effect is generated by the interaction with the series that is not involved in the CLM relationship. This can also be seen from Equations (22) to (24) in the proof. Only in the fourth case with full CLM, the memory transmission changes between Propositions 3 and 4. In this case, the memory in the loss differential is reduced to dz < dx. The third special case (iii) refers to a situation of unbiasedness similar to the last case in Proposition 2. In addition to that, it is assumed that there is balanced CLM as in Proposition 4, where ξa=ξb if at and bt are in a common long memory relationship. Compared to the setting of the previous propositions this is the most ideal situation in terms of forecast accuracy. Here, we have the following result. Proposition 5 (Memory Transmission with Unbiased Forecasts and Balanced CLM). Under Assumptions 1 and 3, and if μy=μ1=μ2and ξy=ξa=ξb, then zt∼LM(dz), with dz={max{d2+max{dx,dη}−1/2, 2max{dx,d2}−1/2, dɛ1},if yt,ŷ1t∼CLM(dx,dx−δ˜)max{d1+max{dx,dη}−1/2, 2max{dx,d1}−1/2, dɛ2},if yt,ŷ2t∼CLM(dx,dx−δ˜)max{max{dx, dy}+max{dɛ1, dɛ2}−1/2, 0},if ŷ1t,ŷ2t∼CLM(dx,dx−δ˜)max{dη+max{dɛ1, dɛ2}−1/2, 2max{dɛ1, dɛ2}−1/2, 0},if yt,ŷ1t∼CLM(dx,dx−δ˜)and yt,ŷ2t∼CLM(dx,dx−δ˜).Here, 0<δ˜≤1/2denotes a generic constant for the reduction in memory. Proof See the Appendix. Proposition 5 shows that the memory of the forecasts and the objective variable can indeed cancel out if the forecasts are unbiased and if they have the same factor loading on xt (i.e., if ξ1=ξ2=ξy). However, in the first two cases, the memory of the error series ɛ1t and ɛ2t imposes a lower bound on the memory of the loss differential. Furthermore, even though the memory can be reduced to zero in the third and fourth case, this situation only occurs if the memory orders of xt, yt and the error series are sufficiently small. Otherwise, the memory is reduced, but does not vanish. Overall, the results in Propositions 2, 3, 4, and 5 show that long memory can be transmitted from forecasts or the forecast objective to the forecast error loss differentials. Our results also show that the biasedness of the forecasts plays an important role for the transmission of dependence to the loss differentials. To get further insights into the mechanisms found in Propositions 2, 3, 4, and 5, let us consider a situation in which two forecasts with different nonzero biases are compared. In the absence of CLM, it is obvious from Proposition 2 that the memory of the loss differential is determined by the maximum of the memory orders of the forecasts and the forecast objective. If one of the forecasts has common long memory with the objective, the same holds true—irrespective of the loadings ξa on the common factor. As can be seen from Proposition 3, even if both forecasts have CLM with the objective, the maximal memory order is transmitted to zt if the factor loadings ξa differ. Only if the factor loadings are equal, the memory is reduced as stated in Proposition 4. If we consider two forecasts that are unbiased in the absence of CLM, it can be seen from Proposition 2 that the memory of the loss differential is lower than that of the original series. The same holds true in the presence of CLM, as covered by Proposition 5. In practical situations, it might be overly restrictive to impose exact unbiasedness (under which memory would be reduced according to Proposition 5). Our empirical application regarding the predictive ability of the VIX serves as an example since it is a biased forecast of future quadratic variation due to the existence of a variance risk premium (see Section 6). Biases can also be caused by estimation errors. This issue might be of less importance in a setup where the estimation period grows at a faster rate than the (pseudo-) out-of-sample period that is used for forecast evaluation. For the DM test, however, it is usually assumed that this is not the case. Otherwise, it could not be used for the comparison of forecasts from nested models due to a degenerated limiting distribution (cf. Giacomini and White (2006) for a discussion). Instead, the sample of size T* is split into an estimation period TE and a forecasting period T such that T*=TE+T and it is assumed that T grows at a faster rate than TE so that TE/T→0 as T*→∞. Therefore, the estimation error shrinks at a lower rate than the growth rate of the evaluation period and it remains relevant, asymptotically. 3.4 Asymptotic and Finite-Sample Behaviour under Long Memory After establishing that forecast error loss differentials may exhibit long memory in various situations, we now consider the effect of long memory on the HAC-based DM test. The following Proposition establishes that the size of the test approaches unity, as T→∞. Thus, the test indicates with probability one that one of the forecasts is superior to the other one, even if both tests perform equally well according to g(·). Note that the test also has an asymptotic rejection probability of one under the alternative. Proposition 6 (DM under Long Memory). For zt∼LM(d)with d∈(0,1/2), the asymptotic size (under H0) of the tHAC-statistic equals unity as T→∞. Proof See the Appendix. This result shows that inference based on HAC estimators is asymptotically invalid under long memory. To explore to what extent this finding also affects the finite-sample performance of the tHAC- and tFB-statistics, we conduct a small-scale Monte Carlo experiment as an illustration. The results shown in Figure 1 are obtained with M = 5000 Monte Carlo repetitions. We simulate samples of T = 50 and T = 2000 observations from a fractionally integrated process using different values of the memory parameter d in the range from 0 to 0.4. The HAC estimator and the fixed-b approach are implemented with the commonly used Bartlett- and Quadratic Spectral (QS) kernels.3 Figure 1. View largeDownload slide Size of the tHAC- and tFB-tests with T∈{50,2000} for different values of the memory parameter d. Figure 1. View largeDownload slide Size of the tHAC- and tFB-tests with T∈{50,2000} for different values of the memory parameter d. We start by commenting on the results for the small sample size of T = 50 in the left panel of Figure 1. As demonstrated by Kiefer and Vogelsang (2005), the fixed-b approach works exceptionally well for the short memory case of d = 0, with the Bartlett and QS kernel achieving approximately equal size control. The tHAC-statistic over-rejects more than the fixed-b approach and, as stated in Andrews (1991), better size control is provided if the Quadratic Spectral kernel is used. If the memory parameter d is positive, we observe that all tests severely over-reject the null hypothesis. For d = 0.4, the size of the HAC-based test is approximately 65% and that of the fixed-b version using the Bartlett kernel is around 40%. We therefore find that the size distortions are not only an asymptotic phenomenon, but they are already severe in samples of just T = 50 observations. Moreover, even for small deviations of d from zero, all tests are over-sized. These findings motivate the use of long memory robust procedures. Continuing with the results for T = 2000 in the right panel of Figure 1, we observe similar findings in general. For the short memory case, size distortions observed in small samples vanish. All tests statistics are well behaved for d = 0. On the contrary, for d > 0 size distortions are stronger compared to T = 50, although the magnitude of the additional distortion is moderate. This feature can be attributed to the slow divergence rate (as given in the proof of Proposition 6) of the test statistic under long memory. 4 Long-Run Variance Estimation under Long Memory Since conventional HAC estimators lead to spurious rejections under long memory, we consider memory robust long-run variance estimators. To the best of our knowledge only two extensions of this kind are available in the literature: the MAC estimator of Robinson (2005) and an extension of the fixed-b estimator from McElroy and Politis (2012). We do not assume that forecasts are obtained from some specific class of model. We merely extend the typical assumptions of Diebold and Mariano (1995) on the loss differentials so that long memory is allowed. 4.1 MAC Estimator The MAC estimator is developed by Robinson (2005) and further explored and extended by Abadir, Distaso, and Giraitis (2009). Albeit stated in a somewhat different form, the same result is derived independently by Phillips and Kim (2007), who consider the long-run variance of a multivariate fractionally integrated process. Robinson (2005) assumes that zt is linear (in the sense of our Equation (1), see also Assumption L in Abadir, Distaso, and Giraitis (2009)) and that for λ→0 its spectral density fulfills f(λ)=b0|λ|−2d+o(|λ|−2d), with b0>0, |λ|≤π, d∈(−1/2,1/2) and b0=limλ→0|λ|2df(λ). Among others, this assumption covers stationary and invertible ARFIMA processes. For notational convenience, here we drop the index z from the spectral density and the memory parameter. A key result for the MAC estimator is that as T→∞ Var(T1/2−dz¯)→b0p(d) with p(d)={2Γ(1−2d)sin(πd)d(1+2d)if d=0,2πif d=0. The case of short memory (d = 0) yields the familiar result that the long-run variance of the sample mean equals 2πb0=2πf(0). Hence, estimation of the long-run variance requires estimation of f(0) in the case of short memory. If long memory is present in the data generating process (DGP), estimation of the long-run variance additionally hinges on the estimation of d. The MAC estimator is therefore given by V̂(d̂,md,m)=b̂m(d̂)p(d̂) . In more detail, the estimation of V works as follows: First, if the estimator for d fulfills the condition d̂−d=op(1/logT), plug-in estimation is valid (cf. Abadir, Distaso, and Giraitis (2009)). Thus, p(d) can simply be estimated through p(d̂). A popular estimator that fulfills this rather weak requirement is the local Whittle estimator with bandwidth md=⌊Tqd⌋, where 0<qd<1 denotes a generic bandwidth parameter and ⌊·⌋ denotes the largest integer smaller than its argument. This estimator is given by d̂LW=argmind∈(−1/2,1/2)RLW(d), where RLW(d)=log(1md∑j=1mdj2dIT(λj))−2dmd∑j=1mdlogj, IT(λj) is the periodogram (which is independent of d̂), IT(λj)=(2πT)−1|∑t=1Texp(itλj)zt|2 and the λj=2πj/T are the Fourier frequencies for j=1,...,⌊T/2⌋. Many other estimation approaches (e.g., log-periodogram estimation, etc.) would be a possibility as well. Since the loss differential in (7) is a linear combination of processes with different memory orders, the local polynomial Whittle plus noise (LPWN) estimator of Frederiksen, Nielsen, and Nielsen (2012) is a particularly useful alternative. This estimator extends the local Whittle estimator by approximating the log-spectrum of possible short memory components and perturbation terms in the vicinity of the origin by polynomials. This leads to a reduction of finite-sample bias. The estimator is consistent for d∈(0,1) and asymptotically normal in the presence of perturbations for d∈(0,0.75), but with the variance inflated by a multiplicative constant compared with the local Whittle estimator. Based on a consistent estimator d̂, as those discussed above, b0 can be estimated consistently by b̂m(d̂)=m−1∑j=1mλj2d̂IT(λj). The bandwidth m is determined according to m=⌊Tq⌋ such that m→∞ and m=o(T/(logT)2). The MAC estimator is consistent as long as d̂→pd and b̂m(d̂)→pb0. These results hold under very weak assumptions—neither linearity of zt nor Gaussianity are required. Under somewhat stronger assumptions the tMAC-statistic is also normal distributed (see Theorem 3.1. of Abadir, Distaso, and Giraitis (2009)): tMAC⇒N(0,1) . The t-statistic using the feasible MAC estimator can be written as tMAC=T1/2−d̂z¯V̂(d̂,md,m), with md and m being the bandwidths for estimation of d and b0, respectively.4 4.2 Extended Fixed-Bandwidth Approach Following up on the work by Kiefer and Vogelsang (2005), McElroy and Politis (2012) extend the fixed-bandwidth approach to long-range dependence. Their approach is similar to the one of Kiefer and Vogelsang (2005) in many respects, as can be seen below. The test statistic suggested by McElroy and Politis (2012) is given by tEFB=T1/2z¯V̂(k,b). In contrast to the tMAC-statistic, the tEFB-statistic involves a scaling of T1/2. This has an effect on the limit distribution, which depends on the memory parameter d. Analogously to the short memory case, the limiting distribution is derived by assuming that a functional central limit theorem for the partial sums of zt applies, so that tEFB⇒Wd(1)Q(k,b,d), where Wd(r) is a fractional Brownian motion and Q(k,b,d) depends on the fractional Brownian bridge W˜d(r)=Wd(r)−rWd(1). Furthermore, Q(k,b,d) depends on the first and second derivatives of the kernel k(·). In more detail, for the Bartlett kernel we have Q(k,b,d)=2b(∫01W˜d(r)2dr−∫01−bW˜d(r+b)W˜d(r)dr) and thus, a similar structure as for the short memory case. Further details and examples can be found in McElroy and Politis (2012). The joint distribution of Wd(1) and Q(k,b,d) is found through their joint Fourier-Laplace transformation, see Fitzsimmons and McElroy (2010). It is symmetric around zero and has a cumulative distribution function which is continuous in d. Besides the similarities to the short memory case, there are some important conceptual differences to the MAC estimator. First, the MAC estimator belongs to the class of “small-b” estimators in the sense that it estimates the long-run variance directly, whereas the fixed-b approach leads also in the long memory case to an estimate of the long-run variance multiplied by a functional of a fractional Brownian bridge. Second, the limiting distribution of the tEFB-statistic is not a standard normal, but rather depending on the chosen kernel k, the fixed-bandwidth parameter b, and the long memory parameter d. While the first two are user-specific, the latter one requires a plug-in estimator, as does the MAC estimator. As a consequence, the critical values are depending on d.McElroy and Politis (2012) offer response curves for various kernels.5 5 Monte Carlo Study This section presents further results on memory transmission to the forecast error loss differentials and the relative performance of the tMAC and tEFB-statistics by means of extensive Monte Carlo simulations. It is divided into three parts. First, we conduct Monte Carlo experiments to verify the results obtained in Propositions 2–5 and to explore whether similar results apply for non-Gaussian processes and under the QLIKE loss function. The second part studies the memory properties of the loss differential in a number of empirically motivated forecasting scenarios. Finally, in the third part we explore the finite-sample size and power properties of the robustified tests discussed above and make recommendations for their practical application. 5.1 Memory Transmission to the Forecast Error Loss Differentials: Beyond MSE and Gaussianity The results on the transmission of long memory from the forecasts or the forecast objective to the loss differentials in Propositions 2–5 are restricted to stationary Gaussian processes and forecasts evaluated using MSE as a loss function. In this section, we first verify the validity of the predictions from our propositions. Furthermore, we study how these results translate to non-Gaussian processes, nonstationary processes, and the QLIKE loss function which we use in our empirical application in Section 6 on volatility forecasting. It is given by QLIKE(ŷit,yt)=logŷit+ytŷit . (10) For a discussion of the role and importance of this loss function in the evaluation of volatility forecasts see Patton (2011). All DGPs are based on fractional integration. Due to the large number of cases in Propositions 2–5, we restrict ourselves to representative situations. The first two DGPs are based on cases (i) and (v) in Proposition 2 that covers situations when the forecasts and the forecast objective are generated from a system without common long memory. We simulate processes of the form at=μa+at*σ̂a*, (11) where at∈{yt,ŷ1t,ŷ2t}, and at*=(1−L)−daɛat. As in Section 3, the starred variable at* is a zero-mean process, whereas at has mean μa and the ɛat are iid. The innovation sequences are either standard normal or t(5)-distributed. The standardization of at* neutralizes the effect of increasing values of the memory parameter d on the process variance and controls the scaling of the mean relative to the variance. The loss differential series zt is then calculated as in (1). We use 5000 Monte Carlo replications and consider sample sizes of T={250,2000}. The first two DPGs for zt are obtained by setting the means μa in (11) as follows DGP1: (μ1,μ2,μy)=(1,−1,0)DGP2: (μ1,μ2,μy)=(0,0,0). The other DGPs represent the last cases of Propositions 3–5. These are based on the fractionally cointegrated system (yt*ŷ1t*ŷ2t*xt)=(100ξy010ξ1001ξ20001)(ηtɛ1tɛ2txt), where ηt, ɛ1t, ɛ2t, and xt are mutually independent and fractionally integrated with parameters dη, dɛ1, dɛ2, and dx. DGPs 3 to 5 are then obtained by selecting the following parameter constellations: DGP3: (μ1,μ2,μy,ξ1,ξ2,ξy)=(1,−1,0,1,2,1.5)DGP4: (μ1,μ2,μy,ξ1,ξ2,ξy)=(1,−1,0,1,1,1)DGP5: (μ1,μ2,μy,ξ1,ξ2,ξy)=(0,0,0,1,1,1). Each of our DGPs 2 to 5 is formulated such that the reduction in the memory parameter is the strongest among all cases covered in the respective proposition. Simulation results for other cases would therefore show an even stronger transmission of memory to the loss differentials. Since the QLIKE criterion is only defined for nonnegative forecasts, we consider a long memory stochastic volatility specification if QLIKE is used and simulate forecasts and forecast objective of the form exp(at/2), whereas the MSE is calculated directly for the at. It should be noted that the loss differential zt is a linear combination of several persistent and antipersistent component series. This is a very challenging setup for the empirical estimation of the memory parameter. We therefore resort to the aforementioned LPWN estimator of Frederiksen, Nielsen, and Nielsen (2012) with a bandwidth of md=⌊T0.65⌋ and a polynomial of degree one for the noise term that can be expected to have the lowest bias in this setup among the available methods to estimate the memory parameters. However, the estimation remains difficult and any mismatch between the theoretical predictions from our propositions and the finite-sample results reported here is likely to be due to the finite-sample bias of the semiparametric estimators. The results for DGPs 1 and 2 are given in Table 1. We start with the discussion of simulation results for cases covered by our theoretical results. Table 1 shows the results for DGPs 1 and 2. Under MSE loss, and with Gaussian innovations, Proposition 2 states that for DGP1 we have dz=0.25 if d1,d2∈{0,0.2} and dz=0.4 if either d1 or d2 is equal to 0.4, in the top left panel. In the bottom left panel results for DGP2 are reported. Proposition 2 states that dz = 0 if d1,d2∈{0,0.2} and dz=0.3, for d1=0.4 or d2=0.4. We can observe that the memory tends to be slightly larger than predicted for small d1 and d2 and it tends to be slightly smaller for dz=0.4. However, the results closely mirror the theoretical results from Proposition 2 in general. Table 1. Monte Carlo averages of estimated memory in the loss differential zt for DGP1 and DGP2 with dy=0.25 MSE QLIKE Gaussian t(5) Gaussian t(5) DGP T d1/d2 0 0.2 0.4 0.6 0 0.2 0.4 0.6 0 0.2 0.4 0.6 0 0.2 0.4 0.6 1 250 0 0.32 0.32 0.35 0.43 0.30 0.31 0.34 0.41 0.29 0.32 0.38 0.44 0.22 0.25 0.32 0.41 0.2 0.33 0.34 0.36 0.43 0.31 0.32 0.34 0.42 0.30 0.31 0.37 0.45 0.23 0.26 0.32 0.40 0.4 0.36 0.36 0.38 0.45 0.33 0.34 0.37 0.43 0.31 0.32 0.37 0.45 0.24 0.27 0.33 0.41 0.6 0.43 0.43 0.45 0.50 0.42 0.41 0.44 0.49 0.36 0.37 0.41 0.48 0.29 0.31 0.36 0.44 2000 0 0.30 0.29 0.32 0.42 0.29 0.28 0.31 0.42 0.29 0.28 0.35 0.46 0.18 0.20 0.28 0.40 0.2 0.29 0.29 0.32 0.41 0.28 0.28 0.31 0.41 0.29 0.28 0.35 0.46 0.18 0.20 0.27 0.40 0.4 0.32 0.32 0.35 0.42 0.32 0.31 0.34 0.41 0.29 0.28 0.35 0.46 0.20 0.21 0.28 0.41 0.6 0.42 0.42 0.43 0.48 0.42 0.41 0.42 0.47 0.35 0.34 0.38 0.48 0.26 0.25 0.31 0.44 2 250 0 0.13 0.15 0.26 0.43 0.10 0.14 0.23 0.41 0.11 0.14 0.26 0.41 0.10 0.13 0.20 0.35 0.2 0.15 0.17 0.26 0.43 0.14 0.17 0.24 0.41 0.15 0.18 0.27 0.41 0.12 0.15 0.21 0.35 0.4 0.26 0.27 0.31 0.43 0.23 0.24 0.29 0.42 0.25 0.27 0.31 0.41 0.20 0.21 0.25 0.36 0.6 0.42 0.42 0.43 0.48 0.41 0.40 0.42 0.48 0.41 0.40 0.41 0.47 0.35 0.35 0.36 0.43 2000 0 0.07 0.11 0.23 0.43 0.07 0.09 0.21 0.41 0.06 0.13 0.27 0.41 0.05 0.08 0.15 0.32 0.2 0.11 0.13 0.23 0.42 0.09 0.11 0.20 0.40 0.13 0.17 0.26 0.40 0.07 0.09 0.17 0.31 0.4 0.23 0.23 0.25 0.41 0.21 0.21 0.24 0.39 0.27 0.26 0.29 0.40 0.16 0.17 0.22 0.33 0.6 0.43 0.42 0.41 0.46 0.41 0.40 0.39 0.45 0.41 0.41 0.40 0.46 0.32 0.31 0.33 0.41 MSE QLIKE Gaussian t(5) Gaussian t(5) DGP T d1/d2 0 0.2 0.4 0.6 0 0.2 0.4 0.6 0 0.2 0.4 0.6 0 0.2 0.4 0.6 1 250 0 0.32 0.32 0.35 0.43 0.30 0.31 0.34 0.41 0.29 0.32 0.38 0.44 0.22 0.25 0.32 0.41 0.2 0.33 0.34 0.36 0.43 0.31 0.32 0.34 0.42 0.30 0.31 0.37 0.45 0.23 0.26 0.32 0.40 0.4 0.36 0.36 0.38 0.45 0.33 0.34 0.37 0.43 0.31 0.32 0.37 0.45 0.24 0.27 0.33 0.41 0.6 0.43 0.43 0.45 0.50 0.42 0.41 0.44 0.49 0.36 0.37 0.41 0.48 0.29 0.31 0.36 0.44 2000 0 0.30 0.29 0.32 0.42 0.29 0.28 0.31 0.42 0.29 0.28 0.35 0.46 0.18 0.20 0.28 0.40 0.2 0.29 0.29 0.32 0.41 0.28 0.28 0.31 0.41 0.29 0.28 0.35 0.46 0.18 0.20 0.27 0.40 0.4 0.32 0.32 0.35 0.42 0.32 0.31 0.34 0.41 0.29 0.28 0.35 0.46 0.20 0.21 0.28 0.41 0.6 0.42 0.42 0.43 0.48 0.42 0.41 0.42 0.47 0.35 0.34 0.38 0.48 0.26 0.25 0.31 0.44 2 250 0 0.13 0.15 0.26 0.43 0.10 0.14 0.23 0.41 0.11 0.14 0.26 0.41 0.10 0.13 0.20 0.35 0.2 0.15 0.17 0.26 0.43 0.14 0.17 0.24 0.41 0.15 0.18 0.27 0.41 0.12 0.15 0.21 0.35 0.4 0.26 0.27 0.31 0.43 0.23 0.24 0.29 0.42 0.25 0.27 0.31 0.41 0.20 0.21 0.25 0.36 0.6 0.42 0.42 0.43 0.48 0.41 0.40 0.42 0.48 0.41 0.40 0.41 0.47 0.35 0.35 0.36 0.43 2000 0 0.07 0.11 0.23 0.43 0.07 0.09 0.21 0.41 0.06 0.13 0.27 0.41 0.05 0.08 0.15 0.32 0.2 0.11 0.13 0.23 0.42 0.09 0.11 0.20 0.40 0.13 0.17 0.26 0.40 0.07 0.09 0.17 0.31 0.4 0.23 0.23 0.25 0.41 0.21 0.21 0.24 0.39 0.27 0.26 0.29 0.40 0.16 0.17 0.22 0.33 0.6 0.43 0.42 0.41 0.46 0.41 0.40 0.39 0.45 0.41 0.41 0.40 0.46 0.32 0.31 0.33 0.41 Table 1. Monte Carlo averages of estimated memory in the loss differential zt for DGP1 and DGP2 with dy=0.25 MSE QLIKE Gaussian t(5) Gaussian t(5) DGP T d1/d2 0 0.2 0.4 0.6 0 0.2 0.4 0.6 0 0.2 0.4 0.6 0 0.2 0.4 0.6 1 250 0 0.32 0.32 0.35 0.43 0.30 0.31 0.34 0.41 0.29 0.32 0.38 0.44 0.22 0.25 0.32 0.41 0.2 0.33 0.34 0.36 0.43 0.31 0.32 0.34 0.42 0.30 0.31 0.37 0.45 0.23 0.26 0.32 0.40 0.4 0.36 0.36 0.38 0.45 0.33 0.34 0.37 0.43 0.31 0.32 0.37 0.45 0.24 0.27 0.33 0.41 0.6 0.43 0.43 0.45 0.50 0.42 0.41 0.44 0.49 0.36 0.37 0.41 0.48 0.29 0.31 0.36 0.44 2000 0 0.30 0.29 0.32 0.42 0.29 0.28 0.31 0.42 0.29 0.28 0.35 0.46 0.18 0.20 0.28 0.40 0.2 0.29 0.29 0.32 0.41 0.28 0.28 0.31 0.41 0.29 0.28 0.35 0.46 0.18 0.20 0.27 0.40 0.4 0.32 0.32 0.35 0.42 0.32 0.31 0.34 0.41 0.29 0.28 0.35 0.46 0.20 0.21 0.28 0.41 0.6 0.42 0.42 0.43 0.48 0.42 0.41 0.42 0.47 0.35 0.34 0.38 0.48 0.26 0.25 0.31 0.44 2 250 0 0.13 0.15 0.26 0.43 0.10 0.14 0.23 0.41 0.11 0.14 0.26 0.41 0.10 0.13 0.20 0.35 0.2 0.15 0.17 0.26 0.43 0.14 0.17 0.24 0.41 0.15 0.18 0.27 0.41 0.12 0.15 0.21 0.35 0.4 0.26 0.27 0.31 0.43 0.23 0.24 0.29 0.42 0.25 0.27 0.31 0.41 0.20 0.21 0.25 0.36 0.6 0.42 0.42 0.43 0.48 0.41 0.40 0.42 0.48 0.41 0.40 0.41 0.47 0.35 0.35 0.36 0.43 2000 0 0.07 0.11 0.23 0.43 0.07 0.09 0.21 0.41 0.06 0.13 0.27 0.41 0.05 0.08 0.15 0.32 0.2 0.11 0.13 0.23 0.42 0.09 0.11 0.20 0.40 0.13 0.17 0.26 0.40 0.07 0.09 0.17 0.31 0.4 0.23 0.23 0.25 0.41 0.21 0.21 0.24 0.39 0.27 0.26 0.29 0.40 0.16 0.17 0.22 0.33 0.6 0.43 0.42 0.41 0.46 0.41 0.40 0.39 0.45 0.41 0.41 0.40 0.46 0.32 0.31 0.33 0.41 MSE QLIKE Gaussian t(5) Gaussian t(5) DGP T d1/d2 0 0.2 0.4 0.6 0 0.2 0.4 0.6 0 0.2 0.4 0.6 0 0.2 0.4 0.6 1 250 0 0.32 0.32 0.35 0.43 0.30 0.31 0.34 0.41 0.29 0.32 0.38 0.44 0.22 0.25 0.32 0.41 0.2 0.33 0.34 0.36 0.43 0.31 0.32 0.34 0.42 0.30 0.31 0.37 0.45 0.23 0.26 0.32 0.40 0.4 0.36 0.36 0.38 0.45 0.33 0.34 0.37 0.43 0.31 0.32 0.37 0.45 0.24 0.27 0.33 0.41 0.6 0.43 0.43 0.45 0.50 0.42 0.41 0.44 0.49 0.36 0.37 0.41 0.48 0.29 0.31 0.36 0.44 2000 0 0.30 0.29 0.32 0.42 0.29 0.28 0.31 0.42 0.29 0.28 0.35 0.46 0.18 0.20 0.28 0.40 0.2 0.29 0.29 0.32 0.41 0.28 0.28 0.31 0.41 0.29 0.28 0.35 0.46 0.18 0.20 0.27 0.40 0.4 0.32 0.32 0.35 0.42 0.32 0.31 0.34 0.41 0.29 0.28 0.35 0.46 0.20 0.21 0.28 0.41 0.6 0.42 0.42 0.43 0.48 0.42 0.41 0.42 0.47 0.35 0.34 0.38 0.48 0.26 0.25 0.31 0.44 2 250 0 0.13 0.15 0.26 0.43 0.10 0.14 0.23 0.41 0.11 0.14 0.26 0.41 0.10 0.13 0.20 0.35 0.2 0.15 0.17 0.26 0.43 0.14 0.17 0.24 0.41 0.15 0.18 0.27 0.41 0.12 0.15 0.21 0.35 0.4 0.26 0.27 0.31 0.43 0.23 0.24 0.29 0.42 0.25 0.27 0.31 0.41 0.20 0.21 0.25 0.36 0.6 0.42 0.42 0.43 0.48 0.41 0.40 0.42 0.48 0.41 0.40 0.41 0.47 0.35 0.35 0.36 0.43 2000 0 0.07 0.11 0.23 0.43 0.07 0.09 0.21 0.41 0.06 0.13 0.27 0.41 0.05 0.08 0.15 0.32 0.2 0.11 0.13 0.23 0.42 0.09 0.11 0.20 0.40 0.13 0.17 0.26 0.40 0.07 0.09 0.17 0.31 0.4 0.23 0.23 0.25 0.41 0.21 0.21 0.24 0.39 0.27 0.26 0.29 0.40 0.16 0.17 0.22 0.33 0.6 0.43 0.42 0.41 0.46 0.41 0.40 0.39 0.45 0.41 0.41 0.40 0.46 0.32 0.31 0.33 0.41 With regard to the cases not covered by the theoretical derivations, we can observe that the results for t-distributed innovations are nearly identical to those obtained for the Gaussian distribution. The same holds true for the Gaussian long memory stochastic volatility model and the QLIKE loss function. If the innovations of the LMSV model are t-distributed, the memory in the loss differential is slightly lower, but still substantial. Finally, in presence of nonstationary long memory with d1 or d2 equal to 0.6, we can observe that the loss differential exhibits long memory with an estimated degree between 0.4 and 0.5. The only exception is when the QLIKE loss function is used for DGP1. Here, we observe some asymmetry in the results, in the sense that the estimated memory parameter of the loss differential is slightly lower if d2 is low, relative to d1. However, the memory transmission is still substantial. The results for DGP3 to DGP5, where forecasts and the forecast objective have common long memory, are shown in Table 2. If we again consider the left column that displays the results for MSE loss and Gaussian innovations, Proposition 3 states for the case of DGP3 that the memory for all d in the stationary range should be dx=0.45. Proposition 4 does not give an exact prediction for DGP4, but states that the memory in the loss differential should be reduced compared with DGP3. Finally, for DGP5, Proposition 5 implies that dz = 0, for d1,d2∈{0,0.2} and dz=0.3 if d1 or d2 equal 0.4. Table 2. Monte Carlo averages of estimated memory in the loss differential zt for DGP3, DGP4, and DGP5 with dη=0.2 MSE QLIKE Gaussian t(5) Gaussian t(5) DGP T dɛ1/dɛ2 0 0.2 0.4 0 0.2 0.4 0 0.2 0.4 0 0.2 0.4 3 250 0 0.31 0.32 0.38 0.29 0.31 0.37 0.29 0.33 0.40 0.27 0.31 0.39 0.2 0.34 0.35 0.39 0.33 0.34 0.38 0.32 0.33 0.41 0.29 0.32 0.39 0.4 0.41 0.42 0.44 0.40 0.40 0.43 0.37 0.38 0.43 0.36 0.37 0.42 2000 0 0.34 0.32 0.38 0.34 0.32 0.38 0.32 0.30 0.39 0.31 0.29 0.38 0.2 0.30 0.30 0.36 0.30 0.30 0.35 0.30 0.29 0.38 0.29 0.29 0.37 0.4 0.39 0.39 0.41 0.38 0.38 0.40 0.37 0.36 0.40 0.36 0.35 0.39 4 250 0 0.29 0.31 0.35 0.27 0.30 0.35 0.28 0.30 0.37 0.24 0.27 0.35 0.2 0.30 0.32 0.36 0.29 0.31 0.35 0.28 0.30 0.38 0.25 0.28 0.35 0.4 0.35 0.36 0.39 0.34 0.35 0.38 0.31 0.32 0.39 0.27 0.30 0.37 2000 0 0.26 0.26 0.33 0.25 0.26 0.33 0.25 0.25 0.35 0.23 0.24 0.33 0.2 0.26 0.26 0.33 0.26 0.25 0.32 0.26 0.26 0.35 0.24 0.24 0.33 0.4 0.33 0.33 0.36 0.33 0.32 0.36 0.29 0.29 0.36 0.27 0.27 0.34 5 250 0 0.12 0.14 0.25 0.11 0.13 0.23 0.10 0.13 0.25 0.10 0.12 0.21 0.2 0.14 0.16 0.26 0.13 0.15 0.23 0.14 0.16 0.25 0.12 0.13 0.22 0.4 0.26 0.25 0.30 0.23 0.24 0.28 0.25 0.25 0.30 0.22 0.22 0.26 2000 0 0.06 0.10 0.23 0.07 0.10 0.21 0.05 0.10 0.25 0.04 0.07 0.19 0.2 0.09 0.11 0.23 0.09 0.11 0.21 0.09 0.12 0.25 0.06 0.09 0.19 0.4 0.23 0.23 0.26 0.22 0.21 0.24 0.25 0.24 0.27 0.19 0.19 0.23 MSE QLIKE Gaussian t(5) Gaussian t(5) DGP T dɛ1/dɛ2 0 0.2 0.4 0 0.2 0.4 0 0.2 0.4 0 0.2 0.4 3 250 0 0.31 0.32 0.38 0.29 0.31 0.37 0.29 0.33 0.40 0.27 0.31 0.39 0.2 0.34 0.35 0.39 0.33 0.34 0.38 0.32 0.33 0.41 0.29 0.32 0.39 0.4 0.41 0.42 0.44 0.40 0.40 0.43 0.37 0.38 0.43 0.36 0.37 0.42 2000 0 0.34 0.32 0.38 0.34 0.32 0.38 0.32 0.30 0.39 0.31 0.29 0.38 0.2 0.30 0.30 0.36 0.30 0.30 0.35 0.30 0.29 0.38 0.29 0.29 0.37 0.4 0.39 0.39 0.41 0.38 0.38 0.40 0.37 0.36 0.40 0.36 0.35 0.39 4 250 0 0.29 0.31 0.35 0.27 0.30 0.35 0.28 0.30 0.37 0.24 0.27 0.35 0.2 0.30 0.32 0.36 0.29 0.31 0.35 0.28 0.30 0.38 0.25 0.28 0.35 0.4 0.35 0.36 0.39 0.34 0.35 0.38 0.31 0.32 0.39 0.27 0.30 0.37 2000 0 0.26 0.26 0.33 0.25 0.26 0.33 0.25 0.25 0.35 0.23 0.24 0.33 0.2 0.26 0.26 0.33 0.26 0.25 0.32 0.26 0.26 0.35 0.24 0.24 0.33 0.4 0.33 0.33 0.36 0.33 0.32 0.36 0.29 0.29 0.36 0.27 0.27 0.34 5 250 0 0.12 0.14 0.25 0.11 0.13 0.23 0.10 0.13 0.25 0.10 0.12 0.21 0.2 0.14 0.16 0.26 0.13 0.15 0.23 0.14 0.16 0.25 0.12 0.13 0.22 0.4 0.26 0.25 0.30 0.23 0.24 0.28 0.25 0.25 0.30 0.22 0.22 0.26 2000 0 0.06 0.10 0.23 0.07 0.10 0.21 0.05 0.10 0.25 0.04 0.07 0.19 0.2 0.09 0.11 0.23 0.09 0.11 0.21 0.09 0.12 0.25 0.06 0.09 0.19 0.4 0.23 0.23 0.26 0.22 0.21 0.24 0.25 0.24 0.27 0.19 0.19 0.23 Table 2. Monte Carlo averages of estimated memory in the loss differential zt for DGP3, DGP4, and DGP5 with dη=0.2 MSE QLIKE Gaussian t(5) Gaussian t(5) DGP T dɛ1/dɛ2 0 0.2 0.4 0 0.2 0.4 0 0.2 0.4 0 0.2 0.4 3 250 0 0.31 0.32 0.38 0.29 0.31 0.37 0.29 0.33 0.40 0.27 0.31 0.39 0.2 0.34 0.35 0.39 0.33 0.34 0.38 0.32 0.33 0.41 0.29 0.32 0.39 0.4 0.41 0.42 0.44 0.40 0.40 0.43 0.37 0.38 0.43 0.36 0.37 0.42 2000 0 0.34 0.32 0.38 0.34 0.32 0.38 0.32 0.30 0.39 0.31 0.29 0.38 0.2 0.30 0.30 0.36 0.30 0.30 0.35 0.30 0.29 0.38 0.29 0.29 0.37 0.4 0.39 0.39 0.41 0.38 0.38 0.40 0.37 0.36 0.40 0.36 0.35 0.39 4 250 0 0.29 0.31 0.35 0.27 0.30 0.35 0.28 0.30 0.37 0.24 0.27 0.35 0.2 0.30 0.32 0.36 0.29 0.31 0.35 0.28 0.30 0.38 0.25 0.28 0.35 0.4 0.35 0.36 0.39 0.34 0.35 0.38 0.31 0.32 0.39 0.27 0.30 0.37 2000 0 0.26 0.26 0.33 0.25 0.26 0.33 0.25 0.25 0.35 0.23 0.24 0.33 0.2 0.26 0.26 0.33 0.26 0.25 0.32 0.26 0.26 0.35 0.24 0.24 0.33 0.4 0.33 0.33 0.36 0.33 0.32 0.36 0.29 0.29 0.36 0.27 0.27 0.34 5 250 0 0.12 0.14 0.25 0.11 0.13 0.23 0.10 0.13 0.25 0.10 0.12 0.21 0.2 0.14 0.16 0.26 0.13 0.15 0.23 0.14 0.16 0.25 0.12 0.13 0.22 0.4 0.26 0.25 0.30 0.23 0.24 0.28 0.25 0.25 0.30 0.22 0.22 0.26 2000 0 0.06 0.10 0.23 0.07 0.10 0.21 0.05 0.10 0.25 0.04 0.07 0.19 0.2 0.09 0.11 0.23 0.09 0.11 0.21 0.09 0.12 0.25 0.06 0.09 0.19 0.4 0.23 0.23 0.26 0.22 0.21 0.24 0.25 0.24 0.27 0.19 0.19 0.23 MSE QLIKE Gaussian t(5) Gaussian t(5) DGP T dɛ1/dɛ2 0 0.2 0.4 0 0.2 0.4 0 0.2 0.4 0 0.2 0.4 3 250 0 0.31 0.32 0.38 0.29 0.31 0.37 0.29 0.33 0.40 0.27 0.31 0.39 0.2 0.34 0.35 0.39 0.33 0.34 0.38 0.32 0.33 0.41 0.29 0.32 0.39 0.4 0.41 0.42 0.44 0.40 0.40 0.43 0.37 0.38 0.43 0.36 0.37 0.42 2000 0 0.34 0.32 0.38 0.34 0.32 0.38 0.32 0.30 0.39 0.31 0.29 0.38 0.2 0.30 0.30 0.36 0.30 0.30 0.35 0.30 0.29 0.38 0.29 0.29 0.37 0.4 0.39 0.39 0.41 0.38 0.38 0.40 0.37 0.36 0.40 0.36 0.35 0.39 4 250 0 0.29 0.31 0.35 0.27 0.30 0.35 0.28 0.30 0.37 0.24 0.27 0.35 0.2 0.30 0.32 0.36 0.29 0.31 0.35 0.28 0.30 0.38 0.25 0.28 0.35 0.4 0.35 0.36 0.39 0.34 0.35 0.38 0.31 0.32 0.39 0.27 0.30 0.37 2000 0 0.26 0.26 0.33 0.25 0.26 0.33 0.25 0.25 0.35 0.23 0.24 0.33 0.2 0.26 0.26 0.33 0.26 0.25 0.32 0.26 0.26 0.35 0.24 0.24 0.33 0.4 0.33 0.33 0.36 0.33 0.32 0.36 0.29 0.29 0.36 0.27 0.27 0.34 5 250 0 0.12 0.14 0.25 0.11 0.13 0.23 0.10 0.13 0.25 0.10 0.12 0.21 0.2 0.14 0.16 0.26 0.13 0.15 0.23 0.14 0.16 0.25 0.12 0.13 0.22 0.4 0.26 0.25 0.30 0.23 0.24 0.28 0.25 0.25 0.30 0.22 0.22 0.26 2000 0 0.06 0.10 0.23 0.07 0.10 0.21 0.05 0.10 0.25 0.04 0.07 0.19 0.2 0.09 0.11 0.23 0.09 0.11 0.21 0.09 0.12 0.25 0.06 0.09 0.19 0.4 0.23 0.23 0.26 0.22 0.21 0.24 0.25 0.24 0.27 0.19 0.19 0.23 As for DGP1 and DGP2, our estimates of dz are roughly in line with the theoretical predictions. For DGP3, where dz should be large, we see that the estimates are a bit lower, but the estimated degree of memory is still considerable. The results for DGP4 are indeed slightly lower than those for DGP3, as predicted by Proposition 4. Finally, for DGP5 we again observe that dz is somewhat overestimated if the true value is low and vice versa. As in Table 1, we see that the results are qualitatively the same if we consider t-distributed innovation sequences and the QLIKE loss function. Additional simulations with dx=0.65 show that the results are virtually identical for d1,d2≤0.4. If either d1=0.6 or d2=0.6, the memory transmission becomes even stronger, which is also in line with the findings for DGP1 and DGP2 in Table 1. Overall, we find that the finite-sample results presented in this section are in line with the theoretical findings from Section 3. Moreover, the practical relevance of the results in Propositions 2–5 extends far beyond the stationary Gaussian case with MSE loss as demonstrated by the finding that the transmission results obtained with t-distributed innovations, non-stationary processes, and the QLIKE loss function are nearly identical. 5.2 Empirical Forecast Scenarios The relevance of memory transmission to the forecast error loss differentials in practice is further examined by considering a number of simple forecast scenarios motivated by typical empirical examples. To ensure that the null hypothesis of equal predictive accuracy holds, we have to construct two competing forecasts that are different from each other, but perform equally well in terms of a loss function—here the MSE. The length of the estimation period equals TE = 250 and the memory parameter estimates are obtained by the LPWN estimator. The first scenario is motivated by the spurious long memory literature. The DGP is a fractionally integrated process with a time-varying mean that is generated by a random level shift process as in Perron and Qu (2010) or Qu (2011). In detail, yt=xt+μtxt=(1−L)−1/4ɛx,tμt=μt−1+πtɛμ,t, where ɛx,t∼iidN(0,1), ɛμ,t∼iidN(0,1), πt∼iidBern(p) and ɛxt, ɛμt and πt are mutually independent.6 It is well known that it can be difficult to distinguish long memory and low frequency contaminations such as structural breaks (cf. Diebold and Inoue (2001) or Granger and Hyung (2004)). Therefore, it is often assumed that the process is either driven by the one or the other, see, for example, Berkes et al. (2006), who suggest a test that allows to test for the null hypothesis of a weakly dependent process with breaks against the alternative of long-range dependence, or Lu and Perron (2010) who demonstrate that a pure level shift process has superior predictive performance compared with ARFIMA and HAR models for the log-absolute returns of the S&P500. See also Varneskov and Perron (2017) for a related recent contribution. In the spirit of this dichotomy, we compare forecasts which solely consider the breaks with those that assume the absence of breaks and predict the process based on a fractionally integrated model (with the memory estimated by the local Whittle method).7 Table 3 shows the results of this exercise. It is clear to see that the average loss differential is close to zero. The estimated memory of the loss differentials is around 0.17 for larger sample sizes. While the classical DM test based on a HAC estimator over-rejects, both the tMAC and the tEFB-statistics control the size well, at least in larger samples. Table 3. Estimated memory of the loss differentials d^z, mean loss differential z¯, and rejection frequencies of the t-statistics for a spurious long memory scenario T d^z z¯ tHAC tMAC tEFB 250 0.113 0.012 0.201 0.110 0.098 500 0.144 0.003 0.227 0.080 0.073 1000 0.171 0.000 0.270 0.057 0.058 2000 0.172 −0.001 0.324 0.046 0.057 T d^z z¯ tHAC tMAC tEFB 250 0.113 0.012 0.201 0.110 0.098 500 0.144 0.003 0.227 0.080 0.073 1000 0.171 0.000 0.270 0.057 0.058 2000 0.172 −0.001 0.324 0.046 0.057 Note: The true DGP is fractionally integrated with random level shifts and the forecasts assume either a pure shift process or a pure long memory process. Table 3. Estimated memory of the loss differentials d^z, mean loss differential z¯, and rejection frequencies of the t-statistics for a spurious long memory scenario T d^z z¯ tHAC tMAC tEFB 250 0.113 0.012 0.201 0.110 0.098 500 0.144 0.003 0.227 0.080 0.073 1000 0.171 0.000 0.270 0.057 0.058 2000 0.172 −0.001 0.324 0.046 0.057 T d^z z¯ tHAC tMAC tEFB 250 0.113 0.012 0.201 0.110 0.098 500 0.144 0.003 0.227 0.080 0.073 1000 0.171 0.000 0.270 0.057 0.058 2000 0.172 −0.001 0.324 0.046 0.057 Note: The true DGP is fractionally integrated with random level shifts and the forecasts assume either a pure shift process or a pure long memory process. As a second scenario, we consider simple predictive regressions based on two regressors that are fractionally cointegrated with the forecast objective. Here xt is fractionally integrated of order d. Then yt=xt+(1−L)−(d−δ)ηtxi,t=xt+(1−L)−(d−δ)ɛi,tŷit=β̂0i+β̂1ixi,t−1, where ηt and the ɛi,t are mutually independent and normally distributed with unit variances, β̂0i and β̂1i are the OLS estimators and 0<δ<d. To resemble processes in the lower nonstationary long memory region (as the realized volatilities in our empirical application) we set d = 0.6. This corresponds to a situation where we forecast realized volatility of the S&P500 with either past values of the VIX or another past realized volatility such as that of a sector index. The cointegration strength is set to δ=0.3. The results are shown in Table 4. Again, one can see that the Monte Carlo averages ( z¯) are close to zero. The tMAC and tEFB-statistics tend to be conservative in larger samples, whereas the tHAC test rejects far too often. The strength of the memory in the loss differential lies roughly at 0.24. Table 4. Estimated memory of the loss differentials d^z, mean loss differential z¯, and rejection frequencies of the t-statistics for comparison of forecasts obtained from predictive regressions where the regressor variables are fractionally cointegrated with the forecast objective T d^z z¯ tHAC tMAC tEFB 250 0.182 0.011 0.289 0.063 0.059 500 0.207 −0.011 0.336 0.040 0.041 1000 0.228 −0.001 0.378 0.023 0.025 2000 0.238 −0.009 0.441 0.016 0.020 T d^z z¯ tHAC tMAC tEFB 250 0.182 0.011 0.289 0.063 0.059 500 0.207 −0.011 0.336 0.040 0.041 1000 0.228 −0.001 0.378 0.023 0.025 2000 0.238 −0.009 0.441 0.016 0.020 Table 4. Estimated memory of the loss differentials d^z, mean loss differential z¯, and rejection frequencies of the t-statistics for comparison of forecasts obtained from predictive regressions where the regressor variables are fractionally cointegrated with the forecast objective T d^z z¯ tHAC tMAC tEFB 250 0.182 0.011 0.289 0.063 0.059 500 0.207 −0.011 0.336 0.040 0.041 1000 0.228 −0.001 0.378 0.023 0.025 2000 0.238 −0.009 0.441 0.016 0.020 T d^z z¯ tHAC tMAC tEFB 250 0.182 0.011 0.289 0.063 0.059 500 0.207 −0.011 0.336 0.040 0.041 1000 0.228 −0.001 0.378 0.023 0.025 2000 0.238 −0.009 0.441 0.016 0.020 Our third scenario is closely related to the previous one. In practice it is hard to distinguish fractionally cointegrated series from fractionally integrated series with highly correlated short-run components (cf. the simulation studies in Hualde and Velasco (2008)). Therefore, our third scenario is similar to the second, but with correlated innovations, yt=(1−L)−dηtxit=(1−L)−dɛi,tand ŷit=β̂0i+β̂1ixi,t−1. Here, all pairwise correlations between ηt and the ɛi,t are ρ=0.4. Furthermore, we set d = 0.4, so that we operate in the stationary long memory region. The situation is the same as in the previous scenarios, with strong long memory of d̂z≈0.3 in the loss differentials, see Table 5. Apparently, the tests are quite conservative for this DGP. This can be attributed to the complicated memory estimation in the forecast error loss differential series via the standard local Whittle estimator, see also our discussion in Section 5.1. Table 5. Estimated memory of the loss differentials d^z, mean loss differential z¯, and rejection frequencies of the t-statistics for comparison of forecasts obtained from predictive regressions where the regressor variables have correlated innovations with the forecast objective T d^z z¯ tHAC tMAC tEFB 250 0.275 0.001 0.364 0.028 0.026 500 0.288 0.001 0.417 0.017 0.020 1000 0.288 0.005 0.445 0.008 0.012 2000 0.287 0.003 0.485 0.004 0.009 T d^z z¯ tHAC tMAC tEFB 250 0.275 0.001 0.364 0.028 0.026 500 0.288 0.001 0.417 0.017 0.020 1000 0.288 0.005 0.445 0.008 0.012 2000 0.287 0.003 0.485 0.004 0.009 Table 5. Estimated memory of the loss differentials d^z, mean loss differential z¯, and rejection frequencies of the t-statistics for comparison of forecasts obtained from predictive regressions where the regressor variables have correlated innovations with the forecast objective T d^z z¯ tHAC tMAC tEFB 250 0.275 0.001 0.364 0.028 0.026 500 0.288 0.001 0.417 0.017 0.020 1000 0.288 0.005 0.445 0.008 0.012 2000 0.287 0.003 0.485 0.004 0.009 T d^z z¯ tHAC tMAC tEFB 250 0.275 0.001 0.364 0.028 0.026 500 0.288 0.001 0.417 0.017 0.020 1000 0.288 0.005 0.445 0.008 0.012 2000 0.287 0.003 0.485 0.004 0.009 Altogether, these results demonstrate that memory transmission can indeed occur in a variety of situations whether that is due to level shifts, cointegration, or correlation—in a nonstationary series, or in a stationary series. 5.3 Size and Power of Long Memory Robust t-Statistics We now turn our attention to the empirical size and power properties of the memory robust tEFB and tMAC-statistics by using the same DGPs as in Section 5.1. Thereby, we reflect the situations covered by our propositions and the distributional properties that are realistic for the forecast error loss differential zt. However, we have to ensure that the loss differentials have zero expectation (size) and that the distance from the null is comparable for different DGPs (power). As DGP1, DGP2, DGP4, and DGP5 are constructed in a symmetric way, we have E[MSE(yt,ŷ1t)−MSE(yt,ŷ2t)]=0 and E[QLIKE(yt,ŷ1t)−QLIKE(yt,ŷ2t)]≠0, due to the asymmetry of the QLIKE loss function. Furthermore, DGP3 is not constructed in a symmetric way, so that E[z˜t]≠0, irrespective of the loss function. We therefore have to correct the means of the loss differentials. In addition to that, different DGPs generate different degrees of long memory. Given that sample means of long memory processes with memory d converge with the rate T1/2−d, we consider the local power to achieve comparable results across DGPs. Let z˜t be generated as in (1), with the ŷit and yt as described in Section 5.1, and z¯=(MT)−1∑i=1M∑t=1Tz˜t, where M denotes the number of Monte Carlo repetitions. Then the loss differentials are obtained via zt=z˜t−z¯+cSD(z˜t)T1/2−dz. (12) The parameter c controls the distance from the null hypothesis (c = 0). Here, each realization of z˜t is centered with the average sample mean from M = 5000 simulations of the respective DGP. Similarly, dz is determined as the Monte Carlo average of the LPWN estimates for the respective setup. In the power simulations, the memory parameters are set to d1=d2=d and dɛ1=dɛ2=d to keep the tables reasonably concise. Table 6 presents the size results for the tMAC-statistic. It tends to be liberal for small T, but generally controls the size well in larger samples. There are, however, two exceptions. First, the test remains liberal for DGP3, even if the sample size increases. This effect is particularly pronounced for d = 0 and if zt is based on the QLIKE loss function. Second, the test is conservative for DGP2, particularly for increasing values of d. With regard to the bandwidth parameters, we find that the size is slightly better controlled with qd=0.65 and q = 0.6. However, the bandwidth choice seems to have limited effects on the size of the test, especially in larger samples. Table 6. Size results of the tMAC-statistic for the DGPs described in Section 5.1 and (12) with Gaussian innovations q 0.5 0.6 MSE QLIKE MSE QLIKE qd T DGP /d 0 0.2 0.4 0 0.2 0.4 0 0.2 0.4 0 0.2 0.4 0.65 250 1 0.10 0.08 0.07 0.10 0.09 0.13 0.09 0.07 0.07 0.10 0.10 0.13 2 0.01 0.02 0.03 0.02 0.04 0.05 0.01 0.03 0.03 0.02 0.04 0.04 3 0.10 0.09 0.09 0.13 0.10 0.09 0.10 0.08 0.09 0.13 0.11 0.10 4 0.10 0.08 0.09 0.10 0.09 0.10 0.09 0.09 0.09 0.10 0.08 0.10 5 0.04 0.05 0.05 0.04 0.06 0.05 0.03 0.04 0.05 0.03 0.05 0.06 2000 1 0.05 0.05 0.04 0.06 0.05 0.08 0.05 0.05 0.03 0.06 0.05 0.07 2 0.02 0.02 0.01 0.02 0.05 0.02 0.01 0.02 0.01 0.02 0.04 0.02 3 0.07 0.05 0.04 0.15 0.11 0.06 0.06 0.05 0.04 0.15 0.11 0.05 4 0.06 0.05 0.04 0.07 0.06 0.06 0.05 0.05 0.04 0.06 0.05 0.05 5 0.04 0.05 0.02 0.04 0.06 0.02 0.03 0.05 0.02 0.03 0.06 0.02 0.8 250 1 0.08 0.07 0.06 0.10 0.09 0.12 0.09 0.06 0.05 0.09 0.08 0.11 2 0.02 0.02 0.02 0.02 0.03 0.04 0.02 0.02 0.02 0.02 0.04 0.04 3 0.10 0.06 0.07 0.12 0.09 0.08 0.09 0.07 0.09 0.12 0.08 0.08 4 0.09 0.07 0.08 0.10 0.08 0.10 0.08 0.06 0.07 0.10 0.08 0.10 5 0.04 0.05 0.03 0.04 0.05 0.04 0.04 0.05 0.03 0.04 0.05 0.03 2000 1 0.06 0.05 0.04 0.06 0.06 0.08 0.05 0.04 0.03 0.06 0.06 0.07 2 0.02 0.01 0.00 0.02 0.04 0.02 0.02 0.01 0.00 0.02 0.04 0.02 3 0.07 0.05 0.06 0.16 0.11 0.07 0.06 0.04 0.05 0.14 0.11 0.06 4 0.06 0.04 0.06 0.06 0.05 0.07 0.05 0.04 0.04 0.06 0.05 0.06 5 0.04 0.04 0.01 0.04 0.05 0.02 0.04 0.04 0.01 0.04 0.05 0.01 q 0.5 0.6 MSE QLIKE MSE QLIKE qd T DGP /d 0 0.2 0.4 0 0.2 0.4 0 0.2 0.4 0 0.2 0.4 0.65 250 1 0.10 0.08 0.07 0.10 0.09 0.13 0.09 0.07 0.07 0.10 0.10 0.13 2 0.01 0.02 0.03 0.02 0.04 0.05 0.01 0.03 0.03 0.02 0.04 0.04 3 0.10 0.09 0.09 0.13 0.10 0.09 0.10 0.08 0.09 0.13 0.11 0.10 4 0.10 0.08 0.09 0.10 0.09 0.10 0.09 0.09 0.09 0.10 0.08 0.10 5 0.04 0.05 0.05 0.04 0.06 0.05 0.03 0.04 0.05 0.03 0.05 0.06 2000 1 0.05 0.05 0.04 0.06 0.05 0.08 0.05 0.05 0.03 0.06 0.05 0.07 2 0.02 0.02 0.01 0.02 0.05 0.02 0.01 0.02 0.01 0.02 0.04 0.02 3 0.07 0.05 0.04 0.15 0.11 0.06 0.06 0.05 0.04 0.15 0.11 0.05 4 0.06 0.05 0.04 0.07 0.06 0.06 0.05 0.05 0.04 0.06 0.05 0.05 5 0.04 0.05 0.02 0.04 0.06 0.02 0.03 0.05 0.02 0.03 0.06 0.02 0.8 250 1 0.08 0.07 0.06 0.10 0.09 0.12 0.09 0.06 0.05 0.09 0.08 0.11 2 0.02 0.02 0.02 0.02 0.03 0.04 0.02 0.02 0.02 0.02 0.04 0.04 3 0.10 0.06 0.07 0.12 0.09 0.08 0.09 0.07 0.09 0.12 0.08 0.08 4 0.09 0.07 0.08 0.10 0.08 0.10 0.08 0.06 0.07 0.10 0.08 0.10 5 0.04 0.05 0.03 0.04 0.05 0.04 0.04 0.05 0.03 0.04 0.05 0.03 2000 1 0.06 0.05 0.04 0.06 0.06 0.08 0.05 0.04 0.03 0.06 0.06 0.07 2 0.02 0.01 0.00 0.02 0.04 0.02 0.02 0.01 0.00 0.02 0.04 0.02 3 0.07 0.05 0.06 0.16 0.11 0.07 0.06 0.04 0.05 0.14 0.11 0.06 4 0.06 0.04 0.06 0.06 0.05 0.07 0.05 0.04 0.04 0.06 0.05 0.06 5 0.04 0.04 0.01 0.04 0.05 0.02 0.04 0.04 0.01 0.04 0.05 0.01 Table 6. Size results of the tMAC-statistic for the DGPs described in Section 5.1 and (12) with Gaussian innovations q 0.5 0.6 MSE QLIKE MSE QLIKE qd T DGP /d 0 0.2 0.4 0 0.2 0.4 0 0.2 0.4 0 0.2 0.4 0.65 250 1 0.10 0.08 0.07 0.10 0.09 0.13 0.09 0.07 0.07 0.10 0.10 0.13 2 0.01 0.02 0.03 0.02 0.04 0.05 0.01 0.03 0.03 0.02 0.04 0.04 3 0.10 0.09 0.09 0.13 0.10 0.09 0.10 0.08 0.09 0.13 0.11 0.10 4 0.10 0.08 0.09 0.10 0.09 0.10 0.09 0.09 0.09 0.10 0.08 0.10 5 0.04 0.05 0.05 0.04 0.06 0.05 0.03 0.04 0.05 0.03 0.05 0.06 2000 1 0.05 0.05 0.04 0.06 0.05 0.08 0.05 0.05 0.03 0.06 0.05 0.07 2 0.02 0.02 0.01 0.02 0.05 0.02 0.01 0.02 0.01 0.02 0.04 0.02 3 0.07 0.05 0.04 0.15 0.11 0.06 0.06 0.05 0.04 0.15 0.11 0.05 4 0.06 0.05 0.04 0.07 0.06 0.06 0.05 0.05 0.04 0.06 0.05 0.05 5 0.04 0.05 0.02 0.04 0.06 0.02 0.03 0.05 0.02 0.03 0.06 0.02 0.8 250 1 0.08 0.07 0.06 0.10 0.09 0.12 0.09 0.06 0.05 0.09 0.08 0.11 2 0.02 0.02 0.02 0.02 0.03 0.04 0.02 0.02 0.02 0.02 0.04 0.04 3 0.10 0.06 0.07 0.12 0.09 0.08 0.09 0.07 0.09 0.12 0.08 0.08 4 0.09 0.07 0.08 0.10 0.08 0.10 0.08 0.06 0.07 0.10 0.08 0.10 5 0.04 0.05 0.03 0.04 0.05 0.04 0.04 0.05 0.03 0.04 0.05 0.03 2000 1 0.06 0.05 0.04 0.06 0.06 0.08 0.05 0.04 0.03 0.06 0.06 0.07 2 0.02 0.01 0.00 0.02 0.04 0.02 0.02 0.01 0.00 0.02 0.04 0.02 3 0.07 0.05 0.06 0.16 0.11 0.07 0.06 0.04 0.05 0.14 0.11 0.06 4 0.06 0.04 0.06 0.06 0.05 0.07 0.05 0.04 0.04 0.06 0.05 0.06 5 0.04 0.04 0.01 0.04 0.05 0.02 0.04 0.04 0.01 0.04 0.05 0.01 q 0.5 0.6 MSE QLIKE MSE QLIKE qd T DGP /d 0 0.2 0.4 0 0.2 0.4 0 0.2 0.4 0 0.2 0.4 0.65 250 1 0.10 0.08 0.07 0.10 0.09 0.13 0.09 0.07 0.07 0.10 0.10 0.13 2 0.01 0.02 0.03 0.02 0.04 0.05 0.01 0.03 0.03 0.02 0.04 0.04 3 0.10 0.09 0.09 0.13 0.10 0.09 0.10 0.08 0.09 0.13 0.11 0.10 4 0.10 0.08 0.09 0.10 0.09 0.10 0.09 0.09 0.09 0.10 0.08 0.10 5 0.04 0.05 0.05 0.04 0.06 0.05 0.03 0.04 0.05 0.03 0.05 0.06 2000 1 0.05 0.05 0.04 0.06 0.05 0.08 0.05 0.05 0.03 0.06 0.05 0.07 2 0.02 0.02 0.01 0.02 0.05 0.02 0.01 0.02 0.01 0.02 0.04 0.02 3 0.07 0.05 0.04 0.15 0.11 0.06 0.06 0.05 0.04 0.15 0.11 0.05 4 0.06 0.05 0.04 0.07 0.06 0.06 0.05 0.05 0.04 0.06 0.05 0.05 5 0.04 0.05 0.02 0.04 0.06 0.02 0.03 0.05 0.02 0.03 0.06 0.02 0.8 250 1 0.08 0.07 0.06 0.10 0.09 0.12 0.09 0.06 0.05 0.09 0.08 0.11 2 0.02 0.02 0.02 0.02 0.03 0.04 0.02 0.02 0.02 0.02 0.04 0.04 3 0.10 0.06 0.07 0.12 0.09 0.08 0.09 0.07 0.09 0.12 0.08 0.08 4 0.09 0.07 0.08 0.10 0.08 0.10 0.08 0.06 0.07 0.10 0.08 0.10 5 0.04 0.05 0.03 0.04 0.05 0.04 0.04 0.05 0.03 0.04 0.05 0.03 2000 1 0.06 0.05 0.04 0.06 0.06 0.08 0.05 0.04 0.03 0.06 0.06 0.07 2 0.02 0.01 0.00 0.02 0.04 0.02 0.02 0.01 0.00 0.02 0.04 0.02 3 0.07 0.05 0.06 0.16 0.11 0.07 0.06 0.04 0.05 0.14 0.11 0.06 4 0.06 0.04 0.06 0.06 0.05 0.07 0.05 0.04 0.04 0.06 0.05 0.06 5 0.04 0.04 0.01 0.04 0.05 0.02 0.04 0.04 0.01 0.04 0.05 0.01 Size results for the tEFB-statistic are displayed in Table 7. To analyze the impact of the bandwidth b and the kernel choice, we set qd=0.65.8 Size performance is more favorable with the MQS kernel than with the Bartlett kernel. Furthermore, it is positively impacted by using a large value of b (0.6 or 0.9). Similar to the tMAC-statistic, we observe that the test is liberal with T = 250, but that the overall performance is very satisfactory for T = 2000. Again, the test tends to be liberal for DGP3—especially for QLIKE. However, if the MQS kernel and a larger b is used, this effect disappears nearly completely. The conservative behavior of the test for DGP2 and large values of d is also the same as for the tMAC-statistic. The tMAC-statistic tends to be perform better than the tEFB-statistic using the Bartlett kernel, but worse when considering the MQS kernel (cf. Tables 6 and 7). Table 7. Size results of the tEFB-statistic for the DGPs described in Section 5.1 and (12) with Gaussian innovations and m=⌊T0.65⌋ MSE QLIKE Kernel MQS Bartlett MQS Bartlett d T DGP /b 0.3 0.6 0.9 0.3 0.6 0.9 0.3 0.6 0.9 0.3 0.6 0.9 0 250 1 0.08 0.06 0.06 0.10 0.10 0.08 0.07 0.06 0.06 0.10 0.10 0.09 2 0.03 0.03 0.03 0.02 0.02 0.02 0.04 0.03 0.03 0.02 0.02 0.02 3 0.12 0.10 0.09 0.16 0.14 0.14 0.11 0.09 0.08 0.15 0.14 0.14 4 0.07 0.06 0.06 0.10 0.09 0.09 0.08 0.06 0.07 0.10 0.10 0.09 5 0.05 0.04 0.04 0.04 0.04 0.04 0.04 0.04 0.05 0.04 0.04 0.04 2000 1 0.06 0.05 0.05 0.07 0.05 0.06 0.06 0.04 0.05 0.07 0.06 0.06 2 0.03 0.03 0.03 0.02 0.02 0.03 0.04 0.03 0.03 0.03 0.03 0.03 3 0.07 0.06 0.06 0.10 0.10 0.09 0.08 0.06 0.07 0.10 0.09 0.10 4 0.06 0.05 0.05 0.06 0.06 0.06 0.07 0.06 0.06 0.08 0.07 0.07 5 0.04 0.04 0.04 0.05 0.05 0.04 0.05 0.04 0.04 0.05 0.05 0.05 0.2 250 1 0.06 0.05 0.05 0.08 0.08 0.07 0.08 0.06 0.06 0.09 0.08 0.08 2 0.04 0.02 0.03 0.03 0.03 0.03 0.04 0.03 0.03 0.04 0.04 0.04 3 0.09 0.07 0.08 0.11 0.10 0.10 0.09 0.07 0.07 0.11 0.10 0.10 4 0.07 0.06 0.06 0.08 0.08 0.07 0.07 0.06 0.05 0.09 0.09 0.08 5 0.05 0.03 0.04 0.04 0.05 0.05 0.05 0.05 0.04 0.05 0.05 0.06 2000 1 0.05 0.04 0.04 0.06 0.05 0.05 0.05 0.04 0.04 0.07 0.05 0.05 2 0.03 0.03 0.03 0.02 0.03 0.03 0.04 0.04 0.04 0.04 0.04 0.05 3 0.07 0.05 0.06 0.08 0.07 0.08 0.07 0.06 0.06 0.07 0.07 0.08 4 0.05 0.04 0.04 0.06 0.06 0.05 0.06 0.05 0.05 0.06 0.06 0.06 5 0.06 0.04 0.05 0.05 0.05 0.06 0.05 0.05 0.05 0.07 0.07 0.06 0.4 250 1 0.05 0.05 0.05 0.07 0.07 0.06 0.10 0.09 0.08 0.14 0.13 0.11 2 0.03 0.03 0.03 0.03 0.03 0.04 0.05 0.04 0.04 0.05 0.04 0.05 3 0.08 0.07 0.07 0.09 0.09 0.08 0.06 0.06 0.05 0.09 0.08 0.08 4 0.08 0.06 0.07 0.09 0.09 0.09 0.09 0.07 0.06 0.11 0.11 0.11 5 0.04 0.03 0.03 0.05 0.05 0.05 0.04 0.04 0.04 0.05 0.05 0.05 2000 1 0.04 0.04 0.04 0.04 0.04 0.04 0.08 0.05 0.06 0.08 0.08 0.07 2 0.02 0.02 0.02 0.02 0.02 0.01 0.03 0.03 0.03 0.03 0.03 0.03 3 0.04 0.04 0.04 0.04 0.04 0.04 0.04 0.04 0.04 0.04 0.04 0.04 4 0.05 0.05 0.04 0.05 0.05 0.05 0.05 0.05 0.05 0.06 0.06 0.06 5 0.03 0.03 0.03 0.02 0.02 0.02 0.03 0.03 0.03 0.03 0.03 0.03 MSE QLIKE Kernel MQS Bartlett MQS Bartlett d T DGP /b 0.3 0.6 0.9 0.3 0.6 0.9 0.3 0.6 0.9 0.3 0.6 0.9 0 250 1 0.08 0.06 0.06 0.10 0.10 0.08 0.07 0.06 0.06 0.10 0.10 0.09 2 0.03 0.03 0.03 0.02 0.02 0.02 0.04 0.03 0.03 0.02 0.02 0.02 3 0.12 0.10 0.09 0.16 0.14 0.14 0.11 0.09 0.08 0.15 0.14 0.14 4 0.07 0.06 0.06 0.10 0.09 0.09 0.08 0.06 0.07 0.10 0.10 0.09 5 0.05 0.04 0.04 0.04 0.04 0.04 0.04 0.04 0.05 0.04 0.04 0.04 2000 1 0.06 0.05 0.05 0.07 0.05 0.06 0.06 0.04 0.05 0.07 0.06 0.06 2 0.03 0.03 0.03 0.02 0.02 0.03 0.04 0.03 0.03 0.03 0.03 0.03 3 0.07 0.06 0.06 0.10 0.10 0.09 0.08 0.06 0.07 0.10 0.09 0.10 4 0.06 0.05 0.05 0.06 0.06 0.06 0.07 0.06 0.06 0.08 0.07 0.07 5 0.04 0.04 0.04 0.05 0.05 0.04 0.05 0.04 0.04 0.05 0.05 0.05 0.2 250 1 0.06 0.05 0.05 0.08 0.08 0.07 0.08 0.06 0.06 0.09 0.08 0.08 2 0.04 0.02 0.03 0.03 0.03 0.03 0.04 0.03 0.03 0.04 0.04 0.04 3 0.09 0.07 0.08 0.11 0.10 0.10 0.09 0.07 0.07 0.11 0.10 0.10 4 0.07 0.06 0.06 0.08 0.08 0.07 0.07 0.06 0.05 0.09 0.09 0.08 5 0.05 0.03 0.04 0.04 0.05 0.05 0.05 0.05 0.04 0.05 0.05 0.06 2000 1 0.05 0.04 0.04 0.06 0.05 0.05 0.05 0.04 0.04 0.07 0.05 0.05 2 0.03 0.03 0.03 0.02 0.03 0.03 0.04 0.04 0.04 0.04 0.04 0.05 3 0.07 0.05 0.06 0.08 0.07 0.08 0.07 0.06 0.06 0.07 0.07 0.08 4 0.05 0.04 0.04 0.06 0.06 0.05 0.06 0.05 0.05 0.06 0.06 0.06 5 0.06 0.04 0.05 0.05 0.05 0.06 0.05 0.05 0.05 0.07 0.07 0.06 0.4 250 1 0.05 0.05 0.05 0.07 0.07 0.06 0.10 0.09 0.08 0.14 0.13 0.11 2 0.03 0.03 0.03 0.03 0.03 0.04 0.05 0.04 0.04 0.05 0.04 0.05 3 0.08 0.07 0.07 0.09 0.09 0.08 0.06 0.06 0.05 0.09 0.08 0.08 4 0.08 0.06 0.07 0.09 0.09 0.09 0.09 0.07 0.06 0.11 0.11 0.11 5 0.04 0.03 0.03 0.05 0.05 0.05 0.04 0.04 0.04 0.05 0.05 0.05 2000 1 0.04 0.04 0.04 0.04 0.04 0.04 0.08 0.05 0.06 0.08 0.08 0.07 2 0.02 0.02 0.02 0.02 0.02 0.01 0.03 0.03 0.03 0.03 0.03 0.03 3 0.04 0.04 0.04 0.04 0.04 0.04 0.04 0.04 0.04 0.04 0.04 0.04 4 0.05 0.05 0.04 0.05 0.05 0.05 0.05 0.05 0.05 0.06 0.06 0.06 5 0.03 0.03 0.03 0.02 0.02 0.02 0.03 0.03 0.03 0.03 0.03 0.03 Table 7. Size results of the tEFB-statistic for the DGPs described in Section 5.1 and (12) with Gaussian innovations and m=⌊T0.65⌋ MSE QLIKE Kernel MQS Bartlett MQS Bartlett d T DGP /b 0.3 0.6 0.9 0.3 0.6 0.9 0.3 0.6 0.9 0.3 0.6 0.9 0 250 1 0.08 0.06 0.06 0.10 0.10 0.08 0.07 0.06 0.06 0.10 0.10 0.09 2 0.03 0.03 0.03 0.02 0.02 0.02 0.04 0.03 0.03 0.02 0.02 0.02 3 0.12 0.10 0.09 0.16 0.14 0.14 0.11 0.09 0.08 0.15 0.14 0.14 4 0.07 0.06 0.06 0.10 0.09 0.09 0.08 0.06 0.07 0.10 0.10 0.09 5 0.05 0.04 0.04 0.04 0.04 0.04 0.04 0.04 0.05 0.04 0.04 0.04 2000 1 0.06 0.05 0.05 0.07 0.05 0.06 0.06 0.04 0.05 0.07 0.06 0.06 2 0.03 0.03 0.03 0.02 0.02 0.03 0.04 0.03 0.03 0.03 0.03 0.03 3 0.07 0.06 0.06 0.10 0.10 0.09 0.08 0.06 0.07 0.10 0.09 0.10 4 0.06 0.05 0.05 0.06 0.06 0.06 0.07 0.06 0.06 0.08 0.07 0.07 5 0.04 0.04 0.04 0.05 0.05 0.04 0.05 0.04 0.04 0.05 0.05 0.05 0.2 250 1 0.06 0.05 0.05 0.08 0.08 0.07 0.08 0.06 0.06 0.09 0.08 0.08 2 0.04 0.02 0.03 0.03 0.03 0.03 0.04 0.03 0.03 0.04 0.04 0.04 3 0.09 0.0