Using simulated currency rainbow options to evaluate covariance matrix forecasts

Using simulated currency rainbow options to evaluate covariance matrix forecasts

Journal of International Financial Markets, Institutions and Money 12 (2002) 216– 230 Using simulated currency rainb...

126KB Sizes 0 Downloads 14 Views

Journal of International Financial Markets, Institutions and Money 12 (2002) 216– 230

Using simulated currency rainbow options to evaluate covariance matrix forecasts Hans N.E. Bystro¨m * Department of Economics, Lund Uni6ersity, P.O. Box 7082, S-220 07 Lund, Sweden Received 5 January 2001; accepted 30 August 2001

Abstract In the literature one can find a number of different methods to evaluate covariance matrix forecasts, and in choosing among these one has to consider what the actual purpose of the forecasts is. In this paper we look at portfolios of currency rainbow options and how simulated trading of such options portfolios can be used as a preference free evaluation measure for the forecasted covariance matrix. The main advantage of using portfolios instead of single options, in addition to making it possible to study multivariate problems of arbitrary size, is the possibility it gives to rely on shorter data series. We apply the methodology to a system of four US dollar exchange rates and compare the relative performance of different forecasting models, among them the fairly new Orthogonal GARCH model. © 2002 Elsevier Science B.V. All rights reserved. Keywords: Forecasting evaluation; Derivatives; Covariance matrix JEL classification: C32; C54; G13

1. Introduction The correct modeling and forecasting of variances and covariances is of great importance in many practical situations, and therefore, it is also important to accurately compare different forecasts and to be able to identify those models that give better forecasts. Evaluation of variance and covariance forecasts can be * Tel.: +46-46-222-7909; fax: +46-46-222-4118. E-mail address: [email protected] (H.N.E. Bystro¨m). 1042-4431/02/$ - see front matter © 2002 Elsevier Science B.V. All rights reserved. PII: S 1 0 4 2 - 4 4 3 1 ( 0 2 ) 0 0 0 0 4 - 5


H.N.E. Bystro¨ m / Int. Fin. Markets, Inst. and Money 12 (2002) 216–230

performed in several ways and the choice of evaluation measure in a particular situation should depend on the actual purpose of the forecasts. An options trader, for instance, uses forecasts of variances (and covariances) to price and hedge different kinds of contingent claims. He should, therefore, acknowledge the role of variances in the pricing and hedging of contingent claims in his evaluation of variance forecasts. Engle et al. (1993) set up a hypothetical derivatives market where actors using different variance forecasts trade simulated options with each other. In such a set up, the idea is that forecasters producing better forecasts should earn higher profits than the other actors in the market. Gibson and Boyer (1998) extends this methodology to covariances by looking at so called ‘rainbow options’, options with more than one underlying asset. They recognize that to price and hedge rainbow options one needs forecasts of the whole covariance matrix. The extension of the Engle et al. (1993) methodology, therefore, gives us a preference free approach, particularly well suited for options traders, to the evaluation of covariance matrix forecasts. While Gibson and Boyer (1998) apply their evaluation technique to a bivariate problem using only one simulated rainbow option, we show in this paper how their method can be extended to a multivariate situation with covariance matrices of arbitrary size by simply using portfolios of simulated rainbow options. In this way one also solves a major problem associated with the technique, the need for long sample periods to get statistically significant results. There are always problems with non-stationarities and structural changes when time series extending far back in time are used, and sometimes asset series longer than a few years do not even exist; the latter problem is particularly important since many of those markets (notably the power markets) where rainbow options are actually traded are fairly young markets. Further, Gibson and Boyer (1998) only look at equity options, but considering the much wider use of rainbow options in the market for foreign exchange, we choose to assess the performance of the evaluation technique in the currency market.1 In addition to presenting simulation results for the portfolio case we apply the evaluation method to a system of four floating US dollar exchange rates, and for comparative reasons we also evaluate forecasts using the standard statistical root mean squared error (RMSE) measure. The forecasts are produced by five different variance/covariance forecasting models; Historical, equally weighted moving average, exponentially weighted moving average (EWMA), pairwise bivariate GARCH, and orthogonal GARCH. The last model, orthogonal GARCH, is a rather new member of the multivariate GARCH family (suggested independently by Ding, 1994; Alexander and Chibumba, 1996, and tested by Bystro¨ m, 1999) and as a side result in this paper we present results of how well orthogonal GARCH performs in the currency market. 1

The existence of real traded rainbow options in the currency market could also make it possible to compare implied volatilities and correlations to different historical forecasts using the simulated options approach for evaluation.

H.N.E. Bystro¨ m / Int. Fin. Markets, Inst. and Money 12 (2002) 216–230


In Section 2, Gibson and Boyer (1998) evaluation approach is extended to portfolios of rainbow options. In Section 3 we simulate the performance of both the portfolio version and the single currency option version. Section 4 defines the empirical setup, presents the different forecasting models, and contains the results. Finally, Section 5 summarizes and concludes the paper.

2. A portfolio version of the forecasting evaluation methodology Gibson and Boyer (1998) evaluate volatility forecasts as well as correlation forecasts by their ability to correctly price different types of two-colored rainbow options; i.e. options with two underlying assets. Their method is an extension to the bivariate case of the univariate method suggested by Engle et al. (1993). By extending Gibson and Boyer (1998) methodology to portfolios of two-colored rainbow options, one can evaluate forecasted covariance matrices of larger dimensions. For instance, the four-by-four covariance matrix associated with four correlated return series can be evaluated by creating an (equally weighted) portfolio of six different two-colored rainbow options, one for each pair of return series. In this paper, we create equally weighted portfolios of one certain type of currency rainbow option, the ‘Outperformance Option’.2 The main reason for us to choose this particular rainbow option is that among all options depending on more than one asset, the Outperformance Option is one of few with an analytical solution for the option price (see Margrabe, 1978).3 Gibson and Boyer (1998) compare the performance of this rainbow option (called Call Spread in their paper) with three other rainbow options. They find all four types of rainbow options to produce the same ranking of their different covariance matrix forecasting models. Referring to these results in Gibson and Boyer (1998) we have chosen to look only at the Outperformance Option in this paper. However, any kind of rainbow option could of course had been used. The Outperformance Option has a payoff function equal to max[0, x1 − x2], where x1 and x2 are the two underlying exchange rates, and the option price depends on both currencies’ variances as well as their covariance. In other words, the whole covariance matrix is needed to accurately price such an option.

2 This option is also called ‘an option to exchange one asset for another’ or ‘at-the-money spread option’. 3 The Margrabe model is an extension of the Black and Scholes model and it assumes non-stochastic variances and covariance. This stands in contrast to what one observes in the real world as well as to our methodology of modelling second moments as time varying. However, the short time to maturity (1 day) of the simulated options in our experiment minimizes the difference in options price between constant volatility and varying volatility models (see Engle et al., 1993). As mentioned by Gibson and Boyer (1998), the use of the same constant volatility option pricing model for all different forecasts should bias the results against finding any differences among forecasting agents.


H.N.E. Bystro¨ m / Int. Fin. Markets, Inst. and Money 12 (2002) 216–230

The relative sensitivity of the price of the Outperformance Option on the variances is similar to the sensitivity on the covariance (see Margrabe, 1978) and a ‘generalist’ forecasting model, that does a good job forecasting both covariances and variances, will, therefore, be identified as best when Outperformance Options are used. Of course, by choosing a rainbow option with a different sensitivity to variances and covariances, or using Gibson and Boyer (1998) option package method, one could tailor the method to be sensitive to either variances or covariances. In the setup described below we follow Gibson and Boyer (1998) but instead of trading a single rainbow option, each actor now trades an equally weighted portfolio of six different rainbow options each day.4 Each day t− 1 in the test period a certain sequence is followed.

2.1. Forecasts Each actor, there are five of them in our setup, makes his forecast of the covariance matrix at t −1 using his particular forecasting model.

2.2. Option pricing Each actor uses his covariance matrix forecast to price 1-day at-the-money options at time t −1 to exchange one currency for a certain amount of an other currency at time t.

2.3. Option trading Actors trade options among themselves at t− 1. Each actor trades six options with each of the other four actors. An actor who finds an other actor’s option to be underpriced buys the option and vice versa. In this way each actor trades 24 options each day, his ‘bank account’ being credited (if a seller) or debited (if a buyer) with the mean (for each option) of the two traders’ options prices.

2.4. Hedging For each of the six different options each trader hedges his exposure to the two underlying currencies by going short or long in these currencies a time t− 1 an amount equal to his option-position’s sensitivity to changes in each underlying currency (delta hedging). His bank account is again credited or debited the amount needed for the trades. It is important to remember that the quality of the different forecasts affects the hedging performance just as well as it affects the pricing performance.


All options are at-the-money currency Outperformance Options and the time to maturity is 1 day.

H.N.E. Bystro¨ m / Int. Fin. Markets, Inst. and Money 12 (2002) 216–230


2.5. Payoff The money in the bank accounts earns 1 day’s interest at the riskfree rates.5 Using actual returns between time t −1 and time t the payoffs (possibly zero) of the options are calculated. The bank accounts are again credited or debited with these payoffs. The hedge positions of the underlying currencies are sold at time t and the bank account is again debited or credited.

2.6. Accounting Going from t− 1 to t the balances of each actors bank account have changed an amount equal to that day’s profit (positive or negative). The last day in the test period the actor with the ‘best’ forecasts should have made the highest accumulated profit measured in US dollars.

3. Simulating the performance of the methodology Again following Gibson and Boyer (1998), we present some simulation results in order to demonstrate the power of the method. Looking at static forecasts we show how Gibson and Boyer (1998) methodology applied to a portfolio of rainbow options gives statistically significant results already when fairly short time series are used. Results for single currency options are presented for comparison. In our creation of four simulated currency series we follow Gibson and Boyer (1998) but instead of using a bivariate normal distribution to create random returns we use a multivariate normal distribution.6 For the single option case, we use a bivariate distribution. There are five agents; agent 1 makes correct volatility and correlation forecasts, agent 2 makes correct correlation forecasts but incorrect volatility forecasts, agent 3 makes correct volatility forecasts but incorrect correlation forecasts, and finally both agent 4 and 5 makes incorrect volatility as well as correlation forecasts. In Table 1 we present the sets of incorrect volatilities and Table 1 Incorrect volatility and correlation forecast sets Incorrect volatility set Incorrect correlation set

[13.2, 13.6, 14.0, 14.3, 14.7, 15.0] [0.50, 0.54, 0.58, 0.62, 0.66, 0.70]

Correct volatility is 14.1% and correct correlation is 0.6.


The interest rate used is the US 30-day Treasury bill rate. For comparative reasons we create returns with the same distribution as Gibson and Boyer (1998); i.e. 0 mean, an annual S.D. of 14.1% and a correlation of 0.6 between all currencies. We repeat the whole trading simulation 100 times (using the same generated data series) for sample lengths of 250, 1000, 1500, and 5000 observations. 6


H.N.E. Bystro¨ m / Int. Fin. Markets, Inst. and Money 12 (2002) 216–230

Table 2 Simulation of Gibson and Boyer (1998) methodology (trading a single currency rainbow option) Forecasting agent¯number of observations





Correct variances as well as covariance No. Mean S.D. Incorrect variances No. Mean S.D. Incorrect covariance No. Mean S.D. Incorrect variances as well as No. covariance Mean S.D. Incorrect variances as well as No. covariance Mean S.D.

47 0.0046 0.0032 31 0.0033 0.0038 9 −0.0023 0.0049 10

80 0.0060 0.0019 13 0.0029 0.0023 4 −0.0019 0.0029 1

84 0.0070 0.0017 16 0.0038 0.0021 0 −0.0027 0.0027 0

93 0.0110 0.0008 7 0.0066 0.0009 0 −0.0048 0.0012 0

−0.0024 0.0050 3

−0.0032 0.0030 0

−0.0043 0.0028 0

−0.0067 0.0012 0

−0.0034 0.0050

−0.0038 0.0030

−0.0039 0.0028

−0.0064 0.0012

No., number of times (out of 100) that each model gave the highest accumulated profit over the sample; mean, mean daily profit; S.D., standard deviation of mean daily profit.

correlations. Each agent making an incorrect volatility or correlation forecast chooses incorrect forecasts at random from the set in Table 1 (the same sets are used for all currencies). Different currencies’ volatility forecasts are assumed to be correlated just as different correlation forecasts are. This behavior is simulated as follows; once the first currency’s volatility (correlation) is chosen from the set, the following are chosen from the subset of volatilities (correlations) in Table 1 that have a bias of the same sign. Volatility and correlation forecasts, however, are assumed to be independent of each other. The results for the single option case are presented in Table 2 and the results for the portfolio approach are presented in Table 3. A comparison of these two tables reveals how a simple generalization of the single option case to a portfolio version significantly relaxes the need for long data series. While up to 5000 observations are needed in the first case, the more modest 1000 observations is sufficient to distinguish correct forecasts from incorrect ones when we deal with portfolios. This observation is perhaps nothing of a surprise; it is only reasonable that including a larger number of data series relaxes the need for length of these very series. The role of the simulations is merely to quantify this idea, however, obvious it might seem. In addition, while Gibson and Boyer (1998) performed their simulations on equity indices, we show how their results remain valid for currencies and currency rainbow options.

H.N.E. Bystro¨ m / Int. Fin. Markets, Inst. and Money 12 (2002) 216–230


4. Evaluating covariance matrix forecasts of four US dollar exchange rates The currency data consists of daily ‘log-returns’ from four US dollar exchange rates over the period 2 January 1990 to 30 April 1999; the US dollar vis-a´ -vis the Australian dollar (AUD), the Canadian dollar (CAD), the Japanese yen (JPY), and the Swiss franc (CHF), respectively. The riskfree interest rates are approximated with 3-month treasury bill rates in each of the five countries.7 Some statistics are shown in Table 4. All the series show excess kurtosis and some skewness, but the Ljung–Box statistics indicate no significant autocorrelations in the index returns. The squared return series’ Ljung – Box tests, however, show large Q-values indicating autocorrelated squared returns and possible GARCH effects. To evaluate the performance of the different forecasting models we divide each time series into an estimation period and a test period. The estimation periods start 2 January 1990 and contains a minimum of 500 days. From 2 January 1992, the start of the test period, onwards, the estimation period is each day extended 1 day by including that day’s return. The test period is 1800 days long in total.

Table 3 Simulation of the extended version of Gibson and Boyer (1998) methodology (trading a portfolio of six currency rainbow options) Forecasting agent¯number of observations Correct variances as well as covariances

Incorrect variances

Incorrect covariances

Incorrect variances as well as covariances

Incorrect variances as well as covariances

No. Mean S.D. No. Mean S.D. No. Mean S.D. No. Mean S.D. No. Mean S.D.





73 0.024 0.010 23 0.014 0.013 2 −0.012 0.016 0 −0.015 0.016 2 −0.012 0.016

94 0.031 0.005 6 0.018 0.006 0 −0.014 0.008 0 −0.019 0.008 0 −0.016 0.008

96 0.030 0.004 4 0.018 0.005 0 −0.013 0.006 0 −0.017 0.006 0 −0.019 0.006

100 0.044 0.002 0 0.026 0.002 0 −0.018 0.003 0 −0.026 0.003 0 −0.027 0.003

No., number of times (out of 100) that each model gave the highest accumulated profit over the sample; mean, mean daily profit; S.D., standard deviation of mean daily profit. 7 All data were retrieved from Findata. Some observations were removed from the original dataset due to different dates of holidays in the four countries.

H.N.E. Bystro¨ m / Int. Fin. Markets, Inst. and Money 12 (2002) 216–230


Table 4 Sample statistics on daily exchange rate changes Statistics





Number of observations Mean (%) S.D. (% on a yearly basis) Skewness Excess kurtosis

2300 0.0080 9.223 0.153 5.16

2300 0.0104 5.117 −0.082 3.17

2300 −0.0075 11.976 −0.802 6.19

2300 0.0001 12.412 −0.288 2.40

11.68 22.40 29.22

9.89 18.46 26.09

7.05 12.14 20.77

11.04 15.99 18.19

186.92 222.67 248.74

129.29 175.80 220.00

234.35 313.61 355.24

99.60 148.32 194.13

Ljung–Box Q(6) Q(12) Q(18) Ljung–Box (squared returns) Q(6) Q(12) Q(18) 2 January 1990–30 April 1999.

4.1. Forecasting models For a financial institution the forecasting of covariances between currencies is important in addition to the forecasting of variances. These covariances and variances are usually, and conveniently, presented in matrix form, the covariance matrix. To forecast such covariance matrices, a number of approaches are in use; regression methods, different Weighted Moving Averages, GARCH models, nonparametric methods, etc. GARCH models are usually difficult to use in multivariate settings, but as suggested in Ding (1994) and in Alexander and Chibumba (1996) and as shown in this article, Orthogonal GARCH might be a powerful alternative to other simpler forecasting techniques.

4.1.1. Historical forecasts As a first forecasting model we take the historical sample variance and covariance (HI) using all past data. This model assumes that conditional moments are equal to their unconditional counterparts. At time t, we forecast | 2t + 1 for each currency as follows | 2t + 1 =

t 1 1 t % y~ − % yj t−1 ~ = 1 t j=1



The expressions for the forecasted covariances look very much the same.

4.1.2. Equally weighted mo6ing a6erage forecasts The 20-day equally weighted moving average model (20-days MA) is very similar to the historical model, but instead of using the whole past we now use only the last 20 days’ returns:

H.N.E. Bystro¨ m / Int. Fin. Markets, Inst. and Money 12 (2002) 216–230

| 2t − 1 =

t t 1 1 % y~ − % yj 19 ~ = t − 19 20 j = t − 19




The expressions for the forecasted covariances again look very much the same.

4.1.3. Exponentially weighted mo6ing a6erage forecasts In order to place less and less weight on observations when we move further away from them, the EWMA model with a smoothing parameter u is also applied. At time t, we forecast | 2t + 1 for each currency as follows 250

| 2t + 1 = (1− u) % u iy 2t − i



Since RiskMetrics (see RiskMetrics™ Technical Document, 1996) put u=0.94 and assumes an unconditional mean= 0 we do the same. A further approximation is the truncation of the infinite history to the 250 past returns

4.1.4. Pairwise bi6ariate GARCH forecasts The univariate GARCH(1,1) model of Bollerslev (1986) is an efficient generalization of the univariate ARCH model suggested by Engle (1982). A well-known problem with GARCH (and ARCH) models is the difficulty in estimation of model parameters when generalizing the model to a multivariate framework. In our case we deal with four currency series and to model them in a unified framework we need to restrict the parameter space somehow.8 In our first GARCH model, all currencies are modeled pairwise within the bivariate constant conditional correlation framework of Bollerslev (1990); the mean equations are specified as AR(2) processes and the conditional variance equations as GARCH(1,1). Each currency pair is modeled with this model and forecasts of variances as well as covariances for the two currencies in question are expressed by (5):9 y1,t = h1,0 +h1,1y1,t − 1 +h1,2y1,t − 2 +m1,t y2,t = h2,0 +h2,1y2,t − 1 +h2,2y2,t − 2 +m2,t


| 21,t =ƒ1,0 +ƒ1,1m 21,t − 1 +ƒ1,2| 21,t − 1 | 22,t = ƒ2,0 + ƒ2,1m 22,t − 1 +ƒ2,2| 22,t − 1 |12,t =z12,t|1,t|2,t


where | and | are the conditional variances of m1,t and m2,t |12,t is the conditional covariance between m1,t and m2,t, and mt =|tut where ut  N(0,1). The time varying 2 1,t

2 2,t

8 Our attempts to fit different fully four-dimensional multivariate GARCH models to this system of currencies have failed. In order to get as efficient estimates as possible and to get estimates of correlations we have chosen a bivariate framework instead of a univariate one. 9 We set up six different bivariate models with U.S. dollar exchange rates (AUD and CAD, AUD and JPY, AUD and CHF, CAD and JPY, CAD and CHF, JPY and CHF) since six correlation estimates are needed. Variance estimates for a particular currency are taken from the first model in which it appears in the listing above.


H.N.E. Bystro¨ m / Int. Fin. Markets, Inst. and Money 12 (2002) 216–230

conditional covariance between the currencies are parameterized to be proportional to the product of the corresponding conditional standard deviations. This assumption greatly simplifies the computational burden in estimation compared with more elaborate multivariate models. In Table 5 we present typical estimation results using the maximum likelihood estimator (BHHH); all GARCH parameters are significant while some of the AR parameters and the correlation coefficients are not.

4.1.5. Orthogonal GARCH forecasts As a second GARCH model we apply the Ding (1994), Alexander and Chibumba (1996) factor GARCH model using orthogonal factors. The main idea is to use principal components analysis to generate a number of orthogonal factors that each can be treated in a simple univariate GARCH framework. In this way one gets around the usual estimation problems connected to multivariate GARCH. We assume there are k return series with t observations (at time t) represented as a t × k matrix, Yt. The t ×k matrix, Pt, of principal components is defined as (6)

Pt = Yt Wt

where Wt is the orthogonal k ×k matrix of eigenvectors of YTt Yt ordered according to size of corresponding eigenvalue. Notice that Pt, just as Wt, is now an orthogonal matrix. By inverting Eq. (6) one gets the principal components representation of the system Yt =Pt WTt .


One can now calculate dt, the variance of Yt at time t, as Table 5 Typical parameters for the Pairwise Bivariate GARCH model and the four US dollar exchange rates (using the first half of the sample in the estimation) AUD hi,0 −0.000087 (0.00014) hi,1 −0.013 (0.031) hi,2 −0.0089 (0.029) ƒi,0×106 5.12 (2.89) ƒi,1 0.13 (0.020) ƒi,2 0.68 (0.051) zaus/can 0.20 (0.023) zaus/yen 0.0042 (0.026) zaus/swfranc 0.060 (0.025) zcan/yen zcan/swfranc zyen/swfranc Figures in parentheses are S.E.




−0.000265(0.00018) 0.0020 (0.030) −0.0083 (0.029) 0.77 (0.40) 0.047 (0.0070) 0.94 (0.0085)

−0.000342 (0.00022) −0.065 (0.027) 0.0040 (0.028) 2.43 (1.29) 0.063 (0.012) 0.90(0.018)

−0.000336(0.00022) −0.063 (0.028) 0.0013 (0.028) 2.50 (1.11) 0.063 (0.013) 0.90 (0.019)

−0.018(0.022) 0.020(0.023) 0.58(0.016)

H.N.E. Bystro¨ m / Int. Fin. Markets, Inst. and Money 12 (2002) 216–230


Table 6 Typical parameters for the orthogonal GARCH models’ four principal components (using the first half of the sample in the estimation) First PC h0×105 −43.0 (25.2) h1 −0.064 (0.027) −0.0021 (0.028) h2 ƒ0×106 2.39 (1.45) ƒ1 0.050 (0.0084) ƒ2 0.93 (0.012)

Second PC

Third PC

Fourth PC

−40.5 (13.8) −0.0061 (0.029) 0.028 (0.030) 2.77 (1.42) 0.012 (0.015) 0.79 (0.026)

−55.2 (12.4) 0.020 (0.026) 0.016 (0.029) 0.17 (0.08) 0.044 (0.0065) 0.95 (0.0081)

−93.4(27.4) −0.077(0.022) −0.047(0.023) 0.0012(0.0007) 0.028(0.0029) 0.97(0.0027)

Figures in parenthesis are S.E.

dt = var(Yt ) = var(Pt WTt ) = Wt Dt WTt


where Dt is a diagonal matrix of principal component variances at t and where Wt is assumed to be known at time t.10 This also enables us to calculate the forecasted covariance matrix dt + 1 ct where ct is the information set at t, by univariate methods; for each principal component the conditional variance of the principal component i, 6art + 1(Pi ct ) can easily be forecasted by for instance any univariate GARCH model. This gives us the Orthogonal GARCH specification. This particular covariance matrix also has the advantage of always being positive definite since Dt is diagonal with positive elements along its diagonal. In this paper we choose to model each principal component with the univariate GARCH(1,1) model described in the previous chapter. Using Eq. (8) we then transform the results from the principal components into the original currency return series. In this way we get a forecast of the whole covariance matrix. Typical maximum likelihood GARCH estimates of the four principal components are presented in Table 6. For all four principal components we get significant and positive ƒ-parameter estimates. Table 7 Mean daily profits from trading options based on covariance matrix forecasts from the five different forecasting models

Mean daily profit S.D.


20-days MA


Orthogonal GARCH

Pairwise GARCH

−0.075 0.025

−0.083 0.021

0.060 0.017

0.071 0.019

0.027 0.019

10 Wt does not change very much from day to day and one can approximate Wt with Wt − 1 without introducing large errors in the calculation of the covariance matrix. This is particularly the case when we calculate the forecasted covariance matrix. In this case we forecast at time t −1 using only information up to t −1 (including Wt − 1).


H.N.E. Bystro¨ m / Int. Fin. Markets, Inst. and Money 12 (2002) 216–230

4.2. Option trading e6aluation In order to evaluate the different forecasts we simulate an options market as described in Section 2. Results from the option evaluation approach (using portfolios of options) are presented in Table 7 and in Fig. 1. The table contains the mean daily profit and its standard deviation, and the graphs show the accumulated profit over the test period. Looking at the whole sample period, orthogonal GARCH turns out to be the winner, indicating orthogonal GARCH to be the better covariance matrix forecaster. However, considering the rather long period of quite low profits for this model relative to the pairwise bivariate GARCH model as well as the EWMA model, it is questionable if orthogonal GARCH should be considered a better performer. In Table 7 this is further indicated by the non-significant difference between orthogonal GARCH and EWMA. The difference between orthogonal GARCH and pairwise bivariate GARCH is larger but barely significant. Both the historical model and the 20-days moving average, however, clearly underperform relative to the others. The difference in mean profit between these two models and the rest of the models is highly significant. The fairly simple EWMA approach used in for instance RiskMetrics™ turns out to be relatively successful compared with the more elaborate GARCH models. One should remember, however, that in our particular market both orthogonal GARCH and pairwise bivariate GARCH suffer from strong assumptions.11 The different currencies in this paper are only weakly correlated and orthogonal GARCH is based on assumptions that partly break down when weakly correlated series are modeled together (see Alexander and Chibumba, 1996). Pairwise bivariate GARCH, on the other hand, is based on the assumption of constant conditional correlations between the four currency returns; an assumption that perhaps is less appropriate in our case as indicated by the many non-significant correlation parameters.

4.3. Statistical e6aluation In addition to the options evaluation methodology above we have also chosen to look at the traditional RMSE over the test period: RMSE =


1 1800 2 −| 2t,ij,forecast)2 % (| 1800 t = 1 t,ij,sample


where | 2t,ij,sample is estimated as the squared daily return or cross product of returns. In Table 8 we present the RMSEs for the whole covariance matrix. The results are fairly similar to those from the options trading strategy.12 Again, the group of 11 It is also a well-known fact that GARCH models generally perform worse in modelling currencies than equities. 12 This is somewhat different from the findings in Gibson and Boyer (1998). They found quite a different ranking for the statistical measure compared with the options trading strategy.

Fig. 1. Accumulated profits from trading portfolios of simulated currency rainbow options.

H.N.E. Bystro¨ m / Int. Fin. Markets, Inst. and Money 12 (2002) 216–230 228


H.N.E. Bystro¨ m / Int. Fin. Markets, Inst. and Money 12 (2002) 216–230

Table 8 RMSE for variance and covariance forecasts

Var. AUD Var. CAD Var. JPY Var. CHF Cov. AUD/CAD Cov. AUD/JPY Cov. AUD/CHF Cov. CAD/JPY Cov. CAD/CHF Cov. JPY/CHF Number of ‘gold’ Number of ‘silver’ Number of ‘bronze’


20-days MA


Orthogonal GARCH

Pairwise GARCH

0.024010 0.006403 0.044589 0.033745 0.007297 0.025238 0.013629 0.010045 0.009306 0.026524 1 2

0.023982 0.006398 0.043860 0.033759 0.007464 0.025087 0.013697 0.010161 0.009362 0.026701 0 1

0.023456 0.006294 0.043271 0.033209 0.007289 0.025036 0.019649 0.010051 0.009277 0.026322 3 3

0.023618 0.006273 0.043523 0.033613 0.007312 0.025125 0.013702 0.010044 0.009312 0.026269 2 2

0.023632 0.006278 0.043146 0.033119 0.007256 0.025245 0.013646 0.010066 0.009340 0.026221 4 2






Bold, gold; underlined, silver; italics, bronze.

more advanced models (EWMA and the two GARCH models) clearly outperforms the simpler 20-days moving average and the historical method, particularly for variances. The ranking within the top-three group is reversed, however, with Pairwise Bivariate GARCH now dominating both orthogonal GARCH and EWMA, at least if the ‘medals’ in Table 8 are given values according to the rules followed by for instance the ‘International Olympic Committee’.13 Simply adding up all medals (perhaps with different weights on different medals) could give a different result, this time with EWMA dominating both the GARCH models. In any case, the differences between the three dominating models are fairly small and picking a winner is probably not possible seen from a statistical point of view.

5. Conclusions In this paper we have shown how the simulated options trading evaluation strategy by Gibson and Boyer (1998) can be extended to portfolios of rainbow options as well as to currencies. The use of portfolios makes us less dependent on very long return series, and simulations with portfolios indicate the option trading evaluation to be a powerful technique even with fairly few observations. An application of the technique to a system of four US dollar exchange rates gives us results from a market where real rainbow options actually are traded. Five 13 One single gold medal is considered better than 1000 silver medals, one silver medal considered better than 1000 bronze medals, etc.

H.N.E. Bystro¨ m / Int. Fin. Markets, Inst. and Money 12 (2002) 216–230


different forecasting models, Historical, 20-day Moving Average, EWMA, orthogonal GARCH, and pairwise bivariate GARCH were used; all but the new orthogonal GARCH well known and widely used forecasting models. Orthogonal GARCH, on the other hand, is a new member of the multivariate GARCH family and evaluation of its performance is of interest in its own. In our particular market, orthogonal GARCH seems to work fine even though both EWMAs and pairwise bivariate GARCH modeling create equally good forecasts. The simpler historical and 20-day averages systematically give less accurate forecasts of both variances and covariances. Overall, the strategy with simulated options gives us a similar ranking of the forecasting models as the standard RMSE measure does.

Acknowledgements The author is particularly grateful for helpful comments received from participants at the 20th International Symposium on Forecasting in Lisboa, Portugal. Financial support from Bankforskningsinstitutet and Crafoordska stiftelsen is also gratefully acknowledged.

References Alexander, C.O., Chibumba, A.M., 1996. Orthogonal GARCH: an empirical validation on equities, foreign exchange and interest rates. unpublished manuscript, University of Sussex. Bollerslev, T., 1986. Generalized autoregressive conditional heteroskedasticity. Journal of Econometrics 31, 307 – 327. Bollerslev, T., 1990. Modeling the coherence in short-run nominal exchange rates: a multivariate generalized ARCH model. The Review of Economics and Statistics 31, 498 – 505. Bystro¨ m, H., 1999. Orthogonal GARCH and Covariance Matrix Forecasting in a Stress Scenario: The Nordic Stock Markets During the Asian Financial Crisis 1997 – 1998. Working Paper 2000: 14, Department of Economics, Lund University. Ding, Z., 1994. Time Series Analysis of Speculative Returns. Ph.D. Thesis, UCSD. Engle, R.F., 1982. Autoregressive conditional heteroscedasticity with estimates of the variance of United Kingdom inflation. Econometrica 50, 987 –1007. Engle, R.F., Hong, C.S., Kane, A., Noh, J., 1993. Arbitrage valuation of variance forecasts with simulated options. Advances in Futures and Options Research 6, 393 – 415. Gibson, M.S., Boyer, B.H., 1998. Evaluating forecasts of correlation using option pricing. Journal of Derivatives, vol. 6 no. 2. Winter, 18 – 38. Margrabe, W., 1978. The value of an option to exchange one asset for another. Journal of Finance 33 (1), 177 – 186. RiskMetrics™ Technical Document, 1996. RiskMetrics™ Technical Document, fourth ed., JP Morgan/ Reuters.