We thank seminar participants at the OCC and the 1996 Chicago Fed Bank Structure Conference for their comments. We also thank Rene Stulz for his suggestions. The views expressed herein are those of the authors and do not necessarily represent the views of the Office of the Comptroller of the Currency or members of its staff. Address correspondence to Barry Schachter, mail stop 6-8, Comptroller of the Currency, 250 E Street SW, Washington, DC 20219; tel: 202-874-5011, fax 202-874-5394, e-mail Barry.Schachter@occ.treas.gov.

In this paper we develop an improvement on one of the more popular methods for Value-at-Risk measurement, the historical simulation approach. The procedure we employ is the following: First, the density of the return on a portfolio is estimated using a non-parametric method, called a Gaussian kernel. Second, we derive an expression for the density of any order statistic of the return distribution. Finally, because the density is not analytic, we employ Gauss-Legendre integration to obtain the moments of the density of the order statistic, the mean being our Value-at-Risk estimate, and the standard deviation providing us with the ability to construct a confidence interval around the estimate. We apply this method to trading portfolios provided by a financial institution.

For several years financial institutions have been searching for the best means to represent the risk exposure of the financial institution=s trading portfolio in a single number. Folklore attributes the inception of this quest to Dennis Weatherstone at J. P. Morgan who was looking for a way to convey meaningful risk exposure information to the financial institution=s board without the need for significant technical expertise on the part of the board members. The appeal of the idea of a simple, risk-revealing statistic has become sufficiently great that it forms the centerpiece both of many risk management systems and proposed regulatory approaches to capital regulation. Despite the popularity of this concept of measuring risk, no consensus has yet developed as to the best implementation of this risk measurement approach. This absence of consensus derives in part from the realization that each method of implementation currently in use has some significant drawbacks. In this paper we develop a means to improve one of the more popular risk measurement methods, the historical simulation approach. We then apply our method to actual trading portfolios of a financial institution.

Every approach to developing a comprehensive risk measurement statistic seeks to extract information from a forecast distribution of the return on the trading portfolio at the end of a given holding period (usually one day). Finance practitioners have focussed their attention on a statistic commonly referred to as Value-at-Risk ("VaR"). VaR is the level of return such that there is a given probability (usually, 5, 2.3, or 1 per cent) of experiencing a return of less than that level. That is, VaR is a point estimate of a given percentile of the cumulative distribution function ("cdf") of the portfolio return. In all cases of which we are aware, this point estimate is the only statistic from the forecast return distribution that is employed in VaR analysis.

This paper is concerned with the VaR estimation method known as historical simulation. Historical simulation, like all VaR estimation methods has three components. The first component is a representation of the return on each position in the portfolio as a function of underlying risk factors. In the case of historical simulation, this representation usually takes the form of an exact sensitivity to each factor (accomplished by revaluing the individual positions). The positions may instead be represented by a linear or nonlinear approximation of the sensitivity to each risk factor, depending on trade-offs the financial institution makes between computational time and accuracy. The second component consists of a model of the changes in the underlying risk factors. In the historical simulation approach potential changes in the risk factors are modeled as being identical to the observed changes in the risk factors over some historical period. This is sometimes referred to as an empirical distribution of factor returns. Modeling the risk factors underlying changes in portfolio value economizes on computation time inasmuch as the number of relevant risk factors is much smaller than the number of instruments in the portfolio. In the third component of the estimation, the VaR is deduced by relating changes in the risk factors to the factor sensitivities of the positions. In the historical simulation approach, this is accomplished by (1) calculating the changes in the values of the positions corresponding to each historically observed change in risk factors, (2) ordering the resulting portfolio value changes from smallest to greatest, and (3) finding the change corresponding to the desired percentile. For example, if 1,000 days of historical risk factor changes are employed the fifth percentile is given by the fiftieth smallest change in the portfolio.

The main strength of the usual implementation of the historical simulation approach is that it is non-parametric; i.e., no specific distributional assumptions about the data are made ex ante, and no distributional parameters need to be estimated. Therefore, the data are allowed to dictate the form of the return distribution. Indeed, Hendricks (1995) in a study using simulated spot foreign exchange portfolios found that with departures from normality in the return distribution, the historical simulation approach provided good estimates of the first percentile of the distribution. Mahoney (1995) obtained a similar result studying simulated spot currency and equity portfolios. While this approach does assume that the historically observed factor changes used in the simulation are taken from independent and identical distributions ("iid") which are the same as the distribution applicable to the forecast, this assumption is common to all VaR estimation approaches.

One shortcoming of the historical simulation approach is the potential for imprecise estimation of VaR, if the historical sampling period is "too short." Hendricks (1995) found that longer historical sample periods resulted in less variability in the VaR estimate. In applying this approach a trade-off must be made between lengthening the sample period and thereby potentially violating the assumption of iid observations, and reducing the precision of the estimate. Evaluating this trade-off is complicated by the fact that the usual application of the historical simulation approach (described above) does not produce a statistical measure of precision. In fact, as Kupiec (1995) notes, typical VaR models of all types lack the ability to measure this precision or goodness-of-fit property ex ante.

A related problem in the historical simulation approach is that the only changes in risk factors that are possible in the forecast distribution are those that are observed in the historical sample period. This problem may be especially significant in the estimation of "tail" probabilities, such as the first or fifth percentile, where the number of observations in the historical sample period that represent draws from the tail of the "true" distribution may be few. Hendricks (1995) found indirect confirmation of this problem in that longer historical sample periods on average produced larger VaR estimates under the historical simulation approach. Also, Kupiec (1995), in a simulation study using return distributions that were normal and Student-t, found that when the return distribution was fat-tailed, the usual historical simulation approach resulted in a VaR that was subject to both high variation and upward bias. He went on to suggest that the problems with this approach do not recommend its use to estimate tail values.

If it were possible to quantify the uncertainty in the estimated VaR for an unknown return distribution, the issues raised by Kupiec, Hendricks and others would be mitigated. The contribution of this paper is to develop a VaR estimator that arises from an estimated portfolio return distribution that is continuous and differentiable, thereby providing added information about the distribution of the desired percentile, including a measure of precision of the estimate (i.e., a standard error). The usefulness of a precision measure goes beyond the point made by Kupiec. With information about the precision of the estimate, it would be easier to evaluate whether large deviations of P/L from the predicted VaR were evidence of model problems. This has an impact on VaR model development and, potentially, regulatory capital allocations as well. Under the Basle market risk rules as proposed in the United States, a bank would be required to maintain additional capital if the daily P/L losses were greater than daily VaR more than 4 times in a year. To the extent that the bank (and its supervisor) can use precision information to explain such exceptions as being unrelated to the quality of the VaR model, the supervisor may elect not to require the greater capital.

Pritzker (1995) has noted that it is possible to compute a standard error in a Monte Carlo based VaR analysis. He suggests that the Monte Carlo standard error can be used to place a confidence interval around the estimate from any VaR model by relating the estimate from the VaR model to a Monte Carlo model estimate and its standard error. This approach, while feasible, may not be desirable for several reasons. A parametric representation will introduce unwanted assumptions about the portfolio return distribution. A nonparametric representation will require bootstrapping from the set of sample observations to obtain a standard error. But this approach cannot generate any information about the tail of the return distribution beyond the smallest sample observation. These issues are explored further in a subsequent section of this paper. In addition, it may not be computationally efficient for risk managers who do not currently use Monte Carlo, to obtain the usual estimate of VaR and then additionally run a Monte Carlo in order to obtain a standard error. For these reasons, we propose a different approach to measuring the precision of VaR that does not suffer from these drawbacks. In fact our approach produces a nonparametric estimate of the continuous pdf of portfolio returns.

The first step - estimating the pdf and cdf of portfolio returns

We use a kernel estimator, which can be thought of as a way of generalizing a histogram constructed with the sample data. Where a histogram results in a density that is piecewise constant, a kernel estimator results in a smooth density. Our kernel attaches a normal pdf to each data point. Note that use of a normal or Gaussian kernel estimator does not make the ultimate estimation of the VaR parametric. Smoothing the data can be done with any continuous shape. As the sample size grows, the net sum of all the smoothed points approaches the true pdf, whatever that may be, irrespective of the method of smoothing the data. This is because the influence of each point becomes arbitrarily small as the sample size grows, so the choice of kernel imposes no restrictions on the results. The smoothing is accomplished by centering each pdf on the data point with a standard deviation, also called the bandwidth, equal to 0.9Fn^-0.2, where F is the standard deviation of the data estimated from the available observations, and n is the sample size. This bandwith is based on a suggestion in Silverman (1986, p. 48). The estimation of the moments of the order statistics appears to be very insensitive to the standard deviation used to estimate the underlying portfolio pdf. Define the pdf of the portfolio return as f and the cdf of the portfolio return as F. Our kernel estimator of the pdf of trading portfolio returns is given by

While we could calculate F, the cdf of the portfolio return distribution, directly from this estimate, we use an empirical cdf for F. The empirical cdf is a step function which increases discretely at every value where a point in the data set occurs. The empirical cdf is easier to compute and shares the property of consistency with a cdf that could be computed from equation 1.

The second step - estimating the distribution of the j-th order statistic

We seek the distribution of the j-th order statistic, i.e., the value such that k data points are at or below that value and n-j above it. When the problem is stated as a percentile, p, then we seek the order statistic given by For example, with 200 data points, the fifth percentile corresponds to the tenth order statistic. When the result is not an integer, it is necessary to round; with 155 data points the fifth percentile corresponds to the eighth order statistic.

Using probability density functions estimated with the kernel density estimator above, we derive the pdf of the j-th order statistic and calculate its mean and variance. The pdf will not be analytic, but its moments can be readily calculated by numerical methods. The mean of that pdf is our estimate of VaR. From the standard error of the estimate we can calculate confidence intervals. In this way we address the problems discussed above associated with the usual historical simulation approach to VaR estimation.

The distribution of the j-th order statistic is derived as follows (Stuart and Ord (1987), 445-446). Let the order statistic whose distribution we seek be called x, with pdf g_j(x) and cdf G_j(x). Then the probability that exactly j of the data are less than or equal to x is:

so that the probability that at least j of the data are less than or equal to x is:

If at least j of the data are less than or equal to x, then the j-th order statistic is less than or equal to x. Thus, equation 3 defines the cdf of the j-th order statistic. The pdf that follows from this by differentiation with respect to x is (after some manipulation, Hoel, Port and Stone (1971), 160-163):

Equation 4 states that j-1 of the data must be less than or equal to x, one must equal x, and the rest must be greater than or equal to x.

To obtain estimates of moments of the pdf of g from equation 4, it is necessary to integrate the pdf numerically. We employ Gaussian quadrature to calculate both the mean of the pdf of the percentile, i.e., our estimate of the Value-at-Risk, and the variance of the pdf, i.e., the standard error of the estimate.

An example

To illustrate this procedure, assume that we collect three observations on X whose distribution is unknown. The three observations are 0.562336303, ­0.872515854, and ­0.555551333. These were generated as random draws from a uniform distribution over [-1,1]. The standard deviation of this sample is 0.615443831. Using a Gaussian kernel we evaluate the pdf of X at -2.0, -1.9, ..., 1.9, 2.0. The bandwidth we use is 0.444638108. The estimated pdf of X at x_i is then given by the following:

This estimated pdf is shown in Figure 1.

In this example it is straightforward to estimate the cdf of X directly from the pdf. This, plus the fact that the empirical cdf with a sample of size three is very coarse, leads us to eschew the empirical cdf for this example, even though it is used in the analysis of the next section. The value of the cdf of X at x_i then is given by the following:

With three observations, the first order statistic in the sample corresponds to the 25th percentile of the of the distribution of X. For ease of exposition, we examine the 25th percentile, rather than the 3rd percentile (as is done in the next section). The pdf of the 25th percentile is evaluated at each of the points -2.5, -2.4, ..., 1.9, 2.0 using equation 4, substituting in equations 5 and 6. The estimated pdf of the percentile is shown in Figure 2.

To estimate the mean and standard deviation of the 25th percentile, the pdf shown in Figure 2 must be integrated numerically. For simplicity of exposition, the mean is estimated using 12-point Gauss-Hermite integration, even though in the next section 128-point Gauss-Legendre integration is employed. The integral being evaluated is replaced by a sum, here with twelve elements, identified by weights, w, and points, y. The points and weights for this example are given in the appendix. The estimated mean is

­0.02405048.

This, therefore is our estimate of "VaR." The estimated variance of the pdf of the percentile is obtained using Gaussian integration as well. In particular, we can write

0.054406.

This variance would be used to construct any desired confidence interval around the VaR estimate.

Next we apply the new estimator. We examine the performance of the new estimator with actual trading portfolio data from one financial institution. We compare the usual historical simulation VaR estimate (the point estimate determined by the average of the third and fourth order statistics in the sample) to the VaR estimate obtained by applying our kernel estimator to equation 3.

Application of the new VaR estimator to actual data

The data used in the following analysis were provided by an active dealer, and they represent actual positions of the financial institution in three trading portfolios at four dates between November 1, 1995 and February 1, 1996. These portfolios are quite diverse, containing positions in various interest rate and f/x sensitive instruments such as swaps, forwards, futures, options, and cash market instruments. For each portfolio the financial institution calculated one hundred simulated changes in portfolio value based on observed changes in levels of various market risk factors, including interest rates and exchange rates, over the one hundred trading days prior to the date of the VaR calculation. For each portfolio we estimate a single VaR using all 100 changes in portfolio value.

Table 1 contains descriptive statistics for the sample returns of the twelve portfolios. The numbers have been re-scaled to help conceal the identity of the financial institution. The portfolio return distributions generally have negative skewness. They are also somewhat fat-tailed relative to the normal distribution (Kurtosis = 3), though only significantly so for one portfolio.

The results of the VaR estimation are shown in Table 2. The usual estimate, calculated as the average of the 3rd and 4th smallest changes in portfolio value is given in the second column. The kernel-based estimate is in the third column. The usual and kernel-based estimates are very close, the kernel-based estimate being more negative in 5 out of 12 cases. The greatest percentage difference in the estimates is observes in portfolio 8, where the usual estimate is about 10.5 percent greater (in absolute value) than the kernel-based estimate.

The standard error of the kernel-based estimates is given in the fourth column. The standard error as a percentage of the estimate varies considerably across portfolios, illustrating the importance of this precision information. Note that with these standard errors we could construct any desired confidence interval around our estimates. In large samples the distribution of the percentile is normal. In this case the desired confidence interval can be created by reference to a normal table and standard t, F, and chi square tests can be performed. However, we do not know the small sample distribution of this estimator of the percentile. If the estimate is not symmetric or normal, then confidence intervals would not be centered on the estimated VaR, and confidence intervals constructed from a normal table will be inefficient. It is important therefore for the kernel-based estimate, or any other estimator, to be normally distributed in small samples.

To examine the small sample properties of the third percentile, we calculated the skewness and Kurtosis of the kernel-based estimate of the VaR (see Stuart and Ord (1987, p. 322)). These results are reported in the last two columns of Table 2. In nine of twelve cases the skewness is estimated to be negative, and in seven the skewness is significantly less than zero. In two cases the skewness is significantly positive. In seven of twelve cases the Kurtosis is greater than three, but in no case is Kurtosis significantly different from three. These results suggest that standard confidence intervals can be constructed for the kernel-based estimate of the VaR, but more accurate confidence intervals could be calculated using the estimate pdf of the VaR directly.

Alternative approaches to obtaining precision information

As noted in an earlier section, resampling of the data could be used to obtain an estimated standard error, though this estimate would be restricted by the range of the observed data, and is therefore downward biased. To examine this alternative to our approach we calculate a standard error for the usual estimate by employing a Monte Carlo simulation for each portfolio consisting of 1,000 samples (with replacement) of size 100 taken from the observations. The results are presented in Table 3. The usual estimate is reproduced from Table 2 for completeness. The standard errors are generally similar to those obtained from the kernel estimator. To examine the small sample properties of the usual estimate, skewness and Kurtosis are calculated as well. Skewness is significantly different from zero in ten out of twelve cases (significantly negative in eight cases). The average skewness over the twelve portfolios is -0.5997 and the standard error of the average is 0.2991. In comparison, the average skewness of the kernel-based estimate is -0.1184 and the standard error of the average is 0.2848. For the usual estimate Kurtosis is significantly less than three in two cases. The skewness and Kurtosis of the usual VaR estimate suggest somewhat greater deviation from the normal than is the case with the kernel-based estimate. Nevertheless, the usual estimate combined with Monte Carlo could be used to construct useful confidence intervals. However, if more information is needed than information about the distribution of the third percentile, for example if the possible losses in the tail of the return distribution is required, such information is available as a by-product of the kernel-based approach, but not from this expanded analysis of the usual estimator.

An alternative that is simpler still would be to impose a parametric assumption that has fewer computational requirements. For example, if it is assumed that the portfolio return is distributed normal, then the VaR corresponding to the third percentile can be obtained immediately from the standard deviation of the portfolio returns. A standard error for the percentile could be calculated using a single Monte Carlo simulation (rather than one for each portfolio as was the case for the usual estimator). We investigate the properties of this alternative in Table 4, which also provides a direct comparison of the estimates and standard errors obtained from the three estimation methods. Normal-based estimated VaR is calculated at 1.88 times the standard deviation of the portfolio returns. The normal-based estimate of the third percentile is generally close to the usual and kernel-based estimates, suggesting that it provides a serviceable point estimate of VaR. The standard error of the normal-based estimate of VaR is calculated by multiplying the standard deviation of portfolio return by 0.21. This multiplier is obtained from the results of a Monte Carlo simulation in which 1,000 samples (with replacement) of size 100 are taken from a standard normal distribution. In this simulation the standard error of the estimated third percentile was approximately 0.21 of the standard error of the distribution. The standard error of the normal-based estimator is generally closer to that of the kernel-based estimator than the usual estimator. Because the underlying portfolio returns are generally negatively skewed, care should be taken if constructing confidence intervals with the normal-based estimator.

We have proposed a new VaR estimator that uses a Gaussian kernel and Gaussian quadrature in estimating moments of the pdf of the p-th percentile of the distribution of the return on a trading portfolio. We offer this estimator as an improvement over the usual approach to historical simulation of VaR because, unlike the usual approach, ours produces a standard error which can be used to gauge the precision of the estimated VaR. We have illustrated the application of our estimator with actual data from three trading portfolios of a financial institution. We evaluated two alternative approaches to obtaining precision information. We found that a standard error for the usual estimate could be obtained from a Monte Carlo simulation. This procedure produced a usable standard error, though it is downward biased. We noted that this approach could not be utilized to obtain additional information about the distribution of portfolio returns, such as the size of possible losses in the tail, while such information is produced as a by-product of the kernel based approach. We found that the point estimate of VaR obtained by assuming the portfolio return distribution was normal was close to the estimates obtained using the other approaches, but we speculate that confidence intervals constructed with the estimated standard errors are unlikely to be reliable because the underlying portfolio distributions are negatively skewed.

Without the additional information provided by a standard error as in the new estimator, the quality of information in the VaR estimate is difficult to assess. The estimator proposed here permits risk managers to construct confidence intervals for making ex ante risk management decisions. The estimator is also useful for evaluating the ex post performance of the VaR estimate, because it provides information which can help explain whether large deviations from VaR are the result of modeling problems (more likely explanation, if the standard error was small) or market conditions (more likely if the standard error was large). This ability may help improve risk management decisions, and it should aid in determining the appropriateness of increased regulatory capital charges for market risk when a bank's actual P/L losses exceed forecast VaR with greater frequency than allowed for under the proposed capital rules.

Beder, Tanya. 1995. VaR: Seductive but Dangerous. Financial Analysts Journal (September-October), 12-24.

Butler, J. S. and Barry Schachter. 1995. Measuring Option Risk Exposure. Unpublished manuscript (December).

Crnkovic, C. And J. Drachman. 1995. A Universal Tool to Discriminate Among Risk Measurement Techniques. Unpublished manuscript, J. P. Morgan (September).

Hendricks. Darrell. 1995. Evaluating Value-at-Risk Models Using Historical Data. Unpublished manuscript, Federal Reserve Bank of New York (July).

Hoel, Paul, Sidney Port and Charles Stone. 1971. Introduction to Probability Theory. New York: Houghton Mifflin Company.

Jackson, Patricia, David Maude and William Perraudin. 1995. Capital Requirements and Value-at-Risk Analysis. Unpublished manuscript.

Jordan, James and Robert Mackay. 1995. Assessing Value at Risk For Equity Portfolios: Implementing Alternative Techniques. Unpublished manuscript, Virginia Tech University (July).

Kupiec, Paul. 1995. Techniques for Verifying the Accuracy of Risk Measurement Models, forthcoming, Journal of Derivatives.

Kupiec, Paul and James O=Brien. 1994. The Use of Bank Risk Measurement Models for Regulatory Capital Purposes. Unpublished manuscript, Board of Governors of the Federal Reserve (March).

Mahoney, James. 1995. Empirical-based Versus Model-based Approaches to Value-at-Risk. Unpublished manuscript, Federal Reserve Bank of New York (September).

Pritzker, Matthew. 1995. Evaluating Value at Risk Methodologies: Accuracy versus Computational Time. Unpublished manuscript, Board of Governors of the Federal Reserve (November).

Silverman, B. 1986. Density Estimation for Statistics and Data Analysis. London: Chapman and Hall.

Smithson, Charles and Lyle Minton. Value-At-Risk. Risk 9 (January), 25-27.

Stuart, Alan and J. Ord. 1987. Kendall=s Advanced Theory of Statistics, volume 1, fifth edition. New York: Oxford University Press.

This Appendix contains the points and weights used in the Gaussian integration performed in the example of section II.

x w wexp(x^2)

­0.314240376254359 0.5701352362625 0.6293078742695

­0.947788391240164 0.2604923102642 0.6396212320203

­1.5976826351526 0.05160798561588 0.6626627732669

­2.27950708050106 0.003905390584629 0.7052203661122

­3.02063702512089 0.000085736870436 0.7866439394633

­3.88972489786978 0.000000265855168 0.9896990470923

0.314240376254359 0.5701352362625 0.6293078742695

0.947788391240164 0.2604923102642 0.6396212320203

1.5976826351526 0.05160798561588 0.6626627732669

2.27950708050106 0.003905390584629 0.7052203661122

3.02063702512089 0.000085736870436 0.7866439394633

3.88972489786978 0.000000265855168 0.9896990470923

TABLE 4
Usual, Kernel-based, and Normal­based estimates of Value-at-Risk for three trading portfolios of a financial institution, each observed at four dates, with Standard Errors of the estimates.
Port/Date	Point Estimate of VaR			Standard Errors of VaR Estimates
	Usual	Normal	Kernel	Usual	Normal	Kernel
01/01	­2909	­2882	­2916	277	297	273
02/01	­4890	­5378	­4493	1395	613	845
03/01	­ 606	­ 671	­591	104	79	65
04/02	­2182	­2059	­2363	337	273	373
05/02	­6157	­6069	­6092	663	647	679
06/02	­ 247	­214	­ 273	60	25	65
07/03	­2000	­2138	­1978	356	206	280
08/03	­3680	­4599	­3326	1126	468	230
09/03	­ 111	­141	­ 107	50	17	7
10/04	­2484	­2566	­2477	175	266	162
11/04	­5723	­5302	­5722	752	555	805
12/04	­ 244	­ 229	­ 257	46	27	50
See Table 1 for data description. The normal-based VaR estimate is -1.88 standard deviations of portfolio dollar returns. The standard error of the normal-based VaR estimate is 0.21 times the standard deviation of the portfolio dollar returns, where the 0.21 is obtained from the standard error of the estimate of the 3rd percentile from a Monte Carlo simulation of 1,000 samples of size 100 from a normal distribution with replacement.