Errors in Functions of the Parameters of Statistically Estimated Models



Errors in Functions of the Parameters of Statistically Estimated Models

Authors

A Daly, ITS, University of Leeds, UK; G de Jong, RAND Europe, NL

Description

Gives and justifies a procedure for calculating errors in parameter ratios, forecasts, consumer surplus etc., derived from statistically estimated models.

Abstract

Discrete choice models are often specified to contain parameters which must be estimated from observed behaviour, often using the maximum likelihood criterion. This criterion has the advantages that it yields minimum-variance, asymptotically unbiased and asymptotically multivariate-normal estimates and also gives asymptotic estimates of the errors associated with those estimates. These error estimates allow analysts to assess the success of their estimation, using techniques such as t ratios or (for non-linear models) asymptotic t ratios.

Once estimated, models can be used in a number of ways, such as:
· the calculation of simple functions of the parameters ? in an important example the calculation of values of time as the ratio of time and cost coefficients;
· the prediction of future behaviour of populations, using aggregation techniques such as sample enumeration;
· the calculation of user benefit measures.
Of course, all of these applications are functions of the estimated parameters and, as such, are subject to the errors associated with the estimation of the parameters. It is important therefore to be able to assess the error associated with statistics derived from the estimated parameters.

Conventionally, two techniques have been used to derive these errors.
· Exploiting the asymptotically multivariate normal distribution of the estimates, text-book results concerning functions of normal variates have been used to derive distributions, and hence error measures, for simple functions of the parameters. This procedures has often been followed to derive error estimates for values of time.
· Sampling procedures have been applied to the estimated parameters, again treating them as multivariate normal distributions, and distributions of the relevant functions of the sampled values have been used to obtain information about the distribution of the function required.
In the paper it is shown that the first approach is inefficient and can better be replaced by the method set out in the paper. The second approach can also be replaced, with a substantial saving in convenience and time, when the functions concerned are not too complex. However, in very complicated situations or for very large models sampling from the parameter distribution remains the only practical approach.

The proposed approach is inspired by the well-known approximation for the variance of a function of random variables as a function of the covariance matrix of those variables and the first derivatives of the function with respect to those variables. It is shown that application of this formula gives the same results for a number of important cases, including the ratio of coefficients, as the ad hoc calculation from first principles given in text books. The formula is however more general and gives more insight into how the error depends on the structure of the problem. Further examples are given of calculations of error in forecasts, pivoted forecasts, forecast changes and consumer surplus measures.

However, the status of these calculations is considerably greater than might be appreciated. First, because the asymptotic normality of the original parameter estimates depends on a second-order approximation, while the calculation of the variance of the function is a first-order approximation, it is possible to prove (under reasonable conditions) that the function is also asymptotically normal. Second, because likelihood could just as well have been maximised with the derived function as one of the original arguments, the estimate of the function is itself a maximum likelihood estimate (again under reasonable conditions), with the properties of minimum variance and asymptotic lack of bias of all such estimates. The paper indicates the conditions under which the properties hold.

These results are then used to throw more light on the commonly-used ?t ratio? and some paradoxes concerning these tests.

The value of this paper is in indicating simple calculation methods that can be used to obtain error measures for many commonly-needed statistics, to show that these error measures often have as much validity as the original parameter estimates and generally to give insight into a number of paradoxes concerning error calculations.

Publisher

Association for European Transport