A Controlled Experiment In Ground-Water Flow Model Calibration
U.S. Geological Survey
1998, Ground Water
Nonlinear regression was introduced to ground-water modeling in the 1970's, but has been used very little to calibrate numerical models of complicated ground-water systems. Apparently, nonlinear regression is thought to be incapable of addressing such complex problems. With what we believe to be the most complicated synthetic test case used for such a study, this work investigates using nonlinear regression in ground-water model calibration.
Results of the study fall into two categories.
First, the study demonstrates how systematic use of a well designed nonlinear regression method can be used to indicate the importance of different types of data and can lead to successive improvement of models and their parameterizations. The method differs from previous methods presented in the ground-water literature in that (1) weighting is more closely related to expected data errors than is usually the case; (2) defined diagnostic statistics allow for more effective evaluation of the available data, the model, and their interaction; and (3) prior information is used more cautiously.
Second, the results challenge some commonly held beliefs about model calibration. These results indicate that (1) field measured values of hydraulic conductivity are not as directly applicable to models as their use in some geostatistical methods imply, (2) a unique model does not necessarily need to be identified to obtain accurate predictions, and (3) in the absence of obvious model bias, model error was normally distributed. The complexity of the test case involved implies that the methods used and conclusions drawn are likely to be powerful in practice.
In this test case, properly used nonlinear regression either produced effective, though nonunique, calibrated models capable of accurately predicting two quantities important to resource management, or provided clear evidence of model or data inadequacy. Conclusions related to demonstrating the use of the nonlinear regression method are as follows.
(1) The method of determining the weights for the weighted least-squares objective function tested in this study was used to correctly detect much smaller measurement error than would normally occur.
(2) The most conclusive indicator of model bias was unrealistic optimal parameter estimates that also had confidence intervals that excluded reasonable values. Nonrandom weighted residuals were a less conclusive indicator of model bias in the present study, but this may not always be the case, especially when models are more biased than those considered here.
(3) Including prior information in the regression diminished model accuracy when used to force optimal parameter values to be reasonable, but improved model accuracy when used to represent the hydraulic-conductivity distribution with more complexity than was supportable with the head and flow observations alone. Excluding all prior information initially allowed for clear evaluation of the contributions of different types of data.
(4) The importance of flow data was clearly demonstrated in this study. This is important because few field studies have measurements representing all of the flow leaving the system, so that problems of completely correlated parameters and the effects of a single correlation-reducing observation, as documented for the CAL0-G1 model, are probably common. These problems can be clearly characterized and understood by first representing ground-water systems very simply, and building complexity as warranted by the data and modeling objectives. Results that challenge some common practices and commonly held beliefs in model calibration include:
(5) The results of this study indicate that, given present technology, hydraulic-conductivity values measured in the field often are not as directly applicable to a numerical model of the system as would be consistent with how this data is commonly used in model calibration. Two aspects of the controlled experiment presented in this paper support this contention.
(7) Weighted residuals (observed minus simulated hydraulic heads, flows, and prior information) resulting from the regression were, in general, random and normally distributed, which is surprising because no errors had been added to the synthetically generated observations. Thus, all error was model error. It is generally thought that if model error dominates measurement error, the regression results are invalid, but the results of this work imply that the dominance of model error does not necessarily produce an inaccurate model if there is no obvious indication of model bias.
Taken with conclusion 2, conclusion 5 appears to produce a dilemma, because unrealistic parameter values are said to indicate a less accurate model, but parameter values cannot be expected to equal measured values because parameter values are accommodating model error. Yet, ranges of realistic parameter values generally are determined based on measurements. A useful resolution is derived by noting that if model error is so large that best fit parameter values are far from measured values, the resulting model is likely to produce less accurate predictions than a model for which the best-fit parameter values are close to measured values. Thus, models with more realistic best-fit parameter values are more likely to be accurate.
Conclusions 4 and 6 suggest that progress in ground-water model calibration is likely to be attained by using available data more effectively and by designing methods to collect new kinds of data, perhaps using regression methods as a guide.
Last Modified: March 17, 1999