PDM Calibration

PDM (Probability Distributed Model) calibration is the means for automatically calibrating the parameters used by a PDM Descriptor in a subcatchment, using observed rainfall, flow, potential evaporation and temperature from a time series database (TSDB) to optimise the set of parameter values used by the PDM descriptor in a particular scenario.

Calibration of a PDM involves the automatic adjustment of model parameters to achieve a model that:

• Provides a good fit between simulated and observed flow, as measured by the objective function.
• Provides a good fit between simulated and observed flow, as determined by careful analysis from the hydrological modeller.
• Has a set of parameter values that can be justified in relation to their conceptual meaning within the model theory.

The Settings tab on the PDM Calibration Dialog allows you to specify the settings in order to calibrate a model, which are then processed using the Calibrate tab on the same dialog.

The output from the calibration is shown as a graph which displays the observed rainfall and flow, together with the simulated soil moisture deficit, baseflow and runoff flow. In addition, the calculations for the objective function and the coefficient of determination (R2) are also displayed on the Calibrate tab.

See Calibrating PDMs for further information about how to calibrate a PDM descriptor.

The objective function and R-squared

The objective function is the function used by InfoWorks ICM to compare simulated and observed flow. If one set of model parameters yields a lower value of the objective function than another, it is deemed to give a 'better fit'. The objective function is, by default, the mean of the squares of the differences (errors) at each time-step. Other choices of objective function are available which can be selected in the Settings tab on the PDM Calibration Dialog.

The value of the objective function will vary with the mean value of the flow as well as the size of the errors, and so cannot be used to compare the performance of models for different subcatchments. One standardised measure of model performance is R2 (also known as the coefficient of determination) which compares the variance of the difference between simulated and observed flow with the variance of the observed flow itself. A value of 1 implies a perfect fit, while a value of 0 implies that the model simulation performs no better than the observed mean. Of course, unlike a linear regression it is entirely possible for the model to perform worse than the observed mean, yielding a (somewhat counter-intuitive) negative value for R2.

The calculated values of the objective function and R2 are displayed on the Calibrate tab which can be used to compare model performance across events.

Automatic optimisation

The optimisation process is essentially a procedure for minimising the value of the objective function. It is based on Nelder and Mead’s simplex search method, and provides effective and robust minimisation of a complicated and discontinuous function.

Automatic optimisation is applied only to those parameters marked with a cross (x) next to their name in the Parameters grid on the Calibrate tab. Assuming there are N such parameters, the search method moves through the N-dimensional parameter space in discrete steps, calculating the value of the objective function after each step. The search starts using the step-size for each parameter that is specified in the Step column of the Parameters grid, and varies as optimisation proceeds, reducing as a possible minimum is approached. The procedure stops when either:

• The upper limit on the number of steps (set in the Steps field) is reached

or

• None of the recently calculated values of the objective function differ by more than the value in the Converge field, and the step size for each parameter has reduced to less than the value in the Tolerance column of the Parameters grid.

In practice, it is often simplest to set small values for Converge and Tolerance, and run the optimisation repeatedly for fixed numbers of steps (say, between 100 and 500) until the parameter values and objective function cease to change appreciably.

Unrestrained application of the automatic optimisation procedure to many parameters at once can quickly lead to a situation in which the first of the calibration criteria listed above (minimal objective function) is met, but the others (sensible-looking fit, meaningful parameter values) are not. A more effective approach is to optimise smaller numbers (2 to 5) of interdependent parameters that are difficult to adjust manually.