In (Most and Will 2008) a model independent measure to assess the model quality was proposed. This measure is the Coefficient of Prognosis (CoP), which is defined as follows

(2–17)

where is the sum of squared prediction errors. These errors are estimated based on cross validation. In the cross validation procedure, the set of support points is mapped to q subsets. Then the approximation model is built by removing subset i from the support points and approximating the subset model output using the remaining point set. This means that the model quality is estimated only at those points which are not used to build the approximation model. Since the prediction error is used instead of the fit, this approach applies to regression and even interpolation models.

The evaluation of the cross validation subsets, which are usually between 5 and 10 sets, causes additional numerical effort in order to calculate the CoP. Nevertheless, for polynomial regression and Moving Least Squares, this additional effort is still quite small since no complex training algorithm is required. For other meta-modeling approaches such as neural networks, Kriging and even Support Vector Regression, the time consuming training algorithm has to be performed for every subset combination.

In Figure 2.7: Convergence of the CoP Measure by Using MLS Approximation Compared to the Polynomial CoD Measure, the convergence of the CoP values for an MLS approximation of the nonlinear coupled function given in Equation 2–12 is shown in comparison to the polynomial CoD. The figure indicates that the CoP values are not over-estimating the approximation quality like the CoD does for a small number of samples.

Figure 2.7: Convergence of the CoP Measure by Using MLS Approximation Compared to the Polynomial CoD Measure

The influence radius of the MLS approximation is found by maximizing the CoP measure. As can be seen in Figure 2.7: Convergence of the CoP Measure by Using MLS Approximation Compared to the Polynomial CoD Measure, the convergence of the approximation quality is much better if only the three most important variables are used in the approximation model.