Luca

Luca is a tool for building and performing a procedure to calibrate parameters for a (hydrologic) model. Luca integrates the following components in a wizard-style user interface:

1. Multiple-objective, step-wise calibration
2. Shuffled Complex Evolution (SCE) global-search parameter optimization; and
3. the Object Modeling System (OMS).

Shuffled Complex Evolution (SCE)

1. Generating points: The set of parameters to be calibrated is considered as a point in N dimension space where N is the number of parameters. SCE generates many points, in which each parameter has a random value within its lower and upper bound values.
2. Assigning criterion values: The OMS model is run with every point (a set of parameters) generated in SCE Step 1 as an input. An objective function that determines how close the simulation results are to observed values is used to calculate a criterion value for each point.
3. Creating complexes: The points are divided into smaller groups called complexes such that points of good and bad criterion values are equally distributed.
4. Complex evolution: Each complex is evolved in the following way: Several points are selected from the complex to construct a sub-complex. In the sub-complex, a new point is generated, and a point that has a bad criterion value is replaced with this new point. This evolution step is repeated several times with different random points in a sub-complex.
5. Combining complexes: All points in the complexes are combined together to be one group.
6. SCE Steps (3) – (5) are called a shuffling loop, which is repeated until the results of the complex evolution meet one of the following end conditions:

• The number of OMS executions reaches the maximum number of model execution
• The percent change in the best criterion value of the current shuffling loop and that of several shuffling loops before is less than a specified percentage

The points converge into a very small region, which is less than 0.1% of the space within the lower and upper bounds of parameters. The number of complexes used in SCE Step 3 decreases by 1 for every shuffling loop. This decrease stops when the number of complexes reaches the minimum number of complex required. The output is the parameter file containing the point (a parameter set) that has the best criterion value.

Luca Wizard Steps

The following calibration wizard steps are provided by Luca. The step number and caption matches the list as shown in the wizard window.

. Start

The user has a choice of starting a new session or choosing a previous one by opening an existing session file. Session files can be created by saving the session in Wizard Step 12.

Opening Session File

When the user opens a session file, the project directory specified in the selected session file is displayed in the project directory in the session file field. The user has an option of changing the project directory but keeping the rest of settings by clicking the Change button and selecting another project directory.

When some fields in the selected session file are invalid or missing, the checkbox Skip to last step and run the calibration using this settings is disabled, and thus the user is not allowed to skip to the last step. However, the user can still proceed to following steps and fill out the missing fields.

. Model and Parameter Files

In this wizard step, the model and parameter files are selected by selecting a Project Simulation. In OMS a Project Simulation is a user-specified combination of a specific model, parameter set, and data set. The Simulations available in the Project Simulation directory are displayed in the Basic Simulation File panel. In the example below, the Basic Simulation prms_test_cc.jsa has been selected. The contents of this Simulation are displayed. The model, chosen from the Project Model directory, is prms_test_cc.jma. In OMS the parameter set can be divided into multiple files to enable easier access to specific groups of parameters. In this example, the efcarson_dates.csp file contains time-specific parameters, such as start and end times of the simulation. The efcarson_files.csp file contains parameters specific to input and output files. The efc1108.csp contains the basin-specific parameters for the model selected.

The efcarson_luca.csp file contains parameter information specific to a Luca application. An example of this file is shown below:

@Properties, Parameter
 created by, od

@Properties, Luca
 created by, od

# Obs data set (static) 
@Property, obs_file,    "${prj.dir}/data/efcarson.csv"
@Property, obs_table,   "efcarson"
@Property, obs_column,  "runoff[0]"

# simulated data set (dynamic)
@Property, sim_file,    "${sim.output}/Luca/efc.csv"
@Property, sim_table,   "Output"
@Property, sim_column,  "basin_cfs"

The Properites tags identify this as a parameter file and a Luca specific file. The Obs data set and simulated data set Properties define the input and output files to be used to obtain observed and predicted variables for objective function computations in Luca. All variables in each file are available for selection in the creation of objective functions used in Luca (see Step 10 below).

. Rounds and Steps

In this Wizard Step, the user is prompted to set the number of rounds and steps.

Rounds and Steps

In the multi-step calibration technique, a step and a round are defined as follows:

Steps

A step is associated with a parameter set, which contains one or more parameter values.

Rounds

A round consists of one or more steps.

Examples:

Assume there are N rounds, each of which contains M steps. The following is how this program performs calibration:

1. In Step 1 of Round 1, a user-defined input parameter file is used, and the parameter values of Step 1 are calibrated based on the SCE algorithm. The calibrated values are written in an output parameter file.
2. In Step 2 of Round 1, the output parameter file in Step 1 of Round 1 is used as an input parameter file. The parameter values are calibrated based on the SCE algorithm, and an output parameter file is produced.
3. The same procedure is repeated for Step 3, 4, ..., N of Round 1.
4. In Step 1 of Round 2, the output parameter file in Step N of Round 1 is used as an input parameter file. The parameter values are calibrated based on the SCE algorithm, and an output parameter file is produced.
5. The same procedure is repeated for the remaining Steps and Rounds.

. Calibration Period

The calibration period is the period for which parameters are calibrated. When selecting the calibration period, you also have to specify the Model Initialization Start Date. The model runs for the period from the selected Model Initialization Start Date to the Calibration End Date, and generates an output file containing values for this period. However, objective function calculation is performed only on the simulated and observed values of the period from the Calibration Start Date to the Calibration End Date. The initialization period is typically set to run the model through at least one wetting and drying cycle in an effort to remove any bias created by the user-defined initial model state variables.

In the example above, the model runs from 1980/10/1 to 1984/9/30, and the objective function calculation is performed using simulated and observe values for the period 1981/10/1 to 1984/9/30.

. Parameter Selection

In Step 3, one Round and two Steps were specified. In Step 5, the parameters for each step are selected. A parameter set for each calibration step is defined by selecting parameters from Available Parameters list. These parameters are automatically displayed based on the input parameter file specified in the Project Simulation selected in Step 2. The user creates a distinct parameter set for each calibration step by clicking the appropriate Step tab and then selecting the desired parameters.

In the panel below, the Step 1 tab and the parameter jh_coef have been selected. The selected parameter is moved to the Selected Parameters list by clicking on the Add button.

To select parameters for Step 2, the Step 2 tab is selected. In the example below, two parameters, soil2gw_max and ssrcoef_sq, have been chosen and moved to the Selected Parameters list by clicking on the Add button. Parameters can be added one at a time, or can be selected in multiples by holding down the Ctrl key when making parameter selections in the Available Parameters list.

6. Calibration Strategy

In Wizard Step 6, the user is prompted to do the following three things for selected parameters in each step:

• Selecting one of the three calibration types:
  • Use the mean value
  • Use the individual values
  • Parameters are binary
• Setting initial parameter values.
• Setting the lower and upper bounds for each parameter.
• (Optional) Saving the modified parameter values in a new parameter file.

Calibration Types

The three calibration types are available for calibrating the parameter values:

Use the mean

The mean, instead of individual values, is calibrated for a given parameter. Every time SCE generates a value for the mean, parameter values are generated based on the new mean such that the value distribution is preserved. See Generation of Parameter Values Based on the Mean at the end of this section to learn how the parameter values are generated.

Use the individual values

The individual values of a given parameter are calibrated. The value distribution will be changed during calibration since each value is calibrated. In the example above, the parameter jh_coef is a monthly parameter and thus has 12 values. Selecting the calibration of individual values assumes that there are observed or estimated values available against which computed values can be compared. In this example, daily or monthly observed or estimated values are needed. Selection of daily or monthly observed or estimated values are discussed in more detail in Step 10. A subset of the individual values can also be selected for calibration by turning on only a selected set of check marks for the desired subset.

Parameters are binary (0, 1)

In the case of a binary parameter, the user needs to choose this option. The parameter values must be either 0 or 1. Calibration will be done in the same way as Use the individual values; the difference is that if the calibrated value is greater than or equal to 0.5, the parameter value is set to 1, and set to 0 otherwise.

Initial Parameter Values

The default parameter values shown in this Wizard Step come from the input parameter file defined by the Simulation. The individual values or the mean values of the parameters specified in this Wizard Step will be used as initial values of one of the points in SCE. If you do not want to specify the initial parameter values or mean values as one of the points, check the box Do not use values from parameter file. In this case, all points are generated by SCE.

Lower and Upper Bounds

The lower and upper bounds for each parameter in every step are used to generate points in SCE. In SCE, a set of selected parameters is considered as a point in N dimension space where N is the number of parameter values in the parameter set. Points are randomly generated by SCE such that each parameter value is within its lower and upper bounds. (See also the brief description of SCE above.)

Restriction:

• Lower bound cannot be greater than the minimum parameter value.
• Upper bound cannot be less than the maximum parameter value.

Actual Bounds

After entering the values for lower and upper bounds, the actual range will be displayed. The actual bounds is exactly the same as the user defined bound if Use the individual values or Parameters are binary is selected for the use of parameter values during calibration. If Use the mean is selected, then the actual bounds are calculated as follows:

where

• [lower bound] = the user defined lower bound
• [upper bound] = the user defined upper bound
• [original mean] = the mean of the values of a given parameter
• min = the minimum value of the values of a given parameter
• max = the maximum value of the values of a given parameter
• C = (absolute value of [lower bound]) + 10

Learn how the equations above are obtained.

Saving Modified Parameter Values

When the initial parameter values are set by modifying the parameter values, the user may want to save the changes in a new parameter file. Luca allows the user to do this as an option. If a file is specified at the bottom of the Wizard Step 6 screen, the parameters with any changes in their values are saved in the file when the user clicks the Next button.

The calibration strategy for the parameters selected in Step 2 is to Use the mean value. The soil2gw_max parameter is a distributed parameter with an individual value for each spatial hydrologic response unit. The generation of distributed parameters using the mean value is described below.

Generation of Parameter Values based on the Mean

If the mean value is calibrated instead of individual values, the parameter values must be regenerated from the calibrated mean value. Each value of a parameter can be generated by the following equation: (1)

where

• [new mean] = the mean value generated by SCE
• [original value] = the original parameter value
• [original mean] = the mean of the original parameter values
• [new value] = a new value, corresponding to [original value], generated based on [new mean]

Equation (1) does not work if [original mean] = 0. To avoid this problem, some constant C is added to each variable in Equation (1). C is a constant that is large enough to ensure that the values of all variables are positive:

Therefore, the equation that determines the new value is:

In Luca, C is defined as: C = [absolute value of the user defined lower bound] + 10

If the absolute value of the user defined lower bound is added to parameter values, it is guaranteed that all of them will be positive because no value is less than the lower bound. Note that 10 is added to C so that the denominator [original mean] + C can never be 0.

7. SCE Control Parameters

In this Wizard Step, SCE control parameters need to be defined for each step. The default values are automatically calculated based on the suggestions in the Duan et al. (1994). The definition and the details on the SCE control parameters are discussed below. For more information on these parameters the user is referred to Duan et al. (1994).

Definition of SCE Control Parameters

Definition of SCE control parameters (See also the brief description of how SCE works above to get a better understanding of the SCE control parameters):

1. Maximum number of model executions: The maximum number of model executions, which is one of the end conditions of SCE. If the number of model executions reaches this maximum number, the program terminates [See # 6 of the brief description of SCE above ].
2. Initial number of model execution: The number of complexes constructed from the initial population of points. The number decreases by 1 for every shuffling loop until it reaches the minimum number (See #6). The program execution time increases as the number of complexes increases.
3. Number of points in each complex: The number of points contained in each complex.
4. Number of points in a sub-complex: The number of points randomly selected from a complex to construct a sub-complex [See # 4 of the brief description of SCE above ].
5. Number of evolution steps before shuffling: The number of times the evolution steps are performed during each shuffling loop [See # 4 of the brief description of SCE above ].
6. Minimum number of complexes required: The minimum number of complexes required for the shuffling loops. The number of complexes decreases by 1 for every shuffling loop until it reaches this minimum number. After that, this minimum number of complexes is kept constant until SCE terminates.
7. Shuffling loops in which the objective function value must change by given percent before optimization is terminated: This value is related to one of the SCE end conditions. If the percent change between the current best objective function value and the objective function value observed this specified number of shuffling loops before is less than the percentage for the criterion value SCE control parameter (See #8), then the program terminates [See SCE Step 6 of how SCE works].
8. Percentage for the objective function value: See the description #7.

SCE Control Parameter Values

N =Number of parameter values. This is the total number of values in the parameter set. For example, if the parameter set has two parameters where one contains 12 values and the other contains 1 value, then N is 13.

8. Number of Objective Functions

For each step, Luca allows the user to use the multi-objective function technique in which the objective function value is calculated from multiple objective functions. In each step, the user must define The number of objective functions to be used in multi-objective functions.

9. Multi-Objective Function Setup

In this Wizard Step, the user is asked to set the following for individual objective functions in multi-objective functions:

• Objective function type
• Weight of each objective function
• Data Subdivide file and value (optional)
• Time step

The user must specify that the objective function value must be either minimized or maximized during the calibration by selecting Minimized or Maximized in the combo box.

Objective Function Type

Each objective function is defined below.

Normalized Root Mean Square Error (NRMSE):

This objective function value is considered good when it is small.

Nash-Sutcliffe (NS):

This objective function value is considered good when it is large (closer to 1.0).

Absolute Difference (ABS):

This objective function value is considered good when it is small.

Absolute Difference log (ABS log):

This objective function value is considered good when it is small.

Pearson's Correlation:

Refer to the function PEARSN in the book Numberical Recipes in Fortran (http://www.nr.com). This objective function value is considered good when it is large.

Weight for Each Objective Function

In Luca, the final objective function value based on multi-objective functions for each step is calculated as follows:

where OFf is the final objective function value, nOF is the number of objective functions for given step, wi is the ith weight associated with the ith objective function (OFi ). The user is prompted to set the weight for each objective function in every step

Data Subdivide

The user has a choice of using a subset of the observed data for model calibration. If the Use Data Subdivide file is chosen, the user is prompted for an input file that consists of the date and a value. When the data value for a given time step is equal to the value listed in the Data Subdivide value, then data for that day is used in model calibration. For example, the user may only want to calibrate runoff on days with irrigation diversions, peak flows, or significant hydrometeorological events.

Data Subdivide File Format

The format of the data_subdivide file is: Year Month Day Value

where Value is the Data Subdivide value, which must be an integer.

Example:

2000 10 1 1
2000 10 2 1
2000 10 3 2
2000 10 4 2
2000 10 5 2
2000 10 6 1
2000 10 7 3
...

If the Data Subdivide technique is used, simulated values and observed values of only desired dates can be used for the objective function calculation. Assume the Data Subdivide value is 2. Then, the simulated and observed values of the dates 2000/10/3 - 2000/10/5 are used, but those of the dates 2000/10/1, 2000/10/2, 2000/10/6, and 2000/10/7 are not used in the above example.

Restriction

If the Data Subdivide technique is used, the following must be satisfied:

• The Data Subdivde file must contain the data from a time period that is the same or longer than the calibration period excluding model initialization period.
• If the observed calibration data set comes from an external source (not statvar file), then the format of the external source file must be the daily format (Year Month Day Value).

Time Step

The time step specified in Wizard Step 9 is used to calculate an objective function value for simulated and observed values. The available time steps are:

• Daily
• Monthly Mean: the arithmetic mean of individual daily values within a given month
• Mean Monthly: the arithmetic mean of the monthly means of a given month during a specific period of years
• Annual Mean: a water year mean

The calibration period must be at least 1 year (excluding the model initialization period) to select Annual Mean.

Number of Days

If the user selects Daily, then the Number of Days must be chosen. A value greater than 1 uses an n-day moving average for objective function calculation. If the Data Subdivide file is used, then Number of Days is automatically set to 1.

Example:

2001 10 1 50.3
2001 10 2 40.1
2001 10 3 55.5
2001 10 4 39.0
2001 10 5 42.3
2001 10 6 41.8
....
2002 9 27 57.1
2002 9 28 55.2
2002 9 29 54.8
2002 9 30 51.0

Assume you want to use the daily data above and Number of Days is set to 3. Then, the program adds up daily values of 2001/10/1, 2001/10/2, and 2001/10/3 (50.3, 40.1, and 55.5), to get 145.9, which is used as a value of the date 2001/10/1. The sum of the values of 2001/10/2, 2001/10/3, 2001/10/4 is used as a value of 2001/10/2, and so forth. There are not enough values for the last two dates, 2002/9/29 and 2002/9/30, so values are not calculated for them.

The daily values below show what values are actually used for the calculation if the daily values above are used and Number of Days is 3.

2001 10 1 145.9
2001 10 2 134.6
2001 10 3 136.8
2001 10 4 123.1
....
2002 9 27 167.1
2002 9 28 161.0

Period

The Period field must be specified if the Monthly Mean, Mean Monthly, or Annual Mean time step is selected. Period allows the user to select a period of months whose monthly mean, mean monthly, or annual mean values are used for objective function calculation. For example, if Period is set to Oct. - Mar., then the monthly mean, mean monthly, or annual mean values for the period from October to March of each year are used for objective function calculation, and values from April to September are ignored. If Period is set to Oct. - Sept., then all monthly mean, mean monthly, or annual mean values are used.

0. Simulated and Observed Variables

In this Wizard Step, the user is prompted to select the simulated variable from a list of variables that are produced by the model and saved in the model output file. The corresponding observed data must be identified and the source must be indicated as either Input File or External Source. If the user chooses Input File, the user must select an available observed variable from the Observed Variable list. If the user chooses External Source, the user must supply the observed data file, which must contain the data covering at least the calibration period, excluding the model initialization period. When this External Source is selected, the observed data type (Value or Range) must be set. In this wizard step, the user is also asked to set the Data Missing Value for observed values.

Observed Data File from External Source

Format for the Value type

If the observed data file contains an observed value for each date, it is the Value type. If the Value type is selected, the file format of the observed data must be selected from several choices:

• Daily (Year Month Day Value): This format can be selected for any time step used for objective function calculation. These daily values in the file are converted into monthly mean, mean monthly, or annual mean values as necessary.
  
  • Example:

        2001 10 1 21.0
        2001 10 2 13.5
        2001 10 3 15.1
        ...
    

• Monthly Mean (year month value): This format can be selected if the time step for objective function calculation is anything except for Daily. These monthly mean values in the file are converted into mean monthly or annual mean value as necessary.
  
  • Example:
    

    2001 10 5.5
    2001 11 9.1
    2001 12 15.2
    ...

• Mean Monthly (month value): This format can be selected only if the time step for objective function calculation is Mean Monthly for all objective functions for a given step. The file does not have to contain values for all 12 months, but must contain mean monthly values of the Period selected in Wizard Step 9.
  
  • Example 1:

     1 43.1
     2 50.4
     ...
    12 49.8

The data can also start with October.

• • Example 2:

      10 55.1
      11 61.4
      12 49.8
       1 43.1
       ...
       9 49.8

• Annual Mean (year value): This format can be selected only if the time step for objective function calculation is Annual Mean for all objective functions in a given step.
  
  • Example:

      2001 2.2
      2002 3.05
      2003 1.4
      ...

NOTE: There is only one space between each field in the examples shown above, but there can be more than one space.

Format for the Range type

Observed data files of the Range type must contain lower and upper bounds for observed values. The file format varies depending on the time step selected for objective function calculation. In order to use the Range type, the same time step must be selected for the all objective functions for a given step.

If the Range type is selected, the format which the file must follow is displayed in the panel:

• Daily (Year Month Day Lower_bound Upper_bound): This is the format to follow when the Daily time step is selected for all objective functions.
  
  • Example:

    2001 10 1 18.1 22.0
    2001 10 2 10.5 14.9
    2001 10 3 12.9 16.0
    ...

• Monthly Mean (Year Month Lower_bound Upper_bound): This is the format to follow when the Monthly Mean time step is selected for all objective functions
  
  • Example:

     2001 10 4.9 5.8
     2001 11 7.8 10.0
     2001 12 14.3 16.2
     ...

• Mean Monthly (Month Lower_bound Upper_bound): This is the format to follow when the Mean Monthly time step is selected for all objective functions.
  
  • Example 1:

     1 40.8 44.2
     1 49.2 52.1
     ...
    12 49.3 50.8

The data can also start with October.

• • Example 2:

      10 50.0 57.1
      11 57.23 64.0
      12 49.3 50.8
       1 40.8 44.2
       ...
       9 49.8

• Annual Mean (Year Lower_bound Upper_bound): This is the format to follow when the Annual Mean time step is selected for all objective functions
  
  • Example:

       2001 1.43 2.86
       2002 2.44 5.1
       2003 0.4 2.0
       ...

Objective Function Calculation with the Range type Observed Data

When the Range type is selected, the observed values used in the objective function calculations are determined based on the range defined by the lower and upper bounds in the observed data file and the simulated value generated by the model. If a simulated value is within the range for a given date, the observed value is set equal to the simulated value. If it is less than the lower bound of the range, the observed value is set to the lower bound. If it is greater than the upper bound, then the observed value is set to the upper bound.

Example: Suppose the observed data file contains the following:

2001 10 1 18.1 22.0
2001 10 2 10.5 14.9
2001 10 3 12.9 16.0
....

and the simulated values generated by the Model are:

2001 10 1 28.3
2001 10 2 12.6
2001 10 3 8.9
....

Then, the observed values that will be used for objective function calculation are:

2001 10 1 22.0 (the same as the upper bound)
2001 10 2 12.6 (the same as the simulated value)
2001 10 3 12.9 (the same as lower bound)
....

1. Output Files & Save

Output Files

Three different types of output files are generated in the <work directory>/output/ directory during the Luca calibration:

• Model Output File
• Calibration Trace File (Optional)
• Output Parameter File

In Wizard Step 11, the name of the model output file is displayed. If the user would like to keep the record of the parameter values and the objective function value for each model execution through the calibration, the checkbox Generate Calibration Trace File must be checked and the calibration trace file name must be specified. Also, the user is asked to set the name of the output parameter file for each step. Note that a separate output parameter file is generated for each step of each round, while only one model output file and calibration trace file are generated during the calibration. See below for the details on each output file.

Session Name

The session name is used distinguish this session from others, and will be used as a prefix for the name of files Luca creates (e.g. output parameter files). Also, this allows you to simultaneously run several calibration procedures as long as each session has a different session name since no session will write the same file.

Example:

Suppose the following:

• Session Name: calib1
• The number of steps: 2
• The number of rounds: 2
• The name of the output parameter file: step1.par (for step 1), step2.par (for step 2)

Then, the name of output files will be:

• Actual Output Parameter File Name:
  • calib1_round1_step1.par (for step1 of round 1)
  • calib1_round1_step2.par (for step2 of round 1)
  • calib1_round2_step1.par (for step1 of round 2)
  • calib1_round2_step2.par (for step2 of round 2)
• The name of the summary file created by Luca, which summarizes a given calibration: calib1_summary.txt
• The model output file: calib1_luca.statvar

Model Output File

The model output file, Statvar file, is not saved for each step of each round. Instead, the model keeps writing the same Statvar file during calibration. (If the session name is "calib1", then the name of the Statvar File is “calib1_luca.statvar.) The user can always reproduce Statvar files of each step of each round using output parameter files created by Luca.

Calibration Trace File

The calibration trace file is a text file containing all objective function values calculated during the calibration regardless of good or bad values, and the corresponding parameter values selected for the calibration. SCE calculates the objective function values many times with many different parameter values in the parameter space in order to find the best values. All of these in all steps of all rounds are stored in this one file. This file gives the information about the changes in objective function values in the parameter space. This file can be huge if the number of steps and rounds are large. The user has to make sure if the enough storage is available.

Calibration Trace File Format:

Each line contains:

Count, Parameter Value 1, Parameter Value 2, ...., Round Number, Step Number, OF Value 1, OF Value 2, ....

Note:

• Comma delimited
• The values of parameters selected for all steps are listed in each line.
• Each OF (Objective Function) value of multi-objective functions is listed. These are values before weighing is applied.

Output Parameter File

The user is allowed to set the name of the output parameter file for each step. Luca keeps parameter files of all steps of all rounds. The output parameter files will be placed in the <work directory>/params/ directory. If the name is set to 'step1.par' for Step 1 and the session name is 'calib1', then the actual parameter file name for Step1 of each round will be:

• calib1_round1_step1.par (for round 1)
• calib1_round2_step1.par (for round 2)
• .....

2. Calibration Run

In the last Wizard Step, the user must click the Run Calibration button at the bottom of the panel to run SCE for every step of every round. While it is running, Luca shows the calibration current status, and the best values and the status for each step of each round including the current best parameter values and the objective function values. After the calibration ends, the SCE end statement is displayed in the box located to the right of the screen. The end statement explains why the SCE is terminated for a given step of a round. The calibration is summarized in a summary file.

Calibration Current Status

The fields in Calibration Current Status tell displays how far the calibration has run. While running the calibration, the model is executed many times, and the objective function value is calculated every time the model is executed. The Objective Function Value field shows the latest objective function value regardless of a good or bad value.

Best Values and Status for each Step of each Round

The fields associated with each step of each round including the following are displayed by clicking on a node in the tree structure located in the left-hand side of the screen:

• Model Execution: This field displays the current number of the model executions for each step of a round.
• bjective Function Value: This field shows the best objective function value each step currently has. In SCE, at the end of every shuffling loop, the best objective function value is determined from objective function values of many points (parameter sets) that are generated by SCE. This best value is shown in this field. After SCE finishes a given step of a round, the final objective function value is displayed.
• Output Parameter File: This field displays the name of the output parameter file generated for each step of each round. Luca generates a new output parameter file for each step of each round in the output directory in your project directory.
• Best Parameter Values (a table at the bottom): This field shows the best parameter values each step currently has. More precisely, the parameter values, associated with the best objective function value shown in the Objective Function Value field explained above, are displayed. If SCE has not started a given step of a round, then the initial parameter values are shown. After SCE finishes a step, then the final parameter values are shown.
• SCE Current Status (a box located to the right): This field displays the SCE current status of each step of each round. When SCE ends, the SCE end statement explaining why SCE terminated is displayed.

SCE End Statement

When SCE finishes at each step of a round, one of the following is displayed as an end statement:

Optimization search terminated because the limit on the maximum number of trials, N, was exceeded. Search was stopped at sub-complex M of complex L in shuffling loop K.

(Note: N, M, L, and K are substituted with real numerical values.)

This end statement is displayed if SCE terminates because the number of model executions in SCE reaches the maximum number of model execution.

N is the user defined value of the SCE control parameter, Maximum number of model executions.

Optimization terminated because the criterion value has not changed N percent in M shuffling loops.

(Note: N and M are substituted with real numerical values.)

This end statement is displayed if SCE terminates because the criterion value (best objective function value) SCE finds in every shuffling loop does not change much. No varying criterion value means that no better parameter values than the current best values can be found. The criterion value is considered not to change much if the percent change in the best criterion value of the current shuffling loop and that of M shuffling loop before is less than N percent.

N and M are the user defined SCE control parameter values. N corresponds to the value of Percentage of the criterion value, and M corresponds to the value of Shuffling loops in which the criterion value must change by given percent before optimization is terminated.

Optimization terminated because the population has converged into N percent of the feasible space.

(Note: N is a normalized geometric mean of parameter ranges.)

This end statement is displayed if SCE terminates because all points (parameter sets) generated in SCE has converged into a very small region. It terminates if the points are gathered in the area that is less than or equal to 0.1 % of the feasible space, the space within the lower and upper bounds of parameters. The value N is the percent area into which the points converged.

References

SCE Related Papers

• Duan, Q., Sorooshian, S. and Gupta, V.K., (1992). Effective and efficient global optimization for conceptual rainfall-runoff models. Water Resources Research 28 (4), 1015-1031.

• Duan, Q., Sorooshian, S. and Gupta, V.K., (1993). A Shuffled Complex Evolution approach for effective and efficient global minimization. J. of Optimization Theory and its Applications, 76 (3), 501-521.

• Duan, Q., Sorooshian, S. and Gupta, V.K., (1994). Optimal use of the SCE-UA global optimization method for calibrating watershed models. Journal of Hydrology, 158 265-284

Step-Wise, Multiple-Objective Calibration Related Papers

• Hay, L.E., Leavesley, G.H., Clark, M.P., Markstrom, S.L., Viger, R.J., and Umemoto, M. (2006). Step-wise, multiple-objective calibration of a hydrologic model for a snowmelt-dominated basin. Journal of the American Water Resources Association.

• Hay, L.E., Leavesley, G.H., and Clark, M.P., (2006). Use of Remotely-Sensed Snow Covered Area in Watershed Model Calibration for the Sprague River, Oregon. Joint 8th Federal Interagency Sedimentation Conference and 3rd Federal Interagency Hydrologic Modeling Conference, Reno, Nevada, April, 2006.