CHAPTER 14
Experiment Manager

The Experiment Manager sets up model runs that SIMPROCESS will run automatically and, optionally, after each run, place the results in the database. This is accomplished by defining experiments in SIMPROCESS, which are stored in Experiments.xml in the SPUser directory. SIMPROCESS reads the information from Experiments.xml, loads the appropriate model, runs the model, then (optionally) commits the results to the database.

The four items on the Experiment menu control the operation of the Experiment Manager.

· Define Experiments

· Run All Experiments

· Run Selected Experiments

· Run Specific Experiment

Defining Experiments

An experiment can have multiple models, a model can have multiple designs, a design can have multiple scenarios, and a scenario can (optionally) have multiple Model Parameters (page 242) and Process Alternatives (page 84). Selecting Define Experiments/Using SIMPROCESS... brings up a dialog experiment information is entered.

If no experiments have been defined, everything is disabled except the Experiment combo box. If there are already experiments defined, the first experiment appears in the Experiment combo box, and all appropriate items are active. To add to or modify an experiment already defined, select an experiment from the Experiment combo box. There may be a slight delay when a predefined experiment is selected. This is because the first model for the experiment must be loaded and then processed.

Entering Experiment Information

There are five fields that require entries for each experiment: Experiment, Selected, Display Error Alerts, Turn Off Animation, and Commit Results To Database. This shows an example experiment called Purchasing. Type the name in the Experiment combo box, then press Enter. If not already active, entering an experiment name causes the remaining items on the Experiment Setup tab and the Model Setup tab to activate.

Once the experiment has been named, then check or uncheck Selected. The default is checked. Selected is used as an option for running experiments. Display Error Alerts defaults to unchecked. If unchecked, then SIMPROCESS will run the experiment without stopping to notify of errors. This means SIMPROCESS will run every combination of model, design, and scenario that it can. Those with errors will be skipped over. If Display Error Alerts is checked, when SIMPROCESS encounters an error when running the experiment, an error dialog will display that will stop the experiment until the error has been acknowledged by the user. Turn Off Animation defaults to checked. If checked, all models in the experiment will run with animation turned off. Commit Results To Database defaults to checked as well. When selected, the results from each run will be placed in the SIMPROCESS database. Multiple experiments can be entered. Although not necessary, it is recommended that model, design, and scenario information be entered before adding another experiment.

Entering Model Information

The next tab is Model Setup. This tab is active when an experiment name is entered or selected on the Experiment Setup tab. To select a model, click the Add Model button. This brings up a file chooser dialog. The complete path and the name of the model selected is added to the Model combo box. All models added are part of the experiment that was selected on the Experiment Setup tab. Although multiple models can be entered before entering design and scenario information, it is recommended that all the design and scenario information (along with parameter and process information, if needed) be entered before entering the next model. If the selected model is moved after being added to an experiment, the model and all associated experiment information (design, scenario, etc.) will be deleted from the experiment the next time Define Experiments/Using SIMPROCESS... is selected.

Entering Design Information

The Design Setup tab is active when a model is selected in the Model combo box on the Model Setup tab. At least one design name is required for each model. The Design Comment and Number of Reps fields are optional. If no value (integer only) is entered for Number of Reps, the last saved model value will be used for each scenario in this design unless there was a previous design for the same model. One design called "5 Reps" is entered for the model C:\Program Files\SIMPROCESS\models\Demos\Purchasing.spm.

Entering Scenario Information

The Scenario Setup tab is active when a design is selected in the Design combo box on the Design Setup tab. At least one scenario must be entered for each design. If creating a new scenario, a name for the scenario must be entered in the Scenario combo box. The Scenario Comment field is optional. At this point no more information is required to run a scenario.

Publishing the results of this scenario run to HTML is optional. If Publish Model To HTML is selected, a value for Publish Directory must be entered. The Publish Directory is the destination directory for the HTML output. The name of the directory created for the HTML files is the model name with any spaces replaced by underscores. If a directory with this name already exists within the selected destination directory, either select another destination directory, or rename, move, or delete the existing directory. For instance, in the example below, the directory of the model Purchasing.spm was selected as the destination directory. This means the publish process will create a directory named Purchasing within the Purchasing directory. If a directory named Purchasing already existed in C:\Program File\SIMPROCESS\models\Demos\Purchasing, an error would occur. So for this example, the HTML files will go in the C:\Program Files\SIMPROCESS\models\Demos\Purchasing\Purchasing directory. Thus, if multiple runs of the same model are to be published, a unique destination directory will need to be assigned for each scenario. Usually, if used at all, only one scenario for each model is published to HTML.

Setting Model Parameter values and Process Alternatives is optional. If these are not set, then the last saved value for each Model Parameter and Process Alternative are used. Model Parameters are set on the Parameter Setup tab, and Process Alternatives are set on the Process Setup tab. These tabs are part of the Scenario Setup tab.

Entering Model Parameter Information

The Parameter Setup tab is used to enter Model Parameter values. The attributes for the selected model that have been designated as Model Parameters appear in the table. When a Model Parameter is selected, its description (comment from the attribute definition), mode (Integer, Real, or String), and default value are displayed in the text fields below the table. Once selected, the value can be changed by typing into the field on the table. Values entered that do not match the mode will not be accepted. The Reset button will reset the selected Model Parameter to its default value. The Reset All button resets all Model Parameters to their default values. In this example there are three Model attributes that can be changed for a scenario. (When changing selected scenarios, there may be a slight lag for the Model Parameter values to update. This is due to the processing required to update the values.)

NOTE

Entering Process Alternative Information

The Process Setup tab is used to set the active alternatives for processes that have more than one alternative. The panel on the left displays the processes in the selected model that have more than one alternative. (Since processes are displayed by name, it is recommended that process names not be duplicated within a model.) Once a process is selected, the alternatives for that process will appear in the panel on the right. If an alternative has already been selected, that alternative will be highlighted. If no alternative is highlighted then the model will run with the alternative that was active during the last save of the model. Select the alternative to be active for the current scenario.

In this example the Centralized, Functional alternative for the Purchasing process will be active for the current scenario.

NOTE

Undo Delete Button

The Undo Delete button is active whenever an experiment, model, design, or scenario has been deleted. The button remains active until used or until the scope of the deleted item has been lost. An experiment can be restored any time before another delete occurs. A model can be restored as long as the experiment to which it belongs remains the selected experiment. A design can be restored as long as the experiment and model to which it belongs remain selected. Finally, a scenario can be restored as long as the experiment, model, and design to which it belongs remain selected.

Running Experiments

Once the experiments have been defined, run the experiments. This section discusses starting experiments, the operation of experiments, how to interact with experiments, and how to generate standard reports from experiments.

Starting Experiments

There are three options for running experiments.

· Run All Experiments

· Run Selected Experiments

· Run Specific Experiment

Run All Experiments

The menu item Run All Experiments causes SIMPROCESS to attempt to run every experiment defined. This option ignores the Selected field.

Run Selected Experiments

When this menu item is selected, SIMPROCESS attempts to run the experiments in which the Selected field is checked.

Run Specific Experiment

This menu item specifies one experiment to run. When selected, enter the experiment name in the dialog.

If no match occurs, an error message appears stating the experiment could not be found in the database. Case does matter when selecting a specific experiment. The case and spelling must match.

Experiment Operation

The Experiment Manager operates as follows when running experiments:

Once an experiment begins, SIMPROCESS begins a sequence of steps. Each step occurs if the previous completes successfully. The steps to running experiments are:

1. Load Experiments.xml.

2. Prompt to save current work if necessary.

3. Enter a loop with sub loops.

Find Experiment.

Find Model.

Find Design.

Find Scenario.

Find Parameters.

Find Process Alternatives.

Load Model.

Verify Design.

Run Model.

Commit results to database.

End loop.

End loop.

End loop.

End loop.

4. Display status message on success of experiments.

Experiment Trace

During experiment execution, the Experiment Manager displays a trace showing the steps that have been completed and the current step. The trace includes the date, time, and a message.

It is important to examine the Experiment Manager trace, since it will give the complete status of the experiments. The above example shows the experiment, alert dialog status (see below), model, design, and scenario. Also, it shows the Process Alternatives that were active and the values of the Model Parameters for the scenario.

Experiment Errors

As explained in the instructions for defining experiments, when the Display Error Alerts field is not selected, interactive error messages are suppressed. However, error messages will appear in the Experiment Manager trace. The Experiment Manager trace shows which experiments completed successfully and which encountered errors. Especially important are the messages that show errors with the Model Parameters.

When the Display Error Alerts field is selected, error dialogs will appear that pause the Experiment until the error is acknowledged.

Also, after the error displays, if there are more experiments to be run, a dialog appears with the option to stop all experiment runs.

If experiments are to be run overnight, it is recommended that the Display Error Alerts field not be selected for every experiment that is to be run. That way SIMPROCESS will run every combination of Experiment, Model, Design, and Scenario that it can. Otherwise, if an error occurs, processing stops until the error is acknowledged.

Interacting With Experiments

A model run by the Experiment Manager can be interacted with in the same manner as a model that was user initiated. This means the animation can be turned on or off, the animation settings can be changed, the model can be navigated, etc.

Experiments can be stopped early by stopping a model run. An experiment cannot be stopped while loading experiment information, loading a model, or committing results to the database. If the Display Error Alerts field is selected, there will be an option to stop completely. Otherwise every run must be stopped to stop all experiment, model, design, and scenario combinations.

Generating Standard Reports From Experiments

If selected, the Experiment Manager automatically commits the run results to the database. If an ASCII version of the Standard Report is needed, go to Define Global Statistics Collection on the Report menu. Select Generate Standard Report after run. This will create a file in the model's directory that will contain the report for every replication, the sum of the replications, and the average of the replications.

Quadralay Corporation http://www.webworks.com Voice: (512) 719-3399 Fax: (512) 719-3606 sales@webworks.com