Options and file formats

In the optimization-process of SMAC, there are several ways to configure the options:

Mandatory:
  • Commandline-options, with which SMAC is called directly (not needed if SMAC is used within Python).
  • Scenario-options, that are specified via a Scenario-object. Either directly in the Python-code or by using a scenario-file.
  • A Parameter Configuration Space (PCS), that provides the legal ranges of parameters to optimize, their types (e.g. int or float) and their default values.
Optional:
  • Instance- and feature-files, that list the instances and features to optimize upon.

Scenario

The scenario-object (smac.scenario.scenario.Scenario) is used to configure SMAC and can be constructed either by providing an actual scenario-object (see SVM-example), or by specifing the options in a scenario file (see SPEAR example). The format of the scenario file is one option per line:

OPTION1 = VALUE1
OPTION2 = VALUE2
...

For boolean options “1” or “true” both evaluate to True. The following assumes that the scenario is created via a scenario-file. If it is generated within custom code, you might not need algo or paramfile.

Options:

abort_on_first_run_crash:
 If true, SMAC will abort if the first run of the target algorithm crashes. Default: True.
algo:Specifies the target algorithm call that SMAC will optimize. Interpreted as a bash-command.
always_race_default:
 Race new incumbents always against default configuration.
cost_for_crash:Defines the cost-value for crashed runs on scenarios with quality as run-obj. Default: 2147483647.0.
cutoff_time:Maximum runtime, after which the target algorithm is cancelled. Required if *run_obj* is runtime.
deterministic:If true, the optimization process will be repeatable.
execdir:Specifies the path to the execution-directory. Default: ..
feature_file:Specifies the file with the instance-features.
initial_incumbent:
 DEFAULT is the default from the PCS. Default: DEFAULT. Must be from: [‘DEFAULT’, ‘RANDOM’].
input_psmac_dirs:
 For parallel SMAC, multiple output-directories are used.
instance_file:Specifies the file with the training-instances.
intensification_percentage:
 The fraction of time to be used on intensification (versus choice of next Configurations). Default: 0.5.
maxR:Maximum number of calls per configuration. Default: 2000.
memory_limit:Maximum available memory the target algorithm can occupy before being cancelled.
minR:Minimum number of calls per configuration. Default: 1.
output_dir:Specifies the output-directory for all emerging files, such as logging and results. Default: smac3-output_2018-01-22_15:05:56_807070.
overall_obj:PARX, where X is an integer defining the penalty imposed on timeouts (i.e. runtimes that exceed the cutoff-time). Default: par10.
paramfile:Specifies the path to the PCS-file.
run_obj:Defines what metric to optimize. When optimizing runtime, cutoff_time is required as well. Must be from: [‘runtime’, ‘quality’].
runcount_limit:Maximum number of algorithm-calls during optimization. Default: inf.
shared_model:Whether to run SMAC in parallel mode.
test_instance_file:
 Specifies the file with the test-instances.
tuner-timeout:Maximum amount of CPU-time used for optimization. Default: inf.
wallclock_limit:
 Maximum amount of wallclock-time used for optimization. Default: inf.

Parameter Configuration Space (PCS)

The Parameter Configuration Space (PCS) defines the legal ranges of the parameters to be optimized and their default values. In the examples-folder you can find several examples for PCS-files. Generally, the format is:

parameter_name categorical {value_1, ..., value_N} [default value]
parameter_name ordinal {value_1, ..., value_N} [default value]
parameter_name integer [min_value, max_value] [default value]
parameter_name integer [min_value, max_value] [default value] log
parameter_name real [min_value, max_value] [default value]
parameter_name real [min_value, max_value] [default value] log

# Conditionals:
child_name | condition [&&,||] condition ...

# Condition Operators:
# parent_x [<, >] parent_x_value (if parameter type is ordinal, integer or real)
# parent_x [==,!=] parent_x_value (if parameter type is categorical, ordinal or integer)
# parent_x in {parent_x_value1, parent_x_value2,...}

# Forbiddens:
{parameter_name_1=value_1, ..., parameter_name_N=value_N}

Instances and Features

To specify instances and features, simply provide text-files in the following format and provide the paths to the instances in the scenario.

Instance-files are text-files with one instance per line. If you want to use training- and test-sets, separate files are expected.

Feature-files are files following the comma-separated-value-format, as can also be seen in the SPEAR-example:

instance name of feature 1 name of feature 2
name of instance 1 value of feature 1 value of feature 2