Scenario

The scenario-object is used to configure SMAC and can be constructed either by providing an acutal scenario-object or by specifing the options in a scenario file.

Create Scenario

Specify the options directly:

from smac.scenario.scenario import Scenario

scenario = Scenario({
    'option': 'value',
    ...
})

or specify the options in a scenario.txt file:

OPTION1 = VALUE1
OPTION2 = VALUE2

Use "true" or "false" for boolean values, respectively.

Finally, create the scenario object by handing over the filename:

from smac.scenario.scenario import Scenario

scenario = Scenario("/path/to/your/scenario.txt")

Options

Following options can be defined within the scenario object:

algo_runs_timelimit

Maximum amount of CPU-time used for optimization. Default: inf.

always_race_default

Race new incumbents always against default configuration.

cost_for_crash

Defines the cost-value for crashed runs on scenarios with quality as run-obj. If multi-objective is used, a list or comma separated string is accepted too. Default: 2147483647.0.

cutoff

Maximum runtime, after which the target algorithm is cancelled. Required if *run_obj* is runtime.

deterministic

If true, SMAC assumes that the target function or algorithm is deterministic (the same static seed of 0 is always passed to the function/algorithm). If false, different random seeds are passed to the target function/algorithm. Default: True.

execdir

Specifies the path to the execution-directory. Default: ..

feature_fn

Specifies the file with the instance-features.

initial_incumbent

DEFAULT is the default from the PCS. Default: DEFAULT.

memory_limit

Maximum available memory the target algorithm can occupy before being cancelled in MB.

multi_objectives

List of string or comma-separated strings of objectives to optimize. Default: cost.

overall_obj

PARX, where X is an integer defining the penalty imposed on timeouts (i.e. runtimes that exceed the cutoff-time). Default: par10.

pcs_fn

Specifies the path to the PCS-file.

run_obj

Defines what metric to optimize. When optimizing runtime, cutoff_time is required as well.

save_results_instantly

If true, runhistory and stats are saved immediately on changes. Otherwise, runhistory and states are only saved once after the optimization process has finished. Default: True.

ta

Specifies the target algorithm call that SMAC will optimize. Interpreted as a bash-command.

ta_run_limit

Maximum number of algorithm-calls during optimization. Default: inf.

test_inst_fn

Specifies the file with the test-instances.

train_inst_fn

Specifies the file with the training-instances.

wallclock_limit

Maximum amount of wallclock-time used for optimization. Default: inf.

Additionally, you can also specify SMAC options with the scenario object:

abort_on_first_run_crash

If true, SMAC will abort if the first run of the target algorithm crashes. Default: True.

acq_opt_challengers

Number of challengers returned by acquisition function optimization. Also influences the number of randomly sampled configurations to optimized the acquisition function Default: 5000.

always_race_default

Race new incumbents always against default configuration.

hydra_iterations

number of hydra iterations. Only active if mode is set to Hydra Default: 3.

input_psmac_dirs

For parallel SMAC, multiple output-directories are used.

intens_adaptive_capping_slackfactork

Slack factor of adpative capping (factor * adpative cutoff). Only active if obj is runtime. If set to very large number it practically deactivates adaptive capping. Default: 1.2.

intens_min_chall

Minimal number of challengers to be considered in each intensification run (> 1). Set to 1 and in combination with very small intensification-percentage. it will deactivate randomly sampled configurations (and hence, extrapolation of random forest will be an issue.) Default: 2.

intensification_percentage

The fraction of time to be used on intensification (versus choice of next Configurations). Default: 0.5.

limit_resources

If true, SMAC will use pynisher to limit time and memory for the target algorithm. Allows SMAC to use all resources available. Applicable only to func TAEs. Set to ‘False’ by default. (Warning: This only works on Linux. Use with caution!)

maxR

Maximum number of calls per configuration. Default: 2000.

minR

Minimum number of calls per configuration. Default: 1.

output_dir

Specifies the output-directory for all emerging files, such as logging and results. Default: smac3-output_2022-02-17_16:01:54_622223.

rand_prob

probablity to run a random configuration instead of configuration optimized on the acquisition function Default: 0.5.

random_configuration_chooser

path to a python module containing a class`RandomConfigurationChooserImpl` implementingthe interface of RandomConfigurationChooser

rf_do_bootstrapping

Use bootstraping in random forest. Default: True.

rf_max_depth

Maximum depth of each tree in the random forest. Default: 20.

rf_min_samples_leaf

Minimum required number of samples in each leaf of a tree in the random forest. Default: 3.

rf_min_samples_split

Minimum number of samples to split for building a tree in the random forest. Default: 3.

rf_num_trees

Number of trees in the random forest (> 1). Default: 10.

rf_ratio_features

Ratio of sampled features in each split ([0.,1.]). Default: 0.8333333333333334.

shared_model

Whether to run SMAC in parallel mode.

sls_max_steps

Maximum number of local search steps in one iteration during the optimization of the acquisition function.

sls_n_steps_plateau_walk

Maximum number of steps on plateaus during the optimization of the acquisition function. Default: 10.

transform_y

Transform all observed cost values via log-transformations or inverse scaling. The subfix “s” indicates that SMAC scales the y-values accordingly to apply the transformation. Default: NONE.

use_ta_time

Instead of measuring SMAC’s wallclock time, only consider time reported by the target algorithm (ta).