Scenario

smac.scenario

Scenario dataclass

Scenario(
    configspace: ConfigurationSpace,
    name: str | None = None,
    output_directory: Path = Path("smac3_output"),
    deterministic: bool = False,
    objectives: str | list[str] = "cost",
    crash_cost: float | list[float] = inf,
    termination_cost_threshold: float | list[float] = inf,
    walltime_limit: float = inf,
    cputime_limit: float = inf,
    trial_walltime_limit: float | None = None,
    trial_memory_limit: int | None = None,
    n_trials: int = 100,
    use_default_config: bool = False,
    instances: list[str] | None = None,
    instance_features: dict[str, list[float]] | None = None,
    adaptive_capping: bool = False,
    adaptive_capping_slackfactor: float | None = None,
    runtime_cutoff: int | None = None,
    min_budget: float | int | None = None,
    max_budget: float | int | None = None,
    seed: int = 0,
    n_workers: int = 1,
)

The scenario manages environment variables and therefore gives context in which frame the optimization is performed.

PARAMETER	DESCRIPTION
configspace	The configuration space from which to sample the configurations. TYPE: ConfigurationSpace
name	The name of the run. If no name is passed, SMAC generates a hash from the meta data. Specify this argument to identify your run easily. TYPE: str | None, defaults to None DEFAULT: None
output_directory	The directory in which to save the output. The files are saved in ./output_directory/name/seed. TYPE: Path, defaults to Path("smac3_output") DEFAULT: Path('smac3_output')
deterministic	If True, only one seed is passed to the target function, assuming deterministic (noise-free) behavior. Otherwise, multiple seeds are passed (if n_seeds > 1) to enable repeated evaluations. For non-deterministic functions, users must specify intensification parameters (e.g., max_config_calls, n_seeds) to ensure reliable performance estimates and optionally model noise in the Gaussian Process surrogate. TYPE: bool, defaults to False DEFAULT: False
objectives	The objective(s) to optimize. This argument is required for multi-objective optimization. TYPE: str | list[str] | None, defaults to "cost" DEFAULT: 'cost'
crash_cost	Defines the cost for a failed trial. In case of multi-objective, each objective can be associated with a different cost. TYPE: float | list[float], defaults to np.inf DEFAULT: inf
termination_cost_threshold	Defines a cost threshold when the optimization should stop. In case of multi-objective, each objective must be associated with a cost. The optimization stops when all objectives crossed the threshold. TYPE: float | list[float], defaults to np.inf DEFAULT: inf
walltime_limit	The maximum time in seconds that SMAC is allowed to run. TYPE: float, defaults to np.inf DEFAULT: inf
cputime_limit	The maximum CPU time in seconds that SMAC is allowed to run. TYPE: float, defaults to np.inf DEFAULT: inf
trial_walltime_limit	The maximum time in seconds that a trial is allowed to run. If not specified, no constraints are enforced. Otherwise, the process will be spawned by pynisher. TYPE: float | None, defaults to None DEFAULT: None
trial_memory_limit	The maximum memory in MB that a trial is allowed to use. If not specified, no constraints are enforced. Otherwise, the process will be spawned by pynisher. TYPE: int | None, defaults to None DEFAULT: None
n_trials	The maximum number of trials (combination of configuration, seed, budget, and instance, depending on the task) to run. TYPE: int, defaults to 100 DEFAULT: 100
use_default_config	If True, the configspace's default configuration is evaluated in the initial design. For historic benchmark reasons, this is False by default. Notice, that this will result in n_configs + 1 for the initial design. Respecting n_trials, this will result in one fewer evaluated configuration in the optimization. TYPE: bool DEFAULT: False
instances	Names of the instances to use. If None, no instances are used. Instances could be dataset names, seeds, subsets, etc. TYPE: list[str] | None, defaults to None DEFAULT: None
instance_features	Instances can be associated with features. For example, meta data of the dataset (mean, var, ...) can be incorporated which are then further used to expand the training data of the surrogate model. TYPE: dict[str, list[float]] | None, defaults to None DEFAULT: None
adaptive_capping	Adaptive capping allows to preemptiveley cancel the evaluation of candidates as soon as their accumulative cost exceed the cost of the current incumbent. If a runtime_cutoff is set, SMAC will allocate budgets that are capped at the runtime_cutoff. TYPE: bool DEFAULT: False
runtime_cutoff	A runtime cutoff can be set to limit the maximum runtime allowed for a single configuration to run solving instances. TYPE: int | None DEFAULT: None
min_budget	The minimum budget (epochs, subset size, number of instances, ...) that is used for the optimization. Use this argument if you use multi-fidelity or instance optimization. TYPE: float | int | None, defaults to None DEFAULT: None
max_budget	The maximum budget (epochs, subset size, number of instances, ...) that is used for the optimization. Use this argument if you use multi-fidelity or instance optimization. TYPE: float | int | None, defaults to None DEFAULT: None
seed	The seed is used to make results reproducible. If seed is -1, SMAC will generate a random seed. TYPE: int, defaults to 0 DEFAULT: 0
n_workers	The number of workers to use for parallelization. If n_workers is greather than 1, SMAC will use Dask to parallelize the optimization. TYPE: int, defaults to 1 DEFAULT: 1

meta property

meta: dict[str, Any]

Returns the meta data of the SMAC run.

Note

Meta data are set when the facade is initialized.

__post_init__

__post_init__() -> None

Checks whether the config is valid.

Source code in smac/scenario.py

def __post_init__(self) -> None:
    """Checks whether the config is valid."""
    # Use random seed if seed is -1
    if self.seed == -1:
        seed = random.randint(0, 999999)
        object.__setattr__(self, "seed", seed)

    # Transform instances to string if they are not
    if self.instances is not None:
        instances = [str(instance) for instance in self.instances]
        object.__setattr__(self, "instances", instances)

    # Transform instance features to string if they are not
    if self.instance_features is not None:
        instance_features = {str(instance): features for instance, features in self.instance_features.items()}
        object.__setattr__(self, "instance_features", instance_features)

    # Validate that we have a runtime cutoff set if adaptive capping slackfactor is given
    if self.adaptive_capping_slackfactor is not None and self.runtime_cutoff is None:
        raise ValueError("If adaptive_capping_slackfactor is set, then runtime_cutoff must be set as well.")

    # Change directory wrt name and seed
    self._change_output_directory()

    # Set empty meta
    object.__setattr__(self, "_meta", {})

count_instance_features

count_instance_features() -> int

Counts the number of instance features.

Source code in smac/scenario.py

def count_instance_features(self) -> int:
    """Counts the number of instance features."""
    # Check whether key of instance features exist
    n_features = 0
    if self.instance_features is not None:
        for k, v in self.instance_features.items():
            if self.instances is None or k not in self.instances:
                raise RuntimeError(f"Instance {k} is not specified in instances.")

            if n_features == 0:
                n_features = len(v)
            else:
                if len(v) != n_features:
                    raise RuntimeError("Instances must have the same number of features.")

    return n_features

count_objectives

count_objectives() -> int

Counts the number of objectives.

Source code in smac/scenario.py

def count_objectives(self) -> int:
    """Counts the number of objectives."""
    if isinstance(self.objectives, list):
        return len(self.objectives)

    return 1

load staticmethod

load(path: Path) -> Scenario

Loads a scenario and the configuration space from a file.

Source code in smac/scenario.py

@staticmethod
def load(path: Path) -> Scenario:
    """Loads a scenario and the configuration space from a file."""
    filename = path / "scenario.json"
    with open(filename, "r") as fh:
        data = json.load(fh)

    # Convert `output_directory` to path object again
    data["output_directory"] = Path(data["output_directory"])
    meta = data["_meta"]
    del data["_meta"]

    # Read configspace
    configspace_filename = path / "configspace.json"
    configspace = ConfigurationSpace.from_json(configspace_filename)

    data["configspace"] = configspace

    scenario = Scenario(**data)
    scenario._set_meta(meta)

    return scenario

make_serializable staticmethod

make_serializable(scenario: Scenario) -> dict[str, Any]

Makes the scenario serializable.

Source code in smac/scenario.py

@staticmethod
def make_serializable(scenario: Scenario) -> dict[str, Any]:
    """Makes the scenario serializable."""
    s = copy.deepcopy(scenario.__dict__)
    del s["configspace"]
    s["output_directory"] = str(s["output_directory"])

    return json.loads(json.dumps(s))

save

save() -> None

Saves internal variables and the configuration space to a file.

Source code in smac/scenario.py

def save(self) -> None:
    """Saves internal variables and the configuration space to a file."""
    if self.meta == {}:
        logger.warning("Scenario will saved without meta data. Please call the facade first to set meta data.")

    if self.name is None:
        raise RuntimeError(
            "Please specify meta data for generating a name. Alternatively, you can specify a name manually."
        )

    self.output_directory.mkdir(parents=True, exist_ok=True)

    data = {}
    for k, v in self.__dict__.items():
        if k in ["configspace", "output_directory"]:
            continue

        data[k] = v

    # Convert `output_directory`
    data["output_directory"] = str(self.output_directory)

    # Save everything
    filename = self.output_directory / "scenario.json"
    with open(filename, "w") as fh:
        json.dump(data, fh, indent=4, cls=NumpyEncoder)

    # Save configspace on its own
    configspace_filename = self.output_directory / "configspace.json"
    self.configspace.to_json(configspace_filename)