Getting Started

In the core, SMAC needs four components (configuration space, target function, scenario and a facade) to run an optimization process, all of which are explained on this page.

Configuration Space

The configuration space defines the search space of the hyperparameters and, therefore, the tunable parameters’ legal ranges and default values.

from ConfigSpace import ConfigSpace

cs = ConfigurationSpace({
    "myfloat": (0.1, 1.5),                # Uniform Float
    "myint": (2, 10),                     # Uniform Integer
    "species": ["mouse", "cat", "dog"],   # Categorical
})

Please see the documentation of ConfigSpace for more details.

Target Function

The target function takes a configuration from the configuration space and returns a performance value. For example, you could use a Neural Network and predict the performance based on the learning rate. Every configuration would (most likely) return a different value. However, SMAC tries to find the best learning rate by trying different and potentially improving configurations.

def train(self, config: Configuration, seed: int) -> float:
    model = MultiLayerPerceptron(learning_rate=config["learning_rate"])
    model.fit(...)
    accuracy = model.validate(...)

    return 1 - accuracy  # SMAC always minimizes (the smaller the better)

Warning

SMAC always minimizes the value returned from the target function.

Note

In general, the arguments of the target function depend on the intensifier. However, in all cases, the first argument must be the configuration (arbitrary argument name is possible here) and a seed. If you specified instances in the scenario, SMAC requires instance as argument additionally. If you use SuccessiveHalving or Hyperband as intensifier but you did not specify instances, SMAC passes budget as argument to the target function. But don’t worry: SMAC will tell you if something is missing or if something is not used.

Warning

SMAC passes either instance or budget to the target function but never both.

Scenario

The Scenario is used to provide environment variables. For example, you want to limit the optimization process by a time limit or want to specify where to save the results.

from smac import Scenario

scenario = Scenario(
    configspace=cs,
    output_directory=Path("your_output_directory")
    walltime_limit=120,  # Limit to two minutes
    n_trials=500,  # Evaluated max 500 trials
    n_workers=8,  # Use eight workers
    ...
)

Facade

A facade is the entry point to SMAC, which constructs a default optimization pipeline for you. SMAC offers various facades, which satisfy many use cases and are crucial to achieving peak performance. The idea behind the facades is to provide a simple interface to SMAC, which is easy to use and understand without diving deep into the material. However, experts are invited to change the components as they please to achieve even better performance potentially. The following table (horizontal scrollable) shows you what is supported and reveals the default components:

Black-Box

Hyperparameter Optimization

Multi-Fidelity

Algorithm Configuration

Random

Hyperband

#Parameters

low

low/medium/high

low/medium/high

low/medium/high

low/medium/high

low/medium/high

Supports Instances

Supports Multi-Fidelity

Initial Design

Sobol

Sobol

Random

Default

Default

Default

Surrogate Model

Gaussian Process

Random Forest

Random Forest

Random Forest

Not used

Not used

Acquisition Function

Expected Improvement

Log Expected Improvement

Log Expected Improvement

Expected Improvement

Not used

Not used

Acquisition Maximier

Local and Sorted Random Search

Local and Sorted Random Search

Local and Sorted Random Search

Local and Sorted Random Search

Local and Sorted Random Search

Local and Sorted Random Search

Intensifier

Default

Default

Hyperband

Hyperband

Default

Hyperband

Runhistory Encoder

Default

Log

Log

Default

Default

Default

Random Design Probability

8.5%

20%

20%

50%

Not used

Not used

Note

The multi-fidelity facade is the closest implementation to BOHB.

Note

We want to emphasize that SMAC is a highly modular optimization framework. The facade accepts many arguments to specify components of the pipeline. Please also note, that in contrast to previous versions, instantiated objects are passed instead of kwargs.

The facades can be imported directely from the smac module.

from smac import BlackBoxFacade as BBFacade
from smac import HyperparameterOptimizationFacade as HPOFacade
from smac import MultiFidelityFacade as MFFacade
from smac import AlgorithmConfigurationFacade as ACFacade
from smac import RandomFacade as RFacade
from smac import HyperbandFacade as HBFacade

smac = HPOFacade(scenario=scenario, target_function=train)
smac = MFFacade(scenario=scenario, target_function=train)
smac = ACFacade(scenario=scenario, target_function=train)
smac = RFacade(scenario=scenario, target_function=train)
smac = HBFacade(scenario=scenario, target_function=train)