Getting Started¶

In the core, SMAC needs four components (configuration space, target function, scenario and a facade) to run an optimization process, all of which are explained on this page.

Configuration Space¶

The configuration space defines the search space of the hyperparameters and, therefore, the tunable parameters’ legal ranges and default values.

from ConfigSpace import ConfigSpace

cs = ConfigurationSpace({
    "myfloat": (0.1, 1.5),                # Uniform Float
    "myint": (2, 10),                     # Uniform Integer
    "species": ["mouse", "cat", "dog"],   # Categorical
})

Please see the documentation of ConfigSpace for more details.

Target Function¶

The target function takes a configuration from the configuration space and returns a performance value. For example, you could use a Neural Network and predict the performance based on the learning rate. Every configuration would (most likely) return a different value. However, SMAC tries to find the best learning rate by trying different and potentially improving configurations.

def train(self, config: Configuration, seed: int) -> float:
    model = MultiLayerPerceptron(learning_rate=config["learning_rate"])
    model.fit(...)
    accuracy = model.validate(...)

    return 1 - accuracy  # SMAC always minimizes (the smaller the better)

Warning

SMAC always minimizes the value returned from the target function.

Note

In general, the arguments of the target function depend on the intensifier. However, in all cases, the first argument must be the configuration (arbitrary argument name is possible here) and a seed. If you specified instances in the scenario, SMAC requires instance as argument additionally. If you use SuccessiveHalving or Hyperband as intensifier but you did not specify instances, SMAC passes budget as argument to the target function. But don’t worry: SMAC will tell you if something is missing or if something is not used.

Warning

SMAC passes either instance or budget to the target function but never both.

Scenario¶

The Scenario is used to provide environment variables. For example, you want to limit the optimization process by a time limit or want to specify where to save the results.

from smac import Scenario

scenario = Scenario(
    configspace=cs,
    output_directory=Path("your_output_directory")
    walltime_limit=120,  # Limit to two minutes
    n_trials=500,  # Evaluated max 500 trials
    n_workers=8,  # Use eight workers
    ...
)

Facade¶

A facade is the entry point to SMAC, which constructs a default optimization pipeline for you. SMAC offers various facades, which satisfy many use cases and are crucial to achieving peak performance. The idea behind the facades is to provide a simple interface to SMAC, which is easy to use and understand without diving deep into the material. However, experts are invited to change the components as they please to achieve even better performance potentially. The following table (horizontal scrollable) shows you what is supported and reveals the default components:

	Black-Box	Hyperparameter Optimization	Multi-Fidelity	Algorithm Configuration	Random	Hyperband
#Parameters	low	low/medium/high	low/medium/high	low/medium/high	low/medium/high	low/medium/high
Supports Instances	❌	✅	✅	✅	❌	✅
Supports Multi-Fidelity	❌	❌	✅	✅	❌	✅
Initial Design	Sobol	Sobol	Random	Default	Default	Default
Surrogate Model	Gaussian Process	Random Forest	Random Forest	Random Forest	Not used	Not used
Acquisition Function	Expected Improvement	Log Expected Improvement	Log Expected Improvement	Expected Improvement	Not used	Not used
Acquisition Maximier	Local and Sorted Random Search	Local and Sorted Random Search	Local and Sorted Random Search	Local and Sorted Random Search	Local and Sorted Random Search	Local and Sorted Random Search
Intensifier	Default	Default	Hyperband	Hyperband	Default	Hyperband
Runhistory Encoder	Default	Log	Log	Default	Default	Default
Random Design Probability	8.5%	20%	20%	50%	Not used	Not used

Note

The multi-fidelity facade is the closest implementation to BOHB.

Note

We want to emphasize that SMAC is a highly modular optimization framework. The facade accepts many arguments to specify components of the pipeline. Please also note, that in contrast to previous versions, instantiated objects are passed instead of kwargs.

The facades can be imported directely from the smac module.

from smac import BlackBoxFacade as BBFacade
from smac import HyperparameterOptimizationFacade as HPOFacade
from smac import MultiFidelityFacade as MFFacade
from smac import AlgorithmConfigurationFacade as ACFacade
from smac import RandomFacade as RFacade
from smac import HyperbandFacade as HBFacade

smac = HPOFacade(scenario=scenario, target_function=train)
smac = MFFacade(scenario=scenario, target_function=train)
smac = ACFacade(scenario=scenario, target_function=train)
smac = RFacade(scenario=scenario, target_function=train)
smac = HBFacade(scenario=scenario, target_function=train)

Package Overview

Minimal Example

SMAC3 Documentation

Getting Started¶

Configuration Space¶

Target Function¶

Scenario¶

Facade¶