smac.acquisition.function

Interfaces

class smac.acquisition.function.AbstractAcquisitionFunction

Bases: object

Abstract base class for acquisition function.

__call__(configurations)

Compute the acquisition value for a given configuration.

Parameters:

configurations (list[Configuration]) – The configurations where the acquisition function should be evaluated.

Returns:

Acquisition values for X

Return type:

np.ndarray [N, 1]

property meta: dict[str, Any]

Returns the meta data of the created object.

property model: AbstractModel | None

Return the used surrogate model in the acquisition function.

property name: str

Returns the full name of the acquisition function.

update(model, **kwargs)

Update the acquisition function attributes required for calculation.

This method will be called after fitting the model, but before maximizing the acquisition function. As an examples, EI uses it to update the current fmin. The default implementation only updates the attributes of the acqusition function which are already present.

Calls _update to update the acquisition function attributes.

Parameters:

• model (AbstractModel) – The model which was used to fit the data.

• kwargs (Any) – Additional arguments to update the specific acquisition function.

Return type:

None

class smac.acquisition.function.EI(xi=0.0, log=False)

Bases: AbstractAcquisitionFunction

Expected Improvement (with or without function values in log space) acquisition function

\(EI(X) := \mathbb{E}\left[ \max\{0, f(\mathbf{X^+}) - f_{t+1}(\mathbf{X}) - \xi \} \right]\), with \(f(X^+)\) as the best location.

Parameters:

• xi (float, defaults to 0.0) – Controls the balance between exploration and exploitation of the acquisition function.

• log (bool, defaults to False) – Whether the function values are in log-space.

_xi

Exploration-exloitation trade-off parameter.

Type:

float

_log

Function values in log-space or not.

Type:

bool

_eta

Current incumbent function value (best value observed so far).

Type:

float

property meta: dict[str, Any]

Returns the meta data of the created object.

property name: str

Returns the full name of the acquisition function.

class smac.acquisition.function.EIPS(xi=0.0)

Bases: EI

Expected Improvement per Second acquisition function

\(EI(X) := \frac{\mathbb{E}\left[\max\{0,f(\mathbf{X^+})-f_{t+1}(\mathbf{X})-\xi\right]\}]}{np.log(r(x))}\), with \(f(X^+)\) as the best location and \(r(x)\) as runtime.

Parameters:

xi (float, defaults to 0.0) – Controls the balance between exploration and exploitation of the acquisition function.

property name: str

Returns the full name of the acquisition function.

class smac.acquisition.function.IntegratedAcquisitionFunction(acquisition_function)

Bases: AbstractAcquisitionFunction

Compute the integrated acquisition function by marginalizing over model hyperparameters

See “Practical Bayesian Optimization of Machine Learning Algorithms” by Jasper Snoek et al. (https://papers.nips.cc/paper/4522-practical-bayesian-optimization-of-machine-learning-algorithms.pdf) for further details.

Parameters:

acquisition_function (AbstractAcquisitionFunction) – Acquisition function to be integrated.

_acquisition_function

Acquisition function to be integrated.

Type:

AbstractAcquisitionFunction

_functions

Holds n (n = number of models) copies of the acquisition function.

Type:

list[AbstractAcquisitionFunction]

_eta

Current incumbent function value.

Type:

float

property meta: dict[str, Any]

Returns the meta data of the created object.

property name: str

Returns the full name of the acquisition function.

class smac.acquisition.function.LCB(beta=1.0)

Bases: AbstractAcquisitionFunction

Computes the lower confidence bound for a given x over the best so far value as acquisition value.

\(LCB(X) = \mu(\mathbf{X}) - \sqrt(\beta_t)\sigma(\mathbf{X})\) [SKKS10]

with

\(\beta_t = 2 \log( |D| t^2 / \beta)\)

\(\text{Input space} D\) \(\text{Number of input dimensions} |D|\) \(\text{Number of data points} t\) \(\text{Exploration/exploitation tradeoff} \beta\)

Returns -LCB(X) as the acquisition_function optimizer maximizes the acquisition value.

Parameters:

beta (float, defaults to 1.0) – Controls the balance between exploration and exploitation of the acquisition function.

_beta

Exploration-exploitation trade-off parameter.

Type:

float

_num_data

Number of data points seen so far.

Type:

int

property meta: dict[str, Any]

Returns the meta data of the created object.

property name: str

Returns the full name of the acquisition function.

class smac.acquisition.function.PI(xi=0.0)

Bases: AbstractAcquisitionFunction

Probability of Improvement

\(P(f_{t+1}(\mathbf{X})\geq f(\mathbf{X^+}))\) \(:= \Phi(\\frac{ \mu(\mathbf{X})-f(\mathbf{X^+}) } { \sigma(\mathbf{X}) })\) with \(f(X^+)\) as the incumbent and \(\Phi\) the cdf of the standard normal.

Parameters:

xi (float, defaults to 0.0) – Controls the balance between exploration and exploitation of the acquisition function.

property meta: dict[str, Any]

Returns the meta data of the created object.

property name: str

Returns the full name of the acquisition function.

class smac.acquisition.function.PriorAcquisitionFunction(acquisition_function, decay_beta, prior_floor=1e-12, discretize=False, discrete_bins_factor=10.0)

Bases: AbstractAcquisitionFunction

Weight the acquisition function with a user-defined prior over the optimum.

See “piBO: Augmenting Acquisition Functions with User Beliefs for Bayesian Optimization” by Carl Hvarfner et al. [HSSL22] for further details.

Parameters:

• decay_beta (float) – Decay factor on the user prior. A solid default value for decay_beta (empirically founded) is scenario.n_trials / 10.

• prior_floor (float, defaults to 1e-12) – Lowest possible value of the prior to ensure non-negativity for all values in the search space.

• discretize (bool, defaults to False) – Whether to discretize (bin) the densities for continous parameters. Triggered for Random Forest models and continous hyperparameters to avoid a pathological case where all Random Forest randomness is removed (RF surrogates require piecewise constant acquisition functions to be well-behaved).

• discrete_bins_factor (float, defaults to 10.0) – If discretizing, the multiple on the number of allowed bins for each parameter.

property meta: dict[str, Any]

Returns the meta data of the created object.

property model: AbstractModel | None

Return the used surrogate model in the acquisition function.

property name: str

Returns the full name of the acquisition function.

class smac.acquisition.function.TS

Bases: AbstractAcquisitionFunction

Thompson Sampling

Warning

Thompson Sampling can only be used together with RandomSearch. Please do not use LocalAndSortedRandomSearch to optimize the TS acquisition function!

:math:`TS(X) ~ mathcal{N}(mu(mathbf{X}),sigma(mathbf{X}))’ Returns -TS(X) as the acquisition_function optimizer maximizes the acquisition value.

Parameters:

xi (float, defaults to 0.0) – TS does not require xi here, we only wants to make it consistent with other acquisition functions.

property name: str

Returns the full name of the acquisition function.

Modules

smac.acquisition.function.abstract_acquisition_function
smac.acquisition.function.confidence_bound
smac.acquisition.function.expected_improvement
smac.acquisition.function.integrated_acquisition_function
smac.acquisition.function.prior_acqusition_function
smac.acquisition.function.probability_improvement
smac.acquisition.function.thompson

previous

smac.acquisition

next

smac.acquisition.function.abstract_acquisition_function