smac.epm.gp_base_prior module

class smac.epm.gp_base_prior.GammaPrior(a: float, scale: float, loc: float, rng: numpy.random.mtrand.RandomState)[source]

Bases: smac.epm.gp_base_prior.Prior

Gamma prior

f(x) = (x-loc)**(a-1) * e**(-(x-loc)) * (1/scale)**a / gamma(a)

Parameters
  • a (float > 0) – shape parameter

  • scale (float > 0) – scale parameter (1/scale corresponds to parameter p in canonical form)

  • loc (float) – mean parameter for the distribution

  • rng (np.random.RandomState) – Random number generator

_gradient(theta: float) → float[source]

As computed by Wolfram Alpha

Parameters

theta (float) – A hyperparameter configuration

Returns

Return type

float

_lnprob(theta: float) → float[source]

Returns the logpdf of theta.

Parameters

theta (float) – Hyperparameter configuration

Returns

Return type

float

_sample_from_prior(n_samples: int) → numpy.ndarray[source]

Returns N samples from the prior.

Parameters

n_samples (int) – The number of samples that will be drawn.

Returns

Return type

np.ndarray

class smac.epm.gp_base_prior.HorseshoePrior(scale: float, rng: numpy.random.mtrand.RandomState)[source]

Bases: smac.epm.gp_base_prior.Prior

Horseshoe Prior as it is used in spearmint

This class is adapted from RoBO:

Klein, A. and Falkner, S. and Mansur, N. and Hutter, F. RoBO: A Flexible and Robust Bayesian Optimization Framework in Python In: NIPS 2017 Bayesian Optimization Workshop

Parameters
  • scale (float) – Scaling parameter. See below how it is influencing the distribution.

  • rng (np.random.RandomState) – Random number generator

_gradient(theta: float) → float[source]

Computes the gradient of the prior with respect to theta.

Parameters

theta ((D,) numpy array) – Hyperparameter configuration

Returns

The gradient of the prior at theta.

Return type

  1. np.array

_lnprob(theta: float) → float[source]

Return the log probability of theta.

Parameters

theta ((D,) numpy array) – A hyperparameter configuration

Returns

Return type

float

_sample_from_prior(n_samples: int) → numpy.ndarray[source]

Returns N samples from the prior.

Parameters

n_samples (int) – The number of samples that will be drawn.

Returns

Return type

np.ndarray

class smac.epm.gp_base_prior.LognormalPrior(sigma: float, rng: numpy.random.mtrand.RandomState, mean: float = 0)[source]

Bases: smac.epm.gp_base_prior.Prior

Log normal prior

This class is adapted from RoBO:

Klein, A. and Falkner, S. and Mansur, N. and Hutter, F. RoBO: A Flexible and Robust Bayesian Optimization Framework in Python In: NIPS 2017 Bayesian Optimization Workshop

Parameters
  • sigma (float) – Specifies the standard deviation of the normal distribution.

  • rng (np.random.RandomState) – Random number generator

  • mean (float) – Specifies the mean of the normal distribution

_gradient(theta: float) → float[source]

Computes the gradient of the prior with respect to theta.

Parameters

theta ((D,) numpy array) – Hyperparameter configuration in log space

Returns

The gradient of the prior at theta.

Return type

  1. np.array

_lnprob(theta: float) → float[source]

Return the log probability of theta

Parameters

theta (float) – A hyperparameter configuration

Returns

Return type

float

_sample_from_prior(n_samples: int) → numpy.ndarray[source]

Returns N samples from the prior.

Parameters

n_samples (int) – The number of samples that will be drawn.

Returns

Return type

np.ndarray

class smac.epm.gp_base_prior.Prior(rng: numpy.random.mtrand.RandomState)[source]

Bases: object

Abstract base class to define the interface for priors of GP hyperparameter.

This class is adapted from RoBO:

Klein, A. and Falkner, S. and Mansur, N. and Hutter, F. RoBO: A Flexible and Robust Bayesian Optimization Framework in Python In: NIPS 2017 Bayesian Optimization Workshop

[16.04.2019]: Whenever lnprob or the gradient is computed for a scalar input, we use math.* rather than np.*

Parameters

rng (np.random.RandomState) – Random number generator

_gradient(theta: float) → float[source]

Computes the gradient of the prior with respect to theta.

Parameters

theta (float) – Hyperparameter configuration in the original space space

Returns

The gradient of the prior at theta.

Return type

float

_lnprob(theta: float) → float[source]

Return the log probability of theta.

Theta must be on the original scale.

Parameters

theta (float) – Hyperparameter configuration on the original scale.

Returns

The log probability of theta

Return type

float

_sample_from_prior(n_samples: int) → numpy.ndarray[source]

Returns n_samples from the prior.

All samples are on a original scale.

Parameters

n_samples (int) – The number of samples that will be drawn.

Returns

Return type

np.ndarray

gradient(theta: float) → float[source]

Computes the gradient of the prior with respect to theta.

Theta must be on the original scale.

Parameters

theta (float) – Hyperparameter configuration in log space

Returns

The gradient of the prior at theta.

Return type

float

lnprob(theta: float) → float[source]

Return the log probability of theta.

Theta must be on a log scale! This method exponentiates theta and calls self._lnprob.

Parameters

theta (float) – Hyperparameter configuration in log space.

Returns

The log probability of theta

Return type

float

sample_from_prior(n_samples: int) → numpy.ndarray[source]

Returns n_samples from the prior.

All samples are on a log scale. This method calls self._sample_from_prior and applies a log transformation to the obtained values.

Parameters

n_samples (int) – The number of samples that will be drawn.

Returns

Return type

np.ndarray

class smac.epm.gp_base_prior.SoftTopHatPrior(lower_bound: float, upper_bound: float, exponent: float, rng: numpy.random.mtrand.RandomState)[source]

Bases: smac.epm.gp_base_prior.Prior

Abstract base class to define the interface for priors of GP hyperparameter.

This class is adapted from RoBO:

Klein, A. and Falkner, S. and Mansur, N. and Hutter, F. RoBO: A Flexible and Robust Bayesian Optimization Framework in Python In: NIPS 2017 Bayesian Optimization Workshop

[16.04.2019]: Whenever lnprob or the gradient is computed for a scalar input, we use math.* rather than np.*

Parameters

rng (np.random.RandomState) – Random number generator

_sample_from_prior(n_samples: int) → numpy.ndarray[source]

Returns N samples from the prior.

Parameters

n_samples (int) – The number of samples that will be drawn.

Returns

Return type

np.ndarray

gradient(theta: float) → float[source]

Computes the gradient of the prior with respect to theta.

Theta must be on the original scale.

Parameters

theta (float) – Hyperparameter configuration in log space

Returns

The gradient of the prior at theta.

Return type

float

lnprob(theta: float) → float[source]

Return the log probability of theta.

Theta must be on a log scale! This method exponentiates theta and calls self._lnprob.

Parameters

theta (float) – Hyperparameter configuration in log space.

Returns

The log probability of theta

Return type

float

class smac.epm.gp_base_prior.TophatPrior(lower_bound: float, upper_bound: float, rng: numpy.random.mtrand.RandomState)[source]

Bases: smac.epm.gp_base_prior.Prior

Tophat prior as it used in the original spearmint code.

This class is adapted from RoBO:

Klein, A. and Falkner, S. and Mansur, N. and Hutter, F. RoBO: A Flexible and Robust Bayesian Optimization Framework in Python In: NIPS 2017 Bayesian Optimization Workshop

Parameters
  • lower_bound (float) – Lower bound of the prior. In original scale.

  • upper_bound (float) – Upper bound of the prior. In original scale.

  • rng (np.random.RandomState) – Random number generator

_lnprob(theta: float) → float[source]

Return the log probability of theta.

Parameters

theta (float) – A hyperparameter configuration

Returns

Return type

float

_sample_from_prior(n_samples: int) → numpy.ndarray[source]

Return n_samples from the prior.

Parameters

n_samples (int) – The number of samples that will be drawn.

Returns

Return type

np.ndarray

gradient(theta: float) → float[source]

Computes the gradient of the prior with respect to theta.

Parameters

theta (float) – Hyperparameter configuration in log space

Returns

The gradient of the prior at theta.

Return type

  1. np.array