smac.intensifier.successive_halving

Classes

SuccessiveHalving(scenario[, ...])

Races multiple challengers against an incumbent using Successive Halving method.

Interfaces

class smac.intensifier.successive_halving.SuccessiveHalving(scenario, instance_seed_pairs=None, instance_order='shuffle_once', incumbent_selection='highest_executed_budget', n_initial_challengers=None, min_challenger=1, eta=3, seed=None, n_seeds=None)

Bases: AbstractParallelIntensifier

Races multiple challengers against an incumbent using Successive Halving method.

Implementation following the description in “BOHB: Robust and Efficient Hyperparameter Optimization at Scale” (Falkner et al. 2018) Supplementary reference: http://proceedings.mlr.press/v80/falkner18a/falkner18a-supp.pdf

Successive Halving intensifier (and Hyperband) can operate on two kinds of budgets:

1. Instances as budget:

When multiple instances are provided or when run objective is “runtime”, this is the criterion used as budget for successive halving iterations i.e., the budget determines how many instances the challengers are evaluated on at a time. Top challengers for the next iteration are selected based on the combined performance across all instances used. If min_budget and max_budget are not provided, then they are set to 1 and total number of available instances respectively by default.

2. Real-valued budget:

This is used when there is only one instance provided and when run objective is “quality”, i.e., budget is a positive, real-valued number that can be passed to the target function as an argument. It can be used to control anything by the target function, Eg: number of epochs for training a neural network. The parameters min_budget and max_budget are required for this type of budget.

This class instantiates SuccessiveHalvingWorker objects on a need basis, that is, to prevent workers from being idle. The actual logic that implements the Successive halving method lies in the worker class.

Parameters:

• scenario (Scenario) –

• instance_seed_pairs (list[tuple[str | None, int]] | None, defaults to None) – This argument is used by Hyperband.

• instance_order (str | None, defaults to shuffle_once) – How to order the instances. Can be set to: * None: Use as is given by the user. * shuffle_once: Shuffle once and use across all Successive Halving run . * shuffle: Shuffle before every Successive Halving run

• incumbent_selection (str, defaults to highest_executed_budget) – How to select the incumbent in Successive Halving. Only active for (real-valued) budgets. Can be set to: * highest_executed_budget: Incumbent is the best in the highest budget run so far. * highest_budget: Incumbent is selected only based on the highest budget. * any_budget: Incumbent is the best on any budget i.e., best performance regardless of budget.

• n_initial_challengers (int | None, defaults to None) – Number of challengers to consider for the initial budget. If not specified, it is calculated internally.

• min_challenger (int, defaults to 1) – Minimal number of challengers to be considered (even if time_bound is exhausted earlier).

• eta (float, defaults to 3) – The “halving” factor after each iteration in a Successive Halving run.

• seed (int | None, defaults to None) – This seed is not the seed used for target function evaluation but rather for sampling next candidates.

• n_seeds (int | None, defaults to None) – The number of seeds to use if the target function is non-deterministic.

calculate_budgets(min_budget, max_budget)

Calculates the budgets for successive halving.

Parameters:

• min_budget (float) – The minimal budget.

• max_budget (float) – The maximal budget.

Return type:

tuple[int, int, list[float]]

Returns:

• max_iterations (int) – How many iterations will be performed.

• n_initial_challengers (int) – How many challengers will be sampled in the first iteration.

• budgets (list[float]) – The budgets for each iteration.

get_target_function_budgets()

Which budgets are used to call the target function.

Return type:

list[Optional[float]]

get_target_function_instances()

Which instances are used to call the target function.

Return type:

list[Optional[str]]

get_target_function_seeds()

Which seeds are used to call the target function.

Return type:

list[int]

property meta: dict[str, Any]

Returns the meta data of the created object.

Return type:

dict[str, Any]

property uses_budgets: bool

If the intensifier needs to make use of budgets.

Return type:

bool

property uses_instances: bool

If the intensifier needs to make use of instances.

Return type:

bool

previous

smac.intensifier.stages

next

smac.intensifier.successive_halving_worker