smac.intensification.successive_halving

Classes

SuccessiveHalving(stats, traj_logger, rng, ...)

Races multiple challengers against an incumbent using Successive Halving method.
class smac.intensification.successive_halving.SuccessiveHalving(stats, traj_logger, rng, instances, instance_specifics=None, cutoff=None, deterministic=False, initial_budget=None, max_budget=None, eta=3, num_initial_challengers=None, run_obj_time=True, n_seeds=None, instance_order='shuffle_once', adaptive_capping_slackfactor=1.2, inst_seed_pairs=None, min_chall=1, incumbent_selection='highest_executed_budget')[source]

Bases: smac.intensification.parallel_scheduling.ParallelScheduler

Races multiple challengers against an incumbent using Successive Halving method.

Implementation following the description in “BOHB: Robust and Efficient Hyperparameter Optimization at Scale” (Falkner et al. 2018) Supplementary reference: http://proceedings.mlr.press/v80/falkner18a/falkner18a-supp.pdf

Successive Halving intensifier (and Hyperband) can operate on two kinds of budgets: 1. ‘Instances’ as budget:

When multiple instances are provided or when run objective is “runtime”, this is the criterion used as budget for successive halving iterations i.e., the budget determines how many instances the challengers are evaluated on at a time. Top challengers for the next iteration are selected based on the combined performance across all instances used.

If initial_budget and max_budget are not provided, then they are set to 1 and total number of available instances respectively by default.

  1. ‘Real-valued’ budget:

    This is used when there is only one instance provided and when run objective is “quality”, i.e., budget is a positive, real-valued number that can be passed to the target algorithm as an argument. It can be used to control anything by the target algorithm, Eg: number of epochs for training a neural network.

    initial_budget and max_budget are required parameters for this type of budget.

Examples for successive halving (and hyperband) can be found here: * Runtime objective and multiple instances (instances as budget): examples/spear_qcp/SMAC4AC_SH_spear_qcp.py * Quality objective and multiple instances (instances as budget): examples/SMAC4MF_sgd_instances.py * Quality objective and single instance (real-valued budget): examples/SMAC4MF_mlp.py

This class instantiates _SuccessiveHalving objects on a need basis, that is, to prevent workers from being idle. The actual logic that implements the Successive halving method lies on the _SuccessiveHalving class.

Parameters

  • stats (smac.stats.stats.Stats) – stats object

  • traj_logger (smac.utils.io.traj_logging.TrajLogger) – TrajLogger object to log all new incumbents

  • rng (np.random.RandomState) –

  • instances (List[str]) – list of all instance ids

  • instance_specifics (Mapping[str, str]) – mapping from instance name to instance specific string

  • cutoff (Optional[int]) – cutoff of TA runs

  • deterministic (bool) – whether the TA is deterministic or not

  • initial_budget (Optional[float]) – minimum budget allowed for 1 run of successive halving

  • max_budget (Optional[float]) – maximum budget allowed for 1 run of successive halving

  • eta (float) – ‘halving’ factor after each iteration in a successive halving run. Defaults to 3

  • num_initial_challengers (Optional[int]) – number of challengers to consider for the initial budget. If None, calculated internally

  • run_obj_time (bool) – whether the run objective is runtime or not (if true, apply adaptive capping)

  • n_seeds (Optional[int]) – Number of seeds to use, if TA is not deterministic. Defaults to None, i.e., seed is set as 0

  • instance_order (Optional[str]) – how to order instances. Can be set to: [None, shuffle_once, shuffle] * None - use as is given by the user * shuffle_once - shuffle once and use across all SH run (default) * shuffle - shuffle before every SH run

  • adaptive_capping_slackfactor (float) – slack factor of adpative capping (factor * adaptive cutoff)

  • inst_seed_pairs (List[Tuple[str, int]], optional) – Do not set this argument, it will only be used by hyperband!

  • min_chall (int) – minimal number of challengers to be considered (even if time_bound is exhausted earlier). This class will raise an exception if a value larger than 1 is passed.

  • incumbent_selection (str) – How to select incumbent in successive halving. Only active for real-valued budgets. Can be set to: [highest_executed_budget, highest_budget, any_budget] * highest_executed_budget - incumbent is the best in the highest budget run so far (default) * highest_budget - incumbent is selected only based on the highest budget * any_budget - incumbent is the best on any budget i.e., best performance regardless of budget

previous

smac.intensification.simple_intensifier

next

smac.optimizer