smac.intensification.intensification

Classes

Intensifier(stats, traj_logger, rng, instances)

Races challengers against an incumbent.

IntensifierStage(value)

Class to define different stages of intensifier.

Exceptions

NoMoreChallengers

Indicates that no more challengers are available for the intensification to proceed.
class smac.intensification.intensification.Intensifier(stats, traj_logger, rng, instances, instance_specifics=None, cutoff=None, deterministic=False, run_obj_time=True, always_race_against=None, run_limit=2147483647, use_ta_time_bound=False, minR=1, maxR=2000, adaptive_capping_slackfactor=1.2, min_chall=2)[source]

Bases: smac.intensification.abstract_racer.AbstractRacer

Races challengers against an incumbent. SMAC’s intensification procedure, in detail:

Procedure 2: Intensify(Θ_new, θ_inc, M, R, t_intensify, Π, cˆ) cˆ(θ, Π’) denotes the empirical cost of θ on the subset of instances Π’ ⊆ Π, based on the runs in R; maxR is a parameter where: Θ_new: Sequence of parameter settings to evaluate, challengers in this class. θ_inc: incumbent parameter setting, incumbent in this class.

1 for i := 1, … , length(Θ_new) do 2 θ_new ← Θ_new[i]

STAGE–>RUN_INCUMBENT

3 if R contains less than maxR runs with configuration θ_inc then 4 Π’ ← {π’∈ Π | R contains less than or equal number of runs using θ_inc and π’ 0 than using θ_inc and any other π’’∈ Π} 5 π ← instance sampled uniformly at random from Π’ 6 s ← seed, drawn uniformly at random 7 R ← ExecuteRun(R, θ_inc, π, s) 8 N ← 1

STAGE–>RUN_CHALLENGER

9 while true do 10 S_missing ← {instance, seed} pairs for which θ_inc was run before, but not θ_new 11 S_torun ← random subset of S_missing of size min(N, size(S_missing)) 12 foreach (π, s) ∈ S_torun do R ← ExecuteRun(R, θ_new, π, s) 13 S_missing ← S_missing \ S_torun 14 Π_common ← instances for which we previously ran both θ_inc and θ_new 15 if cˆ(θ_new, Π_common) > cˆ(θ_inc, Π_common) then break 16 else if S_missing = ∅ then θ_inc ← θ_new; break 17 else N ← 2 · N 18 if time spent in this call to this procedure exceeds t_intensify and i ≥ 2 then break 19 return [R, θ_inc]

Parameters

  • stats (Stats) – stats object

  • traj_logger (TrajLogger) – TrajLogger object to log all new incumbents

  • rng (np.random.RandomState) –

  • instances (List[str]) – list of all instance ids

  • instance_specifics (Mapping[str, str]) – mapping from instance name to instance specific string

  • cutoff (int) – runtime cutoff of TA runs

  • deterministic (bool) – whether the TA is deterministic or not

  • run_obj_time (bool) – whether the run objective is runtime or not (if true, apply adaptive capping)

  • always_race_against (Configuration) – if incumbent changes race this configuration always against new incumbent; can sometimes prevent over-tuning

  • use_ta_time_bound (bool,) – if true, trust time reported by the target algorithms instead of measuring the wallclock time for limiting the time of intensification

  • run_limit (int) – Maximum number of target algorithm runs per call to intensify.

  • maxR (int) – Maximum number of runs per config (summed over all calls to intensifiy).

  • minR (int) – Minimum number of run per config (summed over all calls to intensify).

  • adaptive_capping_slackfactor (float) – slack factor of adpative capping (factor * adpative cutoff)

  • min_chall (int) – minimal number of challengers to be considered (even if time_bound is exhausted earlier)

get_next_challenger(challengers, chooser)[source]

This function returns the next challenger, that should be exercised though lines 8-17.

It does so by populating configs_to_run, which is a pool of configuration from which the racer will sample. Each configuration within configs_to_run, will be intensified on different instances/seed registered in self.to_run as stated in line 11.

A brand new configuration should only be sampled, after all self.to_run instance seed pairs are exhausted.

This method triggers a call to _next_iteration if there are no more configurations to run, for the current intensification loop. This marks the transition to Line 2, where a new configuration to intensify will be drawn from epm/initial challengers.

Parameters

  • challengers (List[Configuration]) – promising configurations

  • chooser (smac.optimizer.epm_configuration_chooser.EPMChooser) – optimizer that generates next configurations to use for racing

Return type

Tuple[Optional[Configuration], bool]

Returns

  • Optional[Configuration] – next configuration to evaluate

  • bool – flag telling if the configuration is newly sampled or one currently being tracked

get_next_run(challengers, incumbent, chooser, run_history, repeat_configs=True, num_workers=1)[source]

This procedure is in charge of generating a RunInfo object to comply with lines 7 (in case stage is stage==RUN_INCUMBENT) or line 12 (In case of stage==RUN_CHALLENGER)

A RunInfo object encapsulates the necessary information for a worker to execute the job, nevertheless, a challenger is not always available. This could happen because no more configurations are available or the new configuration to try was already executed.

To circumvent this, a intent is also returned:

  • (intent=RUN) Run the RunInfo object (Normal Execution

  • (intent=SKIP) Skip this iteration. No challenger is available, in particular

    because challenger is the same as incumbent

Parameters

  • challengers (List[Configuration]) – promising configurations

  • incumbent (Configuration) – incumbent configuration

  • chooser (smac.optimizer.epm_configuration_chooser.EPMChooser) – optimizer that generates next configurations to use for racing

  • run_history (RunHistory) – stores all runs we ran so far

  • repeat_configs (bool) – if False, an evaluated configuration will not be generated again

  • num_workers (int) – the maximum number of workers available at a given time.

Return type

Tuple[RunInfoIntent, RunInfo]

Returns

  • intent (RunInfoIntent) – What should the smbo object do with the runinfo.

  • run_info (RunInfo) – An object that encapsulates necessary information for a config run

process_results(run_info, incumbent, run_history, time_bound, result, log_traj=True)[source]

The intensifier stage will be updated based on the results/status of a configuration execution.

During intensification, the following can happen:

  • Challenger raced against incumbent

  • Also, during a challenger run, a capped exception can be triggered, where no racer post processing is needed

  • A run on the incumbent for more confidence needs to be processed, IntensifierStage.PROCESS_INCUMBENT_RUN

  • The first run results need to be processed (PROCESS_FIRST_CONFIG_RUN)

At the end of any run, checks are done to move to a new iteration.

Parameters

  • run_info (RunInfo) – A RunInfo containing the configuration that was evaluated

  • incumbent (Optional[Configuration]) – best configuration so far, None in 1st run

  • run_history (RunHistory) – stores all runs we ran so far if False, an evaluated configuration will not be generated again

  • time_bound (float) – time in [sec] available to perform intensify

  • result (RunValue) – Contain the result (status and other methadata) of exercising a challenger/incumbent.

  • log_traj (bool) – whether to log changes of incumbents in trajectory

Return type

Tuple[Configuration, float]

Returns

  • incumbent (Configuration()) – current (maybe new) incumbent configuration

  • inc_perf (float) – empirical performance of incumbent configuration

class smac.intensification.intensification.IntensifierStage(value)[source]

Bases: enum.Enum

Class to define different stages of intensifier.

exception smac.intensification.intensification.NoMoreChallengers[source]

Bases: Exception

Indicates that no more challengers are available for the intensification to proceed.

previous

smac.intensification.hyperband

next

smac.intensification.parallel_scheduling