ParallelScheduler(stats: smac.stats.stats.Stats, traj_logger: smac.utils.io.traj_logging.TrajLogger, rng: numpy.random.mtrand.RandomState, instances: List[str], instance_specifics: Mapping[str, numpy.ndarray] = None, cutoff: Optional[float] = None, deterministic: bool = False, initial_budget: Optional[float] = None, max_budget: Optional[float] = None, eta: float = 3, num_initial_challengers: Optional[int] = None, run_obj_time: bool = True, n_seeds: Optional[int] = None, instance_order: Optional[str] = 'shuffle_once', adaptive_capping_slackfactor: float = 1.2, inst_seed_pairs: Optional[List[Tuple[str, int]]] = None, min_chall: int = 1, incumbent_selection: str = 'highest_executed_budget')¶
Common Racer class for Intensifiers that will schedule configurations on a parallel fashion.
This class instantiates intensifier objects on a need basis, that is, to prevent workers from being idle. This intensifier objects will give configurations to run
stats (smac.stats.stats.Stats) – stats object
traj_logger (smac.utils.io.traj_logging.TrajLogger) – TrajLogger object to log all new incumbents
rng (np.random.RandomState) –
instances (typing.List[str]) – list of all instance ids
instance_specifics (typing.Mapping[str,np.ndarray]) – mapping from instance name to instance specific string
cutoff (typing.Optional[int]) – cutoff of TA runs
deterministic (bool) – whether the TA is deterministic or not
initial_budget (typing.Optional[float]) – minimum budget allowed for 1 run of successive halving
max_budget (typing.Optional[float]) – maximum budget allowed for 1 run of successive halving
eta (float) – ‘halving’ factor after each iteration in a successive halving run. Defaults to 3
num_initial_challengers (typing.Optional[int]) – number of challengers to consider for the initial budget. If None, calculated internally
run_obj_time (bool) – whether the run objective is runtime or not (if true, apply adaptive capping)
n_seeds (typing.Optional[int]) – Number of seeds to use, if TA is not deterministic. Defaults to None, i.e., seed is set as 0
instance_order (typing.Optional[str]) – how to order instances. Can be set to: [None, shuffle_once, shuffle] * None - use as is given by the user * shuffle_once - shuffle once and use across all SH run (default) * shuffle - shuffle before every SH run
adaptive_capping_slackfactor (float) – slack factor of adpative capping (factor * adaptive cutoff)
inst_seed_pairs (typing.List[typing.Tuple[str, int]], optional) – Do not set this argument, it will only be used by hyperband!
min_chall (int) – minimal number of challengers to be considered (even if time_bound is exhausted earlier). This class will raise an exception if a value larger than 1 is passed.
incumbent_selection (str) – How to select incumbent in successive halving. Only active for real-valued budgets. Can be set to: [highest_executed_budget, highest_budget, any_budget] * highest_executed_budget - incumbent is the best in the highest budget run so far (default) * highest_budget - incumbent is selected only based on the highest budget * any_budget - incumbent is the best on any budget i.e., best performance regardless of budget
_add_new_instance(num_workers: int) → bool¶
Decides if it is possible to add a new intensifier instance, and adds it. If a new intensifier instance is added, True is returned, else False.
- num_workers: int
the maximum number of workers available at a given time.
- Return type
Whether or not a successive halving instance was added
_get_intensifier_ranking(intensifier: smac.intensification.abstract_racer.AbstractRacer) → Tuple[int, int]¶
Given a intensifier, returns how advance it is. This metric will be used to determine what priority to assign to the intensifier
intensifier (AbstractRacer) – Intensifier to rank based on run progress
ranking (int) – the higher this number, the faster the intensifier will get the running resources. For hyperband we can use the sh_intensifier stage, for example
tie_breaker (int) – The configurations that have been launched to break ties. For example, in the case of Successive Halving it can be the number of configurations launched
_sort_instances_by_stage(instances: Dict[int, smac.intensification.abstract_racer.AbstractRacer]) → List[int]¶
This procedure dictates what SH to prioritize in launching jobs. It prioritizes resource allocation to SH instances that have higher stages. In case of tie, we prioritize the SH instance with more launched configs
instances (typing.Dict[int, AbstractRacer]) – Dict with the instances to prioritize
The order in which to query for new jobs
- Return type
get_next_run(challengers: Optional[List[ConfigSpace.configuration_space.Configuration]], incumbent: ConfigSpace.configuration_space.Configuration, chooser: Optional[smac.optimizer.epm_configuration_chooser.EPMChooser], run_history: smac.runhistory.runhistory.RunHistory, repeat_configs: bool = False, num_workers: int = 1) → Tuple[smac.intensification.abstract_racer.RunInfoIntent, smac.runhistory.runhistory.RunInfo]¶
This procedure decides from which instance to pick a config, in order to determine the next run.
To prevent having idle workers, this procedure creates new instances up to the maximum number of workers available.
If no new intensifier instance can be created and all intensifier objects need to wait for more data, this procedure sends a wait request to smbo.
challengers (typing.List[Configuration]) – promising configurations
incumbent (Configuration) – incumbent configuration
chooser (smac.optimizer.epm_configuration_chooser.EPMChooser) – optimizer that generates next configurations to use for racing
run_history (smac.runhistory.runhistory.RunHistory) – stores all runs we ran so far
repeat_configs (bool) – if False, an evaluated configuration will not be generated again
num_workers (int) – the maximum number of workers available at a given time.
intent (RunInfoIntent) – Indicator of how to consume the RunInfo object
run_info (RunInfo) – An object that encapsulates the minimum information to evaluate a configuration
process_results(run_info: smac.runhistory.runhistory.RunInfo, incumbent: Optional[ConfigSpace.configuration_space.Configuration], run_history: smac.runhistory.runhistory.RunHistory, time_bound: float, result: smac.runhistory.runhistory.RunValue, log_traj: bool = True) → Tuple[ConfigSpace.configuration_space.Configuration, float]¶
The intensifier stage will be updated based on the results/status of a configuration execution.
To do so, this procedures redirects the result argument, to the respective intensifier object that generated the original config.
Also, an incumbent will be determined. This determination is done using the complete run history, so we rely on the current intensifier choice of incumbent. That is, no need to go over each instance to get the incumbent, as there is no local runhistory
run_info (RunInfo) – A RunInfo containing the configuration that was evaluated
incumbent (typing.Optional[Configuration]) – Best configuration seen so far
run_history (RunHistory) – stores all runs we ran so far if False, an evaluated configuration will not be generated again
time_bound (float) – time in [sec] available to perform intensify
result (RunValue) – Contain the result (status and other methadata) of exercising a challenger/incumbent.
log_traj (bool) – Whether to log changes of incumbents in trajectory
incumbent (Configuration) – current (maybe new) incumbent configuration
inc_perf (float) – empirical performance of incumbent configuration