deepcave.evaluators.mo_ablation

# Ablation Paths

This module evaluates the ablation paths.

Ablation Paths is a method to analyze the importance of hyperparameters in a configuration space. Starting from a default configuration, the default configuration is iteratively changed to the incumbent configuration by changing one hyperparameter at a time, choosing the hyperparameter that leads to the largest improvement in the objective function at each step.

## Classes:

  • Ablation: Provide an evaluator of the ablation paths.

Classes

MOAblation(run)

Provide an evaluator of the ablation paths.
class deepcave.evaluators.mo_ablation.MOAblation(run)[source]

Bases: Ablation

Provide an evaluator of the ablation paths.

Override: Multi-Objective case

Properties

runAbstractRun

The run to analyze.

csConfigurationSpace

The configuration space of the run.

hp_namesList[str]

A list of the hyperparameter names.

performancesOptional[Dict[Any, Any]]

A dictionary containing the performances for each HP.

improvementsOptional[Dict[Any, Any]]

A dictionary containing the improvements over the respective previous step for each HP.

objectivesOptional[Union[Objective, List[Objective]]]

The objective(s) of the run.

default_configConfigurations

The default configuration of this configuration space. Gets changed step by step towards the incumbent configuration.

ablation(budget, incumbent_config, def_cost, hp_it, weighting)[source]

Calculate the ablation importance for each hyperparameter.

Parameters:

  • budget (Optional[Union[int, float]]) – The budget of the run.

  • incumbent_config (Any) – The incumbent configuration.

  • def_cost (Any) – The default cost.

  • hp_it (List[str]) – A list of the HPs that still have to be looked at.

  • weighting (np.ndarray[Any, Any]) – The weighting of the objective values.

Returns:

continue_ablation, max_hp, max_hp_performance, max_hp_std

Return type:

Tuple[Any, Any, Any, Any]

calculate(objectives, budget=None, n_trees=50, seed=0)[source]

Calculate the MO ablation path performances and improvements.

Parameters:

  • objectives (Optional[Union[Objective, List[Objective]]]) – The objective(s) to be considered.

  • budget (Optional[Union[int, float]]) – The budget to be considered. If None, all budgets of the run are considered. Default is None.

  • n_trees (int) – The number of trees for the surrogate model. Default is 50.

  • seed (int) – The seed for the surrogate model. Default is 0.

Return type:

None

calculate_ablation_path(df, objectives_normed, weighting, budget)[source]

Calculate the ablation path performances.

Parameters:

  • df (pd.DataFrame) – Dataframe with encoded data.

  • objectives_normed (List[str]) – The normed objective names to be considered.

  • weighting (np.ndarray) – The weighting of the objective values.

  • budget (Optional[Union[int, float]]) – The budget to be considered. If None, all budgets of the run are considered. Default is None.

Returns:

df – Dataframe with results of the ablation calculation.

Return type:

pd.DataFrame

get_importances()[source]

Return the importance scores.

Returns:

Dictionary with Hyperparameter names and the corresponding importance scores and variances.

Return type:

Dict

Raises:

RuntimeError – If the important scores are not calculated.

predict(cfg, weighting)[source]

Predict the performance of the input configuration.

The model results are weighted by the input weightings and summed.

Parameters:

  • cfg (Dict) – Configuration.

  • weighting (List[float]) – Weightings.

Return type:

Tuple[float, float]

Returns:

  • mean (float) – The mean of the weighted sum of predictions.

  • var (float) – The variance of the weighted sum of predictions.

previous

deepcave.evaluators.lpi

next

deepcave.evaluators.mo_fanova