cave.plot.configurator_footprint module

class cave.plot.configurator_footprint.ConfiguratorFootprintPlotter(scenario: smac.scenario.scenario.Scenario, rhs: smac.runhistory.runhistory.RunHistory, incs: list = None, final_incumbent=None, rh_labels=None, max_plot: int = -1, contour_step_size=0.2, use_timeslider: bool = False, num_quantiles: int = 10, timeslider_log: bool = True, output_dir: str = None)[source]

Bases: object

Creating an interactive plot, visualizing the configuration search space. The runhistories are correlated to the individual runs. Each run consists of a runhistory (in the smac-format), a list of incumbents If the dict “additional_info” in the RunValues of the runhistory contains a nested dict with additional_info[“timestamps”][“finished”], using those timestamps to sort data

  • scenario (Scenario) – scenario

  • rhs (List[RunHistory]) – runhistories from configurator runs, only data collected during optimization (no validation!)

  • incs (List[List[Configuration]]) – incumbents per run, last entry is final incumbent

  • final_incumbent (Configuration) – final configuration (best of all runs)

  • max_plot (int) – maximum number of configs to plot, if -1 plot all

  • contour_step_size (float) – step size of meshgrid to compute contour of fitness landscape

  • use_timeslider (bool) – whether or not to have a time_slider-widget on cfp-plot INCREASES FILE-SIZE DRAMATICALLY

  • num_quantiles (int) – number of quantiles for the slider/ number of static pictures

  • timeslider_log (bool) – whether to use a logarithmic scale for the timeslider/quantiles

  • output_dir (str) – output directory

get_conf_matrix(rh, incs)[source]

Iterates through runhistory to get a matrix of configurations (in vector representation), a list of configurations and the number of runs per configuration in a quantiled manner.

  • rh (RunHistory) – smac.runhistory

  • incs (List[List[Configuration]]) – incumbents of configurator runs, last entry is final incumbent


  • conf_matrix (np.array) – matrix of configurations in vector representation

  • conf_list (np.array) – list of all Configuration objects that appeared in runhistory the order of this list is used to determine all kinds of properties in the plotting (but is arbitrarily determined)

  • runs_per_quantile (np.array) – numpy array of runs per configuration per quantile

  • labels (List[str]) – labels for timeslider (i.e. wallclock-times)

get_depth(cs: ConfigSpace.configuration_space.ConfigurationSpace, param: str)[source]

Get depth in configuration space of a given parameter name breadth search until reaching a leaf for the first time

  • cs (ConfigurationSpace) – ConfigurationSpace to get parents of a parameter

  • param (str) – name of parameter to inspect

get_distance(conf_matrix, cs: ConfigSpace.configuration_space.ConfigurationSpace)[source]

Computes the distance between all pairs of configurations.

  • conf_matrx (np.array) – numpy array with cols as parameter values

  • cs (ConfigurationSpace) – ConfigurationSpace to get conditionalities


dists – np.array with distances between configurations i,j in dists[i,j] or dists[j,i]

Return type



Compute multi-dimensional scaling (using sklearn MDS) – nonlinear scaling


dists (np.array) – full matrix of distances between all configurations


scaled coordinates in 2-dim room

Return type


get_pred_surface(rh, X_scaled, conf_list: list, contour_step_size)[source]

fit epm on the scaled input dimension and return data to plot a contour plot of the empirical performance

  • rh (RunHistory) – runhistory

  • X_scaled (np.array) – configurations in scaled 2dim

  • conf_list (list) – list of Configuration objects

  • contour_step_size (float) – step-size for contour


contour_data – x, y, Z for contour plots

Return type

(np.array, np.array, np.array)

plot(X, conf_list: list, runs_per_quantile, inc_list: list = None, contour_data=None, use_timeslider=False, use_checkbox=True, timeslider_labels=None)[source]

plots sampled configuration in 2d-space; uses bokeh for interactive plot saves results in self.output, if set

  • X (np.array) – np.array with 2-d coordinates for each configuration

  • conf_list (list) – list of ALL configurations in the same order as X

  • runs_per_quantile (list[np.array]) – configurator-run to be analyzed, as a np.array with the number of target-algorithm-runs per config per quantile.

  • inc_list (list) – list of incumbents (Configuration)

  • contour_data (list) – contour data (xx,yy,Z)

  • use_timeslider (bool) – whether or not to have a time_slider-widget on cfp-plot INCREASES FILE-SIZE DRAMATICALLY

  • use_checkbox (bool) – have checkboxes to toggle individual runs


  • (script, div) (str) – script and div of the bokeh-figure

  • over_time_paths (List[str]) – list with paths to the different quantiled timesteps of the configurator run (for static evaluation)

reduce_runhistory(rh: smac.runhistory.runhistory.RunHistory, max_configs: int, keep=None)[source]

Reduce configs to desired number, by default just drop the configs with the fewest runs.

  • rh (RunHistory) – runhistory that is to be reduced

  • max_configs (int) – if > -1 reduce runhistory to at most max_configs

  • keep (List[Configuration]) – list of configs that should be kept for sure (e.g. default, incumbents)


rh – reduced runhistory

Return type



Uses available Configurator-data to perform a MDS, estimate performance data and plot the configurator footprint.