cave.analyzer.configurator.parallel_coordinates module

class cave.analyzer.configurator.parallel_coordinates.ParallelCoordinates(runscontainer, pc_sort_by: str = None, params: Union[int, List[str]] = None, n_configs: int = None, max_runs_epm: int = None)

Bases: cave.analyzer.base_analyzer.BaseAnalyzer

Previously used by Golovin et al. to study the frequency of chosen parameter settings in black-box-optimization. Each line corresponds to one configuration in the runhistory and shows the parameter settings and the corresponding (estimated) average cost. To handle large configuration spaces with hundreds of parameters, the (at most) 10 most important parameters based on a fANOVA parameter importance analysis are plotted. To emphasize better configurations, the performance is encoded in the color of each line, ranging from blue to red. These plots provide insights into whether the configurator focused on specific parameter values and how these correlate to their costs.

NOTE: the given RunHistory should contain only optimization and no validation to analyze the explored parameter-space.

This function prepares the data from a SMAC-related format (using runhistories and parameters) to a more general format (using a dataframe). The resulting dataframe is passed to the parallel_coordinates-routine

Parameters

• params (Union[int, List[str]]) – either directly the parameters to displayed or the number of parameters (will try to define the most important ones

• n_configs (int) – number of configs to be plotted

• pc_sort_by (str) – defines the pimp-method by which to choose the plotted parameters

• max_runs_epm (int) – maximum number of runs to train the epm with. this should prevent MemoryErrors

_plot_budget(df)

_preprocess()

_preprocess_budget(original_rh: smac.runhistory.runhistory.RunHistory, validated_rh: smac.runhistory.runhistory.RunHistory, validator: smac.utils.validate.Validator, scenario: smac.scenario.scenario.Scenario, default: ConfigSpace.configuration_space.Configuration, incumbent: ConfigSpace.configuration_space.Configuration, param_imp: Union[None, Dict[str, float]], output_dir: str, cs: ConfigSpace.configuration_space.ConfigurationSpace, runtime: bool = False)

Preprocess data and save in self.data to enable fast replots

original_rh: RunHistory

runhistory that should contain only runs that were executed during search

validated_rh: RunHistory

runhistory that may contain as many runs as possible, also external runs. this runhistory will be used to build the EPM

validator: Validator

validator to be used to estimate costs for configurations

scenario: Scenario

scenario object to take instances from

default, incumbent: Configuration

default and incumbent, they will surely be displayed

param_imp: Union[None, Dict[str->float]

if given, maps parameter-names to importance

output_dir: str

output directory for plots

cs: ConfigurationSpace

parameter configuration space to be visualized

runtime: boolean

runtime will be on logscale

classmethod check_for_bokeh(d)

Check if there is bokeh-plots in the output of this analyzer by checking the result-dictionary for the bokeh keyword.

get_html(d=None, tooltip=None)

General reports in html-format, to be easily integrated in html-code. WORKS ALSO FOR BOKEH-OUTPUT.

Parameters

• d (Dictionary) – a dictionary that will be later turned into a website

• tooltip (string) – tooltip to be displayed in report. optional, will overwrite the docstrings that are used by default.

Returns

script, div – header and body part of html-code

Return type

str, str

get_jupyter()

Depending on analysis, this creates jupyter-notebook compatible output.

get_name()

plot_bokeh(return_components=False)

If components is specified, will return script, div-tuple