cave.analyzer.performance.plot_ecdf module

class cave.analyzer.performance.plot_ecdf.PlotECDF(runscontainer)[source]

Bases: cave.analyzer.base_analyzer.BaseAnalyzer

Depicts cost distributions over the set of instances. Since these are empirical distributions, the plots show step functions. These plots provide insights into how well configurations perform up to a certain threshold. For runtime scenarios this shows the probability of solving all instances from the set in a given timeframe. On the left side the training-data is scattered, on the right side the test-data is scattered.

Plot the cumulated distribution functions for given configurations, plots will share y-axis and if desired x-axis. Saves plot to file.

_plot_ecdf(default: ConfigSpace.configuration_space.Configuration, incumbent: ConfigSpace.configuration_space.Configuration, rh: smac.runhistory.runhistory.RunHistory, train: List[str], test: List[str], cutoff, output_dir: str)[source]
Parameters

  • incumbent (default,) – configurations to be compared

  • rh (RunHistory) – runhistory to use for cost-estimations

  • test (train,) – lists with corresponding instances

  • cutoff (Union[None, int]) – cutoff for target algorithms, if set

  • output_dir (str) – directory to save plots in

classmethod check_for_bokeh(d)

Check if there is bokeh-plots in the output of this analyzer by checking the result-dictionary for the bokeh keyword.

get_html(d=None, tooltip=None) → Tuple[str, str]

General reports in html-format, to be easily integrated in html-code. WORKS ALSO FOR BOKEH-OUTPUT.

Parameters

  • d (Dictionary) – a dictionary that will be later turned into a website

  • tooltip (string) – tooltip to be displayed in report. optional, will overwrite the docstrings that are used by default.

Returns

script, div – header and body part of html-code

Return type

str, str

get_jupyter()

Depending on analysis, this creates jupyter-notebook compatible output.

get_name()[source]
plot_bokeh()

This function should recreate the bokeh-plot from scratch with as little overhead as possible. This is needed to show the bokeh plot in jupyter AND save it to the webpage. The bokeh plot needs to be recreated to be displayed in different outputs for reasons beyond out control. So save all analysis results in the class and simply redo the plotting with this function. This function needs to be called if bokeh-plots are to be displayed in notebook AND saved to html-result.