Depicts cost distributions over the set of instances. Since these are empirical distributions, the plots show step functions. These plots provide insights into how well configurations perform up to a certain threshold. For runtime scenarios this shows the probability of solving all instances from the set in a given timeframe. On the left side the training-data is scattered, on the right side the test-data is scattered.
Plot the cumulated distribution functions for given configurations, plots will share y-axis and if desired x-axis. Saves plot to file.
_plot_ecdf(default: ConfigSpace.configuration_space.Configuration, incumbent: ConfigSpace.configuration_space.Configuration, rh: smac.runhistory.runhistory.RunHistory, train: List[str], test: List[str], cutoff, output_dir: str)¶
incumbent (default,) – configurations to be compared
rh (RunHistory) – runhistory to use for cost-estimations
test (train,) – lists with corresponding instances
cutoff (Union[None, int]) – cutoff for target algorithms, if set
output_dir (str) – directory to save plots in
get_html(d=None, tooltip=None) → Tuple[str, str]¶
General reports in html-format, to be easily integrated in html-code. ALSO FOR BOKEH-OUTPUT.
d (Dictionary) – a dictionary that will be later turned into a website
script, div – header and body part of html-code
- Return type
Depending on analysis, this creates jupyter-notebook compatible output.
This function needs to be called if bokeh-plots are to be displayed in notebook AND saved to webpage.