cave.analyzer.cost_over_time module

class cave.analyzer.cost_over_time.CostOverTime(scenario, output_dir, rh: smac.runhistory.runhistory.RunHistory, runs: List[cave.reader.configurator_run.ConfiguratorRun], block_epm: bool = False, bohb_results=None, average_over_runs: bool = True, output_fn: str = 'performance_over_time.png', validator: Union[None, smac.utils.validate.Validator] = None, cot_inc_traj='racing')[source]

Bases: cave.analyzer.base_analyzer.BaseAnalyzer

Plot performance over time, using all trajectory entries where max_time = max(wallclock_limit, the highest recorded time)

Parameters
  • scenario (smac.scenario.scenario.Scenario) – scenario object with necessary information

  • output_dir (str) – output-directory for smac-object

  • rh (smac.runhistory.runhistory.RunHistory) – runhistory to use

  • runs (List[ConfiguratorRun]) – list of configurator-runs

  • block_epm (bool) – if block_epm, only use given runs to estimate cost

  • average_over_runs (bool) – if True, average over plots. if False, all runs are treated individually with checkboxes

  • output_fn (str) – path to output-png for this analysis

  • validator (Validator or None) – if given, use this epm to estimate costs for the individual incumbents (EPM)

  • cot_inc_traj (str) – from [‘racing’, ‘minimum’, ‘prefer_higher_budget’], defines incumbent trajectory from hpbandster result

get_html(d=None, tooltip=None)[source]

General reports in html-format, to be easily integrated in html-code. ALSO FOR BOKEH-OUTPUT.

Returns

script, div – header and body part of html-code

Return type

str, str

get_jupyter()[source]

Depending on analysis, this creates jupyter-notebook compatible output.

get_plots()[source]
get_static_plots() → List[str]

Returns plot-paths, if any are available

Returns

plot_paths – returns list of strings

Return type

List[str]

get_table()

Get table, if available

plot()[source]

Plot performance over time, using all trajectory entries. max_time denotes max(wallclock_limit, highest recorded time).

class cave.analyzer.cost_over_time.Line(name, time, mean, upper, lower, config)

Bases: tuple

Create new instance of Line(name, time, mean, upper, lower, config)

property config

Alias for field number 5

count(value) → integer -- return number of occurrences of value
index(value[, start[, stop]]) → integer -- return first index of value.

Raises ValueError if the value is not present.

property lower

Alias for field number 4

property mean

Alias for field number 2

property name

Alias for field number 0

property time

Alias for field number 1

property upper

Alias for field number 3