cave.analyzer.cost_over_time module

class cave.analyzer.cost_over_time.CostOverTime(runscontainer: cave.reader.runs_container.RunsContainer, incumbent_trajectory: str = None, average_over_runs: bool = None)[source]

Bases: cave.analyzer.base_analyzer.BaseAnalyzer

Depicts the average cost of the best so far found configuration (using all trajectory data) over the time spent by the configurator (including target algorithm runs and the overhead generated by the configurator) If the curve flattens out early, it indicates that too much time was spent for the configurator run; whereas a curve that is still improving at the end of the budget indicates that one should increase the configuration budget. The plotted standard deviation gives the uncertainty over multiple configurator runs.

Plot performance over time, using all trajectory entries where max_time = max(wallclock_limit, the highest recorded time)

_get_all_runs(validator, runs, rh)[source]

get a list of Line-objects

_get_avg(validator, runs, rh)[source]
_get_bohb_line(validator, runs, rh, budget=None)[source]
_get_mean_var_time(validator, traj, use_epm, rh)[source]
Parameters
  • validator (Validator) – validator (smac-based)

  • traj (List[Configuraton]) – trajectory to set in validator

  • use_epm (bool) – validated or not (no need to use epm if validated)

  • rh (RunHistory) –

    ??

Returns

  • mean, var

  • times (List[float]) – times to plot (x-values)

  • configs

classmethod check_for_bokeh(d)
get_html(d=None, tooltip=None)[source]

General reports in html-format, to be easily integrated in html-code. ALSO FOR BOKEH-OUTPUT.

Parameters

d (Dictionary) – a dictionary that will be later turned into a website

Returns

script, div – header and body part of html-code

Return type

str, str

get_jupyter()[source]

Depending on analysis, this creates jupyter-notebook compatible output.

get_name()[source]
get_plots()[source]
plot()[source]

Plot performance over time, using all trajectory entries. max_time denotes max(wallclock_limit, highest recorded time).

plot_bokeh()

This function needs to be called if bokeh-plots are to be displayed in notebook AND saved to webpage.

class cave.analyzer.cost_over_time.Line(name, time, mean, upper, lower, config)

Bases: tuple

Create new instance of Line(name, time, mean, upper, lower, config)

_asdict()

Return a new OrderedDict which maps field names to their values.

_fields = ('name', 'time', 'mean', 'upper', 'lower', 'config')
classmethod _make(iterable, new=<built-in method __new__ of type object>, len=<built-in function len>)

Make a new Line object from a sequence or iterable

_replace(**kwds)

Return a new Line object replacing specified fields with new values

_source = "from builtins import property as _property, tuple as _tuple\nfrom operator import itemgetter as _itemgetter\nfrom collections import OrderedDict\n\nclass Line(tuple):\n 'Line(name, time, mean, upper, lower, config)'\n\n __slots__ = ()\n\n _fields = ('name', 'time', 'mean', 'upper', 'lower', 'config')\n\n def __new__(_cls, name, time, mean, upper, lower, config):\n 'Create new instance of Line(name, time, mean, upper, lower, config)'\n return _tuple.__new__(_cls, (name, time, mean, upper, lower, config))\n\n @classmethod\n def _make(cls, iterable, new=tuple.__new__, len=len):\n 'Make a new Line object from a sequence or iterable'\n result = new(cls, iterable)\n if len(result) != 6:\n raise TypeError('Expected 6 arguments, got %d' % len(result))\n return result\n\n def _replace(_self, **kwds):\n 'Return a new Line object replacing specified fields with new values'\n result = _self._make(map(kwds.pop, ('name', 'time', 'mean', 'upper', 'lower', 'config'), _self))\n if kwds:\n raise ValueError('Got unexpected field names: %r' % list(kwds))\n return result\n\n def __repr__(self):\n 'Return a nicely formatted representation string'\n return self.__class__.__name__ + '(name=%r, time=%r, mean=%r, upper=%r, lower=%r, config=%r)' % self\n\n def _asdict(self):\n 'Return a new OrderedDict which maps field names to their values.'\n return OrderedDict(zip(self._fields, self))\n\n def __getnewargs__(self):\n 'Return self as a plain tuple. Used by copy and pickle.'\n return tuple(self)\n\n name = _property(_itemgetter(0), doc='Alias for field number 0')\n\n time = _property(_itemgetter(1), doc='Alias for field number 1')\n\n mean = _property(_itemgetter(2), doc='Alias for field number 2')\n\n upper = _property(_itemgetter(3), doc='Alias for field number 3')\n\n lower = _property(_itemgetter(4), doc='Alias for field number 4')\n\n config = _property(_itemgetter(5), doc='Alias for field number 5')\n\n"
property config

Alias for field number 5

count(value) → integer -- return number of occurrences of value
index(value[, start[, stop]]) → integer -- return first index of value.

Raises ValueError if the value is not present.

property lower

Alias for field number 4

property mean

Alias for field number 2

property name

Alias for field number 0

property time

Alias for field number 1

property upper

Alias for field number 3