cave.reader.configurator_run module

class cave.reader.configurator_run.ConfiguratorRun(folder: str, ta_exec_dir: str, file_format: str = 'SMAC3', validation_format: str = 'NONE')[source]

Bases: smac.facade.smac_facade.SMAC

ConfiguratorRuns load and maintain information about individual configurator runs. There are three supported formats: SMAC3, SMAC2 and CSV This class is responsible for providing a scenario, a runhistory and a trajectory and handling original/validated data appropriately.

Initialize scenario, runhistory and incumbent from folder, execute init-method of SMAC facade (so you could simply use SMAC-instances instead)

Parameters
  • folder (string) – output-dir of this run

  • ta_exec_dir (string) – if the execution directory for the SMAC-run differs from the cwd, there might be problems loading instance-, feature- or PCS-files in the scenario-object. since instance- and PCS-files are necessary, specify the path to the execution-dir of SMAC here

  • file_format (string) – from [SMAC2, SMAC3, BOHB, CSV]

  • validation_format (string) – from [SMAC2, SMAC3, CSV, NONE], in which format to look for validated data

get_X_y()

Simple interface to obtain all data in runhistory in X, y format.

Uses smac.runhistory.runhistory2epm.AbstractRunHistory2EPM.get_X_y().

Returns

  • X (numpy.ndarray) – matrix of all configurations (+ instance features)

  • y (numpy.ndarray) – vector of cost values; can include censored runs

  • cen (numpy.ndarray) – vector of bools indicating whether the y-value is censored

get_incumbent()[source]
get_runhistory()
Returns the runhistory (i.e., all evaluated configurations and

the results).

Returns

Runhistory

Return type

smac.runhistory.runhistory.RunHistory

get_tae_runner()

Returns target algorithm evaluator (TAE) object which can run the target algorithm given a configuration

Returns

TAE

Return type

smac.tae.execute_ta_run.ExecuteTARun

get_trajectory()

Returns the trajectory (i.e., all incumbent configurations over time).

Returns

Trajectory

Return type

List of TrajEntry

optimize()

Optimizes the algorithm provided in scenario (given in constructor)

Returns

incumbent – Best found configuration

Return type

Configuration

validate(config_mode: Union[List[ConfigSpace.configuration_space.Configuration], numpy.ndarray, str] = 'inc', instance_mode: Union[List[str], str] = 'train+test', repetitions: int = 1, use_epm: bool = False, n_jobs: int = -1, backend: str = 'threading')

Create validator-object and run validation, using scenario-information, runhistory from smbo and tae_runner from intensify

Parameters
  • config_mode (str or list<Configuration>) – string or directly a list of Configuration str from [def, inc, def+inc, wallclock_time, cpu_time, all] time evaluates at cpu- or wallclock-timesteps of: [max_time/2^0, max_time/2^1, max_time/2^3, …, default] with max_time being the highest recorded time

  • instance_mode (string) – what instances to use for validation, from [train, test, train+test]

  • repetitions (int) – number of repetitions in nondeterministic algorithms (in deterministic will be fixed to 1)

  • use_epm (bool) – whether to use an EPM instead of evaluating all runs with the TAE

  • n_jobs (int) – number of parallel processes used by joblib

  • backend (string) – what backend to be used by joblib

Returns

runhistory – runhistory containing all specified runs

Return type

RunHistory