cave.reader.configurator_run module

class cave.reader.configurator_run.ConfiguratorRun(folder: str, ta_exec_dir: str, file_format: str = 'SMAC3', validation_format: str = 'NONE')[source]

Bases: smac.facade.smac_facade.SMAC

ConfiguratorRuns load and maintain information about individual configurator runs. There are three supported formats: SMAC3, SMAC2 and CSV This class is responsible for providing a scenario, a runhistory and a trajectory and handling original/validated data appropriately.

Initialize scenario, runhistory and incumbent from folder, execute init-method of SMAC facade (so you could simply use SMAC-instances instead)

  • folder (string) – output-dir of this run
  • ta_exec_dir (string) – if the execution directory for the SMAC-run differs from the cwd, there might be problems loading instance-, feature- or PCS-files in the scenario-object. since instance- and PCS-files are necessary, specify the path to the execution-dir of SMAC here
  • file_format (string) – from [SMAC2, SMAC3, BOHB, CSV]
  • validation_format (string) – from [SMAC2, SMAC3, CSV, NONE], in which format to look for validated data

Simple interface to obtain all data in runhistory in X, y format.

Uses smac.runhistory.runhistory2epm.AbstractRunHistory2EPM.get_X_y().

  • X (numpy.ndarray) – matrix of all configurations (+ instance features)
  • y (numpy.ndarray) – vector of cost values; can include censored runs
  • cen (numpy.ndarray) – vector of bools indicating whether the y-value is censored
Returns the runhistory (i.e., all evaluated configurations and
the results).
Return type:smac.runhistory.runhistory.RunHistory

Returns target algorithm evaluator (TAE) object which can run the target algorithm given a configuration

Return type:smac.tae.execute_ta_run.ExecuteTARun

Returns the trajectory (i.e., all incumbent configurations over time).

Return type:List of TrajEntry

Optimizes the algorithm provided in scenario (given in constructor)

Returns:incumbent – Best found configuration
Return type:Configuration
validate(config_mode: Union[List[ConfigSpace.configuration_space.Configuration], numpy.ndarray, str] = 'inc', instance_mode: Union[List[str], str] = 'train+test', repetitions: int = 1, use_epm: bool = False, n_jobs: int = -1, backend: str = 'threading')

Create validator-object and run validation, using scenario-information, runhistory from smbo and tae_runner from intensify

  • config_mode (str or list<Configuration>) – string or directly a list of Configuration str from [def, inc, def+inc, wallclock_time, cpu_time, all] time evaluates at cpu- or wallclock-timesteps of: [max_time/2^0, max_time/2^1, max_time/2^3, …, default] with max_time being the highest recorded time
  • instance_mode (string) – what instances to use for validation, from [train, test, train+test]
  • repetitions (int) – number of repetitions in nondeterministic algorithms (in deterministic will be fixed to 1)
  • use_epm (bool) – whether to use an EPM instead of evaluating all runs with the TAE
  • n_jobs (int) – number of parallel processes used by joblib
  • backend (string) – what backend to be used by joblib

runhistory – runhistory containing all specified runs

Return type: