Convert hpbandster-results into smac-format, aggregating parallel runs along the budgets, so it is treated as one run with the same budgets. Throws ValueError when budgets of individual runs dont match.
folders (List[str]) – list of runs to consider
output_dir (str) – path to CAVE’s output-directory
result (hpbandster.core.result) – BOHB-result in original format
paths (List[str]) – paths to converted data
budgets (List[int]) – budgets, corresponding to paths
Returns the best configurations over time
budget (string) – TODO
result (Result) – result object with runs
dictionary with all the config IDs, the times the runs finished, their respective budgets, and corresponding losses
- Return type
get_trajectory(folder2result, output_path, scenario, rh, budget=None)¶
If budget is specified, get trajectory for only that budget. Else use hpbandster’s averaging. If multiple results are specified, sort by times_finished and only add to combined trajectory if loss is better
hpbandster2smac(folder2result, cs: ConfigSpace.configuration_space.ConfigurationSpace, backup_cs, output_dir: str)¶
Reading hpbandster-result-object and creating RunHistory and trajectory… treats each budget as an individual ‘smac’-run, creates an output-directory with subdirectories for each budget.
folder2result (Dict(str : hpbandster.core.result.Result)) – folder mapping to bohb’s result-objects
cs (ConfigurationSpace) – the configuration space
backup_cs (List[ConfigurationSpace]) – if loading a configuration fails, try configspaces from this list until succeed
output_dir (str) – the output-dir to save the smac-runs to
Will try to load the configspace. If it’s a pcs-file, backup_cs will be a list containing all possible combinations of interpretation for Categoricals. If this issue will be fixed, we can drop this procedure.