cave.plot.algorithm_footprint module

class cave.plot.algorithm_footprint.AlgorithmFootprintPlotter(rh: smac.runhistory.runhistory.RunHistory, train_inst_feat, test_inst_feat, algorithms, cutoff=inf, output_dir=None, rng=None)[source]

Bases: object

Class that provides the algorithmic footprints after “Measuring algorithm footprints in instance space” (Kate Smith-Miles, Kate Smith-Miles)

General procedure:

  • label for each algorithm each instance with the same metric

  • map the instances onto a plane using pca

NOTE: The terms ‘algorithm’ and ‘config/configuration’ will be used synonymous throughout the class.

Parameters

  • rh (RunHistory) – runhistory to take cost from

  • test_inst_feat (train_inst_feat,) – instances names mapped to features

  • algorithms (List[Tuple(Configuration, str)]) – list with configs and descriptive names

  • cutoff (int) – cutoff (if available)

  • output_dir (str) – output directory

_get_cost(algorithm, instance=None)[source]

Return cost according to (possibly EPM-)validated runhistory.

Parameters

  • algorithm (Configuration) – config

  • instance (str) – instance name

_get_good_bad(conf, insts=[])[source]

Creates a list of indices for good and bad instances for a configuration.

Parameters

  • conf (Configuration) – configuration for which to plot good vs bad

  • insts (List[str]) – instances to be plotted

Returns

outpath – output path

Return type

str

_label_instances(epsilon=0.95)[source]

Returns dictionary with a label for each instance.

Returns

labels – maps instance-names (strings) to label (floats)

Return type

Dict[str:float]

_reduce_dim(feature_array, n=2)[source]

Expects feature-array (not dict!), performs a PCA

Parameters

  • feature_array (np.array) – array containing features in order of self.inst_names

  • n (int) – target dimension for pca, 2 or 3

Returns

feature_array_nd – array with pca’ed features (n-dimensional)

Return type

np.array

footprint(a, density_threshold, purity_threshold)[source]

Calculating the footprint within a portfolio using convex hulls that depend on density and purity thresholds. (algorithm 1 in Smith-Miles 2014)

We use 3 ways to refer to an instance here: name: the name (unique!) of the instance feat2d: the position as np.array tup: the tuple-version of feat2d (hashable…)

Parameters

  • a (Configuration) – configuration to get footprint of

  • density_threshold (float) – minimum density that regions must show to be merged

  • purity_threshold (float) – minimum purity (percentage of good instance) that regions must show to be merged

Returns

footprint – the size of all resulting convex hulls

Return type

float

get_clusters(features_2d)[source]

Mapping instances to clusters, using silhouette-scores to determine number of cluster.

Returns

paths – paths to plots

Return type

List[str]

plot3d()[source]

Plot 3d-version of the algorithm footprint from four different angles.

plot_interactive_footprint()[source]

Use bokeh to create an interactive algorithm footprint with zoom and hover tooltips. Should avoid problems with overplotting (since we can zoom) and provide better information about instances.

plot_points_per_cluster()[source]

Plot good versus bad for passed config per cluster.

Parameters

  • conf (Configuration) – configuration for which to plot good vs bad

  • out (str) – output path

Returns

outpaths – output paths per cluster

Return type

List[str]