Note

Go to the end to download the full example code. or to run this example in your browser via Binder

Support Vector Machine with Cross-Validation

An example of optimizing a simple support vector machine on the IRIS dataset. We use the hyperparameter optimization facade, which uses a random forest as its surrogate model. It is able to scale to higher evaluation budgets and a higher number of dimensions. Also, you can use mixed data types as well as conditional hyperparameters.

[INFO][abstract_initial_design.py:147] Using 5 initial design configurations and 0 additional configurations.
/home/runner/work/SMAC3/SMAC3/examples/1_basics/2_svm_cv.py:57: DeprecationWarning: Please use `dict(config)` instead of `config.get_dictionary()` or use it as a dictionary directly if needed.
  config_dict = config.get_dictionary()
[INFO][abstract_intensifier.py:516] Added config 16b2dc as new incumbent because there are no incumbents yet.
/home/runner/work/SMAC3/SMAC3/examples/1_basics/2_svm_cv.py:57: DeprecationWarning: Please use `dict(config)` instead of `config.get_dictionary()` or use it as a dictionary directly if needed.
  config_dict = config.get_dictionary()
/home/runner/work/SMAC3/SMAC3/examples/1_basics/2_svm_cv.py:57: DeprecationWarning: Please use `dict(config)` instead of `config.get_dictionary()` or use it as a dictionary directly if needed.
  config_dict = config.get_dictionary()
[WARNING][abstract_runner.py:134] Target function returned infinity or nothing at all. Result is treated as CRASHED and cost is set to inf.
[WARNING][abstract_runner.py:140] Traceback: Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/smac/runner/target_function_runner.py", line 186, in run
    rval = self(config_copy, target_function, kwargs)
  File "/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/smac/runner/target_function_runner.py", line 259, in __call__
    return algorithm(config, **algorithm_kwargs)
  File "/home/runner/work/SMAC3/SMAC3/examples/1_basics/2_svm_cv.py", line 59, in train
    config_dict["gamma"] = config_dict["gamma_value"] if config_dict["gamma"] == "value" else "auto"
KeyError: 'gamma'


[WARNING][abstract_runner.py:134] Target function returned infinity or nothing at all. Result is treated as CRASHED and cost is set to inf.
[WARNING][abstract_runner.py:140] Traceback: Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/smac/runner/target_function_runner.py", line 186, in run
    rval = self(config_copy, target_function, kwargs)
  File "/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/smac/runner/target_function_runner.py", line 259, in __call__
    return algorithm(config, **algorithm_kwargs)
  File "/home/runner/work/SMAC3/SMAC3/examples/1_basics/2_svm_cv.py", line 59, in train
    config_dict["gamma"] = config_dict["gamma_value"] if config_dict["gamma"] == "value" else "auto"
KeyError: 'gamma'


/home/runner/work/SMAC3/SMAC3/examples/1_basics/2_svm_cv.py:57: DeprecationWarning: Please use `dict(config)` instead of `config.get_dictionary()` or use it as a dictionary directly if needed.
  config_dict = config.get_dictionary()
/home/runner/work/SMAC3/SMAC3/examples/1_basics/2_svm_cv.py:57: DeprecationWarning: Please use `dict(config)` instead of `config.get_dictionary()` or use it as a dictionary directly if needed.
  config_dict = config.get_dictionary()
/home/runner/work/SMAC3/SMAC3/examples/1_basics/2_svm_cv.py:57: DeprecationWarning: Please use `dict(config)` instead of `config.get_dictionary()` or use it as a dictionary directly if needed.
  config_dict = config.get_dictionary()
/home/runner/work/SMAC3/SMAC3/examples/1_basics/2_svm_cv.py:57: DeprecationWarning: Please use `dict(config)` instead of `config.get_dictionary()` or use it as a dictionary directly if needed.
  config_dict = config.get_dictionary()
/home/runner/work/SMAC3/SMAC3/examples/1_basics/2_svm_cv.py:57: DeprecationWarning: Please use `dict(config)` instead of `config.get_dictionary()` or use it as a dictionary directly if needed.
  config_dict = config.get_dictionary()
/home/runner/work/SMAC3/SMAC3/examples/1_basics/2_svm_cv.py:57: DeprecationWarning: Please use `dict(config)` instead of `config.get_dictionary()` or use it as a dictionary directly if needed.
  config_dict = config.get_dictionary()
[INFO][abstract_intensifier.py:595] Added config abecc0 and rejected config 16b2dc as incumbent because it is not better than the incumbents on 3 instances:
/home/runner/work/SMAC3/SMAC3/examples/1_basics/2_svm_cv.py:57: DeprecationWarning: Please use `dict(config)` instead of `config.get_dictionary()` or use it as a dictionary directly if needed.
  config_dict = config.get_dictionary()
/home/runner/work/SMAC3/SMAC3/examples/1_basics/2_svm_cv.py:57: DeprecationWarning: Please use `dict(config)` instead of `config.get_dictionary()` or use it as a dictionary directly if needed.
  config_dict = config.get_dictionary()
/home/runner/work/SMAC3/SMAC3/examples/1_basics/2_svm_cv.py:57: DeprecationWarning: Please use `dict(config)` instead of `config.get_dictionary()` or use it as a dictionary directly if needed.
  config_dict = config.get_dictionary()
/home/runner/work/SMAC3/SMAC3/examples/1_basics/2_svm_cv.py:57: DeprecationWarning: Please use `dict(config)` instead of `config.get_dictionary()` or use it as a dictionary directly if needed.
  config_dict = config.get_dictionary()
[WARNING][abstract_runner.py:134] Target function returned infinity or nothing at all. Result is treated as CRASHED and cost is set to inf.
[WARNING][abstract_runner.py:140] Traceback: Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/smac/runner/target_function_runner.py", line 186, in run
    rval = self(config_copy, target_function, kwargs)
  File "/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/smac/runner/target_function_runner.py", line 259, in __call__
    return algorithm(config, **algorithm_kwargs)
  File "/home/runner/work/SMAC3/SMAC3/examples/1_basics/2_svm_cv.py", line 59, in train
    config_dict["gamma"] = config_dict["gamma_value"] if config_dict["gamma"] == "value" else "auto"
KeyError: 'gamma'


[WARNING][abstract_runner.py:134] Target function returned infinity or nothing at all. Result is treated as CRASHED and cost is set to inf.
[WARNING][abstract_runner.py:140] Traceback: Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/smac/runner/target_function_runner.py", line 186, in run
    rval = self(config_copy, target_function, kwargs)
  File "/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/smac/runner/target_function_runner.py", line 259, in __call__
    return algorithm(config, **algorithm_kwargs)
  File "/home/runner/work/SMAC3/SMAC3/examples/1_basics/2_svm_cv.py", line 59, in train
    config_dict["gamma"] = config_dict["gamma_value"] if config_dict["gamma"] == "value" else "auto"
KeyError: 'gamma'


[WARNING][abstract_runner.py:134] Target function returned infinity or nothing at all. Result is treated as CRASHED and cost is set to inf.
[WARNING][abstract_runner.py:140] Traceback: Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/smac/runner/target_function_runner.py", line 186, in run
    rval = self(config_copy, target_function, kwargs)
  File "/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/smac/runner/target_function_runner.py", line 259, in __call__
    return algorithm(config, **algorithm_kwargs)
  File "/home/runner/work/SMAC3/SMAC3/examples/1_basics/2_svm_cv.py", line 59, in train
    config_dict["gamma"] = config_dict["gamma_value"] if config_dict["gamma"] == "value" else "auto"
KeyError: 'gamma'


/home/runner/work/SMAC3/SMAC3/examples/1_basics/2_svm_cv.py:57: DeprecationWarning: Please use `dict(config)` instead of `config.get_dictionary()` or use it as a dictionary directly if needed.
  config_dict = config.get_dictionary()
/home/runner/work/SMAC3/SMAC3/examples/1_basics/2_svm_cv.py:57: DeprecationWarning: Please use `dict(config)` instead of `config.get_dictionary()` or use it as a dictionary directly if needed.
  config_dict = config.get_dictionary()
/home/runner/work/SMAC3/SMAC3/examples/1_basics/2_svm_cv.py:57: DeprecationWarning: Please use `dict(config)` instead of `config.get_dictionary()` or use it as a dictionary directly if needed.
  config_dict = config.get_dictionary()
/home/runner/work/SMAC3/SMAC3/examples/1_basics/2_svm_cv.py:57: DeprecationWarning: Please use `dict(config)` instead of `config.get_dictionary()` or use it as a dictionary directly if needed.
  config_dict = config.get_dictionary()
/home/runner/work/SMAC3/SMAC3/examples/1_basics/2_svm_cv.py:57: DeprecationWarning: Please use `dict(config)` instead of `config.get_dictionary()` or use it as a dictionary directly if needed.
  config_dict = config.get_dictionary()
/home/runner/work/SMAC3/SMAC3/examples/1_basics/2_svm_cv.py:57: DeprecationWarning: Please use `dict(config)` instead of `config.get_dictionary()` or use it as a dictionary directly if needed.
  config_dict = config.get_dictionary()
[WARNING][abstract_runner.py:134] Target function returned infinity or nothing at all. Result is treated as CRASHED and cost is set to inf.
[WARNING][abstract_runner.py:140] Traceback: Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/smac/runner/target_function_runner.py", line 186, in run
    rval = self(config_copy, target_function, kwargs)
  File "/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/smac/runner/target_function_runner.py", line 259, in __call__
    return algorithm(config, **algorithm_kwargs)
  File "/home/runner/work/SMAC3/SMAC3/examples/1_basics/2_svm_cv.py", line 59, in train
    config_dict["gamma"] = config_dict["gamma_value"] if config_dict["gamma"] == "value" else "auto"
KeyError: 'gamma'


[WARNING][abstract_runner.py:134] Target function returned infinity or nothing at all. Result is treated as CRASHED and cost is set to inf.
[WARNING][abstract_runner.py:140] Traceback: Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/smac/runner/target_function_runner.py", line 186, in run
    rval = self(config_copy, target_function, kwargs)
  File "/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/smac/runner/target_function_runner.py", line 259, in __call__
    return algorithm(config, **algorithm_kwargs)
  File "/home/runner/work/SMAC3/SMAC3/examples/1_basics/2_svm_cv.py", line 59, in train
    config_dict["gamma"] = config_dict["gamma_value"] if config_dict["gamma"] == "value" else "auto"
KeyError: 'gamma'


[WARNING][abstract_runner.py:134] Target function returned infinity or nothing at all. Result is treated as CRASHED and cost is set to inf.
[WARNING][abstract_runner.py:140] Traceback: Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/smac/runner/target_function_runner.py", line 186, in run
    rval = self(config_copy, target_function, kwargs)
  File "/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/smac/runner/target_function_runner.py", line 259, in __call__
    return algorithm(config, **algorithm_kwargs)
  File "/home/runner/work/SMAC3/SMAC3/examples/1_basics/2_svm_cv.py", line 59, in train
    config_dict["gamma"] = config_dict["gamma_value"] if config_dict["gamma"] == "value" else "auto"
KeyError: 'gamma'


[WARNING][abstract_runner.py:134] Target function returned infinity or nothing at all. Result is treated as CRASHED and cost is set to inf.
[WARNING][abstract_runner.py:140] Traceback: Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/smac/runner/target_function_runner.py", line 186, in run
    rval = self(config_copy, target_function, kwargs)
  File "/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/smac/runner/target_function_runner.py", line 259, in __call__
    return algorithm(config, **algorithm_kwargs)
  File "/home/runner/work/SMAC3/SMAC3/examples/1_basics/2_svm_cv.py", line 59, in train
    config_dict["gamma"] = config_dict["gamma_value"] if config_dict["gamma"] == "value" else "auto"
KeyError: 'gamma'


[WARNING][abstract_runner.py:134] Target function returned infinity or nothing at all. Result is treated as CRASHED and cost is set to inf.
[WARNING][abstract_runner.py:140] Traceback: Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/smac/runner/target_function_runner.py", line 186, in run
    rval = self(config_copy, target_function, kwargs)
  File "/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/smac/runner/target_function_runner.py", line 259, in __call__
    return algorithm(config, **algorithm_kwargs)
  File "/home/runner/work/SMAC3/SMAC3/examples/1_basics/2_svm_cv.py", line 59, in train
    config_dict["gamma"] = config_dict["gamma_value"] if config_dict["gamma"] == "value" else "auto"
KeyError: 'gamma'


[WARNING][abstract_runner.py:134] Target function returned infinity or nothing at all. Result is treated as CRASHED and cost is set to inf.
[WARNING][abstract_runner.py:140] Traceback: Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/smac/runner/target_function_runner.py", line 186, in run
    rval = self(config_copy, target_function, kwargs)
  File "/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/smac/runner/target_function_runner.py", line 259, in __call__
    return algorithm(config, **algorithm_kwargs)
  File "/home/runner/work/SMAC3/SMAC3/examples/1_basics/2_svm_cv.py", line 59, in train
    config_dict["gamma"] = config_dict["gamma_value"] if config_dict["gamma"] == "value" else "auto"
KeyError: 'gamma'


/home/runner/work/SMAC3/SMAC3/examples/1_basics/2_svm_cv.py:57: DeprecationWarning: Please use `dict(config)` instead of `config.get_dictionary()` or use it as a dictionary directly if needed.
  config_dict = config.get_dictionary()
/home/runner/work/SMAC3/SMAC3/examples/1_basics/2_svm_cv.py:57: DeprecationWarning: Please use `dict(config)` instead of `config.get_dictionary()` or use it as a dictionary directly if needed.
  config_dict = config.get_dictionary()
/home/runner/work/SMAC3/SMAC3/examples/1_basics/2_svm_cv.py:57: DeprecationWarning: Please use `dict(config)` instead of `config.get_dictionary()` or use it as a dictionary directly if needed.
  config_dict = config.get_dictionary()
/home/runner/work/SMAC3/SMAC3/examples/1_basics/2_svm_cv.py:57: DeprecationWarning: Please use `dict(config)` instead of `config.get_dictionary()` or use it as a dictionary directly if needed.
  config_dict = config.get_dictionary()
/home/runner/work/SMAC3/SMAC3/examples/1_basics/2_svm_cv.py:57: DeprecationWarning: Please use `dict(config)` instead of `config.get_dictionary()` or use it as a dictionary directly if needed.
  config_dict = config.get_dictionary()
/home/runner/work/SMAC3/SMAC3/examples/1_basics/2_svm_cv.py:57: DeprecationWarning: Please use `dict(config)` instead of `config.get_dictionary()` or use it as a dictionary directly if needed.
  config_dict = config.get_dictionary()
/home/runner/work/SMAC3/SMAC3/examples/1_basics/2_svm_cv.py:57: DeprecationWarning: Please use `dict(config)` instead of `config.get_dictionary()` or use it as a dictionary directly if needed.
  config_dict = config.get_dictionary()
/home/runner/work/SMAC3/SMAC3/examples/1_basics/2_svm_cv.py:57: DeprecationWarning: Please use `dict(config)` instead of `config.get_dictionary()` or use it as a dictionary directly if needed.
  config_dict = config.get_dictionary()
/home/runner/work/SMAC3/SMAC3/examples/1_basics/2_svm_cv.py:57: DeprecationWarning: Please use `dict(config)` instead of `config.get_dictionary()` or use it as a dictionary directly if needed.
  config_dict = config.get_dictionary()
/home/runner/work/SMAC3/SMAC3/examples/1_basics/2_svm_cv.py:57: DeprecationWarning: Please use `dict(config)` instead of `config.get_dictionary()` or use it as a dictionary directly if needed.
  config_dict = config.get_dictionary()
/home/runner/work/SMAC3/SMAC3/examples/1_basics/2_svm_cv.py:57: DeprecationWarning: Please use `dict(config)` instead of `config.get_dictionary()` or use it as a dictionary directly if needed.
  config_dict = config.get_dictionary()
/home/runner/work/SMAC3/SMAC3/examples/1_basics/2_svm_cv.py:57: DeprecationWarning: Please use `dict(config)` instead of `config.get_dictionary()` or use it as a dictionary directly if needed.
  config_dict = config.get_dictionary()
/home/runner/work/SMAC3/SMAC3/examples/1_basics/2_svm_cv.py:57: DeprecationWarning: Please use `dict(config)` instead of `config.get_dictionary()` or use it as a dictionary directly if needed.
  config_dict = config.get_dictionary()
/home/runner/work/SMAC3/SMAC3/examples/1_basics/2_svm_cv.py:57: DeprecationWarning: Please use `dict(config)` instead of `config.get_dictionary()` or use it as a dictionary directly if needed.
  config_dict = config.get_dictionary()
/home/runner/work/SMAC3/SMAC3/examples/1_basics/2_svm_cv.py:57: DeprecationWarning: Please use `dict(config)` instead of `config.get_dictionary()` or use it as a dictionary directly if needed.
  config_dict = config.get_dictionary()
[WARNING][abstract_runner.py:134] Target function returned infinity or nothing at all. Result is treated as CRASHED and cost is set to inf.
[WARNING][abstract_runner.py:140] Traceback: Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/smac/runner/target_function_runner.py", line 186, in run
    rval = self(config_copy, target_function, kwargs)
  File "/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/smac/runner/target_function_runner.py", line 259, in __call__
    return algorithm(config, **algorithm_kwargs)
  File "/home/runner/work/SMAC3/SMAC3/examples/1_basics/2_svm_cv.py", line 59, in train
    config_dict["gamma"] = config_dict["gamma_value"] if config_dict["gamma"] == "value" else "auto"
KeyError: 'gamma'


[WARNING][abstract_runner.py:134] Target function returned infinity or nothing at all. Result is treated as CRASHED and cost is set to inf.
[WARNING][abstract_runner.py:140] Traceback: Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/smac/runner/target_function_runner.py", line 186, in run
    rval = self(config_copy, target_function, kwargs)
  File "/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/smac/runner/target_function_runner.py", line 259, in __call__
    return algorithm(config, **algorithm_kwargs)
  File "/home/runner/work/SMAC3/SMAC3/examples/1_basics/2_svm_cv.py", line 59, in train
    config_dict["gamma"] = config_dict["gamma_value"] if config_dict["gamma"] == "value" else "auto"
KeyError: 'gamma'


[WARNING][abstract_runner.py:134] Target function returned infinity or nothing at all. Result is treated as CRASHED and cost is set to inf.
[WARNING][abstract_runner.py:140] Traceback: Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/smac/runner/target_function_runner.py", line 186, in run
    rval = self(config_copy, target_function, kwargs)
  File "/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/smac/runner/target_function_runner.py", line 259, in __call__
    return algorithm(config, **algorithm_kwargs)
  File "/home/runner/work/SMAC3/SMAC3/examples/1_basics/2_svm_cv.py", line 59, in train
    config_dict["gamma"] = config_dict["gamma_value"] if config_dict["gamma"] == "value" else "auto"
KeyError: 'gamma'


/home/runner/work/SMAC3/SMAC3/examples/1_basics/2_svm_cv.py:57: DeprecationWarning: Please use `dict(config)` instead of `config.get_dictionary()` or use it as a dictionary directly if needed.
  config_dict = config.get_dictionary()
/home/runner/work/SMAC3/SMAC3/examples/1_basics/2_svm_cv.py:57: DeprecationWarning: Please use `dict(config)` instead of `config.get_dictionary()` or use it as a dictionary directly if needed.
  config_dict = config.get_dictionary()
[INFO][abstract_intensifier.py:595] Added config e50e31 and rejected config abecc0 as incumbent because it is not better than the incumbents on 3 instances:
[INFO][smbo.py:320] Finished 50 trials.
[INFO][smbo.py:328] Configuration budget is exhausted:
[INFO][smbo.py:329] --- Remaining wallclock time: inf
[INFO][smbo.py:330] --- Remaining cpu time: inf
[INFO][smbo.py:331] --- Remaining trials: 0
/home/runner/work/SMAC3/SMAC3/examples/1_basics/2_svm_cv.py:57: DeprecationWarning: Please use `dict(config)` instead of `config.get_dictionary()` or use it as a dictionary directly if needed.
  config_dict = config.get_dictionary()
/home/runner/work/SMAC3/SMAC3/examples/1_basics/2_svm_cv.py:57: DeprecationWarning: Please use `dict(config)` instead of `config.get_dictionary()` or use it as a dictionary directly if needed.
  config_dict = config.get_dictionary()
/home/runner/work/SMAC3/SMAC3/examples/1_basics/2_svm_cv.py:57: DeprecationWarning: Please use `dict(config)` instead of `config.get_dictionary()` or use it as a dictionary directly if needed.
  config_dict = config.get_dictionary()
Default cost: 0.03333333333333344
/home/runner/work/SMAC3/SMAC3/examples/1_basics/2_svm_cv.py:57: DeprecationWarning: Please use `dict(config)` instead of `config.get_dictionary()` or use it as a dictionary directly if needed.
  config_dict = config.get_dictionary()
/home/runner/work/SMAC3/SMAC3/examples/1_basics/2_svm_cv.py:57: DeprecationWarning: Please use `dict(config)` instead of `config.get_dictionary()` or use it as a dictionary directly if needed.
  config_dict = config.get_dictionary()
/home/runner/work/SMAC3/SMAC3/examples/1_basics/2_svm_cv.py:57: DeprecationWarning: Please use `dict(config)` instead of `config.get_dictionary()` or use it as a dictionary directly if needed.
  config_dict = config.get_dictionary()
Incumbent cost: 0.013333333333333308


import numpy as np
from ConfigSpace import Categorical, Configuration, ConfigurationSpace, Float, Integer
from ConfigSpace.conditions import InCondition
from sklearn import datasets, svm
from sklearn.model_selection import cross_val_score

from smac import HyperparameterOptimizationFacade, Scenario

__copyright__ = "Copyright 2021, AutoML.org Freiburg-Hannover"
__license__ = "3-clause BSD"


# We load the iris-dataset (a widely used benchmark)
iris = datasets.load_iris()


class SVM:
    @property
    def configspace(self) -> ConfigurationSpace:
        # Build Configuration Space which defines all parameters and their ranges
        cs = ConfigurationSpace(seed=0)

        # First we create our hyperparameters
        kernel = Categorical("kernel", ["linear", "poly", "rbf", "sigmoid"], default="poly")
        C = Float("C", (0.001, 1000.0), default=1.0, log=True)
        shrinking = Categorical("shrinking", [True, False], default=True)
        degree = Integer("degree", (1, 5), default=3)
        coef = Float("coef0", (0.0, 10.0), default=0.0)
        gamma = Categorical("gamma", ["auto", "value"], default="auto")
        gamma_value = Float("gamma_value", (0.0001, 8.0), default=1.0, log=True)

        # Then we create dependencies
        use_degree = InCondition(child=degree, parent=kernel, values=["poly"])
        use_coef = InCondition(child=coef, parent=kernel, values=["poly", "sigmoid"])
        use_gamma = InCondition(child=gamma, parent=kernel, values=["rbf", "poly", "sigmoid"])
        use_gamma_value = InCondition(child=gamma_value, parent=gamma, values=["value"])

        # Add hyperparameters and conditions to our configspace
        cs.add([kernel, C, shrinking, degree, coef, gamma, gamma_value])
        cs.add([use_degree, use_coef, use_gamma, use_gamma_value])

        return cs

    def train(self, config: Configuration, seed: int = 0) -> float:
        """Creates a SVM based on a configuration and evaluates it on the
        iris-dataset using cross-validation."""
        config_dict = config.get_dictionary()
        if "gamma" in config:
            config_dict["gamma"] = config_dict["gamma_value"] if config_dict["gamma"] == "value" else "auto"
            config_dict.pop("gamma_value", None)

        classifier = svm.SVC(**config_dict, random_state=seed)
        scores = cross_val_score(classifier, iris.data, iris.target, cv=5)
        cost = 1 - np.mean(scores)

        return cost


if __name__ == "__main__":
    classifier = SVM()

    # Next, we create an object, holding general information about the run
    scenario = Scenario(
        classifier.configspace,
        n_trials=50,  # We want to run max 50 trials (combination of config and seed)
    )

    # We want to run the facade's default initial design, but we want to change the number
    # of initial configs to 5.
    initial_design = HyperparameterOptimizationFacade.get_initial_design(scenario, n_configs=5)

    # Now we use SMAC to find the best hyperparameters
    smac = HyperparameterOptimizationFacade(
        scenario,
        classifier.train,
        initial_design=initial_design,
        overwrite=True,  # If the run exists, we overwrite it; alternatively, we can continue from last state
    )

    incumbent = smac.optimize()

    # Get cost of default configuration
    default_cost = smac.validate(classifier.configspace.get_default_configuration())
    print(f"Default cost: {default_cost}")

    # Let's calculate the cost of the incumbent
    incumbent_cost = smac.validate(incumbent)
    print(f"Incumbent cost: {incumbent_cost}")

Total running time of the script: (0 minutes 2.978 seconds)

previous

Quadratic Function

next

Ask-and-Tell