Note
Click here to download the full example code or to run this example in your browser via Binder
Random Search¶
A crucial feature of auto-sklearn is automatically optimizing the hyperparameters through SMAC, introduced here. Additionally, it is possible to use random search instead of SMAC, as demonstrated in the example below. Furthermore, the example also demonstrates how to use Random Online Aggressive Racing (ROAR) as yet another alternative optimizatino strategy. Both examples are intended to show how the optimization strategy in auto-sklearn can be adapted.
from pprint import pprint
import sklearn.model_selection
import sklearn.datasets
import sklearn.metrics
from smac.facade.roar_facade import ROAR
from smac.scenario.scenario import Scenario
import autosklearn.classification
Data Loading¶
X, y = sklearn.datasets.load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = sklearn.model_selection.train_test_split(
X, y, random_state=1
)
Fit a classifier using ROAR¶
def get_roar_object_callback(
scenario_dict,
seed,
ta,
ta_kwargs,
metalearning_configurations,
n_jobs,
dask_client,
multi_objective_algorithm, # This argument will be ignored as ROAR does not yet support multi-objective optimization
multi_objective_kwargs,
):
"""Random online adaptive racing."""
if n_jobs > 1 or (dask_client and len(dask_client.nthreads()) > 1):
raise ValueError(
"Please make sure to guard the code invoking Auto-sklearn by "
"`if __name__ == '__main__'` and remove this exception."
)
scenario = Scenario(scenario_dict)
return ROAR(
scenario=scenario,
rng=seed,
tae_runner=ta,
tae_runner_kwargs=ta_kwargs,
run_id=seed,
dask_client=dask_client,
n_jobs=n_jobs,
)
automl = autosklearn.classification.AutoSklearnClassifier(
time_left_for_this_task=60,
per_run_time_limit=15,
tmp_folder="/tmp/autosklearn_random_search_example_tmp",
initial_configurations_via_metalearning=0,
# The callback to get the SMAC object
get_smac_object_callback=get_roar_object_callback,
)
automl.fit(X_train, y_train, dataset_name="breast_cancer")
print("#" * 80)
print("Results for ROAR.")
# Print the final ensemble constructed by auto-sklearn via ROAR.
pprint(automl.show_models(), indent=4)
predictions = automl.predict(X_test)
# Print statistics about the auto-sklearn run such as number of
# iterations, number of models failed with a time out.
print(automl.sprint_statistics())
print("Accuracy score", sklearn.metrics.accuracy_score(y_test, predictions))
Fitting to the training data: 0%| | 0/60 [00:00<?, ?it/s, The total time budget for this task is 0:01:00]
Fitting to the training data: 2%|1 | 1/60 [00:01<00:59, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 3%|3 | 2/60 [00:02<00:58, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 5%|5 | 3/60 [00:03<00:57, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 7%|6 | 4/60 [00:04<00:56, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 8%|8 | 5/60 [00:05<00:55, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 10%|# | 6/60 [00:06<00:54, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 12%|#1 | 7/60 [00:07<00:53, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 13%|#3 | 8/60 [00:08<00:52, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 15%|#5 | 9/60 [00:09<00:51, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 17%|#6 | 10/60 [00:10<00:50, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 18%|#8 | 11/60 [00:11<00:49, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 20%|## | 12/60 [00:12<00:48, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 22%|##1 | 13/60 [00:13<00:47, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 23%|##3 | 14/60 [00:14<00:46, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 25%|##5 | 15/60 [00:15<00:45, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 27%|##6 | 16/60 [00:16<00:44, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 28%|##8 | 17/60 [00:17<00:43, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 30%|### | 18/60 [00:18<00:42, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 32%|###1 | 19/60 [00:19<00:41, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 33%|###3 | 20/60 [00:20<00:40, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 35%|###5 | 21/60 [00:21<00:39, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 37%|###6 | 22/60 [00:22<00:38, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 38%|###8 | 23/60 [00:23<00:37, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 40%|#### | 24/60 [00:24<00:36, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 42%|####1 | 25/60 [00:25<00:35, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 43%|####3 | 26/60 [00:26<00:34, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 45%|####5 | 27/60 [00:27<00:33, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 47%|####6 | 28/60 [00:28<00:32, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 48%|####8 | 29/60 [00:29<00:31, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 50%|##### | 30/60 [00:30<00:30, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 52%|#####1 | 31/60 [00:31<00:29, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 53%|#####3 | 32/60 [00:32<00:28, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 55%|#####5 | 33/60 [00:33<00:27, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 57%|#####6 | 34/60 [00:34<00:26, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 58%|#####8 | 35/60 [00:35<00:25, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 60%|###### | 36/60 [00:36<00:24, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 62%|######1 | 37/60 [00:37<00:23, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 63%|######3 | 38/60 [00:38<00:22, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 65%|######5 | 39/60 [00:39<00:21, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 67%|######6 | 40/60 [00:40<00:20, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 68%|######8 | 41/60 [00:41<00:19, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 70%|####### | 42/60 [00:42<00:18, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 72%|#######1 | 43/60 [00:43<00:17, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 73%|#######3 | 44/60 [00:44<00:16, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 75%|#######5 | 45/60 [00:45<00:15, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 77%|#######6 | 46/60 [00:46<00:14, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 78%|#######8 | 47/60 [00:47<00:13, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 80%|######## | 48/60 [00:48<00:12, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 82%|########1 | 49/60 [00:49<00:11, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 83%|########3 | 50/60 [00:50<00:10, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 85%|########5 | 51/60 [00:51<00:09, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 100%|##########| 60/60 [00:51<00:00, 1.17it/s, The total time budget for this task is 0:01:00]
################################################################################
Results for ROAR.
{ 2: { 'balancing': Balancing(random_state=1),
'classifier': <autosklearn.pipeline.components.classification.ClassifierChoice object at 0x7f2afd3e1160>,
'cost': 0.028368794326241176,
'data_preprocessor': <autosklearn.pipeline.components.data_preprocessing.DataPreprocessorChoice object at 0x7f2af0a52df0>,
'ensemble_weight': 0.1,
'feature_preprocessor': <autosklearn.pipeline.components.feature_preprocessing.FeaturePreprocessorChoice object at 0x7f2afd3e1a60>,
'model_id': 2,
'rank': 3,
'sklearn_classifier': RandomForestClassifier(max_features=5, n_estimators=512, n_jobs=1,
random_state=1, warm_start=True)},
3: { 'balancing': Balancing(random_state=1),
'classifier': <autosklearn.pipeline.components.classification.ClassifierChoice object at 0x7f2afd3e1e80>,
'cost': 0.07092198581560283,
'data_preprocessor': <autosklearn.pipeline.components.data_preprocessing.DataPreprocessorChoice object at 0x7f2afd48d6a0>,
'ensemble_weight': 0.06,
'feature_preprocessor': <autosklearn.pipeline.components.feature_preprocessing.FeaturePreprocessorChoice object at 0x7f2afd3e12b0>,
'model_id': 3,
'rank': 11,
'sklearn_classifier': PassiveAggressiveClassifier(C=0.10318256510142626, average=True, max_iter=32,
random_state=1, tol=0.0013607858153657413,
warm_start=True)},
4: { 'balancing': Balancing(random_state=1, strategy='weighting'),
'classifier': <autosklearn.pipeline.components.classification.ClassifierChoice object at 0x7f2af50ed790>,
'cost': 0.08510638297872342,
'data_preprocessor': <autosklearn.pipeline.components.data_preprocessing.DataPreprocessorChoice object at 0x7f2af6c35d00>,
'ensemble_weight': 0.08,
'feature_preprocessor': <autosklearn.pipeline.components.feature_preprocessing.FeaturePreprocessorChoice object at 0x7f2af51a0220>,
'model_id': 4,
'rank': 16,
'sklearn_classifier': PassiveAggressiveClassifier(C=0.000505280701827856, max_iter=32, random_state=1,
tol=0.06421873693006744, warm_start=True)},
5: { 'balancing': Balancing(random_state=1, strategy='weighting'),
'classifier': <autosklearn.pipeline.components.classification.ClassifierChoice object at 0x7f2afcd45b20>,
'cost': 0.021276595744680882,
'data_preprocessor': <autosklearn.pipeline.components.data_preprocessing.DataPreprocessorChoice object at 0x7f2af42752b0>,
'ensemble_weight': 0.02,
'feature_preprocessor': <autosklearn.pipeline.components.feature_preprocessing.FeaturePreprocessorChoice object at 0x7f2afcd45df0>,
'model_id': 5,
'rank': 2,
'sklearn_classifier': HistGradientBoostingClassifier(early_stopping=True,
l2_regularization=1.4469619760002166e-09,
learning_rate=0.12309321429220763, max_iter=256,
max_leaf_nodes=10, min_samples_leaf=17,
n_iter_no_change=16, random_state=1,
validation_fraction=None, warm_start=True)},
6: { 'balancing': Balancing(random_state=1),
'classifier': <autosklearn.pipeline.components.classification.ClassifierChoice object at 0x7f2af4670b80>,
'cost': 0.028368794326241176,
'data_preprocessor': <autosklearn.pipeline.components.data_preprocessing.DataPreprocessorChoice object at 0x7f2aede90eb0>,
'ensemble_weight': 0.02,
'feature_preprocessor': <autosklearn.pipeline.components.feature_preprocessing.FeaturePreprocessorChoice object at 0x7f2af4670cd0>,
'model_id': 6,
'rank': 8,
'sklearn_classifier': LinearDiscriminantAnalysis(tol=0.08622878828999171)},
7: { 'balancing': Balancing(random_state=1),
'classifier': <autosklearn.pipeline.components.classification.ClassifierChoice object at 0x7f2af5208d90>,
'cost': 0.36879432624113473,
'data_preprocessor': <autosklearn.pipeline.components.data_preprocessing.DataPreprocessorChoice object at 0x7f2af703e160>,
'ensemble_weight': 0.04,
'feature_preprocessor': <autosklearn.pipeline.components.feature_preprocessing.FeaturePreprocessorChoice object at 0x7f2af6e91e50>,
'model_id': 7,
'rank': 26,
'sklearn_classifier': BernoulliNB(alpha=0.020898102999400082)},
8: { 'balancing': Balancing(random_state=1, strategy='weighting'),
'classifier': <autosklearn.pipeline.components.classification.ClassifierChoice object at 0x7f2af6d67310>,
'cost': 0.028368794326241176,
'data_preprocessor': <autosklearn.pipeline.components.data_preprocessing.DataPreprocessorChoice object at 0x7f2af4b59550>,
'ensemble_weight': 0.06,
'feature_preprocessor': <autosklearn.pipeline.components.feature_preprocessing.FeaturePreprocessorChoice object at 0x7f2af6d676a0>,
'model_id': 8,
'rank': 9,
'sklearn_classifier': PassiveAggressiveClassifier(C=5.4951544593926646e-05, average=True, max_iter=16,
random_state=1, tol=0.012344344043531127,
warm_start=True)},
10: { 'balancing': Balancing(random_state=1),
'classifier': <autosklearn.pipeline.components.classification.ClassifierChoice object at 0x7f2af09ef4f0>,
'cost': 0.16312056737588654,
'data_preprocessor': <autosklearn.pipeline.components.data_preprocessing.DataPreprocessorChoice object at 0x7f2af4eccdf0>,
'ensemble_weight': 0.02,
'feature_preprocessor': <autosklearn.pipeline.components.feature_preprocessing.FeaturePreprocessorChoice object at 0x7f2af09ef640>,
'model_id': 10,
'rank': 22,
'sklearn_classifier': ExtraTreesClassifier(criterion='entropy', max_features=2, min_samples_leaf=11,
min_samples_split=19, n_estimators=512, n_jobs=1,
random_state=1, warm_start=True)},
11: { 'balancing': Balancing(random_state=1, strategy='weighting'),
'classifier': <autosklearn.pipeline.components.classification.ClassifierChoice object at 0x7f2afd4f7130>,
'cost': 0.028368794326241176,
'data_preprocessor': <autosklearn.pipeline.components.data_preprocessing.DataPreprocessorChoice object at 0x7f2af03a1940>,
'ensemble_weight': 0.02,
'feature_preprocessor': <autosklearn.pipeline.components.feature_preprocessing.FeaturePreprocessorChoice object at 0x7f2af4b14ca0>,
'model_id': 11,
'rank': 6,
'sklearn_classifier': LinearDiscriminantAnalysis(shrinkage='auto', solver='lsqr',
tol=0.039339489121391846)},
12: { 'balancing': Balancing(random_state=1),
'classifier': <autosklearn.pipeline.components.classification.ClassifierChoice object at 0x7f2af47b96d0>,
'cost': 0.13475177304964536,
'data_preprocessor': <autosklearn.pipeline.components.data_preprocessing.DataPreprocessorChoice object at 0x7f2af70997c0>,
'ensemble_weight': 0.08,
'feature_preprocessor': <autosklearn.pipeline.components.feature_preprocessing.FeaturePreprocessorChoice object at 0x7f2af7235a60>,
'model_id': 12,
'rank': 20,
'sklearn_classifier': LinearDiscriminantAnalysis(tol=0.0004182629077658861)},
13: { 'balancing': Balancing(random_state=1),
'classifier': <autosklearn.pipeline.components.classification.ClassifierChoice object at 0x7f2af545acd0>,
'cost': 0.36879432624113473,
'data_preprocessor': <autosklearn.pipeline.components.data_preprocessing.DataPreprocessorChoice object at 0x7f2aee6b89a0>,
'ensemble_weight': 0.02,
'feature_preprocessor': <autosklearn.pipeline.components.feature_preprocessing.FeaturePreprocessorChoice object at 0x7f2af545a3d0>,
'model_id': 13,
'rank': 25,
'sklearn_classifier': ExtraTreesClassifier(bootstrap=True, criterion='entropy', max_features=1,
min_samples_leaf=20, min_samples_split=20,
n_estimators=512, n_jobs=1, random_state=1,
warm_start=True)},
14: { 'balancing': Balancing(random_state=1),
'classifier': <autosklearn.pipeline.components.classification.ClassifierChoice object at 0x7f2aef23dd30>,
'cost': 0.028368794326241176,
'data_preprocessor': <autosklearn.pipeline.components.data_preprocessing.DataPreprocessorChoice object at 0x7f2af6d252b0>,
'ensemble_weight': 0.1,
'feature_preprocessor': <autosklearn.pipeline.components.feature_preprocessing.FeaturePreprocessorChoice object at 0x7f2aef23d280>,
'model_id': 14,
'rank': 5,
'sklearn_classifier': LinearDiscriminantAnalysis(shrinkage='auto', solver='lsqr',
tol=0.013943862590174107)},
15: { 'balancing': Balancing(random_state=1, strategy='weighting'),
'classifier': <autosklearn.pipeline.components.classification.ClassifierChoice object at 0x7f2af0410c40>,
'cost': 0.16312056737588654,
'data_preprocessor': <autosklearn.pipeline.components.data_preprocessing.DataPreprocessorChoice object at 0x7f2aeda5b5b0>,
'ensemble_weight': 0.06,
'feature_preprocessor': <autosklearn.pipeline.components.feature_preprocessing.FeaturePreprocessorChoice object at 0x7f2afd49baf0>,
'model_id': 15,
'rank': 23,
'sklearn_classifier': DecisionTreeClassifier(class_weight='balanced', criterion='entropy',
max_depth=1, min_samples_leaf=3, min_samples_split=13,
random_state=1)},
17: { 'balancing': Balancing(random_state=1, strategy='weighting'),
'classifier': <autosklearn.pipeline.components.classification.ClassifierChoice object at 0x7f2af7705700>,
'cost': 0.07801418439716312,
'data_preprocessor': <autosklearn.pipeline.components.data_preprocessing.DataPreprocessorChoice object at 0x7f2af70aa400>,
'ensemble_weight': 0.08,
'feature_preprocessor': <autosklearn.pipeline.components.feature_preprocessing.FeaturePreprocessorChoice object at 0x7f2af48f5190>,
'model_id': 17,
'rank': 14,
'sklearn_classifier': PassiveAggressiveClassifier(C=0.04885675602027957, average=True, max_iter=64,
random_state=1, tol=0.0002678322187543955,
warm_start=True)},
18: { 'balancing': Balancing(random_state=1),
'classifier': <autosklearn.pipeline.components.classification.ClassifierChoice object at 0x7f2af6e4cb50>,
'cost': 0.1063829787234043,
'data_preprocessor': <autosklearn.pipeline.components.data_preprocessing.DataPreprocessorChoice object at 0x7f2afd445820>,
'ensemble_weight': 0.02,
'feature_preprocessor': <autosklearn.pipeline.components.feature_preprocessing.FeaturePreprocessorChoice object at 0x7f2af6e4cee0>,
'model_id': 18,
'rank': 18,
'sklearn_classifier': SGDClassifier(alpha=0.0002548862311968591, average=True,
eta0=4.278565071229579e-05, learning_rate='constant',
loss='squared_hinge', max_iter=1024, random_state=1,
tol=5.482673674397374e-05, warm_start=True)},
20: { 'balancing': Balancing(random_state=1),
'classifier': <autosklearn.pipeline.components.classification.ClassifierChoice object at 0x7f2b11091310>,
'cost': 0.07092198581560283,
'data_preprocessor': <autosklearn.pipeline.components.data_preprocessing.DataPreprocessorChoice object at 0x7f2af07100d0>,
'ensemble_weight': 0.02,
'feature_preprocessor': <autosklearn.pipeline.components.feature_preprocessing.FeaturePreprocessorChoice object at 0x7f2afd5f1040>,
'model_id': 20,
'rank': 12,
'sklearn_classifier': AdaBoostClassifier(algorithm='SAMME',
base_estimator=DecisionTreeClassifier(max_depth=9),
learning_rate=0.21684295401135145, n_estimators=72,
random_state=1)},
21: { 'balancing': Balancing(random_state=1, strategy='weighting'),
'classifier': <autosklearn.pipeline.components.classification.ClassifierChoice object at 0x7f2af0a01f40>,
'cost': 0.24113475177304966,
'data_preprocessor': <autosklearn.pipeline.components.data_preprocessing.DataPreprocessorChoice object at 0x7f2af7616ee0>,
'ensemble_weight': 0.02,
'feature_preprocessor': <autosklearn.pipeline.components.feature_preprocessing.FeaturePreprocessorChoice object at 0x7f2af0a015b0>,
'model_id': 21,
'rank': 24,
'sklearn_classifier': DecisionTreeClassifier(class_weight='balanced', max_depth=30,
min_samples_leaf=4, min_samples_split=18,
random_state=1)},
23: { 'balancing': Balancing(random_state=1, strategy='weighting'),
'classifier': <autosklearn.pipeline.components.classification.ClassifierChoice object at 0x7f2af3ff2850>,
'cost': 0.07801418439716312,
'data_preprocessor': <autosklearn.pipeline.components.data_preprocessing.DataPreprocessorChoice object at 0x7f2af70b2a30>,
'ensemble_weight': 0.04,
'feature_preprocessor': <autosklearn.pipeline.components.feature_preprocessing.FeaturePreprocessorChoice object at 0x7f2af3ff2100>,
'model_id': 23,
'rank': 13,
'sklearn_classifier': MLPClassifier(activation='tanh', alpha=0.02522490081077779, beta_1=0.999,
beta_2=0.9, early_stopping=True, hidden_layer_sizes=(35,),
learning_rate_init=0.026536043936270744, max_iter=32,
n_iter_no_change=32, random_state=1, verbose=0, warm_start=True)},
24: { 'balancing': Balancing(random_state=1, strategy='weighting'),
'classifier': <autosklearn.pipeline.components.classification.ClassifierChoice object at 0x7f2af4b2b820>,
'cost': 0.028368794326241176,
'data_preprocessor': <autosklearn.pipeline.components.data_preprocessing.DataPreprocessorChoice object at 0x7f2af6bf15b0>,
'ensemble_weight': 0.08,
'feature_preprocessor': <autosklearn.pipeline.components.feature_preprocessing.FeaturePreprocessorChoice object at 0x7f2af4b2bdf0>,
'model_id': 24,
'rank': 4,
'sklearn_classifier': LinearDiscriminantAnalysis(shrinkage='auto', solver='lsqr',
tol=0.001678686694320821)},
25: { 'balancing': Balancing(random_state=1, strategy='weighting'),
'classifier': <autosklearn.pipeline.components.classification.ClassifierChoice object at 0x7f2afd2aa760>,
'cost': 0.021276595744680882,
'data_preprocessor': <autosklearn.pipeline.components.data_preprocessing.DataPreprocessorChoice object at 0x7f2af0b83dc0>,
'ensemble_weight': 0.04,
'feature_preprocessor': <autosklearn.pipeline.components.feature_preprocessing.FeaturePreprocessorChoice object at 0x7f2afd2aa910>,
'model_id': 25,
'rank': 1,
'sklearn_classifier': HistGradientBoostingClassifier(early_stopping=True,
l2_regularization=2.1928852428306468e-08,
learning_rate=0.29557366541255203, max_iter=64,
max_leaf_nodes=70, min_samples_leaf=24,
random_state=1,
validation_fraction=0.13252036572629863,
warm_start=True)},
27: { 'balancing': Balancing(random_state=1, strategy='weighting'),
'classifier': <autosklearn.pipeline.components.classification.ClassifierChoice object at 0x7f2afd2668b0>,
'cost': 0.375886524822695,
'data_preprocessor': <autosklearn.pipeline.components.data_preprocessing.DataPreprocessorChoice object at 0x7f2af547bd60>,
'ensemble_weight': 0.02,
'feature_preprocessor': <autosklearn.pipeline.components.feature_preprocessing.FeaturePreprocessorChoice object at 0x7f2af4554e50>,
'model_id': 27,
'rank': 27,
'sklearn_classifier': BernoulliNB(alpha=9.144520523203797)}}
auto-sklearn results:
Dataset name: breast_cancer
Metric: accuracy
Best validation score: 0.978723
Number of target algorithm runs: 27
Number of successful target algorithm runs: 27
Number of crashed target algorithm runs: 0
Number of target algorithms that exceeded the time limit: 0
Number of target algorithms that exceeded the memory limit: 0
Accuracy score 0.958041958041958
Fit a classifier using Random Search¶
def get_random_search_object_callback(
scenario_dict,
seed,
ta,
ta_kwargs,
metalearning_configurations,
n_jobs,
dask_client,
multi_objective_algorithm, # This argument will be ignored as ROAR does not yet support multi-objective optimization
multi_objective_kwargs,
):
"""Random search"""
if n_jobs > 1 or (dask_client and len(dask_client.nthreads()) > 1):
raise ValueError(
"Please make sure to guard the code invoking Auto-sklearn by "
"`if __name__ == '__main__'` and remove this exception."
)
scenario_dict["minR"] = len(scenario_dict["instances"])
scenario_dict["initial_incumbent"] = "RANDOM"
scenario = Scenario(scenario_dict)
return ROAR(
scenario=scenario,
rng=seed,
tae_runner=ta,
tae_runner_kwargs=ta_kwargs,
run_id=seed,
dask_client=dask_client,
n_jobs=n_jobs,
)
automl = autosklearn.classification.AutoSklearnClassifier(
time_left_for_this_task=60,
per_run_time_limit=15,
tmp_folder="/tmp/autosklearn_random_search_example_tmp",
initial_configurations_via_metalearning=0,
# Passing the callback to get the SMAC object
get_smac_object_callback=get_random_search_object_callback,
)
automl.fit(X_train, y_train, dataset_name="breast_cancer")
print("#" * 80)
print("Results for random search.")
# Print the final ensemble constructed by auto-sklearn via random search.
pprint(automl.show_models(), indent=4)
# Print statistics about the auto-sklearn run such as number of
# iterations, number of models failed with a time out.
print(automl.sprint_statistics())
predictions = automl.predict(X_test)
print("Accuracy score", sklearn.metrics.accuracy_score(y_test, predictions))
Fitting to the training data: 0%| | 0/60 [00:00<?, ?it/s, The total time budget for this task is 0:01:00]
Fitting to the training data: 2%|1 | 1/60 [00:01<00:59, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 3%|3 | 2/60 [00:02<00:58, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 5%|5 | 3/60 [00:03<00:57, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 7%|6 | 4/60 [00:04<00:56, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 8%|8 | 5/60 [00:05<00:55, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 10%|# | 6/60 [00:06<00:54, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 12%|#1 | 7/60 [00:07<00:53, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 13%|#3 | 8/60 [00:08<00:52, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 15%|#5 | 9/60 [00:09<00:51, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 17%|#6 | 10/60 [00:10<00:50, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 18%|#8 | 11/60 [00:11<00:49, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 20%|## | 12/60 [00:12<00:48, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 22%|##1 | 13/60 [00:13<00:47, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 23%|##3 | 14/60 [00:14<00:46, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 25%|##5 | 15/60 [00:15<00:45, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 27%|##6 | 16/60 [00:16<00:44, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 28%|##8 | 17/60 [00:17<00:43, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 30%|### | 18/60 [00:18<00:42, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 32%|###1 | 19/60 [00:19<00:41, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 33%|###3 | 20/60 [00:20<00:40, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 35%|###5 | 21/60 [00:21<00:39, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 37%|###6 | 22/60 [00:22<00:38, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 38%|###8 | 23/60 [00:23<00:37, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 40%|#### | 24/60 [00:24<00:36, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 42%|####1 | 25/60 [00:25<00:35, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 43%|####3 | 26/60 [00:26<00:34, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 45%|####5 | 27/60 [00:27<00:33, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 47%|####6 | 28/60 [00:28<00:32, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 48%|####8 | 29/60 [00:29<00:31, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 50%|##### | 30/60 [00:30<00:30, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 52%|#####1 | 31/60 [00:31<00:29, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 53%|#####3 | 32/60 [00:32<00:28, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 55%|#####5 | 33/60 [00:33<00:27, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 57%|#####6 | 34/60 [00:34<00:26, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 58%|#####8 | 35/60 [00:35<00:25, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 60%|###### | 36/60 [00:36<00:24, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 62%|######1 | 37/60 [00:37<00:23, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 63%|######3 | 38/60 [00:38<00:22, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 65%|######5 | 39/60 [00:39<00:21, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 67%|######6 | 40/60 [00:40<00:20, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 68%|######8 | 41/60 [00:41<00:19, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 70%|####### | 42/60 [00:42<00:18, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 72%|#######1 | 43/60 [00:43<00:17, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 73%|#######3 | 44/60 [00:44<00:16, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 75%|#######5 | 45/60 [00:45<00:15, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 77%|#######6 | 46/60 [00:46<00:14, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 78%|#######8 | 47/60 [00:47<00:13, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 80%|######## | 48/60 [00:48<00:12, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 82%|########1 | 49/60 [00:49<00:11, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 83%|########3 | 50/60 [00:50<00:10, 1.00s/it, The total time budget for this task is 0:01:00]
Fitting to the training data: 100%|##########| 60/60 [00:50<00:00, 1.20it/s, The total time budget for this task is 0:01:00]
################################################################################
Results for random search.
{ 3: { 'balancing': Balancing(random_state=1, strategy='weighting'),
'classifier': <autosklearn.pipeline.components.classification.ClassifierChoice object at 0x7f2af6aba400>,
'cost': 0.08510638297872342,
'data_preprocessor': <autosklearn.pipeline.components.data_preprocessing.DataPreprocessorChoice object at 0x7f2afd2aae80>,
'ensemble_weight': 0.04,
'feature_preprocessor': <autosklearn.pipeline.components.feature_preprocessing.FeaturePreprocessorChoice object at 0x7f2af6aba190>,
'model_id': 3,
'rank': 15,
'sklearn_classifier': PassiveAggressiveClassifier(C=0.000505280701827856, max_iter=32, random_state=1,
tol=0.06421873693006744, warm_start=True)},
4: { 'balancing': Balancing(random_state=1, strategy='weighting'),
'classifier': <autosklearn.pipeline.components.classification.ClassifierChoice object at 0x7f2af4f066a0>,
'cost': 0.021276595744680882,
'data_preprocessor': <autosklearn.pipeline.components.data_preprocessing.DataPreprocessorChoice object at 0x7f2afc03c730>,
'ensemble_weight': 0.06,
'feature_preprocessor': <autosklearn.pipeline.components.feature_preprocessing.FeaturePreprocessorChoice object at 0x7f2af6dcd370>,
'model_id': 4,
'rank': 2,
'sklearn_classifier': HistGradientBoostingClassifier(early_stopping=True,
l2_regularization=1.4469619760002166e-09,
learning_rate=0.12309321429220763, max_iter=256,
max_leaf_nodes=10, min_samples_leaf=17,
n_iter_no_change=16, random_state=1,
validation_fraction=None, warm_start=True)},
5: { 'balancing': Balancing(random_state=1),
'classifier': <autosklearn.pipeline.components.classification.ClassifierChoice object at 0x7f2af0b83be0>,
'cost': 0.028368794326241176,
'data_preprocessor': <autosklearn.pipeline.components.data_preprocessing.DataPreprocessorChoice object at 0x7f2af4b676d0>,
'ensemble_weight': 0.14,
'feature_preprocessor': <autosklearn.pipeline.components.feature_preprocessing.FeaturePreprocessorChoice object at 0x7f2af0b83af0>,
'model_id': 5,
'rank': 4,
'sklearn_classifier': RandomForestClassifier(max_features=5, n_estimators=512, n_jobs=1,
random_state=1, warm_start=True)},
6: { 'balancing': Balancing(random_state=1),
'classifier': <autosklearn.pipeline.components.classification.ClassifierChoice object at 0x7f2af5350cd0>,
'cost': 0.028368794326241176,
'data_preprocessor': <autosklearn.pipeline.components.data_preprocessing.DataPreprocessorChoice object at 0x7f2afd2c4850>,
'ensemble_weight': 0.06,
'feature_preprocessor': <autosklearn.pipeline.components.feature_preprocessing.FeaturePreprocessorChoice object at 0x7f2af5350040>,
'model_id': 6,
'rank': 5,
'sklearn_classifier': LinearDiscriminantAnalysis(tol=0.08622878828999171)},
7: { 'balancing': Balancing(random_state=1),
'classifier': <autosklearn.pipeline.components.classification.ClassifierChoice object at 0x7f2af75c6fa0>,
'cost': 0.36879432624113473,
'data_preprocessor': <autosklearn.pipeline.components.data_preprocessing.DataPreprocessorChoice object at 0x7f2af3ff2550>,
'ensemble_weight': 0.04,
'feature_preprocessor': <autosklearn.pipeline.components.feature_preprocessing.FeaturePreprocessorChoice object at 0x7f2af09f6a60>,
'model_id': 7,
'rank': 25,
'sklearn_classifier': BernoulliNB(alpha=0.020898102999400082)},
9: { 'balancing': Balancing(random_state=1, strategy='weighting'),
'classifier': <autosklearn.pipeline.components.classification.ClassifierChoice object at 0x7f2af49b15e0>,
'cost': 0.12056737588652477,
'data_preprocessor': <autosklearn.pipeline.components.data_preprocessing.DataPreprocessorChoice object at 0x7f2afda88af0>,
'ensemble_weight': 0.06,
'feature_preprocessor': <autosklearn.pipeline.components.feature_preprocessing.FeaturePreprocessorChoice object at 0x7f2af49b1fa0>,
'model_id': 9,
'rank': 18,
'sklearn_classifier': PassiveAggressiveClassifier(C=0.09581923415311244, max_iter=64, random_state=1,
tol=0.000145918046557035, warm_start=True)},
10: { 'balancing': Balancing(random_state=1),
'classifier': <autosklearn.pipeline.components.classification.ClassifierChoice object at 0x7f2afc0135b0>,
'cost': 0.16312056737588654,
'data_preprocessor': <autosklearn.pipeline.components.data_preprocessing.DataPreprocessorChoice object at 0x7f2af6bf1820>,
'ensemble_weight': 0.06,
'feature_preprocessor': <autosklearn.pipeline.components.feature_preprocessing.FeaturePreprocessorChoice object at 0x7f2afc013220>,
'model_id': 10,
'rank': 22,
'sklearn_classifier': ExtraTreesClassifier(criterion='entropy', max_features=2, min_samples_leaf=11,
min_samples_split=19, n_estimators=512, n_jobs=1,
random_state=1, warm_start=True)},
11: { 'balancing': Balancing(random_state=1, strategy='weighting'),
'classifier': <autosklearn.pipeline.components.classification.ClassifierChoice object at 0x7f2af09f6970>,
'cost': 0.028368794326241176,
'data_preprocessor': <autosklearn.pipeline.components.data_preprocessing.DataPreprocessorChoice object at 0x7f2af41bc070>,
'ensemble_weight': 0.06,
'feature_preprocessor': <autosklearn.pipeline.components.feature_preprocessing.FeaturePreprocessorChoice object at 0x7f2af0410430>,
'model_id': 11,
'rank': 8,
'sklearn_classifier': LinearDiscriminantAnalysis(shrinkage='auto', solver='lsqr',
tol=0.039339489121391846)},
12: { 'balancing': Balancing(random_state=1),
'classifier': <autosklearn.pipeline.components.classification.ClassifierChoice object at 0x7f2af70b2250>,
'cost': 0.13475177304964536,
'data_preprocessor': <autosklearn.pipeline.components.data_preprocessing.DataPreprocessorChoice object at 0x7f2aedc93820>,
'ensemble_weight': 0.08,
'feature_preprocessor': <autosklearn.pipeline.components.feature_preprocessing.FeaturePreprocessorChoice object at 0x7f2af70b2e80>,
'model_id': 12,
'rank': 19,
'sklearn_classifier': LinearDiscriminantAnalysis(tol=0.0004182629077658861)},
13: { 'balancing': Balancing(random_state=1),
'classifier': <autosklearn.pipeline.components.classification.ClassifierChoice object at 0x7f2af4a9d130>,
'cost': 0.36879432624113473,
'data_preprocessor': <autosklearn.pipeline.components.data_preprocessing.DataPreprocessorChoice object at 0x7f2af4665310>,
'ensemble_weight': 0.02,
'feature_preprocessor': <autosklearn.pipeline.components.feature_preprocessing.FeaturePreprocessorChoice object at 0x7f2af4a9dd90>,
'model_id': 13,
'rank': 24,
'sklearn_classifier': ExtraTreesClassifier(bootstrap=True, criterion='entropy', max_features=1,
min_samples_leaf=20, min_samples_split=20,
n_estimators=512, n_jobs=1, random_state=1,
warm_start=True)},
14: { 'balancing': Balancing(random_state=1),
'classifier': <autosklearn.pipeline.components.classification.ClassifierChoice object at 0x7f2af4bc3220>,
'cost': 0.028368794326241176,
'data_preprocessor': <autosklearn.pipeline.components.data_preprocessing.DataPreprocessorChoice object at 0x7f2af48f51c0>,
'ensemble_weight': 0.04,
'feature_preprocessor': <autosklearn.pipeline.components.feature_preprocessing.FeaturePreprocessorChoice object at 0x7f2af4bc31f0>,
'model_id': 14,
'rank': 3,
'sklearn_classifier': LinearDiscriminantAnalysis(shrinkage='auto', solver='lsqr',
tol=0.013943862590174107)},
15: { 'balancing': Balancing(random_state=1, strategy='weighting'),
'classifier': <autosklearn.pipeline.components.classification.ClassifierChoice object at 0x7f2af03a1bb0>,
'cost': 0.16312056737588654,
'data_preprocessor': <autosklearn.pipeline.components.data_preprocessing.DataPreprocessorChoice object at 0x7f2af4c0bc10>,
'ensemble_weight': 0.04,
'feature_preprocessor': <autosklearn.pipeline.components.feature_preprocessing.FeaturePreprocessorChoice object at 0x7f2afc03a1f0>,
'model_id': 15,
'rank': 21,
'sklearn_classifier': DecisionTreeClassifier(class_weight='balanced', criterion='entropy',
max_depth=1, min_samples_leaf=3, min_samples_split=13,
random_state=1)},
16: { 'balancing': Balancing(random_state=1, strategy='weighting'),
'classifier': <autosklearn.pipeline.components.classification.ClassifierChoice object at 0x7f2afd3bdb50>,
'cost': 0.06382978723404253,
'data_preprocessor': <autosklearn.pipeline.components.data_preprocessing.DataPreprocessorChoice object at 0x7f2aedda8b20>,
'ensemble_weight': 0.02,
'feature_preprocessor': <autosklearn.pipeline.components.feature_preprocessing.FeaturePreprocessorChoice object at 0x7f2afd3bd430>,
'model_id': 16,
'rank': 9,
'sklearn_classifier': GaussianNB()},
17: { 'balancing': Balancing(random_state=1, strategy='weighting'),
'classifier': <autosklearn.pipeline.components.classification.ClassifierChoice object at 0x7f2af0b8ea60>,
'cost': 0.07801418439716312,
'data_preprocessor': <autosklearn.pipeline.components.data_preprocessing.DataPreprocessorChoice object at 0x7f2af3ec6bb0>,
'ensemble_weight': 0.02,
'feature_preprocessor': <autosklearn.pipeline.components.feature_preprocessing.FeaturePreprocessorChoice object at 0x7f2afd64a0d0>,
'model_id': 17,
'rank': 12,
'sklearn_classifier': PassiveAggressiveClassifier(C=0.04885675602027957, average=True, max_iter=64,
random_state=1, tol=0.0002678322187543955,
warm_start=True)},
18: { 'balancing': Balancing(random_state=1),
'classifier': <autosklearn.pipeline.components.classification.ClassifierChoice object at 0x7f2afcd45430>,
'cost': 0.1063829787234043,
'data_preprocessor': <autosklearn.pipeline.components.data_preprocessing.DataPreprocessorChoice object at 0x7f2af4a952e0>,
'ensemble_weight': 0.06,
'feature_preprocessor': <autosklearn.pipeline.components.feature_preprocessing.FeaturePreprocessorChoice object at 0x7f2afcd45700>,
'model_id': 18,
'rank': 17,
'sklearn_classifier': SGDClassifier(alpha=0.0002548862311968591, average=True,
eta0=4.278565071229579e-05, learning_rate='constant',
loss='squared_hinge', max_iter=1024, random_state=1,
tol=5.482673674397374e-05, warm_start=True)},
19: { 'balancing': Balancing(random_state=1, strategy='weighting'),
'classifier': <autosklearn.pipeline.components.classification.ClassifierChoice object at 0x7f2af6c71c70>,
'cost': 0.09219858156028371,
'data_preprocessor': <autosklearn.pipeline.components.data_preprocessing.DataPreprocessorChoice object at 0x7f2af7645be0>,
'ensemble_weight': 0.02,
'feature_preprocessor': <autosklearn.pipeline.components.feature_preprocessing.FeaturePreprocessorChoice object at 0x7f2af6c719d0>,
'model_id': 19,
'rank': 16,
'sklearn_classifier': AdaBoostClassifier(base_estimator=DecisionTreeClassifier(max_depth=7),
learning_rate=0.035867722843024794, n_estimators=70,
random_state=1)},
20: { 'balancing': Balancing(random_state=1),
'classifier': <autosklearn.pipeline.components.classification.ClassifierChoice object at 0x7f2afd4f7130>,
'cost': 0.07092198581560283,
'data_preprocessor': <autosklearn.pipeline.components.data_preprocessing.DataPreprocessorChoice object at 0x7f2af6c35d90>,
'ensemble_weight': 0.02,
'feature_preprocessor': <autosklearn.pipeline.components.feature_preprocessing.FeaturePreprocessorChoice object at 0x7f2af464b340>,
'model_id': 20,
'rank': 10,
'sklearn_classifier': AdaBoostClassifier(algorithm='SAMME',
base_estimator=DecisionTreeClassifier(max_depth=9),
learning_rate=0.21684295401135145, n_estimators=72,
random_state=1)},
23: { 'balancing': Balancing(random_state=1, strategy='weighting'),
'classifier': <autosklearn.pipeline.components.classification.ClassifierChoice object at 0x7f2af7258220>,
'cost': 0.07801418439716312,
'data_preprocessor': <autosklearn.pipeline.components.data_preprocessing.DataPreprocessorChoice object at 0x7f2af51a0160>,
'ensemble_weight': 0.02,
'feature_preprocessor': <autosklearn.pipeline.components.feature_preprocessing.FeaturePreprocessorChoice object at 0x7f2af7258fd0>,
'model_id': 23,
'rank': 13,
'sklearn_classifier': MLPClassifier(activation='tanh', alpha=0.02522490081077779, beta_1=0.999,
beta_2=0.9, early_stopping=True, hidden_layer_sizes=(35,),
learning_rate_init=0.026536043936270744, max_iter=32,
n_iter_no_change=32, random_state=1, verbose=0, warm_start=True)},
24: { 'balancing': Balancing(random_state=1, strategy='weighting'),
'classifier': <autosklearn.pipeline.components.classification.ClassifierChoice object at 0x7f2af6dfc2e0>,
'cost': 0.028368794326241176,
'data_preprocessor': <autosklearn.pipeline.components.data_preprocessing.DataPreprocessorChoice object at 0x7f2af7235ac0>,
'ensemble_weight': 0.06,
'feature_preprocessor': <autosklearn.pipeline.components.feature_preprocessing.FeaturePreprocessorChoice object at 0x7f2af6dfc970>,
'model_id': 24,
'rank': 6,
'sklearn_classifier': LinearDiscriminantAnalysis(shrinkage='auto', solver='lsqr',
tol=0.001678686694320821)},
25: { 'balancing': Balancing(random_state=1, strategy='weighting'),
'classifier': <autosklearn.pipeline.components.classification.ClassifierChoice object at 0x7f2af4e47b50>,
'cost': 0.021276595744680882,
'data_preprocessor': <autosklearn.pipeline.components.data_preprocessing.DataPreprocessorChoice object at 0x7f2afc02fd00>,
'ensemble_weight': 0.02,
'feature_preprocessor': <autosklearn.pipeline.components.feature_preprocessing.FeaturePreprocessorChoice object at 0x7f2af4e47520>,
'model_id': 25,
'rank': 1,
'sklearn_classifier': HistGradientBoostingClassifier(early_stopping=True,
l2_regularization=2.1928852428306468e-08,
learning_rate=0.29557366541255203, max_iter=64,
max_leaf_nodes=70, min_samples_leaf=24,
random_state=1,
validation_fraction=0.13252036572629863,
warm_start=True)},
26: { 'balancing': Balancing(random_state=1, strategy='weighting'),
'classifier': <autosklearn.pipeline.components.classification.ClassifierChoice object at 0x7f2af03c1220>,
'cost': 0.08510638297872342,
'data_preprocessor': <autosklearn.pipeline.components.data_preprocessing.DataPreprocessorChoice object at 0x7f2aef70c550>,
'ensemble_weight': 0.02,
'feature_preprocessor': <autosklearn.pipeline.components.feature_preprocessing.FeaturePreprocessorChoice object at 0x7f2af7309460>,
'model_id': 26,
'rank': 14,
'sklearn_classifier': KNeighborsClassifier(n_neighbors=80, p=1)},
27: { 'balancing': Balancing(random_state=1, strategy='weighting'),
'classifier': <autosklearn.pipeline.components.classification.ClassifierChoice object at 0x7f2afd4a1e20>,
'cost': 0.375886524822695,
'data_preprocessor': <autosklearn.pipeline.components.data_preprocessing.DataPreprocessorChoice object at 0x7f2af7435ac0>,
'ensemble_weight': 0.04,
'feature_preprocessor': <autosklearn.pipeline.components.feature_preprocessing.FeaturePreprocessorChoice object at 0x7f2af41bc370>,
'model_id': 27,
'rank': 26,
'sklearn_classifier': BernoulliNB(alpha=9.144520523203797)}}
auto-sklearn results:
Dataset name: breast_cancer
Metric: accuracy
Best validation score: 0.978723
Number of target algorithm runs: 26
Number of successful target algorithm runs: 26
Number of crashed target algorithm runs: 0
Number of target algorithms that exceeded the time limit: 0
Number of target algorithms that exceeded the memory limit: 0
Accuracy score 0.972027972027972
Total running time of the script: ( 2 minutes 3.285 seconds)
