Games
The games module contains all the game-theoretic explanation games of HyperSHAP.
AblationGame
¶
Bases: AbstractHPIGame
The ablation game generates local explanations for hyperparameter configurations.
It does so by evaluating all potential ablation paths, switching from a baseline configuration to an optimized configuration value by value. It leverages a surrogate model to estimate performance based on blended configurations.
Source code in src/hypershap/games/ablation.py
34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 | |
__init__(explanation_task, n_workers=None, verbose=None)
¶
Initialize an AblationGame.
explanation_task (AblationExplanationTask): An instance of AblationExplanationTask
providing access to the baseline configuration, configuration of interest,
and the surrogate model for evaluation.
n_workers (int | None): The number of worker threads to use for parallel
evaluation of coalitions. Defaults to None, which disables parallelization.
Using more workers can significantly speed up the computation of Shapley values.
The maximum number of workers is capped by the number of coalitions.
verbose (bool | None): A boolean indicating whether to print verbose messages
during computation. Defaults to None. When set to True, the method prints
debugging information and progress updates.
Source code in src/hypershap/games/ablation.py
42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 | |
evaluate_single_coalition(coalition)
¶
Evaluate a single coalition (combination of baseline and optimized hyperparameters).
This method blends the baseline and optimized configurations based on the provided coalition, then uses the surrogate model to estimate the performance of the blended configuration.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
coalition
|
ndarray
|
A Boolean array indicating which hyperparameters should be taken from the baseline configuration (False) and which from the configuration of interest (True). |
required |
Returns:
| Name | Type | Description |
|---|---|---|
float |
float
|
The performance score of the blended configuration as estimated by the surrogate model. |
Source code in src/hypershap/games/ablation.py
65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 | |
AbstractHPIGame
¶
Bases: Game
Abstract base class for Hyperparameter Importance Games (HPIGames).
Represents a game-theoretic framework for analyzing the importance of
hyperparameters for HPO. It leverages the shapiq library to compute
Shapley values and analyze coalition behavior.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
explanation_task
|
ExplanationTask
|
The |
required |
n_workers
|
int | None
|
The number of worker threads to use for parallel evaluation of coalitions. Defaults to 1. Using more workers can significantly speed up the computation of Shapley values. |
None
|
verbose
|
bool | None
|
A boolean indicating whether to print verbose messages during computation. Defaults to False. |
None
|
Source code in src/hypershap/games/abstract.py
26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 | |
__init__(explanation_task, n_workers=None, verbose=None)
¶
Initialize the Hyperparameter Interaction Game (HPIGame).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
explanation_task
|
ExplanationTask
|
The |
required |
n_workers
|
int | None
|
The number of worker threads to use for parallel evaluation of coalitions. Defaults to None meaning no parallelization. Using more workers can significantly speed up the computation of Shapley values. The maximum number of workers is capped by the number of coalitions. |
None
|
verbose
|
bool | None
|
A boolean indicating whether to print verbose messages during computation. Defaults to None. When set to True, the method prints debugging information and progress updates. |
None
|
Source code in src/hypershap/games/abstract.py
44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 | |
evaluate_single_coalition(coalition)
abstractmethod
¶
Evaluate the value of a single coalition.
This method must be implemented by subclasses.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
coalition
|
ndarray
|
A boolean array representing a coalition of hyperparameters. |
required |
Returns:
| Type | Description |
|---|---|
float
|
The value of the coalition (a float). |
Source code in src/hypershap/games/abstract.py
131 132 133 134 135 136 137 138 139 140 141 142 143 | |
get_hyperparameter_names()
¶
Return a list of the names of the hyperparameters.
Returns:
| Type | Description |
|---|---|
list[str]
|
A list of strings, where each string is the name of a hyperparameter. |
Source code in src/hypershap/games/abstract.py
154 155 156 157 158 159 160 161 | |
get_num_hyperparameters()
¶
Return the number of hyperparameters being considered.
Returns:
| Type | Description |
|---|---|
int
|
The number of hyperparameters (an integer). |
Source code in src/hypershap/games/abstract.py
145 146 147 148 149 150 151 152 | |
value_function(coalitions)
¶
Calculate the value of a list of coalitions.
This method handles both single-worker and multi-worker scenarios for efficient computation.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
coalitions
|
ndarray
|
A NumPy array representing a list of coalitions. Each coalition is a Boolean array indicating which hyperparameters are included in that coalition. |
required |
Returns:
| Type | Description |
|---|---|
ndarray
|
A NumPy array containing the values of the input coalitions. |
Source code in src/hypershap/games/abstract.py
92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 | |
MistunabilityGame
¶
Bases: SearchBasedGame
Game representing the mistunability of hyperparameters.
Source code in src/hypershap/games/tunability.py
151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 | |
__init__(explanation_task, cs_searcher=None, n_workers=None, verbose=None)
¶
Initialize the mistunability game.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
explanation_task
|
MistunabilityExplanationTask
|
The explanation task containing the configuration space and surrogate model. |
required |
cs_searcher
|
ConfigSpaceSearcher | None
|
The configuration space searcher. If None, a RandomConfigSpaceSearcher is used by default. |
None
|
n_workers
|
int | None
|
The number of worker threads to use for parallel evaluation of coalitions. Defaults to None meaning no parallelization. Using more workers can significantly speed up the computation of Shapley values. The maximum number of workers is capped by the number of coalitions. |
None
|
verbose
|
bool | None
|
A boolean indicating whether to print verbose messages during computation. Defaults to None. When set to True, the method prints debugging information and progress updates. |
None
|
Source code in src/hypershap/games/tunability.py
154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 | |
MultiBaselineAblationGame
¶
Bases: AbstractHPIGame
The multi-baseline ablation game generates local explanations for hyperparameter configurations.
It does so by considering multiple baseline configurations as starting points.
Source code in src/hypershap/games/ablation.py
107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 | |
__init__(explanation_task, aggregation=Aggregation.AVG, n_workers=None, verbose=None)
¶
Initialize a MultiBaselineAblationGame.
n_workers (int | None): The number of worker threads to use for parallel evaluation of coalitions. Defaults to None, which disables parallelization. Using more workers can significantly speed up the computation of Shapley values. The maximum number of workers is capped by the number of coalitions. verbose (bool | None): A boolean indicating whether to print verbose messages during computation. Defaults to None. When set to True, the method prints debugging information and progress updates.
Source code in src/hypershap/games/ablation.py
113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 | |
evaluate_single_coalition(coalition)
¶
Evaluate a single coalition (combination of baseline and optimized hyperparameters).
Source code in src/hypershap/games/ablation.py
146 147 148 149 | |
MultiDataHPIGame
¶
Bases: AbstractHPIGame
The multi-data game generalizes an explanation game to multiple datasets.
Source code in src/hypershap/games/multi_data.py
20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 | |
__init__(explanation_task, base_game, aggregation)
¶
Initialize the multi-data game wrapper.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
explanation_task
|
ExplanationTask
|
The explanation task containing the configuration space and surrogate model. |
required |
base_game
|
AbstractHPIGame
|
The base game instance. |
required |
aggregation
|
Aggregation
|
The aggregation method to use. |
required |
Source code in src/hypershap/games/multi_data.py
23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 | |
evaluate_single_coalition(coalition)
¶
Evaluate the multi-data game on the coalition.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
coalition
|
ndarray
|
The coalition to evaluate. |
required |
Returns: The value of the multi-data game on the coalition.
Source code in src/hypershap/games/multi_data.py
53 54 55 56 57 58 59 60 61 62 63 | |
OptimizerBiasGame
¶
Bases: AbstractHPIGame
An explanation game to measure bias in hyperparameter optimizers.
This class extends the AbstractHPIGame base class and is specifically designed
to quantify how much an optimizer is biased toward tuning certain hyperparameters.
To this end a set of diverse optimizers is used as a reference.
Attributes:
| Name | Type | Description |
|---|---|---|
explanation_task |
OptimizerBiasExplanationTask
|
The task that defines the game. This includes the configuration sapce, the surrogate model and a baseline configuration, as well as the optimizer of interest and the ensemble of diverse optimizers. |
Methods:
| Name | Description |
|---|---|
evaluate_single_coalition |
Computes the marginal contribution of a coalition (subset of features) by comparing the optimizer of interest against an optimizer ensemble. This method is called internally during the game's main evaluation process. |
Note
The game evaluates coalitions based on the difference between the outcome using the primary optimizer and the maximum outcome achieved by any optimizer in the provided ensemble.
Source code in src/hypershap/games/optimizerbias.py
31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 | |
__init__(explanation_task, n_workers=None, verbose=None)
¶
Initialize an instance of OptimizerBiasGame.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
explanation_task
|
OptimizerBiasExplanationTask
|
The task that contains all necessary information for defining the game. This includes the configuration space, the surrogate model, the optimizer of interest, and the ensemble of diverse optimizers. |
required |
n_workers
|
int | None
|
The number of worker threads to use for parallel evaluation of coalitions. Defaults to None meaning no parallelization. Using more workers can significantly speed up the computation of Shapley values. The maximum number of workers is capped by the number of coalitions. |
None
|
verbose
|
bool | None
|
A boolean indicating whether to print verbose messages during computation. Defaults to None. When set to True, the method prints debugging information and progress updates. |
None
|
Example
from hypershap.task import OptimizerBiasExplanationTask
Create an explanation task instance with specific parameters¶
expl_task = OptimizerBiasExplanationTask(...) game = OptimizerBiasGame(expl_task)
Source code in src/hypershap/games/optimizerbias.py
55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 | |
evaluate_single_coalition(coalition)
¶
Evaluate a single coalition by comparing against an optimizer ensemble.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
coalition
|
ndarray
|
A binary array indicating which hyperparameters are included in the coalition. The array has shape (n_hyperparameters,) where True means hyperparameters is included and False otherwise. |
required |
Returns:
| Name | Type | Description |
|---|---|---|
float |
float
|
The marginal contribution of the coalition. It is computed as the difference between the outcome when using the optimizer of interest with the given coalition, versus the best outcome achievable by any optimizer in the ensemble (including the one of interest). |
Note
This method overrides a base class abstract method and must be implemented to provide specific game logic. The value returned here is used for computing Shapley values or other cooperative game theory measures.
Example
coalition = np.array([1, 0, 1]) # Hyperparameters 0 and 2 are included marginal_contribution = game.evaluate_single_coalition(coalition) print(marginal_contribution) # Outputs the difference in outcomes for this coalition.
Source code in src/hypershap/games/optimizerbias.py
88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 | |
SearchBasedGame
¶
Bases: AbstractHPIGame
Base class for games that rely on searching the configuration space.
Source code in src/hypershap/games/tunability.py
40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 | |
__init__(explanation_task, cs_searcher, n_workers=None, verbose=None)
¶
Initialize the search-based game.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
explanation_task
|
BaselineExplanationTask
|
The explanation task containing the configuration space and surrogate model. |
required |
cs_searcher
|
ConfigSpaceSearcher
|
The configuration space searcher. If None, a RandomConfigSpaceSearcher is used by default. |
required |
n_workers
|
int | None
|
The number of worker threads to use for parallel evaluation of coalitions. Defaults to None meaning no parallelization. Using more workers can significantly speed up the computation of Shapley values. The maximum number of workers is capped by the number of coalitions. |
None
|
verbose
|
bool | None
|
A boolean indicating whether to print verbose messages during computation. Defaults to None. When set to True, the method prints debugging information and progress updates. |
None
|
Source code in src/hypershap/games/tunability.py
43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 | |
evaluate_single_coalition(coalition)
¶
Evaluate the value of a single coalition using the configuration space searcher.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
coalition
|
ndarray
|
A boolean array indicating which hyperparameters are constrained by the coalition. |
required |
Returns:
| Type | Description |
|---|---|
float
|
The value of the coalition based on the search results. |
Source code in src/hypershap/games/tunability.py
68 69 70 71 72 73 74 75 76 77 78 79 | |
SensitivityGame
¶
Bases: SearchBasedGame
Game representing the sensitivity of hyperparameters.
Source code in src/hypershap/games/tunability.py
116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 | |
__init__(explanation_task, cs_searcher=None, n_workers=None, verbose=None)
¶
Initialize the sensitivity game.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
explanation_task
|
SensitivityExplanationTask
|
The explanation task containing the configuration space and surrogate model. |
required |
cs_searcher
|
ConfigSpaceSearcher | None
|
The configuration space searcher. If None, a RandomConfigSpaceSearcher is used by default. |
None
|
n_workers
|
int | None
|
The number of worker threads to use for parallel evaluation of coalitions. Defaults to None meaning no parallelization. Using more workers can significantly speed up the computation of Shapley values. The maximum number of workers is capped by the number of coalitions. |
None
|
verbose
|
bool | None
|
A boolean indicating whether to print verbose messages during computation. Defaults to None. When set to True, the method prints debugging information and progress updates. |
None
|
Source code in src/hypershap/games/tunability.py
119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 | |
TunabilityGame
¶
Bases: SearchBasedGame
Game representing the tunability of hyperparameters.
Source code in src/hypershap/games/tunability.py
82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 | |
__init__(explanation_task, cs_searcher=None, n_workers=None, verbose=None)
¶
Initialize the tunability game.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
explanation_task
|
TunabilityExplanationTask
|
The explanation task containing the configuration space and surrogate model. |
required |
cs_searcher
|
ConfigSpaceSearcher | None
|
The configuration space searcher. If None, a RandomConfigSpaceSearcher is used by default. |
None
|
n_workers
|
int | None
|
The number of worker threads to use for parallel evaluation of coalitions. Defaults to None meaning no parallelization. Using more workers can significantly speed up the computation of Shapley values. The maximum number of workers is capped by the number of coalitions. |
None
|
verbose
|
bool | None
|
A boolean indicating whether to print verbose messages during computation. Defaults to None. When set to True, the method prints debugging information and progress updates. |
None
|
Source code in src/hypershap/games/tunability.py
85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 | |