cave.utils.statistical_tests module

cave.utils.statistical_tests.paired_permutation(data1, data2, rng, num_permutations=10000, logger=None)

Test for significance using paired permutation.

Parameters

• data2 (data1,) – ordered results for instances for two different configurations

• rng (np.RandomState) – random number generator

• num_permutations (int) – number of permutations performed

• logger (Logger) – logger-instance write debugs to

Returns

p – p-value for statistical test

Return type

float

cave.utils.statistical_tests.paired_t_student(data1, data2, logger=None)

Test for significance using paired t-test.

Parameters

• data2 (data1,) – ordered results for instances for two different configurations

• logger (logging.Logger) – to log scores, if given

Returns

p – p-value for statistical test

Return type

float