cave.utils.statistical_tests module

cave.utils.statistical_tests.paired_permutation(data1, data2, rng, num_permutations=10000, logger=None)[source]

Test for significance using paired permutation.

Parameters:
  • data2 (data1,) – ordered results for instances for two different configurations
  • rng (np.RandomState) – random number generator
  • num_permutations (int) – number of permutations performed
  • logger (Logger) – logger-instance write debugs to
Returns:

p – p-value for statistical test

Return type:

float

cave.utils.statistical_tests.paired_t_student(data1, data2, logger=None)[source]

Test for significance using paired t-test.

Parameters:
  • data2 (data1,) – ordered results for instances for two different configurations
  • logger (logging.Logger) – to log scores, if given
Returns:

p – p-value for statistical test

Return type:

float