cave.utils.statistical_tests module

cave.utils.statistical_tests.paired_permutation(data1, data2, rng, num_permutations=10000, logger=None)[source]

Test for significance using paired permutation.

Parameters
  • data2 (data1,) – ordered results for instances for two different configurations

  • rng (np.RandomState) – random number generator

  • num_permutations (int) – number of permutations performed

  • logger (Logger) – logger-instance write debugs to

Returns

p – p-value for statistical test

Return type

float

cave.utils.statistical_tests.paired_t_student(data1, data2, logger=None)[source]

Test for significance using paired t-test.

Parameters
  • data2 (data1,) – ordered results for instances for two different configurations

  • logger (logging.Logger) – to log scores, if given

Returns

p – p-value for statistical test

Return type

float