JAHS-Bench-201 Leaderboards
The leaderboards as explained in our paper. To add an entry please create a pull request changing docs/leaderboards.md.
Make sure to follow our evaluation protocol and to add a URL to material that explains how to reproduce you results.
Single Objective
Black-box
CIFAR-10
| Rank |
Accuracy ± SE |
Name |
URL |
| 1 |
90.68 ± 0.16 |
random search |
JAHS-Bench-201 |
Colorectal-Histology
| Rank |
Accuracy ± SE |
Name |
URL |
| 1 |
94.76 ± 0.08 |
random search |
JAHS-Bench-201 |
Fashion-MNIST
| Rank |
Accuracy ± SE |
Name |
URL |
| 1 |
95.09 ± 0.05 |
random search |
JAHS-Bench-201 |
Cost-aware
CIFAR-10
| Rank |
Accuracy ± SE |
Name |
URL |
| 1 |
90.68 ± 0.16 |
random search |
JAHS-Bench-201 |
Colorectal-Histology
| Rank |
Accuracy ± SE |
Name |
URL |
| 1 |
94.76 ± 0.08 |
random search |
JAHS-Bench-201 |
Fashion-MNIST
| Rank |
Accuracy ± SE |
Name |
URL |
| 1 |
95.09 ± 0.05 |
random search |
JAHS-Bench-201 |
Multi-fidelity
CIFAR-10
Colorectal-Histology
Fashion-MNIST
Multi multi-fidelity
CIFAR-10
Colorectal-Histology
Fashion-MNIST
Multi Objective
Black-box
CIFAR-10
| Rank |
Hypervolume ± SE |
Name |
URL |
| 1 |
1171.15 ± 2.27 |
random search |
JAHS-Bench-201 |
Colorectal-Histology
| Rank |
Hypervolume ± SE |
Name |
URL |
| 1 |
278.6 ± 2.23 |
random search |
JAHS-Bench-201 |
Fashion-MNIST
| Rank |
Hypervolume ± SE |
Name |
URL |
| 1 |
231.88 ± 3.86 |
random search |
JAHS-Bench-201 |
Cost-aware
CIFAR-10
| Rank |
Hypervolume ± SE |
Name |
URL |
| 1 |
1171.15 ± 2.27 |
random search |
JAHS-Bench-201 |
Colorectal-Histology
| Rank |
Hypervolume ± SE |
Name |
URL |
| 1 |
278.6 ± 2.23 |
random search |
JAHS-Bench-201 |
Fashion-MNIST
| Rank |
Hypervolume ± SE |
Name |
URL |
| 1 |
231.88 ± 3.86 |
random search |
JAHS-Bench-201 |
Multi-fidelity
CIFAR-10
| Rank |
Hypervolume ± SE |
Name |
URL |
| 1 |
1080.26 ± 1.36 |
Hyperband |
JAHS-Bench-201 |
Colorectal-Histology
| Rank |
Hypervolume ± SE |
Name |
URL |
| 1 |
347.52 ± 1.96 |
Hyperband |
JAHS-Bench-201 |
Fashion-MNIST
| Rank |
Hypervolume ± SE |
Name |
URL |
| 1 |
277.52 ± 1.07 |
Hyperband |
JAHS-Bench-201 |
Multi multi-fidelity
CIFAR-10
| Rank |
Hypervolume ± SE |
Name |
URL |
| 1 |
899.92 ± 1.54 |
Hyperband |
JAHS-Bench-201 |
Colorectal-Histology
| Rank |
Hypervolume ± SE |
Name |
URL |
| 1 |
330.76 ± 0.22 |
Hyperband |
JAHS-Bench-201 |
Fashion-MNIST
| Rank |
Hypervolume ± SE |
Name |
URL |
| 1 |
306.36 ± 0.19 |
Hyperband |
JAHS-Bench-201 |