Benchmark Overview

DACBench contains a range of benchmarks in different categories and from different domains. There is a range of highly configurable, cheap to run benchmarks that often also include a ground truth solution. We recommend using these as an introduction to DAC, to verify new algorithms and to generate detailed insights. They are both based on artificial functions and real algorithms:

• Function Approximation (Artificial Benchmark): Function approximation in multiple dimensions with importance weighting.

• Luby (Artificial Benchmark): Learning the Luby sequence.

• ToySGD (Artificial Benchmark): Controlling the learning rate in gradient descent.

• Theory benchmark with ground truth: RLS algorithm on the LeadingOnes problem.

Beyond these smaller scale problems we know a lot about, DACBench also contains less interpretable algorithms with larger scopes. These are oftentimes noisier, harder to debug and more costly to run and thus present a real challenge for DAC algorithms:

• CMA-ES: Step-size adpation and algorithm component selection for CMA-ES.

• SGD-DL: Learning rate adaption for neural networks.

Our benchmarks are based on the gymnasium interface for Reinforcement Learning. That means to run a benchmark, you need to create an environment of that benchmark to then interact with it. We include examples of this interaction between environment and DAC methods in our GitHub repository. To instantiate a benchmark environment, run:

from dacbench.benchmarks import FunctionApproximationBenchmark
bench = FunctionApproximationBenchmark()
benchmark_env = bench.get_environment()

Alternatively, if you do not plan on modifying the benchmark configuration, you can also use our the default version in the gymnasium registry:

import gymnasium as gym
import dacbench
environment = gym.make("FunctionApproximation-v0")

Abstract Benchmark.

class dacbench.abstract_benchmark.AbstractBenchmark(config_path=None, config: objdict | None = None)

Bases: ABC

Abstract template for benchmark classes.

__eq__(other)

Check for equality.

classmethod class_to_str()

Get string name from class.

dejson_wrappers(wrapper_list)

Load wrapper from list.

Parameters:

wrapper_list (list) – wrapper description to load

dictify_json(dict_list)

Json to dict structure for gym spaces.

Parameters:

dict_list (list) – list of dicts

classmethod from_json(json_config)

Get config from json dict.

get_config()

Return current configuration.

Returns:

dict: Current config

abstract get_environment()

Make benchmark environment.

Returns:

gym.Env: Benchmark environment

jsonify_dict_space(dict_space)

Gym spaces to json dict.

Parameters:

dict_space (dict) – space dict

jsonify_wrappers()

Write wrapper description to list.

Returns:

list

list_to_space(space_list)

Make gym space from list.

Parameters:

space_list (list) – list to space-ify

load_config(config: objdict)

Load config.

Parameters:

config (objdict) – config to load

read_config_file(path)

Read configuration from file.

Parameters:

path (str) – Path to config file

register_wrapper(wrap_func)

Register wrapper.

Parameters:

wrap_func (function) – wrapper init function

save_config(path)

Write config to path.

serialize_config()

Save configuration to json.

Parameters:

path (str) – File to save config to

set_action_space(kind, args)

Change action space.

Parameters:

• kind (str) – Name of action space class

• args (list) – List of arguments to pass to action space class

set_observation_space(kind, args, data_type)

Change observation_space.

Parameters:

• kind (str) – Name of observation space class

• args (list) – List of arguments to pass to observation space class

• data_type (type) – Data type of observation space

set_seed(seed)

Set environment seed.

Parameters:

seed (int) – New seed

space_to_list(space)

Make list from gym space.

Parameters:

space (gym.spaces.Space) – space to parse

to_json()

Write config to json.

class dacbench.abstract_benchmark.objdict

Bases: dict

Modified dict to make config changes more flexible.

__delattr__(name)

Delete attribute.

__eq__(other)

Check for equality.

__getattr__(name)

Get attribute.

__ne__(other)

Check for inequality.

__setattr__(name, value)

Set attribute.

copy()

Copy self.

Abstract Environment.

class dacbench.abstract_env.AbstractEnv(config)

Bases: ABC, Env

Abstract template for environments.

get_inst_id()

Return instance ID.

Returns:

int: ID of current instance

get_instance()

Return current instance.

Returns:

type flexible: Currently used instance

get_instance_set()

Return instance set.

Returns:

list: List of instances

abstract reset(seed: int | None = None)

Reset environment.

Parameters:

• seed – Seed for the environment

• Returns

• --------

• state – Environment state

• info (dict) – Additional metainfo

reset_(seed=0, options=None, instance=None, instance_id=None, scheme=None)

Pre-reset function for progressing through the instance set. Will either use round robin, random or no progression scheme.

seed(seed=None, seed_action_space=False)

Set rng seed.

Parameters:

• seed – seed for rng

• seed_action_space (bool, default False) – if to seed the action space as well

set_inst_id(inst_id)

Change current instance ID.

Parameters:

inst_id (int) – New instance index

set_instance(instance)

Change currently used instance.

Parameters:

instance – New instance

set_instance_set(inst_set)

Change instance set.

Parameters:

inst_set (list) – New instance set

abstract step(action)

Execute environment step.

Parameters:

• action – Action to take

• Returns

• --------

• state – Environment state

• reward – Environment reward

• terminated (bool) – Run finished flag

• truncated (bool) – Run timed out flag

• info (dict) – Additional metainfo

step_()

Pre-step function for step count and cutoff.

Returns:

bool: End of episode

use_next_instance(instance=None, instance_id=None, scheme=None)

Changes instance according to chosen instance progession.

Parameters:

• instance – Instance specification for potentional new instances

• instance_id – ID of the instance to switch to

• scheme – Update scheme for this progression step (either round robin, random or no progression)

use_test_set()

Change to test instance set.

use_training_set()

Change to training instance set.

class dacbench.abstract_env.AbstractMADACEnv(config)

Bases: AbstractEnv

Multi-Agent version of DAC environment.

property agent_selection

Current agent.

property infos

Current infos per agent.

last()

Get current step data.

Returns:

np.array, float, bool, bool, dict

multi_agent_reset(seed: int | None = None)

Reset env, but don’t return observations.

Parameters:

seed (int) – seed to use

multi_agent_step(action)

Step for a single hyperparameter.

Parameters:

action – the action in the current agent’s dimension

property num_agents

Current number of agents.

register_agent(agent_id)

Add agent.

Parameters:

agent_id (int) – id of the agent to add

remove_agent(agent_id)

Remove agent.

Parameters:

agent_id (int) – id of the agent to remove

property rewards

Current rewards values per agent.

property terminations

Current termination values per agent.

property truncations

Current truncation values per agent.

previous

Using DACBench Containers

next

The Function Approximation Toy Benchmark