arlbench package

Subpackages

• arlbench.autorl package
  • Submodules
  • arlbench.autorl.autorl_env module
    • AutoRLEnv
      • AutoRLEnv.ALGORITHMS
      • AutoRLEnv.action_space
      • AutoRLEnv.checkpoints
      • AutoRLEnv.config
      • AutoRLEnv.config_space
      • AutoRLEnv.eval()
      • AutoRLEnv.get_algorithm_init_kwargs()
      • AutoRLEnv.hpo_config
      • AutoRLEnv.objectives
      • AutoRLEnv.observation_space
      • AutoRLEnv.reset()
      • AutoRLEnv.step()
  • arlbench.autorl.checkpointing module
    • Checkpointer
      • Checkpointer.MRP_FILE
      • Checkpointer.NODES_FILE
      • Checkpointer.SCALARS_FILE
      • Checkpointer.load()
      • Checkpointer.load_buffer()
      • Checkpointer.save()
      • Checkpointer.save_buffer()
  • arlbench.autorl.objectives module
    • DiscountedReward
      • DiscountedReward.KEY
      • DiscountedReward.RANK
      • DiscountedReward.__call__()
      • DiscountedReward.get_spec()
    • DiscountedTrainReward
      • DiscountedTrainReward.KEY
      • DiscountedTrainReward.RANK
      • DiscountedTrainReward.__call__()
      • DiscountedTrainReward.get_spec()
    • Emissions
      • Emissions.KEY
      • Emissions.RANK
      • Emissions.__call__()
      • Emissions.get_spec()
    • Objective
      • Objective.KEY
      • Objective.RANK
      • Objective.__call__()
      • Objective.__lt__()
      • Objective.__new__()
      • Objective.get_spec()
    • Reward
      • Reward.KEY
      • Reward.RANK
      • Reward.__call__()
      • Reward.get_spec()
    • Runtime
      • Runtime.KEY
      • Runtime.RANK
      • Runtime.__call__()
      • Runtime.get_spec()
    • TrainReward
      • TrainReward.KEY
      • TrainReward.RANK
      • TrainReward.__call__()
      • TrainReward.get_spec()
    • discount_rewards()
  • arlbench.autorl.state_features module
    • GradInfo
      • GradInfo.KEY
      • GradInfo.__call__()
      • GradInfo.get_state_space()
    • LossInfo
      • LossInfo.KEY
      • LossInfo.__call__()
      • LossInfo.get_state_space()
    • PredictionInfo
      • PredictionInfo.KEY
      • PredictionInfo.__call__()
      • PredictionInfo.get_state_space()
    • StateFeature
      • StateFeature.KEY
      • StateFeature.__call__()
      • StateFeature.__new__()
      • StateFeature.get_state_space()
    • WeightInfo
      • WeightInfo.KEY
      • WeightInfo.__call__()
      • WeightInfo.get_state_space()
  • Module contents
    • AutoRLEnv
      • AutoRLEnv.ALGORITHMS
      • AutoRLEnv.action_space
      • AutoRLEnv.checkpoints
      • AutoRLEnv.config
      • AutoRLEnv.config_space
      • AutoRLEnv.eval()
      • AutoRLEnv.get_algorithm_init_kwargs()
      • AutoRLEnv.hpo_config
      • AutoRLEnv.objectives
      • AutoRLEnv.observation_space
      • AutoRLEnv.reset()
      • AutoRLEnv.step()
• arlbench.core package
  • Subpackages
    • arlbench.core.algorithms package
      • Subpackages
      • Submodules
      • arlbench.core.algorithms.algorithm module
      • arlbench.core.algorithms.buffers module
      • arlbench.core.algorithms.common module
      • arlbench.core.algorithms.prioritised_item_buffer module
      • Module contents
    • arlbench.core.environments package
      • Submodules
      • arlbench.core.environments.autorl_env module
      • arlbench.core.environments.brax_env module
      • arlbench.core.environments.envpool_env module
      • arlbench.core.environments.gymnasium_env module
      • arlbench.core.environments.gymnax_env module
      • arlbench.core.environments.make_env module
      • arlbench.core.environments.xland_env module
      • Module contents
    • arlbench.core.wrappers package
      • Submodules
      • arlbench.core.wrappers.flatten_observation module
      • arlbench.core.wrappers.wrapper module
      • Module contents
  • Submodules
  • arlbench.core.running_statistics module
    • Array
      • Array.dtype
      • Array.shape
    • NestedMeanStd
      • NestedMeanStd.mean
      • NestedMeanStd.replace()
      • NestedMeanStd.std
    • RunningStatisticsState
      • RunningStatisticsState.count
      • RunningStatisticsState.replace()
      • RunningStatisticsState.summed_variance
    • denormalize()
    • init_state()
    • normalize()
    • update()
  • Module contents
• arlbench.utils package
  • Submodules
  • arlbench.utils.common module
    • config_space_to_gymnasium_space()
    • config_space_to_yaml()
    • gymnasium_space_to_gymnax_space()
    • recursive_concat()
    • save_defaults_to_yaml()
    • tuple_concat()
  • Module contents
    • config_space_to_gymnasium_space()
    • config_space_to_yaml()
    • gymnasium_space_to_gymnax_space()
    • recursive_concat()
    • save_defaults_to_yaml()
    • tuple_concat()

Submodules

arlbench.arlbench module

This module provides a function to run ARLBench using a given config.

arlbench.arlbench.run_arlbench(cfg, logger=None)

Run ARLBench using the given config and return objective(s).

Return type:

float | tuple | list

Module contents

Top-level package for ARLBench.

class arlbench.AutoRLEnv(config=None)

Bases: Env

Automated Reinforcement Learning (gynmasium-like) Environment.

With each reset, the algorithm state is (re-)initialized. If a checkpoint path is passed to reset, the agent state is initialized with the checkpointed state.

In each step, one iteration of training is performed with the current hyperparameter configuration (= action).

ALGORITHMS = {'dqn': <class 'arlbench.core.algorithms.dqn.dqn.DQN'>, 'ppo': <class 'arlbench.core.algorithms.ppo.ppo.PPO'>, 'sac': <class 'arlbench.core.algorithms.sac.sac.SAC'>}

property action_space: Space

Returns the hyperparameter configuration spaces as gymnasium space.

Returns:

Hyperparameter configuration space.

Return type:

gymnasium.spaces.Space

property checkpoints: list[str]

Returns a list of created checkpoints for this AutoRL environment.

Returns:

List of checkpoint paths.

Return type:

list[str]

property config: dict

Returns the AutoRL configuration.

Returns:

AutoRL configuration.

Return type:

dict

property config_space: ConfigurationSpace

Returns the hyperparameter configuration spaces as ConfigSpace.

Returns:

Hyperparameter configuration space.

Return type:

ConfigurationSpace

eval(num_eval_episodes)

Evaluates the algorithm using its current training state.

Parameters:

num_eval_episodes (int) – Number of evaluation episodes to run.

Returns:

Array of evaluation return for each episode.

Return type:

np.ndarray

get_algorithm_init_kwargs(init_rng)

Returns the algorithm initialization parameters.

Returns:

Dictionary of algorithm initialization parameters.

Return type:

Dict

property hpo_config: Configuration

Returns the current hyperparameter configuration stored in the AutoRL environment..

Returns:

Hyperparameter configuration.

Return type:

Configuration

property objectives: list[str]

Returns configured objectives.

Returns:

List of objectives.

Return type:

list[str]

property observation_space: Space

Returns a gymnasium spaces of state features (observations).

Returns:

Gynasium space.

Return type:

gymnasium.spaces.Space

reset()

Resets the AutoRL environment and current algorithm state.

Returns:

Empty observation and state information.

Return type:

tuple[ObservationT, InfoT]

step(action, checkpoint_path=None, n_total_timesteps=None, n_eval_steps=None, n_eval_episodes=None, seed=None)

Performs one iteration of RL training.

Parameters:

• action (Configuration | dict) – Hyperparameter configuration to use for training.

• n_total_timesteps (int | None, optional) – Number of total training steps. Defaults to None.

• n_eval_steps (int | None, optional) – Number of evaluations during training. Defaults to None.

• n_eval_episodes (int | None, optional) – Number of episodes to run per evalution during training. Defaults to None.

• seed (int | None, optional) – Random seed. Defaults to None. If None, seed of the AutoRL environment is used.

Raises:

ValueError – Error is raised if step() is called before reset() was called.

Returns:

State information, objectives, terminated, truncated, additional information.

Return type:

tuple[ObservationT, ObjectivesT, bool, bool, InfoT]

arlbench.run_arlbench(cfg, logger=None)

Run ARLBench using the given config and return objective(s).

Return type:

float | tuple | list