arlbench.autorl.objectives

This module contains the objectives for the AutoRL environment.

Functions

discount_rewards(rewards[, gamma])

Compute discounted rewards.

Classes

DiscountedReward(*args, **kwargs)

Discounted reward objective for the AutoRL environment.

DiscountedTrainReward(*args, **kwargs)

Discounted reward objective for the AutoRL environment.

Emissions(*args, **kwargs)

Emissions objective for the AutoRL environment.

Objective(*args, **kwargs)

An abstract optimization objective for the AutoRL environment.

Reward(*args, **kwargs)

Reward objective for the AutoRL environment.

Runtime(*args, **kwargs)

Runtime objective for the AutoRL environment.

TrainReward(*args, **kwargs)

Reward objective for the AutoRL environment.
class arlbench.autorl.objectives.DiscountedReward(*args, **kwargs)[source]

Bases: Objective

Discounted reward objective for the AutoRL environment. It measures the last discounted evaluation rewards.

static __call__(train_func, objectives, optimize_objectives, gamma=0.99, default_arg='mean')[source]

Wraps the training function with the reward mean calculation.

Return type:

Callable[[DQNRunnerState | PPORunnerState | SACRunnerState, PrioritisedTrajectoryBufferState, int | None, int | None, int | None], tuple[DQNState, DQNTrainingResult] | tuple[PPOState, PPOTrainingResult] | tuple[SACState, SACTrainingResult]]

static get_spec()[source]

Returns a dictionary containing the specification of the objective.

Return type:

dict

class arlbench.autorl.objectives.DiscountedTrainReward(*args, **kwargs)[source]

Bases: Objective

Discounted reward objective for the AutoRL environment. It measures the mean of the last discounted training rewards.

static __call__(train_func, objectives, optimize_objectives, gamma=0.99, default_arg='mean')[source]

Wraps the training function with the reward mean calculation.

Return type:

Callable[[DQNRunnerState | PPORunnerState | SACRunnerState, PrioritisedTrajectoryBufferState, int | None, int | None, int | None], tuple[DQNState, DQNTrainingResult] | tuple[PPOState, PPOTrainingResult] | tuple[SACState, SACTrainingResult]]

static get_spec()[source]

Returns a dictionary containing the specification of the objective.

Return type:

dict

class arlbench.autorl.objectives.Emissions(*args, **kwargs)[source]

Bases: Objective

Emissions objective for the AutoRL environment. It measures the emissions during the training using code carbon.

static __call__(train_func, objectives, optimize_objectives)[source]

Wraps the training function with the emissions calculation.

Return type:

Callable[[DQNRunnerState | PPORunnerState | SACRunnerState, PrioritisedTrajectoryBufferState, int | None, int | None, int | None], tuple[DQNState, DQNTrainingResult] | tuple[PPOState, PPOTrainingResult] | tuple[SACState, SACTrainingResult]]

static get_spec()[source]

Returns a dictionary containing the specification of the objective.

Return type:

dict

class arlbench.autorl.objectives.Objective(*args, **kwargs)[source]

Bases: ABC

An abstract optimization objective for the AutoRL environment.

It can be wrapped around the training function to calculate the objective. We do this be overriding the __new__() function. It allows us to imitate the behaviour of a basic function while keeping the advantages of a static class.

abstract static __call__(train_func, objectives, optimize_objectives)[source]

Wraps the training function with the objective calculation.

Parameters:

  • train_func (TrainFunc) – Training function to wrap.

  • objectives (dict) – Dictionary to store objective.

  • optimize_objectives (str) – Whether to minimize/maximize the objectve.

Returns:

Training function.

Return type:

TrainFunc

__lt__(other)[source]

Implements “less-than” comparison between two objectives. Used for sorting based on objective rank.

Parameters:

other (Objective) – Other Objective to compare to.

Returns:

Whether this Objective is less than the other Objective.

Return type:

bool

static __new__(cls, *args, **kwargs)[source]

Creates a new instance of this objective and directly wraps the train function.

This is done by first creating an object and subsequently calling self.__call__().

Returns:

Wrapped training function.

Return type:

TrainFunc

abstract static get_spec()[source]

Returns a dictionary containing the specification of the objective.

Returns:

Specification.

Return type:

dict

class arlbench.autorl.objectives.Reward(*args, **kwargs)[source]

Bases: Objective

Reward objective for the AutoRL environment. And applies an aggregation function.

static __call__(train_func, objectives, optimize_objectives, default_arg='mean')[source]

Wraps the training function with the reward mean calculation.

Return type:

Callable[[DQNRunnerState | PPORunnerState | SACRunnerState, PrioritisedTrajectoryBufferState, int | None, int | None, int | None], tuple[DQNState, DQNTrainingResult] | tuple[PPOState, PPOTrainingResult] | tuple[SACState, SACTrainingResult]]

static get_spec()[source]

Returns a dictionary containing the specification of the objective.

Return type:

dict

class arlbench.autorl.objectives.Runtime(*args, **kwargs)[source]

Bases: Objective

Runtime objective for the AutoRL environment. It measures the total training runtime.

static __call__(train_func, objectives, optimize_objectives)[source]

Wraps the training function with the runtime calculation.

Return type:

Callable[[DQNRunnerState | PPORunnerState | SACRunnerState, PrioritisedTrajectoryBufferState, int | None, int | None, int | None], tuple[DQNState, DQNTrainingResult] | tuple[PPOState, PPOTrainingResult] | tuple[SACState, SACTrainingResult]]

static get_spec()[source]

Returns a dictionary containing the specification of the objective.

Return type:

dict

class arlbench.autorl.objectives.TrainReward(*args, **kwargs)[source]

Bases: Objective

Reward objective for the AutoRL environment. It measures the mean of the last training rewards.

static __call__(train_func, objectives, optimize_objectives, default_arg='mean')[source]

Wraps the training function with the reward mean calculation.

Return type:

Callable[[DQNRunnerState | PPORunnerState | SACRunnerState, PrioritisedTrajectoryBufferState, int | None, int | None, int | None], tuple[DQNState, DQNTrainingResult] | tuple[PPOState, PPOTrainingResult] | tuple[SACState, SACTrainingResult]]

static get_spec()[source]

Returns a dictionary containing the specification of the objective.

Return type:

dict

arlbench.autorl.objectives.discount_rewards(rewards, gamma=0.99)[source]

Compute discounted rewards.

previous

arlbench.autorl.checkpointing

next

arlbench.autorl.state_features