arlbench.autorl.objectives

This module contains the objectives for the AutoRL environment.

Classes

Emissions(*args, **kwargs)	Emissions objective for the AutoRL environment.
Objective(*args, **kwargs)	An abstract optimization objective for the AutoRL environment.
RewardMean(*args, **kwargs)	Reward objective for the AutoRL environment.
RewardStd(*args, **kwargs)	Reward objective for the AutoRL environment.
Runtime(*args, **kwargs)	Runtime objective for the AutoRL environment.

class arlbench.autorl.objectives.Emissions(*args, **kwargs)

Bases: Objective

Emissions objective for the AutoRL environment. It measures the emissions during the training using code carbon.

static __call__(train_func, objectives, optimize_objectives)

Wraps the training function with the emissions calculation.

Return type:

Callable[[DQNRunnerState | PPORunnerState | SACRunnerState, PrioritisedTrajectoryBufferState, int | None, int | None, int | None], tuple[DQNState, DQNTrainingResult] | tuple[PPOState, PPOTrainingResult] | tuple[SACState, SACTrainingResult]]

static get_spec()

Returns a dictionary containing the specification of the objective.

Return type:

dict

class arlbench.autorl.objectives.Objective(*args, **kwargs)

Bases: ABC

An abstract optimization objective for the AutoRL environment.

It can be wrapped around the training function to calculate the objective. We do this be overriding the __new__() function. It allows us to imitate the behaviour of a basic function while keeping the advantages of a static class.

abstract static __call__(train_func, objectives, optimize_objectives)

Wraps the training function with the objective calculation.

Parameters:

• train_func (TrainFunc) – Training function to wrap.

• objectives (dict) – Dictionary to store objective.

• optimize_objectives (str) – Whether to minimize/maximize the objectve.

Returns:

Training function.

Return type:

TrainFunc

__lt__(other)

Implements “less-than” comparison between two objectives. Used for sorting based on objective rank.

Parameters:

other (Objective) – Other Objective to compare to.

Returns:

Whether this Objective is less than the other Objective.

Return type:

bool

static __new__(cls, *args, **kwargs)

Creates a new instance of this objective and directly wraps the train function.

This is done by first creating an object and subsequently calling self.__call__().

Returns:

Wrapped training function.

Return type:

TrainFunc

abstract static get_spec()

Returns a dictionary containing the specification of the objective.

Returns:

Specification.

Return type:

dict

class arlbench.autorl.objectives.RewardMean(*args, **kwargs)

Bases: Objective

Reward objective for the AutoRL environment. It measures the mean of the last evaluation rewards.

static __call__(train_func, objectives, optimize_objectives)

Wraps the training function with the reward mean calculation.

Return type:

Callable[[DQNRunnerState | PPORunnerState | SACRunnerState, PrioritisedTrajectoryBufferState, int | None, int | None, int | None], tuple[DQNState, DQNTrainingResult] | tuple[PPOState, PPOTrainingResult] | tuple[SACState, SACTrainingResult]]

static get_spec()

Returns a dictionary containing the specification of the objective.

Return type:

dict

class arlbench.autorl.objectives.RewardStd(*args, **kwargs)

Bases: Objective

Reward objective for the AutoRL environment. It measures the standard deviation of the last evaluation rewards.

static __call__(train_func, objectives, optimize_objectives)

Wraps the training function with the reward standard deviation calculation.

Return type:

Callable[[DQNRunnerState | PPORunnerState | SACRunnerState, PrioritisedTrajectoryBufferState, int | None, int | None, int | None], tuple[DQNState, DQNTrainingResult] | tuple[PPOState, PPOTrainingResult] | tuple[SACState, SACTrainingResult]]

static get_spec()

Returns a dictionary containing the specification of the objective.

Return type:

dict

class arlbench.autorl.objectives.Runtime(*args, **kwargs)

Bases: Objective

Runtime objective for the AutoRL environment. It measures the total training runtime.

static __call__(train_func, objectives, optimize_objectives)

Wraps the training function with the runtime calculation.

Return type:

Callable[[DQNRunnerState | PPORunnerState | SACRunnerState, PrioritisedTrajectoryBufferState, int | None, int | None, int | None], tuple[DQNState, DQNTrainingResult] | tuple[PPOState, PPOTrainingResult] | tuple[SACState, SACTrainingResult]]

static get_spec()

Returns a dictionary containing the specification of the objective.

Return type:

dict

previous

arlbench.autorl.checkpointing

next

arlbench.autorl.state_features