Regularized Evolution

Bases: MetaOptimizer

RegularizedEvolution is a class that implements the Regularized Evolution algorithm for Neural Architecture Search (NAS).

Attributes:

Name	Type	Description
using_step_function	bool	Flag indicating the absence of a step function for this optimizer.
config	CfgNode	Configuration node with settings for the search process.
epochs	int	Number of epochs for the search process.
sample_size	int	The number of architectures to sample for each population.
population_size	int	The maximum size of the population in the evolutionary search.
performance_metric	Metric	The performance metric for evaluating the architectures.
dataset	str	The dataset to be used for evaluation.
population	collections.deque	A queue to hold the population of architectures.
history	torch.nn.ModuleList	A list to store the history of architectures.

__init__(config)

Initializes the Regularized Evolution class with configuration settings.

Parameters:

Name	Type	Description	Default
config	CfgNode	Configuration settings for the search process.	required

adapt_search_space(search_space, scope=None, dataset_api=None, **kwargs)

Adapts the search space for regularized evolution search.

Parameters:

Name	Type	Description	Default
search_space	Graph	The search space to be adapted.	required
scope	str	The scope for the search. Defaults to None.	None
dataset_api	dict	API for the dataset. Defaults to None.	None

get_checkpointables()

Gets the models that can be checkpointed.

Returns:

Name	Type	Description
dict		A dictionary with "model" as the key and the history of architectures as the value.

get_final_architecture()

Gets the final (best) architecture from the search.

Returns:

Name	Type	Description
Graph		The best architecture found during the search.

get_model_size()

Gets the size of the model in terms of the number of parameters.

Returns:

Name	Type	Description
float		The size of the model in MB.

get_op_optimizer()

Gets the optimizer for the operations. This method is not implemented in this class and raises an error when called.

Raises:

Type	Description
NotImplementedError	Always, because this method is not implemented in this class.

new_epoch(epoch)

Starts a new epoch in the search process.

Parameters:

Name	Type	Description	Default
epoch	int	The current epoch number.	required

test_statistics()

Reports the test statistics.

Returns:

Name	Type	Description
float		The raw performance metric for the best architecture.

train_statistics(report_incumbent=True)

Reports the statistics after training.

Parameters:

Name	Type	Description	Default
report_incumbent	bool	Whether to report the incumbent or the most recent architecture. Defaults to True.	True

Returns:

Name	Type	Description
tuple		A tuple containing the training accuracy, validation accuracy, test accuracy, and training time.