Local Search

Bases: MetaOptimizer

LocalSearch is a class for conducting local search in Neural Architecture Search (NAS) methods. It selects a random architecture, generates its neighborhood, and moves to a neighbor if it has better performance. If no better neighbors are found, a new random architecture is selected.

Attributes:

Name	Type	Description
using_step_function	bool	Flag indicating the absence of a step function for this optimizer.
config	CfgNode	Configuration settings for the search process.
epochs	int	Number of epochs for the search process.
performance_metric	Metric	The performance metric for evaluating the architectures.
dataset	str	The dataset to be used for evaluation.
num_init	int	Number of initial random architectures.
nbhd	list	A list to store the neighborhood of the current architecture.
chosen	Graph	The currently chosen architecture.
best_arch	Graph	The best architecture found so far.
history	torch.nn.ModuleList	A list to store the history of architectures.
newest_child_idx	int	The index of the most recently added child architecture in the history.

__init__(config)

Initializes the LocalSearch class with configuration settings.

Parameters:

Name	Type	Description	Default
config	CfgNode	Configuration settings for the search process.	required

adapt_search_space(search_space, scope=None, dataset_api=None)

Adapts the search space for the local search.

Parameters:

Name	Type	Description	Default
search_space	Graph	The search space to be adapted.	required
scope	str	The scope for the search. Defaults to None.	None
dataset_api	dict	API for the dataset. Defaults to None.	None

get_checkpointables()

Gets the models that can be checkpointed.

Returns:

Name	Type	Description
dict		A dictionary with "model" as the key and the history of architectures as the value.

get_final_architecture()

Gets the final (best) architecture from the search.

Returns:

Name	Type	Description
Graph		The best architecture found during the search.

get_model_size()

Gets the size of the model.

Returns:

Name	Type	Description
float		The size of the model in megabytes (MB).

get_op_optimizer()

Gets the optimizer for the operations. This method is not implemented in this class and raises an error when called.

Raises:

Type	Description
NotImplementedError	Always, because this method is not implemented in this class.

new_epoch(epoch)

Starts a new epoch in the search process, performing local search on the chosen architecture.

Parameters:

Name	Type	Description	Default
epoch	int	The current epoch number.	required

test_statistics()

Reports the test statistics.

Returns:

Name	Type	Description
float		The raw performance metric for the best architecture.

train_statistics(report_incumbent=True)

Reports the statistics after training.

Parameters:

Name	Type	Description	Default
report_incumbent	bool	Whether to report the incumbent or the most recent architecture. Defaults to True.	True

Returns:

Name	Type	Description
tuple		A tuple containing the training accuracy, validation accuracy, and test accuracy, and training time.