Transbench101

Bases: Graph

Implementation of the transbench 101 search space.

This class creates a neural network architecture represented as a graph. It is designed for the Transbench 101 benchmark, providing an interface for search space and task-specific configuration.

Attributes:

Name	Type	Description
OPTIMIZER_SCOPE	List[str]	A list of stages to be considered by the optimizer.
QUERYABLE	bool	Boolean indicating whether the class is queryable or not.

__init__(dataset='jigsaw', use_small_model=True, create_graph=False, n_classes=10, in_channels=3)

Initialize the TransBench101SearchSpaceMicro class.

Parameters:

Name	Type	Description	Default
dataset	str	The dataset name. Default is 'jigsaw'.	'jigsaw'
use_small_model	bool	Use a small model if True. Default is True.	True
create_graph	bool	Whether to create a graph upon initialization. Default is False.	False
n_classes	int	Number of classes. Default is 10.	10
in_channels	int	Number of input channels. Default is 3.	3

encode(encoding_type='adjacency_one_hot')

Encode the current architecture.

This function encodes the current architecture based on the specified encoding type.

Parameters:

Name	Type	Description	Default
encoding_type	str	Type of encoding to perform. Defaults to "adjacency_one_hot".	'adjacency_one_hot'

Returns:

Name	Type	Description
Various		Encoded representation of the architecture.

encode_spec(encoding_type='adjacency_one_hot')

Encode the architecture based on the specified encoding type.

This function encodes the architecture for the 'TransBench101SearchSpaceMicro' specifically based on the encoding type provided.

Parameters:

Name	Type	Description	Default
encoding_type	str	Type of encoding to perform. Defaults to "adjacency_one_hot".	'adjacency_one_hot'

Returns:

Name	Type	Description
Various		Encoded representation of the architecture.

Raises:

Type	Description
NotImplementedError	If the encoding type is not supported for the current search space.

forward_before_global_avg_pool(x)

Forward pass method before the Global Average Pooling operation.

This function determines which forward method to call based on the dataset and graph creation status and returns the output accordingly.

Parameters:

Name	Type	Description	Default
x	torch.Tensor	Input tensor to forward propagate.	required

Returns:

Type	Description
	torch.Tensor: Output tensor before the Global Average Pooling operation.

Raises:

Type	Description
Exception	If the method is not implemented for the current dataset and NASLib graph settings.

get_arch_iterator(dataset_api=None)

Get an iterator for all possible architectures.

This function returns an iterator that generates all possible combinations of operation indices.

Parameters:

Name	Type	Description	Default
dataset_api	dict	API for querying dataset-related info. Defaults to None.	None

Returns:

Name	Type	Description
iterator		An iterator over all possible architectures.

get_hash()

Get a hashable representation of the current architecture.

This function returns a tuple of operation indices, which serves as a unique identifier for the architecture.

Returns:

Name	Type	Description
tuple		Tuple of operation indices.

get_loss_fn()

Get the appropriate loss function based on the dataset.

This function returns the loss function that should be used for training based on the dataset.

Returns:

Name	Type	Description
function		Loss function suitable for the dataset.

get_nbhd(dataset_api=None)

Get all neighbors of the current architecture.

This function returns a list of all possible neighbor architectures by varying a single operation index on each edge.

Parameters:

Name	Type	Description	Default
dataset_api		API for querying dataset-related info. Defaults to None.	None

Returns:

Name	Type	Description
list		List of neighboring architectures.

get_op_indices()

Gets the operation indices of the graph.

Returns:

Name	Type	Description
list		List of operation indices.

Raises:

Type	Description
NotImplementedError	If neither op_indices nor the model is set.

get_type()

Get the type of search space.

This function returns a string representing the type of search space.

Returns:

Name	Type	Description
str		Type of the search space, in this case, 'transbench101_micro'.

mutate(parent, dataset_api=None)

Mutate a single operation index from the parent's architecture.

This function performs a mutation operation on the parent's architecture by randomly selecting one edge and changing its operation index. The mutated architecture is then updated in the NASlib object.

Parameters:

Name	Type	Description	Default
parent		The parent architecture object.	required
dataset_api		API for querying dataset-related info. Defaults to None.	None

Returns:

Name	Type	Description
None		Updates the object in place.

query(metric=None, dataset=None, path=None, epoch=-1, full_lc=False, dataset_api=None)

Queries results from transbench 101 based on the specified metric, dataset, and other parameters.

Parameters:

Name	Type	Description	Default
metric	Metric	The metric to query.	None
dataset	str	The dataset to query for.	None
path	str	The path to load the results from.	None
epoch	int	The epoch number to query for. Defaults to -1.	-1
full_lc	bool	Flag to retrieve the full learning curve. Defaults to False.	False
dataset_api	dict	The dataset API to use for querying.	None

Returns:

Name	Type	Description
Any		The queried result based on the metric and dataset.

Raises:

Type	Description
NotImplementedError	If querying for Metric.ALL or if the dataset API is not passed.

sample_random_architecture(dataset_api=None, load_labeled=False)

Sample a random valid architecture.

This function samples a random architecture and ensures its validity before updating the object accordingly.

Parameters:

Name	Type	Description	Default
dataset_api	dict	API for querying dataset-related info. Defaults to None.	None
load_labeled	bool	Whether to load a labeled architecture. Defaults to False.	False

Returns:

Name	Type	Description
None		Updates the object in place.

sample_random_labeled_architecture()

Sample a random labeled architecture.

This function samples a random architecture from the list of labeled architectures, and updates the object accordingly.

Returns:

Name	Type	Description
None		Updates the object in place.

set_op_indices(op_indices)

Set the operation indices and update the architecture accordingly.

This function updates the operation indices and the edges in the NASlib object based on the given op_indices.

Parameters:

Name	Type	Description	Default
op_indices	list	List of operation indices to set.	required

Returns:

Name	Type	Description
None		Updates the object in place.

set_spec(op_indices, dataset_api=None)

Unify the setter for operation indices across different search spaces.

This function simply calls set_op_indices to set the operation indices. Used for code consistency across different search spaces.

Parameters:

Name	Type	Description	Default
op_indices	list	List of operation indices to set.	required
dataset_api		API for querying dataset-related info. Defaults to None.	None

Returns:

Name	Type	Description
None		Updates the object in place.

Bases: Graph

Implementation of the TransBench 101 search space, with an interface to the tabular benchmark of TransBench 101.

Attributes:

Name	Type	Description
OPTIMIZER_SCOPE	List[str]	Defines the optimization scope.
QUERYABLE	bool	Defines if the class object is queryable.

__init__(dataset='jigsaw', *arg, **kwargs)

Initializes the TransBench101SearchSpaceMacro class.

Parameters:

Name	Type	Description	Default
dataset	str	Dataset used in the search space. Defaults to 'jigsaw'.	'jigsaw'
*arg		Variable length arguments.	()
**kwargs		Arbitrary keyword arguments.	{}

encode(encoding_type=EncodingType.ADJACENCY_ONE_HOT)

Encode the architecture based on the specified encoding type.

Parameters:

Name	Type	Description	Default
encoding_type	EncodingType	The type of encoding to use. Defaults to EncodingType.ADJACENCY_ONE_HOT.	EncodingType.ADJACENCY_ONE_HOT

Returns:

Type	Description
	The encoded architecture.

forward_before_global_avg_pool(x)

Execute a forward pass until either the layer before global average pooling or the layer before the last convolutional layer.

Parameters:

Name	Type	Description	Default
x	torch.Tensor	Input tensor.	required

Returns:

Type	Description
	torch.Tensor: Output tensor before the specified layer.

get_hash()

Get a tuple hash of the operation indices.

Returns:

Name	Type	Description
Tuple		Tuple containing the operation indices.

get_loss_fn()

Get the appropriate loss function based on the dataset attribute.

Returns:

Type	Description
	A PyTorch loss function.

get_nbhd(dataset_api=None)

Get all neighbors of the architecture.

Parameters:

Name	Type	Description	Default
dataset_api		API for the dataset.	None

Returns:

Type	Description
	List of neighbor architectures, each wrapped in a PyTorch module.

get_op_indices()

Get the operation indices.

Returns:

Name	Type	Description
Any		The operation indices.

Raises:

Type	Description
ValueError	If op_indices is not set.

get_type()

Get the type of the search space.

Returns:

Name	Type	Description
str		The type of the search space, in this case, 'transbench101_macro'.

mutate(parent, dataset_api=None)

Mutate one operation from the parent's operation indices and update the naslib object.

Parameters:

Name	Type	Description	Default
parent	TransBench101SearchSpaceMacro	Parent architecture from which to mutate.	required
dataset_api		API for the dataset.	None

Returns:

Name	Type	Description
None		Updates the object in place with the mutated operations.

query(metric=None, dataset=None, path=None, epoch=-1, full_lc=False, dataset_api=None)

Query results from TransBench 101.

Parameters:

Name	Type	Description	Default
metric	Metric	Type of metric to be queried.	None
dataset	str	Dataset to query.	None
path	str	Path for query.	None
epoch	int	Epoch number. Defaults to -1.	-1
full_lc	bool	Flag for full learning curve. Defaults to False.	False
dataset_api	dict	API for the dataset. Must be provided.	None

Returns:

Name	Type	Description
Any		Query result based on the metric and other optional parameters.

Raises:

Type	Description
NotImplementedError	For unsupported metrics or missing dataset API.

sample_random_architecture(dataset_api=None, load_labeled=False)

Sample a random architecture and update the edges in the naslib object accordingly.

Parameters:

Name	Type	Description	Default
dataset_api		API for the dataset.	None
load_labeled		Whether to load a labeled architecture. Defaults to False.	False

Returns:

Type	Description
	A sampled architecture if load_labeled is True, otherwise updates the object in place.

sample_random_labeled_architecture()

Sample a random labeled architecture.

Returns:

Type	Description
	A sampled architecture.

Raises:

Type	Description
AssertionError	If labeled architectures are not provided.

set_op_indices(op_indices)

Set the operation indices and update the edges in the naslib object accordingly.

Parameters:

Name	Type	Description	Default
op_indices		The new operation indices.	required

set_spec(op_indices, dataset_api=None)

Set the specifications for the search space.

Parameters:

Name	Type	Description	Default
op_indices		The new operation indices.	required
dataset_api		API for the dataset.	None