autoweka

Class Experiment

java.lang.Object

autoweka.XmlSerializable

autoweka.Experiment

public class Experiment
extends XmlSerializable

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	Experiment.TrajectoryPointExtra Class capturing any of the extra dataset partitions that should be run on all the trajectory points for analysis

Field Summary

Fields

Modifier and Type	Field and Description
java.util.List<java.lang.String>	allowedClassifiers Forces Auto-WEKA to only use the list of classifiers here - if it is empty, then Auto-WEKA will try to use everything that it can
boolean	attributeSelection Boolean indicating if Attribute/Feature selection should happen
float	attributeSelectionTimeout How many seconds should be spent performing attribute selection?
java.util.List<java.lang.String>	callString The list of command line arguments that are used to invoke the SMBO method - this is set by the ExperimentConstructor
java.lang.String	datasetString The string that indicates what dataset is going to be used
java.util.List<java.lang.String>	envVariables A list of var=values that are set in environment variables before the SMBO method is executed - these are set by the ExperimentConstructor
java.lang.String	extraPropsString Any extra properties that are associated with the experiment, these are generally for the constructor or your wrappers
java.lang.String	instanceGenerator The class name of the instance generator
java.lang.String	instanceGeneratorArgs The property string that contains the construction arguments for the instance generator
java.lang.String	memory The string passed to the Xmx argument of a sub process limiting the RAM that WEKA will have
java.lang.String	name The name of the experiment, designed to be human readable and contain useful ID information
java.lang.String	resultMetric The name of the result metric to use - see ClassiferResult.Metric
float	trainTimeout How many seconds should be spent training a set of hyperparameters on a specific partition of the training and test data?.
java.lang.String	trajectoryParserClassName The name of the class that's going to be used to parse the trajectory - this is set by the ExperimentConstructor
java.util.List<Experiment.TrajectoryPointExtra>	trajectoryPointExtras For analysis experiements, these extra classifier evaluations will be done for every point along the trajectory.
float	tunerTimeout How many seconds should be spent for this experiment overall? (IE the SMBO method's budget)
java.lang.String	type A helper variable that indicates the type of the experiment - this is set by the ExperimentConstructor

Constructor Summary

Constructors

Constructor and Description
Experiment()

Method Summary

Modifier and Type	Method and Description
static Experiment	createFromFolder(java.io.File folder)
static Experiment	fromXML(java.io.InputStream xml) Static method to load up an experiment from XML.
static Experiment	fromXML(java.lang.String filename) Static method to load up an experiment from XML.
static void	main(java.lang.String[] args) Main method to actually run an experiment with a given seed.
void	validate() Throws a runtime exception if the experiment contains some sort of crazy values.

Methods inherited from class autoweka.XmlSerializable

fromXML, fromXML, populateObjectFromCMDParams, populateObjectFromCMDParams, toXML, toXML

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

name

public java.lang.String name

The name of the experiment, designed to be human readable and contain useful ID information

type

public java.lang.String type

A helper variable that indicates the type of the experiment - this is set by the ExperimentConstructor

resultMetric

public java.lang.String resultMetric

The name of the result metric to use - see ClassiferResult.Metric

instanceGenerator

public java.lang.String instanceGenerator

The class name of the instance generator

instanceGeneratorArgs

public java.lang.String instanceGeneratorArgs

The property string that contains the construction arguments for the instance generator

datasetString

public java.lang.String datasetString

The string that indicates what dataset is going to be used

attributeSelection

public boolean attributeSelection

Boolean indicating if Attribute/Feature selection should happen

attributeSelectionTimeout

public float attributeSelectionTimeout

How many seconds should be spent performing attribute selection?

trajectoryParserClassName

public java.lang.String trajectoryParserClassName

The name of the class that's going to be used to parse the trajectory - this is set by the ExperimentConstructor

callString

public java.util.List<java.lang.String> callString

The list of command line arguments that are used to invoke the SMBO method - this is set by the ExperimentConstructor

envVariables

public java.util.List<java.lang.String> envVariables

A list of var=values that are set in environment variables before the SMBO method is executed - these are set by the ExperimentConstructor

tunerTimeout

public float tunerTimeout

How many seconds should be spent for this experiment overall? (IE the SMBO method's budget)

trainTimeout

public float trainTimeout

How many seconds should be spent training a set of hyperparameters on a specific partition of the training and test data?. Note that this is more of a guideline, Auto-WEKA has quite a bit of slack built into to accomodate classifiers that can return partial results, or ones that decide to offload training onto their evaluation phase.

memory

public java.lang.String memory

The string passed to the Xmx argument of a sub process limiting the RAM that WEKA will have

extraPropsString

public java.lang.String extraPropsString

Any extra properties that are associated with the experiment, these are generally for the constructor or your wrappers

allowedClassifiers

public java.util.List<java.lang.String> allowedClassifiers

Forces Auto-WEKA to only use the list of classifiers here - if it is empty, then Auto-WEKA will try to use everything that it can

trajectoryPointExtras

public java.util.List<Experiment.TrajectoryPointExtra> trajectoryPointExtras

For analysis experiements, these extra classifier evaluations will be done for every point along the trajectory. For example, if we want to look at the Testing performance all the way along, we would add a new TrajectoryPointExtra that contains an instance string of 'default', since that causes Auto-WEKA to preserve the provided training/test split

Constructor Detail

Experiment

public Experiment()

Method Detail

validate

public void validate()

Throws a runtime exception if the experiment contains some sort of crazy values.

fromXML

public static Experiment fromXML(java.lang.String filename)

Static method to load up an experiment from XML.

Parameters:

filename - The name of the file.

Returns:

The experiment.

fromXML

public static Experiment fromXML(java.io.InputStream xml)

Static method to load up an experiment from XML.

Parameters:

xml - The input stream.

Returns:

The experiment.

main

public static void main(java.lang.String[] args)

Main method to actually run an experiment with a given seed. This requires two command line arguments, the path to the experiment folder (but not the actual .experiment itself) and the seed on which you want to run this experiment. The seed is passed to the SMBO method to allow for easy paralellization of each SMBO method.

Parameters:

args - Any arguments.

createFromFolder

public static Experiment createFromFolder(java.io.File folder)