weka.classifiers.functions
Class SMO

java.lang.Object
  |
  +--weka.classifiers.Classifier
        |
        +--weka.classifiers.functions.SMO

All Implemented Interfaces:

java.lang.Cloneable, OptionHandler, java.io.Serializable, WeightedInstancesHandler

public class SMO

extends Classifier

implements WeightedInstancesHandler

Implements John C. Platt's sequential minimal optimization algorithm for training a support vector classifier using polynomial or RBF kernels. This implementation globally replaces all missing values and transforms nominal attributes into binary ones. It also normalizes all attributes by default. (Note that the coefficients in the output are based on the normalized/standardized data, not the original data.) Multi-class problems are solved using pairwise classification. To obtain proper probability estimates, use the option that fits logistic regression models to the outputs of the support vector machine. In the multi-class case the predicted probabilities will be coupled using Hastie and Tibshirani's pairwise coupling method. Note: for improved speed standardization should be turned off when operating on SparseInstances.

For more information on the SMO algorithm, see

J. Platt (1998). Fast Training of Support Vector Machines using Sequential Minimal Optimization. Advances in Kernel Methods - Support Vector Learning, B. Schölkopf, C. Burges, and A. Smola, eds., MIT Press.

S.S. Keerthi, S.K. Shevade, C. Bhattacharyya, K.R.K. Murthy, Improvements to Platt's SMO Algorithm for SVM Classifier Design. Neural Computation, 13(3), pp 637-649, 2001.

Valid options are:

-C num
The complexity constant C. (default 1)

-E num
The exponent for the polynomial kernel. (default 1)

-G num
Gamma for the RBF kernel. (default 0.01)

-N <0|1|2>
Whether to 0=normalize/1=standardize/2=neither. (default 0=normalize)

-F
Feature-space normalization (only for non-linear polynomial kernels).

-O
Use lower-order terms (only for non-linear polynomial kernels).

-R
Use the RBF kernel. (default poly)

-A num
Sets the size of the kernel cache. Should be a prime number. (default 1000003)

-T num
Sets the tolerance parameter. (default 1.0e-3)

-P num
Sets the epsilon for round-off error. (default 1.0e-12)

-M
Fit logistic models to SVM outputs.

-V num
Number of folds for cross-validation used to generate data for logistic models. (default -1, use training data) -W num
Random number seed for cross-validation. (default 1)

Version:

$Revision: 1.50 $

Author:

Eibe Frank (eibe@cs.waikato.ac.nz), Shane Legg (shane@intelligenesis.net) (sparse vector code), Stuart Inglis (stuart@reeltwo.com) (sparse vector code)

See Also:

Serialized Form

Field Summary

static int	FILTER_NONE
static int	FILTER_NORMALIZE The filter to apply to the training data
static int	FILTER_STANDARDIZE
static Tag[]	TAGS_FILTER

Constructor Summary

SMO()

Method Summary

java.lang.String[][][]	attributeNames() Returns the attribute names.
double[][]	bias() Returns the bias of each binary SMO.
void	buildClassifier(Instances insts) Method for building the classifier.
java.lang.String	buildLogisticModelsTipText() Returns the tip text for this property
java.lang.String	cacheSizeTipText() Returns the tip text for this property
java.lang.String[]	classAttributeNames()
java.lang.String	cTipText() Returns the tip text for this property
double[]	distributionForInstance(Instance inst) Estimates class probabilities for given instance.
java.lang.String	epsilonTipText() Returns the tip text for this property
java.lang.String	exponentTipText() Returns the tip text for this property
java.lang.String	featureSpaceNormalizationTipText() Returns the tip text for this property
java.lang.String	filterTypeTipText() Returns the tip text for this property
java.lang.String	gammaTipText() Returns the tip text for this property
boolean	getBuildLogisticModels() Get the value of buildLogisticModels.
double	getC() Get the value of C.
int	getCacheSize() Get the size of the kernel cache
double	getEpsilon() Get the value of epsilon.
double	getExponent() Get the value of exponent.
boolean	getFeatureSpaceNormalization() Check whether feature spaces is being normalized.
SelectedTag	getFilterType() Gets how the training data will be transformed.
double	getGamma() Get the value of gamma.
boolean	getLowerOrderTerms() Check whether lower-order terms are being used.
int	getNumFolds() Get the value of numFolds.
java.lang.String[]	getOptions() Gets the current settings of the classifier.
int	getRandomSeed() Get the value of randomSeed.
double	getToleranceParameter() Get the value of tolerance parameter.
boolean	getUseRBF() Check if the RBF kernel is to be used.
java.lang.String	globalInfo() Returns a string describing classifier
java.util.Enumeration	listOptions() Returns an enumeration describing the available options.
java.lang.String	lowerOrderTermsTipText() Returns the tip text for this property
static void	main(java.lang.String[] argv) Main method for testing this class.
int	numClassAttributeValues()
java.lang.String	numFoldsTipText() Returns the tip text for this property
int[]	obtainVotes(Instance inst) Returns an array of votes for the given instance.
double[]	pairwiseCoupling(double[][] n, double[][] r) Implements pairwise coupling.
java.lang.String	randomSeedTipText() Returns the tip text for this property
void	setBuildLogisticModels(boolean newbuildLogisticModels) Set the value of buildLogisticModels.
void	setC(double v) Set the value of C.
void	setCacheSize(int v) Set the value of the kernel cache.
void	setEpsilon(double v) Set the value of epsilon.
void	setExponent(double v) Set the value of exponent.
void	setFeatureSpaceNormalization(boolean v) Set whether feature space is normalized.
void	setFilterType(SelectedTag newType) Sets how the training data will be transformed.
void	setGamma(double v) Set the value of gamma.
void	setLowerOrderTerms(boolean v) Set whether lower-order terms are to be used.
void	setNumFolds(int newnumFolds) Set the value of numFolds.
void	setOptions(java.lang.String[] options) Parses a given list of options.
void	setRandomSeed(int newrandomSeed) Set the value of randomSeed.
void	setToleranceParameter(double v) Set the value of tolerance parameter.
void	setUseRBF(boolean v) Set if the RBF kernel is to be used.
int[][][]	sparseIndices() Returns the indices in sparse format.
double[][][]	sparseWeights() Returns the weights in sparse format.
java.lang.String	toleranceParameterTipText() Returns the tip text for this property
java.lang.String	toString() Prints out the classifier.
void	turnChecksOff() Turns off checks for missing values, etc.
void	turnChecksOn() Turns on checks for missing values, etc.
java.lang.String	useRBFTipText() Returns the tip text for this property

Methods inherited from class weka.classifiers.Classifier

classifyInstance, debugTipText, forName, getDebug, makeCopies, setDebug

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

FILTER_NORMALIZE

public static final int FILTER_NORMALIZE

The filter to apply to the training data

See Also:

Constant Field Values

FILTER_STANDARDIZE

public static final int FILTER_STANDARDIZE

See Also:

Constant Field Values

FILTER_NONE

public static final int FILTER_NONE

See Also:

Constant Field Values

TAGS_FILTER

public static final Tag[] TAGS_FILTER

Constructor Detail

SMO

public SMO()

Method Detail

globalInfo

public java.lang.String globalInfo()

Returns a string describing classifier

Returns:

a description suitable for displaying in the explorer/experimenter gui

turnChecksOff

public void turnChecksOff()

Turns off checks for missing values, etc. Use with caution.

turnChecksOn

public void turnChecksOn()

Turns on checks for missing values, etc.

buildClassifier

public void buildClassifier(Instances insts)
                     throws java.lang.Exception

Method for building the classifier. Implements a one-against-one wrapper for multi-class problems.

Specified by:

buildClassifier in class Classifier

Parameters:

insts - the set of training instances

Throws:

java.lang.Exception - if the classifier can't be built successfully

distributionForInstance

public double[] distributionForInstance(Instance inst)
                                 throws java.lang.Exception

Estimates class probabilities for given instance.

Overrides:

distributionForInstance in class Classifier

Parameters:

inst - the instance to be classified

Returns:

an array containing the estimated membership probabilities of the test instance in each class or the numeric prediction

Throws:

java.lang.Exception - if distribution could not be computed successfully

pairwiseCoupling

public double[] pairwiseCoupling(double[][] n,
                                 double[][] r)

Implements pairwise coupling.

Parameters:

n - the sum of weights used to train each model

r - the probability estimate from each model

Returns:

the coupled estimates

obtainVotes

public int[] obtainVotes(Instance inst)
                  throws java.lang.Exception

Returns an array of votes for the given instance.

Parameters:

inst - the instance

Returns:

array of votex

Throws:

java.lang.Exception - if something goes wrong

sparseWeights

public double[][][] sparseWeights()

Returns the weights in sparse format.

sparseIndices

public int[][][] sparseIndices()

Returns the indices in sparse format.

bias

public double[][] bias()

Returns the bias of each binary SMO.

numClassAttributeValues

public int numClassAttributeValues()

classAttributeNames

public java.lang.String[] classAttributeNames()

attributeNames

public java.lang.String[][][] attributeNames()

Returns the attribute names.

listOptions

public java.util.Enumeration listOptions()

Returns an enumeration describing the available options.

Specified by:

listOptions in interface OptionHandler

Overrides:

listOptions in class Classifier

Returns:

an enumeration of all the available options.

setOptions

public void setOptions(java.lang.String[] options)
                throws java.lang.Exception

Parses a given list of options. Valid options are:

-C num
The complexity constant C. (default 1)

-E num
The exponent for the polynomial kernel. (default 1)

-G num
Gamma for the RBF kernel. (default 0.01)

-N <0|1|2>
Whether to 0=normalize/1=standardize/2=neither. (default 0=normalize)

-F
Feature-space normalization (only for non-linear polynomial kernels).

-O
Use lower-order terms (only for non-linear polynomial kernels).

-R
Use RBF kernel (default poly).

-A num
Sets the size of the kernel cache. Should be a prime number. (default 1000003)

-T num
Sets the tolerance parameter. (default 1.0e-3)

-P num
Sets the epsilon for round-off error. (default 1.0e-12)

-M
Fit logistic models to SVM outputs.

-V num
Number of folds for cross-validation used to generate data for logistic models. (default -1, use training data) -W num
Random number seed. (default 1)

Specified by:

setOptions in interface OptionHandler

Overrides:

setOptions in class Classifier

Parameters:

options - the list of options as an array of strings

Throws:

java.lang.Exception - if an option is not supported

getOptions

public java.lang.String[] getOptions()

Gets the current settings of the classifier.

Specified by:

getOptions in interface OptionHandler

Overrides:

getOptions in class Classifier

Returns:

an array of strings suitable for passing to setOptions

exponentTipText

public java.lang.String exponentTipText()

Returns the tip text for this property

Returns:

tip text for this property suitable for displaying in the explorer/experimenter gui

getExponent

public double getExponent()

Get the value of exponent.

Returns:

Value of exponent.

setExponent

public void setExponent(double v)

Set the value of exponent. If linear kernel is used, rescaling and lower-order terms are turned off.

Parameters:

v - Value to assign to exponent.

gammaTipText

public java.lang.String gammaTipText()

Returns the tip text for this property

Returns:

tip text for this property suitable for displaying in the explorer/experimenter gui

getGamma

public double getGamma()

Get the value of gamma.

Returns:

Value of gamma.

setGamma

public void setGamma(double v)

Set the value of gamma.

Parameters:

v - Value to assign to gamma.

cTipText

public java.lang.String cTipText()

Returns the tip text for this property

Returns:

tip text for this property suitable for displaying in the explorer/experimenter gui

getC

public double getC()

Get the value of C.

Returns:

Value of C.

setC

public void setC(double v)

Set the value of C.

Parameters:

v - Value to assign to C.

toleranceParameterTipText

public java.lang.String toleranceParameterTipText()

Returns the tip text for this property

Returns:

tip text for this property suitable for displaying in the explorer/experimenter gui

getToleranceParameter

public double getToleranceParameter()

Get the value of tolerance parameter.

Returns:

Value of tolerance parameter.

setToleranceParameter

public void setToleranceParameter(double v)

Set the value of tolerance parameter.

Parameters:

v - Value to assign to tolerance parameter.

epsilonTipText

public java.lang.String epsilonTipText()

Returns the tip text for this property

Returns:

tip text for this property suitable for displaying in the explorer/experimenter gui

getEpsilon

public double getEpsilon()

Get the value of epsilon.

Returns:

Value of epsilon.

setEpsilon

public void setEpsilon(double v)

Set the value of epsilon.

Parameters:

v - Value to assign to epsilon.

cacheSizeTipText

public java.lang.String cacheSizeTipText()

Returns the tip text for this property

Returns:

tip text for this property suitable for displaying in the explorer/experimenter gui

getCacheSize

public int getCacheSize()

Get the size of the kernel cache

Returns:

Size of kernel cache.

setCacheSize

public void setCacheSize(int v)

Set the value of the kernel cache.

Parameters:

v - Size of kernel cache.

filterTypeTipText

public java.lang.String filterTypeTipText()

Returns the tip text for this property

Returns:

tip text for this property suitable for displaying in the explorer/experimenter gui

getFilterType

public SelectedTag getFilterType()

Gets how the training data will be transformed. Will be one of FILTER_NORMALIZE, FILTER_STANDARDIZE, FILTER_NONE.

Returns:

the filtering mode

setFilterType

public void setFilterType(SelectedTag newType)

Sets how the training data will be transformed. Should be one of FILTER_NORMALIZE, FILTER_STANDARDIZE, FILTER_NONE.

Parameters:

newType - the new filtering mode

useRBFTipText

public java.lang.String useRBFTipText()

Returns the tip text for this property

Returns:

tip text for this property suitable for displaying in the explorer/experimenter gui

getUseRBF

public boolean getUseRBF()

Check if the RBF kernel is to be used.

Returns:

true if RBF

setUseRBF

public void setUseRBF(boolean v)

Set if the RBF kernel is to be used.

Parameters:

v - true if RBF

featureSpaceNormalizationTipText

public java.lang.String featureSpaceNormalizationTipText()

Returns the tip text for this property

Returns:

tip text for this property suitable for displaying in the explorer/experimenter gui

getFeatureSpaceNormalization

public boolean getFeatureSpaceNormalization()
                                     throws java.lang.Exception

Check whether feature spaces is being normalized.

Returns:

true if feature space is normalized.

java.lang.Exception

setFeatureSpaceNormalization

public void setFeatureSpaceNormalization(boolean v)
                                  throws java.lang.Exception

Set whether feature space is normalized.

Parameters:

v - true if feature space is to be normalized.

java.lang.Exception

lowerOrderTermsTipText

public java.lang.String lowerOrderTermsTipText()

Returns the tip text for this property

Returns:

tip text for this property suitable for displaying in the explorer/experimenter gui

getLowerOrderTerms

public boolean getLowerOrderTerms()

Check whether lower-order terms are being used.

Returns:

Value of lowerOrder.

setLowerOrderTerms

public void setLowerOrderTerms(boolean v)

Set whether lower-order terms are to be used. Defaults to false if a linear machine is built.

Parameters:

v - Value to assign to lowerOrder.

buildLogisticModelsTipText

public java.lang.String buildLogisticModelsTipText()

Returns the tip text for this property

Returns:

tip text for this property suitable for displaying in the explorer/experimenter gui

getBuildLogisticModels

public boolean getBuildLogisticModels()

Get the value of buildLogisticModels.

Returns:

Value of buildLogisticModels.

setBuildLogisticModels

public void setBuildLogisticModels(boolean newbuildLogisticModels)

Set the value of buildLogisticModels.

Parameters:

newbuildLogisticModels - Value to assign to buildLogisticModels.

numFoldsTipText

public java.lang.String numFoldsTipText()

Returns the tip text for this property

Returns:

tip text for this property suitable for displaying in the explorer/experimenter gui

getNumFolds

public int getNumFolds()

Get the value of numFolds.

Returns:

Value of numFolds.

setNumFolds

public void setNumFolds(int newnumFolds)

Set the value of numFolds.

Parameters:

newnumFolds - Value to assign to numFolds.

randomSeedTipText

public java.lang.String randomSeedTipText()

Returns the tip text for this property

Returns:

tip text for this property suitable for displaying in the explorer/experimenter gui

getRandomSeed

public int getRandomSeed()

Get the value of randomSeed.

Returns:

Value of randomSeed.

setRandomSeed

public void setRandomSeed(int newrandomSeed)

Set the value of randomSeed.

Parameters:

newrandomSeed - Value to assign to randomSeed.

toString

public java.lang.String toString()

Prints out the classifier.

Overrides:

toString in class java.lang.Object

Returns:

a description of the classifier as a string

main

public static void main(java.lang.String[] argv)

Main method for testing this class.