Python API¶
This section includes information for using the pure Python API of bob.learn.linear
.
Summary¶
Classes¶
bob.learn.linear.Machine 
A linear classifier, see C. 
bob.learn.linear.PCATrainer 
Sets a linear machine to perform the Principal Component Analysis (PCA; a.k.a. 
bob.learn.linear.FisherLDATrainer 
Trains a bob.learn.linear.Machine to perform Fisher’s Linear Discriminant Analysis (LDA). 
bob.learn.linear.WCCNTrainer 
Trains a linear machine to perform WithinClass Covariance 
bob.learn.linear.WhiteningTrainer 
Trains a linear bob.learn.linear.Machine to perform Cholesky whitening. 
bob.learn.linear.CGLogRegTrainer 
Trains a linear machine to perform Linear Logistic Regression 
bob.learn.linear.BICMachine 
This machine is designed to classify image difference vectors to be 
bob.learn.linear.BICTrainer 
A trainer for a bob.learn.linear.BICMachine 
bob.learn.linear.GFKMachine ([hdf5]) 
Geodesic flow Kernel (GFK) Machine. 
bob.learn.linear.GFKTrainer ([...]) 
Trains the Geodesic Flow Kernel (GFK) that models the domain shift from a certain source linear subspace to a certain target linear subspaces . 
Functions¶
bob.learn.linear.get_config () 
Returns a string containing the configuration information. 
bob.learn.linear.bic_intra_extra_pairs (...) 
Computes intraclass and extraclass pairs from given training data. 
bob.learn.linear.bic_intra_extra_pairs_between_factors (...) 
Computes intraclass and extraclass pairs from given training data, where only pairs between the first and second factors are considered. 
Reference¶

class
bob.learn.linear.
BICMachine
¶ Bases:
object
This machine is designed to classify image difference vectors to be either intrapersonal or extrapersonal
There are two possible implementations of the BIC:
 ‘The Bayesian Intrapersonal/Extrapersonal Classifier’ from Teixeira [Teixeira2003]. A full projection of the data is performed. No prior for the classes has to be selected.
 ‘Face Detection and Recognition using Maximum Likelihood Classifiers on Gabor Graphs’ from Guenther and Wuertz [Guenther2009].Only mean and variance of the difference vectors are calculated. There is no subspace truncation and no priors.
What kind of machine is used is dependent on the way, this class is trained via the
bob.learn.linear.BICTrainer
.[Teixeira2003] (1, 2, 3) Marcio Luis Teixeira. The Bayesian intrapersonal/extrapersonal classifier, Colorado State University, 2003. [Guenther2009] (1, 2) Manuel Guenther and Rolf P. Wuertz. Face detection and recognition using maximum likelihood classifiers on Gabor graphs, International Journal of Pattern Recognition and Artificial Intelligence, 23(3):433461, 2009. Constructor Documentation:
 bob.learn.linear.BICMachine ([use_DFFS])
 bob.learn.linear.BICMachine (bic)
 bob.learn.linear.BICMachine (hdf5)
Creates a BIC Machine
Parameters:
use_DFFS
: bool[default:False
] Use the Distance From Feature Space measure as described in [Teixeira2003]bic
:bob.learn.linear.BICMachine
Another machine to copyhdf5
:bob.io.base.HDF5File
An HDF5 file open for readingClass Members:

forward
(input) → score¶ Computes the BIC or IEC score for the given input vector, which results of a comparison vector of two (facial) images
The resulting value is returned as a single float value. The score itself is the loglikelihood score of the given input vector belonging to the intrapersonal class.
Note
the
__call__
method is an alias for this oneParameters:
input
: array_like (float, 1D)The input vector, which is the result of comparing to (facial) imagesReturns:
score
: floatThe loglikelihood that the giveninput
belongs to the intrapersonal class

input_size
¶ int <– The expected input dimensionality, readonly

is_similar_to
(other[, r_epsilon][, a_epsilon]) → similar¶ Compares this BICMachine with the
other
one to be approximately the sameThe optional values
r_epsilon
anda_epsilon
refer to the relative and absolute precision, similarly tonumpy.allclose()
.Parameters:
other
:bob.learn.linear.BICMachine
The other BICMachine to compare withr_epsilon
: float[Default:1e5
] The relative precisiona_epsilon
: float[Default:1e8
] The absolute precisionReturns:
similar
: boolTrue
if theother
machine is similar to this one, otherwiseFalse

load
(hdf5) → None¶ Loads the BIC machine from the given HDF5 file
Parameters:
hdf5
:bob.io.base.HDF5File
An HDF5 file opened for reading

save
(hdf5) → None¶ Saves the BIC machine to the given HDF5 file
Parameters:
hdf5
:bob.io.base.HDF5File
An HDF5 file open for writing

use_DFFS
¶ bool <– Use the Distance From Feature Space during forwarding?

class
bob.learn.linear.
BICTrainer
¶ Bases:
object
A trainer for a
bob.learn.linear.BICMachine
It trains either a BIC model (including projection matrix and eigenvalues) [Teixeira2003] or an IEC model (containing mean and variance only) [Guenther2009]. See
bob.learn.linear.BICMachine
for more details.Constructor Documentation:
 bob.learn.linear.BICTrainer ()
 bob.learn.linear.BICTrainer (intra_dim, extra_dim)
Creates a BIC Trainer
There are two ways of creating a BIC trainer. When you specify the
intra_dim
andextra_dim
subspaces, a BIC model will be created, otherwise an IEC model is created.Parameters:
intra_dim
: intThe subspace dimensionality of the intrapersonal classextra_dim
: intThe subspace dimensionality of the extrapersonal classClass Members:

train
(intra_differences, extra_differences[, machine]) → machine¶ Trains the given machine to classify intrapersonal (image) difference vectors vs. extrapersonal ones
The given difference vectors might be the result of any (image) comparison function, e.g., the pixel difference of two images. In any case, all distance vectors must have the same length.
Parameters:
intra_differences
: array_like (float, 2D)The input vectors, which are the result of intrapersonal (facial image) comparisons, in shape(#features, length)
extra_differences
: array_like (float, 2D)The input vectors, which are the result of extrapersonal (facial image) comparisons, in shape(#features, length)
machine
:bob.learn.linear.BICMachine
The machine to be trainedReturns:
machine
:bob.learn.linear.BICMachine
A newly generated and trained BIC machine, where the bob.lear.linear.BICMachine.use_DFFS flag is set toFalse

class
bob.learn.linear.
CGLogRegTrainer
¶ Bases:
object
Trains a linear machine to perform Linear Logistic Regression
The training stage will place the resulting weights (and bias) in a linear machine with a single output dimension. For details about Linear Logistic Regression, please see:
 A comparison of numerical optimizers for logistic regression, T. Minka, (See Microsoft Research paper)
 FoCal, https://sites.google.com/site/nikobrummer/focal
Constructor Documentation:
 bob.learn.linear.CGLogRegTrainer ([prior], [convergence_threshold], [max_iterations], [reg], [mean_std_norm])
 bob.learn.linear.CGLogRegTrainer (other)
Creates a new trainer to perform Linear Logistic Regression
There are two initializers for objects of this class. In the first variant, the user passes the discrete training parameters, including the classes prior, convergence threshold and the maximum number of conjugate gradient (CG) iterations among other parameters. If
mean_std_norm
is set toTrue
, your input data will be mean/standarddeviation normalized and the according values will be set as normalization factors to the resulting machine. The second initialization form copy constructs a new trainer from an existing one.Parameters:
prior
: float[Default:0.5
] The synthetic prior (should be in range )convergence_threshold
: float[Default:1e5
] The convergence threshold for the conjugate gradient algorithmmax_iterations
: int[Default:10000
] The maximum number of iterations for the conjugate gradient algorithmreg
: float[Default:0.
] The regularization factor lambda. If you set this to the value of0.
, then the algorithm will apply no regularization whatsoevermean_std_norm
: bool[Default:False
] Performs mean and standarddeviation normalization (whitening) of the input data before training the (resulting)bob.learn.linear.Machine
. Setting this toTrue
is recommended for large data sets with significant amplitude variations between dimensionsother
:CGLogRegTrainer
If you decide to copy construct from another object of the same type, pass it using this parameterClass Members:

convergence_threshold
¶ float <– The convergence threshold for the conjugate gradient algorithm

max_iterations
¶ int <– The maximum number of iterations for the conjugate gradient algorithm

mean_std_norm
¶ bool <– Perform whitening on input data?
If set to
True
, performs mean and standarddeviation normalization (whitening) of the input data before training the (resulting) Machine. Setting this toTrue
is recommended for large data sets with significant amplitude variations between dimensions

prior
¶ float <– The synthetic prior (should be in range )

reg
¶ float <– The regularization factor lambda
If you set this to the value of
0.
, the algorithm will apply no regularization whatsoever.

train
(negatives, positives[, machine]) → machine¶ Trains a linear machine to perform linear logistic regression
The resulting machine will have the same number of inputs as columns in
negatives
andpositives
and a single output. This method always returns a machine, which will be identical to the one provided (if the user passed one) or a new one allocated internally.Parameters:
negatives, positives
: array_like(2D, float)negatives
andpositives
should be arrays organized in such a way that every row corresponds to a new observation of the phenomena (i.e., a new sample) and every column corresponds to a different featuremachine
:bob.learn.linear.Machine
The user may provide or not a machine that will be set by this method. If provided, the machine should have 1 output and the number of inputs matching the number of columns in the input data arraysReturns:
machine
:bob.learn.linear.Machine
The trained linear machine; identical to themachine
parameter, if given

class
bob.learn.linear.
FisherLDATrainer
¶ Bases:
object
Trains a
bob.learn.linear.Machine
to perform Fisher’s Linear Discriminant Analysis (LDA).LDA finds the projection matrix W that allows us to linearly project the data matrix X to another (sub) space in which the betweenclass and withinclass variances are jointly optimized: the betweenclass variance is maximized while the withclass is minimized. The (inverse) cost function for this criteria can be posed as the following:
where:
the transformation matrix that converts X into the LD spacethe betweenclass scatter; it has dimensions (X.shape[0], X.shape[0]) and is defined as , with equal to the number of classes.the withinclass scatter; it also has dimensions (X.shape[0], X.shape[0]) and is defined as , with equal to the number of classes and a set representing all samples for class .the class k empirical mean, defined asthe overall set empirical mean, defined asNote
A scatter matrix equals the covariance matrix if we remove the division factor.
Because this cost function is convex, you can just find its maximum by solving . This problem can be reformulated as finding the eigenvalues () that solve the following condition:
The respective eigenvectors that correspond to the eigenvalues form W.
Constructor Documentation:
 bob.learn.linear.FisherLDATrainer ([use_pinv, strip_to_rank])
 bob.learn.linear.FisherLDATrainer (other)
Constructs a new FisherLDATrainer
Objects of this class can be initialized in two ways. In the first variant, the user creates a new trainer from discrete flags indicating a couple of optional parameters. If
use_pinv
is set toTrue
, use the pseudoinverse to calculate and then perform eigen value decomposition (using LAPACK’sdgeev
) instead of using (the more numerically stable) LAPACK’sdsyvgd
to solve the generalized symmetricdefinite eigenproblem of the form .Note
Using the pseudoinverse for LDA is only recommended if you cannot make it work using the default method (via
dsyvg
). It is slower and requires more machine memory to store partial values of the pseudoinverse and the dot product .strip_to_rank
specifies how to calculate the final size of the tobetrainedbob.learn.linear.Machine
. The default setting (True
), makes the trainer return only the K1 eigenvalues/vectors limiting the output to the rank of . If you set this value toFalse
, the it returns all eigenvalues/vectors of , including the ones that are supposed to be zero.The second initialization variant allows the user to deep copy an object of the same type creating a new identical object.
Parameters:
use_pinv
: bool[Default:False
] use the pseudoinverse to calculate ?strip_to_rank
: bool[Default:True
] return only the nonzero eigenvalues/vectorsother
:FisherLDATrainer
The trainer to copyconstructClass Members:

output_size
(X) → size¶ Returns the expected size of the output (or the number of eigenvalues returned) given the data
This number could be either (where is number of classes) or the number of columns (features) in
X
, depending on the setting ofstrip_to_rank
. This method should be used to setup linear machines and input vectors prior to feeding them into this trainer.The value of
X
should be a sequence over as many 2D 64bit floating point number arrays as classes in the problem. All arrays will be checked for conformance (identical number of columns). To accomplish this, either prepare a list with all your class observations organized in 2D arrays or pass a 3D array in which the first dimension (depth) contains as many elements as classes you want to discriminate.Parameters:
X
: [array_like(2D, floats)] or array_like(3D, floats)The input data, separated to contain the training data per class in the first dimensionReturns:
size
: intThe number of eigenvectors/values that will be created in a call totrain()
, given the same input dataX

strip_to_rank
¶ bool <– Only return the nonzero eigenvalues/vectors?
If
True
, strip the resulting LDA projection matrix to keep only the eigenvectors with nonzero eigenvalues. Otherwise the full projection matrix is returned.

train
(X[, machine]) → machine, eigen_values¶ Trains a given machine to perform Fisher/LDA discrimination
After this method has been called, an input
machine
(or one allocated internally) will have the eigenvectors of the product, arranged by decreasing energy. Each input data set represents data from a given input class. This method also returns the eigenvalues allowing you to implement your own compression scheme.The user may provide or not an object of type
bob.learn.linear.Machine
that will be set by this method. If provided, machine should have the correct number of inputs and outputs matching, respectively, the number of columns in the input data arraysX
and the output of the methodoutput_size()
.The value of
X
should be a sequence over as many 2D 64bit floating point number arrays as classes in the problem. All arrays will be checked for conformance (identical number of columns). To accomplish this, either prepare a list with all your class observations organized in 2D arrays or pass a 3D array in which the first dimension (depth) contains as many elements as classes you want to discriminate.Note
We set at most
output_size()
eigenvalues and vectors on the passed machine. You can compress the machine output further usingMachine.resize()
if necessary.Parameters:
X
: [array_like(2D, floats)] or array_like(3D, floats)The input data, separated to contain the training data per class in the first dimensionmachine
:bob.learn.linear.Machine
The machine to be trained; this machine will be returned by this functionReturns:
machine
:bob.learn.linear.Machine
The machine that has been trained; if given, identical to themachine
parametereigen_values
: array_like(1D, floats)The eigenvalues of the LDA projection.

use_pinv
¶ bool <– Use the pseudoinverse?
If
True
, use the pseudoinverse to calculate and then perform the eigen value decomposition (using LAPACK’sdgeev
) instead of using (the more numerically stable) LAPACK’sdsyvgd
to solve the generalized symmetricdefinite eigenproblem of the form .

class
bob.learn.linear.
GFKMachine
(hdf5=None)[source]¶ Bases:
object
Geodesic flow Kernel (GFK) Machine.
This is output of the
bob.learn.linear.GFKTrainer

compute_binetcouchy_distance
()[source]¶ Compute the BinetCouchy distance between source () and target () subspaces in a Grassman which is defined as the following:

compute_principal_angles
()[source]¶ Compute the principal angles between source () and target () subspaces in a Grassman which is defined as the following:
,

load
(hdf5)[source]¶ Loads the machine from the given HDF5 file
Parameters
 hdf5:
bob.io.base.HDF5File
 An HDF5 file opened for reading
 hdf5:

save
(hdf5)[source]¶ Saves the machine to the given HDF5 file
Parameters
 hdf5:
bob.io.base.HDF5File
 An HDF5 file opened for writing
 hdf5:


class
bob.learn.linear.
GFKTrainer
(number_of_subspaces=1, subspace_dim_source=0.99, subspace_dim_target=0.99, eps=1e20)[source]¶ Bases:
object
Trains the Geodesic Flow Kernel (GFK) that models the domain shift from a certain source linear subspace to a certain target linear subspaces .
GFK models the source domain and the target domain with ddimensional linear subspaces and embeds them onto a Grassmann manifold. Specifically, let denote the basis of the PCA subspaces for each of the two domains, respectively. The Grassmann manifold is the collection of all ddimensional subspaces of the feature vector space .
The geodesic flow between on the manifold parameterizes a path connecting the two subspaces. In the beginning of the flow, the subspace is similar to that of the source domain and in the end of the flow, the subspace is similar to that of the target. The original feature is projected into these subspaces and forms a feature vector of infinite dimensions:
.
Using the new feature representation for learning, will force the classifiers to NOT lean towards either the source domain or the target domain, or in other words, will force the classifier to use domaininvariant features. The infinitedimensional feature vector is handled conveniently by their inner product that gives rise to a positive semidefinite kernel defined on the original features,
.
The matrix G can be computed efficiently using singular value decomposition. Moreover, computing the kernel does not require any labeled data.
More details can be found in:
Gong, Boqing, et al. “Geodesic flow kernel for unsupervised domain adaptation.” Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on. IEEE, 2012.
A very good intuition can be found in: http://wwwscf.usc.edu/~boqinggo/domainadaptation.html#gfk_section
Constructor Documentation:
 bob.learn.linear.GFKTrainer (number_of_subspaces, subspace_dim_source, subspace_dim_target, eps)
Parameters
 number_of_subspaces: int
 Number of subspaces for the transfer learning. If set to 1, this value will be estimated automatically. For more information check, Section 3.4.
 subspace_dim_source: float
 Energy kept in the source linear subspace
 subspace_dim_target: float
 Energy kept in the target linear subspace
 eps: float
 Floor value

get_best_d
(Ps, Pt, Pst)[source]¶ Get the best value for the number of subspaces
For more details, read section 3.4 of the paper.
 Parameters
Ps: Source subspace
Pt: Target subspace
Pst: Source + Target subspace

train
(source_data, target_data, norm_inputs=True)[source]¶ Trains the GFK (
bob.learn.linear.GFKMachine
) Parameters
 source_data:
numpy.array()
 Data from the source domain
 target_data:
numpy.array()
 Data from the target domain
 source_data:
Returns
machine:bob.learn.linear.GFKMachine

class
bob.learn.linear.
Machine
¶ Bases:
object
A linear classifier, see C. M. Bishop, ‘Pattern Recognition and Machine Learning’, chapter 4 for more details.
The basic matrix operation performed for projecting the input to the output is: (with being the vector of machine weights and the input data vector). The weights matrix is therefore organized columnwise. In this scheme, each column of the weights matrix can be interpreted as vector to which the input is projected. The number of columns of the weights matrix determines the number of outputs this linear machine will have. The number of rows is the number of allowed inputs it can process.
Input and output is always performed on 1D arrays with 64bit floating point numbers.
Constructor Documentation:
 bob.learn.linear.Machine ([input_size], [output_size]))
 bob.learn.linear.Machine (weights)
 bob.learn.linear.Machine (config)
 bob.learn.linear.Machine (other)
Creates a new linear machine
A linear machine can be constructed in different ways. In the first form, the user specifies optional input and output vector sizes. The machine is remains uninitialized. With the second form, the user passes a 2D array with 64bit floats containing weight matrix to be used as the
weights
matrix by the new machine. In the third form the user passes abob.io.base.HDF5File
opened for reading, which points to the machine information to be loaded in memory. Finally, in the last form (copy constructor), the user passes anotherbob.learn.linear.Machine
that will be deep copied.Parameters:
input_size
: int[Default: 0] The dimensionality of the input data that should be projectedoutput_size
: int[Default: 0] The dimensionality of the output dataweights
: array_like(2D, float)A weight matrix to initialize theweights
config
:bob.io.base.HDF5File
The HDF5 file open for readingother
:bob.learn.linear.Machine
The machine to copy constructClass Members:

activation
¶ bob.learn.activation.Activation
or one of its derivatives <– The activation functionBy default, the activation function is the
bob.learn.activation.Identity
function.

biases
¶ array_like(1D, float) <– Bias to the output units of this linear machine
These values will be added to the output before the
activation
is applied. Must have the same size asshape
[1]

forward
(input[, output]) → output¶ Projects
input
through its internal weights and biasesThe
input
(andoutput
) arrays can be either 1D or 2D 64bit float arrays. If one provides a 1D array, theoutput
array, if provided, should also be 1D, matching the output size of this machine. If one provides a 2D array, it is considered a set of vertically stacked 1D arrays (one input per row) and a 2D array is produced or expected inoutput
. Theoutput
array in this case shall have the same number of rows as theinput
array and as many columns as the output size for this machine.Note
The
__call__
method is an alias for this method.Parameters:
input
: array_like(1D or 2D, float)The array that should be projected; must be compatible withshape
[0]output
: array_like(1D or 2D, float)The output array that will be filled. If given, must be compatible withinput
andshape
[1]Returns:
output
: array_like(1D or 2D, float)The projected data; identical to theoutput
parameter, if given

input_divide
¶ array_like(1D, float) <– Input division factor
These data will be divided by
input_divide
before feeding it through theweights
matrix. The division is applied just after subtraction. Must have the same size asshape
[0]. By default, it is set to 1.

input_subtract
¶ array_like(1D, float) <– Input subtraction factor
These values will be subtracted before feeding data through the
weights
matrix. Must have the same size asshape
[0]. By default, it is set to 0.

is_similar_to
(other[, r_epsilon][, a_epsilon]) → similar¶ Compares this LinearMachine with the
other
one to be approximately the sameThe optional values
r_epsilon
anda_epsilon
refer to the relative and absolute precision for theweights
,biases
and any other values internal to this machine.Parameters:
other
:bob.learn.linear.Machine
The other machine to compare withr_epsilon
: float[Default:1e5
] The relative precisiona_epsilon
: float[Default:1e8
] The absolute precisionReturns:
similar
: boolTrue
if theother
machine is similar to this one, otherwiseFalse

load
(hdf5) → None¶ Loads the machine from the given HDF5 file
Parameters:
hdf5
:bob.io.base.HDF5File
An HDF5 file opened for reading

resize
(input, output) → None¶ Resizes the machine
If either the input or output increases in size, the weights and other factors should be considered uninitialized. If the size is preserved or reduced, already initialized values will not be changed.
Note
Use this method to force data compression. All will work out given most relevant factors to be preserved are organized on the top of the weight matrix. In this way, reducing the system size will suppress less relevant projections.
Parameters:
input
: intThe input dimension to be setoutput
: intThe output dimension to be set

save
(hdf5) → None¶ Saves the machine to the given HDF5 file
Parameters:
hdf5
:bob.io.base.HDF5File
An HDF5 file open for writing

shape
¶ (int, int) <– The size of the
weights
matrixA tuple that represents the size of the input vector followed by the size of the output vector in the format
(input, output)
.

weights
¶ array_like(2D, float) <– Weight matrix to which the input is projected to
The output of the projection is fed subject to bias and activation before being output

class
bob.learn.linear.
PCATrainer
¶ Bases:
object
Sets a linear machine to perform the Principal Component Analysis (PCA; a.k.a. KarhunenLoeve Transform – KLT) on a given dataset using either Singular Value Decomposition (SVD, the default) or the Covariance Matrix Method
The training stage will place the resulting principal components in the linear machine and set it up to extract the variable means automatically. As an option, you may preset the trainer so that the normalization performed by the resulting linear machine also divides the variables by the standard deviation of each variable ensemble. The principal components correspond the direction of the data in which its points are maximally spread.
Computing these principal components is equivalent to computing the eigenvectors for the covariance matrix extracted from the data matrix . The covariance matrix for the data is computed using the equation below:
where is the number of rows in (that is, the number of samples).
Once you are in possession of , it suffices to compute the eigenvectors , solving the linear equation:
In this trainer, we make use of LAPACK’s
dsyevd
to solve the above equation, if you choose to use the Covariance Method for extracting the principal components of your data matrix .By default though, this class will perform PC extraction using Singular Value Decomposition (SVD). SVD is a factorization technique that allows for the decomposition of a matrix , with size (m,n) into 3 other matrices in this way:
where:
unitary matrix of size (m,m)  a.k.a., left singular vectors ofrectangular diagonal matrix with nonnegative real numbers, size (m,n)(the conjugate transpose of ) unitary matrix of size (n,n), a.k.a. right singular vectors ofWe can use this property to avoid the computation of the covariance matrix of the data matrix , if we note the following:
If has zero mean, we can conclude by inspection that the matrix obtained by SVD contains the eigenvectors of the covariance matrix of () and corresponds to its eigen values.
Note
Our implementation uses LAPACK’s
dgesdd
to compute the solution to this linear equation.The corresponding
bob.learn.linear.Machine
and returned eigenvalues of , are presorted in descending order (the first eigenvector  or column  of the weight matrix in thebob.learn.linear.Machine
corresponds to the highest eigenvalue obtained).Note
One question you should pose yourself is which of the methods to choose. Here is some advice: you should prefer the covariance method over SVD when the number of samples (rows of ) is greater than the number of features (columns of ). It provides a faster execution path in that case. Otherwise, use the default SVD method.
References:
 Eigenfaces for Recognition, Turk & Pentland, Journal of Cognitive Neuroscience (1991) Volume: 3, Issue: 1, Publisher: MIT Press, Pages: 7186
 http://en.wikipedia.org/wiki/Singular_value_decomposition
 http://en.wikipedia.org/wiki/Principal_component_analysis
 http://www.netlib.org/lapack/double/dsyevd.f
 http://www.netlib.org/lapack/double/dgesdd.f
Constructor Documentation:
 bob.learn.linear.PCATrainer ([use_svd])
 bob.learn.linear.PCATrainer (other)
Constructs a new PCA trainer
There are two initializers for objects of this class. In the first variant, the user can pass a flag indicating if the trainer should use SVD (default) or the covariance method for PCA extraction. The second initialization form copy constructs a new trainer from an existing one.
Parameters:
use_svd
: bool[Default:True
] Use SVD for computing the PCA?other
:PCATrainer
The trainer to copyconstructClass Members:

output_size
(X) → size¶ Calculates the maximum possible rank for the covariance matrix of the given
X
Returns the maximum number of nonzero eigen values that can be generated by this trainer, given
X
. This number (K) depends on the size of X and is calculated as follows , with being the number of rows indata
(samples) and the number of columns (or features).This method should be used to setup linear machines and input vectors prior to feeding them into the
train()
function.Parameters:
X
: array_like(2D, floats)The input data that should be trained onReturns:
size
: intThe number of eigenvectors/values that will be created in a call totrain()
, given the same input dataX

safe_svd
¶ bool <– Use the safe LAPACK SVD function?
If the
use_svd
flag is enabled, this flag will indicate which LAPACK SVD function to use (dgesvd
if set toTrue
,dgesdd
otherwise). By default, this flag is set toFalse
upon construction, which makes this trainer use the fastest possible SVD decomposition.

train
(X[, machine]) → machine, eigen_values¶ Trains a linear machine to perform the PCA (aka. KLT)
The resulting machine will have the same number of inputs as columns in
X
and eigenvectors, where , with being the number of rows inX
(samples) and the number of columns (or features). The vectors are arranged by decreasing eigenvalue automatically – there is no need to sort the results.The user may provide or not an object of type
bob.learn.linear.Machine
that will be set by this method. If provided, machine should have the correct number of inputs and outputs matching, respectively, the number of columns in the input data arrayX
and the output of the methodoutput_size()
.The input data matrix
X
should correspond to a 64bit floating point array organized in such a way that every row corresponds to a new observation of the phenomena (i.e., a new sample) and every column corresponds to a different feature.This method returns a tuple consisting of the trained machine and a 1D 64bit floating point array containing the eigenvalues calculated while computing the KLT. The eigenvalue ordering matches that of eigenvectors set in the machine.
Parameters:
X
: array_like(2D, floats)The input data to train onmachine
:bob.learn.linear.Machine
The machine to be trained; this machine will be returned by this functionReturns:
machine
:bob.learn.linear.Machine
The machine that has been trained; if given, identical to themachine
parametereigen_values
: array_like(1D, floats)The eigenvalues of the PCA projection.

use_svd
¶ bool <– Use the SVD to compute PCA?
This flag determines if this trainer will use the SVD method (set it to
True
) to calculate the principal components or the Covariance method (set it toFalse
).

class
bob.learn.linear.
WCCNTrainer
¶ Bases:
object
Trains a linear machine to perform WithinClass Covariance Normalization (WCCN)
WCCN finds the projection matrix W that allows us to linearly project the data matrix X to another (sub) space such that:
where is an upper triangular matrix computed using Cholesky Decomposition:
where:
the number of classesthe withinclass scatter; it also has dimensions(X.shape[0], X.shape[0])
and is defined as , with being a set representing all samples for class k.the class k empirical mean, defined asReferences:
 Withinclass covariance normalization for SVMbased speaker recognition, Andrew O. Hatch, Sachin Kajarekar, and Andreas Stolcke, In INTERSPEECH, 2006.
 http://en.wikipedia.org/wiki/Cholesky_decomposition
Constructor Documentation:
 bob.learn.linear.WCCNTrainer ()
 bob.learn.linear.WCCNTrainer (other)
Constructs a new trainer to train a linear machine to perform WCCN
Parameters:
other
:WCCNTrainer
Another WCCN trainer to copyClass Members:

train
(X[, machine]) → machine¶ Trains a linear machine using WCCN
The value of
X
should be a sequence over as many 2D 64bit floating point number arrays as classes in the problem. All arrays will be checked for conformance (identical number of columns). To accomplish this, either prepare a list with all your class observations organized in 2D arrays or pass a 3D array in which the first dimension (depth) contains as many elements as classes you want to train for.The resulting machine will have the same number of inputs and outputs as columns in any of
X
‘s matrices.The user may provide or not an object of type
bob.learn.linear.Machine
that will be set by this method. In such a case, the machine should have a shape that matches(X.shape[1], X.shape[1])
. If the user does not provide a machine to be set, then a new one will be allocated internally. In both cases, the resulting machine is always returned.Parameters:
X
: [array_like(2D,float)] or array_like(3D, float)The training data arranged by classmachine
:bob.learn.linear.Machine
A preallocated machine to be trained; may be omittedReturns:
machine
:bob.learn.linear.Machine
The trained machine; identical to themachine
parameter, if specified

class
bob.learn.linear.
WhiteningTrainer
¶ Bases:
object
Trains a linear
bob.learn.linear.Machine
to perform Cholesky whitening.The whitening transformation is a decorrelation method that converts the covariance matrix of a set of samples into the identity matrix . This effectively linearly transforms random variables such that the resulting variables are uncorrelated and have the same variances as the original random variables. This transformation is invertible. The method is called the whitening transform because it transforms the input matrix closer towards white noise (let’s call it ):
with:
where is the projection matrix that allows us to linearly project the data matrix to another (sub) space such that:
is computed using Cholesky decomposition:
References:
 https://rtmath.net/help/html/e9c12dc0e8134ca9aaa382340f1c5d24.htm
 http://en.wikipedia.org/wiki/Cholesky_decomposition
Constructor Documentation:
 bob.learn.linear.WhiteningTrainer ()
 bob.learn.linear.WhiteningTrainer (other)
Constructs a new whitening trainer
Parameters:
other
:WhiteningTrainer
Another whitening trainer to copyClass Members:

train
(X[, machine]) → machine¶ Trains a linear machine to perform Cholesky whitening
The user may provide or not an object of type
bob.learn.linear.Machine
that will be set by this method. In such a case, the machine should have a shape that matches(X.shape[1], X.shape[1])
. If the user does not provide a machine to be set, then a new one will be allocated internally. In both cases, the resulting machine is always returned by this method.The input data matrix should correspond to a 64bit floating point 2D array organized in such a way that every row corresponds to a new observation of the phenomena (i.e., a new sample) and every column corresponds to a different feature.
Parameters:
X
: array_like(2D, float)The training datamachine
:bob.learn.linear.Machine
A preallocated machine to be trained; may be omittedReturns:
machine
:bob.learn.linear.Machine
The trained machine; identical to themachine
parameter, if specified

bob.learn.linear.
bic_intra_extra_pairs
(training_data) → intra_pairs, extra_pairs[source]¶ Computes intraclass and extraclass pairs from given training data.
The
training_data
should be aligned in a list of sublists, where each sublist contains the data of one class. This function will return two lists of tuples of data, where the first list contains tuples of the same class, while the second list contains tuples of different classes. These tuples can be used to compute difference vectors, which then can be fed into theBICTrainer.train()
method.Note
In general, many more
extra_pairs
thanintra_pairs
are returned.Warning
This function actually returns a two lists of pairs of references to the given data. Even for relatively low numbers of classes and elements per class, the returned lists may contain billions of pairs, which require huge amounts of memory.
Keyword parameters
 training_data : [[object]]
 The training data, where the data for each class are enclosed in one list.
Return values
 intra_pairs : [(object, object)]
 A list of tuples of data, where both data belong to the same class, where each data element is a reference to one element of the given
training_data
.  extra_pairs : [(object, object)]
 A list of tuples of data, where both data belong to different classes, where each data element is a reference to one element of the given
training_data
.

bob.learn.linear.
bic_intra_extra_pairs_between_factors
(first_factor, second_factor) → intra_pairs, extra_pairs[source]¶ Computes intraclass and extraclass pairs from given training data, where only pairs between the first and second factors are considered.
Both
first_factor
andsecond_factor
should be aligned in a list of sublists, where corresponding sublist contains the data of one class. Both lists need to contain the same classes in the same order; empty classes (empty lists) are allowed. This function will return two lists of tuples of data, where the first list contains tuples of the same class, while the second list contains tuples of different classes. These tuples can be used to compute difference vectors, which then can be fed into theBICTrainer.train()
method.Note
In general, many more
extra_pairs
thanintra_pairs
are returned.Warning
This function actually returns a two lists of pairs of references to the given data. Even for relatively low numbers of classes and elements per class, the returned lists may contain billions of pairs, which require huge amounts of memory.
Keyword parameters
 first_factor : [[object]]
 The training data for the first factor, where the data for each class are enclosed in one list.
 second_factor : [[object]]
 The training data for the second factor, where the data for each class are enclosed in one list.
Must have the same size as
first_factor
.
Return values
 intra_pairs : [(array_like, array_like)]
 A list of tuples of data, where both data belong to the same class, but different factors.
 extra_pairs : [(array_like, array_like)]
 A list of tuples of data, where both data belong to different classes and different factors.