Documentation>C API
Functions
svm.c File Reference

Support Vector Machines (SVM) - Implementation. More...

`#include "svm.h"`
`#include "mathop.h"`
`#include <string.h>`

## Functions

VlSvm * | vl_svm_new (VlSvmSolverType type, double const *data, vl_size dimension, vl_size numData, double const *labels, double lambda)
| Create a new object with plain data. More...
VlSvm * | vl_svm_new_with_dataset (VlSvmSolverType solver, VlSvmDataset *dataset, double const *labels, double lambda)
| Create a new object with a dataset. More...
VlSvm * | vl_svm_new_with_abstract_data (VlSvmSolverType solver, void *data, vl_size dimension, vl_size numData, double const *labels, double lambda)
| Create a new object with abstract data. More...
void | vl_svm_delete (VlSvm *self)
| Delete object. More...
void | vl_svm_set_epsilon (VlSvm *self, double epsilon)
| Set the convergence threshold. More...
double | vl_svm_get_epsilon (VlSvm const *self)
| Get the convergence threshold. More...
void | vl_svm_set_bias_learning_rate (VlSvm *self, double rate)
| Set the bias learning rate. More...
double | vl_svm_get_bias_learning_rate (VlSvm const *self)
| Get the bias leraning rate. More...
void | vl_svm_set_bias_multiplier (VlSvm *self, double b)
| Set the bias multiplier. More...
double | vl_svm_get_bias_multiplier (VlSvm const *self)
| Get the bias multiplier. More...
void | vl_svm_set_iteration_number (VlSvm *self, vl_uindex n)
| Set the current iteratio number. More...
vl_size | vl_svm_get_iteration_number (VlSvm const *self)
| Get the current iteration number. More...
void | vl_svm_set_max_num_iterations (VlSvm *self, vl_size n)
| Set the maximum number of iterations. More...
vl_size | vl_svm_get_max_num_iterations (VlSvm const *self)
| Get the maximum number of iterations. More...
void | vl_svm_set_diagnostic_frequency (VlSvm *self, vl_size f)
| Set the diagnostic frequency. More...
vl_size | vl_svm_get_diagnostic_frequency (VlSvm const *self)
| Get the diagnostic frequency. More...
VlSvmSolverType | vl_svm_get_solver (VlSvm const *self)
| Get the SVM solver type. More...
void | vl_svm_set_lambda (VlSvm *self, double lambda)
| Set the regularizer parameter lambda. More...
double | vl_svm_get_lambda (VlSvm const *self)
| Get the regularizer parameter lambda. More...
void | vl_svm_set_weights (VlSvm *self, double const *weights)
| Set the data weights. More...
double const * | vl_svm_get_weights (VlSvm const *self)
| Get the data weights. More...
vl_size | vl_svm_get_dimension (VlSvm *self)
| Get the model dimenison. More...
vl_size | vl_svm_get_num_data (VlSvm *self)
| Get the number of data samples. More...
double const * | vl_svm_get_model (VlSvm const *self)
| Get the SVM model. More...
void | vl_svm_set_model (VlSvm *self, double const *model)
| Set the SVM model. More...
void | vl_svm_set_bias (VlSvm *self, double b)
| Set the SVM bias. More...
double | vl_svm_get_bias (VlSvm const *self)
| Get the value of the bias. More...
VlSvmStatistics const * | vl_svm_get_statistics (VlSvm const *self)
| Get the solver statistics. More...
double const * | vl_svm_get_scores (VlSvm const *self)
| Get the scores of the data points. More...
void | vl_svm_set_diagnostic_function (VlSvm *self, VlSvmDiagnosticFunction f, void *data)
| Set the diagnostic function callback. More...
void | vl_svm_set_data_functions (VlSvm *self, VlSvmInnerProductFunction inner, VlSvmAccumulateFunction acc)
| Set the data functions. More...
void | vl_svm_set_loss_function (VlSvm *self, VlSvmLossFunction f)
| Set the loss function callback. More...
void | vl_svm_set_loss_derivative_function (VlSvm *self, VlSvmLossFunction f)
| Set the loss derivative function callback. More...
void | vl_svm_set_conjugate_loss_function (VlSvm *self, VlSvmLossFunction f)
| Set the conjugate loss function callback. More...
void | vl_svm_set_dca_update_function (VlSvm *self, VlSvmDcaUpdateFunction f)
| Set the DCA update function callback. More...
void | vl_svm_set_loss (VlSvm *self, VlSvmLossType loss)
| Set the loss function to one of the default types. More...
double | vl_svm_hinge_loss (double inner, double label)
| SVM hinge loss. More...
double | vl_svm_hinge_loss_derivative (double inner, double label)
| SVM hinge loss derivative. More...
double | vl_svm_hinge_conjugate_loss (double u, double label)
| SVM hinge loss conjugate. More...
double | vl_svm_hinge_dca_update (double alpha, double inner, double norm2, double label)
| SVM hinge loss DCA update. More...
double | vl_svm_hinge2_loss (double inner, double label)
| SVM square hinge loss. More...
double | vl_svm_hinge2_loss_derivative (double inner, double label)
| SVM square hinge loss derivative. More...
double | vl_svm_hinge2_conjugate_loss (double u, double label)
| SVM square hinge loss conjugate. More...
double | vl_svm_hinge2_dca_update (double alpha, double inner, double norm2, double label)
| SVM square hinge loss DCA update. More...
double | vl_svm_l1_loss (double inner, double label)
| SVM l1 loss. More...
double | vl_svm_l1_loss_derivative (double inner, double label)
| SVM l1 loss derivative. More...
double | vl_svm_l1_conjugate_loss (double u, double label)
| SVM l1 loss conjugate. More...
double | vl_svm_l1_dca_update (double alpha, double inner, double norm2, double label)
| SVM l1 loss DCA update. More...
double | vl_svm_l2_loss (double inner, double label)
| SVM l2 loss. More...
double | vl_svm_l2_loss_derivative (double inner, double label)
| SVM l2 loss derivative. More...
double | vl_svm_l2_conjugate_loss (double u, double label)
| SVM l2 loss conjugate. More...
double | vl_svm_l2_dca_update (double alpha, double inner, double norm2, double label)
| SVM l2 loss DCA update. More...
double | vl_svm_logistic_loss (double inner, double label)
| SVM l2 loss. More...
double | vl_svm_logistic_loss_derivative (double inner, double label)
| SVM l2 loss derivative. More...
double | vl_svm_logistic_conjugate_loss (double u, double label)
| SVM l2 loss conjugate. More...
double | vl_svm_logistic_dca_update (double alpha, double inner, double norm2, double label)
| SVM l2 loss DCA update. More...
void | _vl_svm_update_statistics (VlSvm *self)
| Update SVM statistics. More...
void | vl_svm_train (VlSvm *self)
| Run the SVM solver. More...

## Detailed Description

Author
  Milan Sulc
  Daniele Perrone
  Andrea Vedaldi

## Function Documentation

## ◆ _vl_svm_update_statistics()

void _vl_svm_update_statistics | ( | VlSvm * | self | ) |

Parameters

  self | object.

## ◆ vl_svm_delete()

void vl_svm_delete | ( | VlSvm * | self | ) |

Parameters

  self | object.

See also
  vl_svm_new

## ◆ vl_svm_get_bias()

double vl_svm_get_bias | ( | VlSvm const * | self | ) |

Parameters

  self | object.

Returns
  bias \(b\).

The value of the bias returned already include the effect of bias mutliplier.

## ◆ vl_svm_get_bias_learning_rate()

double vl_svm_get_bias_learning_rate | ( | VlSvm const * | self | ) |

Parameters

  self | object

Returns
  bias learning rate.

## ◆ vl_svm_get_bias_multiplier()

double vl_svm_get_bias_multiplier | ( | VlSvm const * | self | ) |

Parameters

  self | object.

Returns
  bias multiplier.

## ◆ vl_svm_get_diagnostic_frequency()

vl_size vl_svm_get_diagnostic_frequency | ( | VlSvm const * | self | ) |

Parameters

  self | object.

Returns
  diagnostic frequency.

## ◆ vl_svm_get_dimension()

vl_size vl_svm_get_dimension | ( | VlSvm * | self | ) |

Parameters

  self | object.

Returns
  model dimension.

This is the dimensionality of the weight vector \(\bw\).

## ◆ vl_svm_get_epsilon()

double vl_svm_get_epsilon | ( | VlSvm const * | self | ) |

Parameters

  self | object

Returns
  epsilon threshold.

## ◆ vl_svm_get_iteration_number()

vl_size vl_svm_get_iteration_number | ( | VlSvm const * | self | ) |

Parameters

  self | object.

Returns
  current iteration number.

## ◆ vl_svm_get_lambda()

double vl_svm_get_lambda | ( | VlSvm const * | self | ) |

Parameters

  self | object.

Returns
  diagnostic frequency.

## ◆ vl_svm_get_max_num_iterations()

vl_size vl_svm_get_max_num_iterations | ( | VlSvm const * | self | ) |

Parameters

  self | object.

Returns
  maximum number of iterations.

## ◆ vl_svm_get_model()

double const* vl_svm_get_model | ( | VlSvm const * | self | ) |

Parameters

  self | object.

Returns
  model.

This is the weight vector \(\bw\).

## ◆ vl_svm_get_num_data()

vl_size vl_svm_get_num_data | ( | VlSvm * | self | ) |

Parameters

  self | object.

Returns
  model number of data samples

This is the dimensionality of the weight vector \(\bw\).

## ◆ vl_svm_get_scores()

double const* vl_svm_get_scores | ( | VlSvm const * | self | ) |

Parameters

  self | object.

Returns
  vector of scores.

After training or during the diagnostic callback, this function can be used to retrieve the scores of the points, i.e. \(\langle \bx_i, \bw \rangle + b\).

## ◆ vl_svm_get_solver()

VlSvmSolverType vl_svm_get_solver | ( | VlSvm const * | self | ) |

Parameters

  self | object.

Returns
  SVM solver type.

## ◆ vl_svm_get_statistics()

VlSvmStatistics const* vl_svm_get_statistics | ( | VlSvm const * | self | ) |

Parameters

  self | object.

Returns
  statistics.

## ◆ vl_svm_get_weights()

double const* vl_svm_get_weights | ( | VlSvm const * | self | ) |

Parameters

  self | object.

Returns
  data weights.

## ◆ vl_svm_hinge2_conjugate_loss()

double vl_svm_hinge2_conjugate_loss | ( | double | u,
|  | double | label
| ) |  |

Parameters

  u | dual variable.
  label | label value.

Returns
  conjugate loss.

## ◆ vl_svm_hinge2_dca_update()

double vl_svm_hinge2_dca_update | ( | double | alpha,
|  | double | inner,
|  | double | norm2,
|  | double | label
| ) |  |

Parameters

  alpha | current value of the dual variable.
  inner | inner product \(\bw^\top \bx\) of the sample with the SVM model.
  norm2 | normalization factor \(\|\bx\|^2/\lambda n\).
  label | label \(y\) of the sample.

Returns
  incremental update \(\Delta\alpha\) of the dual variable.

See also
  Stochastic Dual Coordinate Ascent

## ◆ vl_svm_hinge2_loss()

double vl_svm_hinge2_loss | ( | double | inner,
|  | double | label
| ) |  |

Parameters

  inner | inner product between sample and model \(\bw^\top \bx\).
  label | sample label \(y\).

Returns
  value of the loss.

The interface is the same for a loss function, its derivative, or the conjugate loss.

See also
  SVM fundamentals

## ◆ vl_svm_hinge2_loss_derivative()

double vl_svm_hinge2_loss_derivative | ( | double | inner,
|  | double | label
| ) |  |

Parameters

  inner | inner product between sample and model \(\bw^\top \bx\).
  label | sample label \(y\).

Returns
  value of the loss.

The interface is the same for a loss function, its derivative, or the conjugate loss.

See also
  SVM fundamentals

## ◆ vl_svm_hinge_conjugate_loss()

double vl_svm_hinge_conjugate_loss | ( | double | u,
|  | double | label
| ) |  |

Parameters

  u | dual variable.
  label | label value.

Returns
  conjugate loss.

## ◆ vl_svm_hinge_dca_update()

double vl_svm_hinge_dca_update | ( | double | alpha,
|  | double | inner,
|  | double | norm2,
|  | double | label
| ) |  |

Parameters

  alpha | current value of the dual variable.
  inner | inner product \(\bw^\top \bx\) of the sample with the SVM model.
  norm2 | normalization factor \(\|\bx\|^2/\lambda n\).
  label | label \(y\) of the sample.

Returns
  incremental update \(\Delta\alpha\) of the dual variable.

See also
  Stochastic Dual Coordinate Ascent

## ◆ vl_svm_hinge_loss()

double vl_svm_hinge_loss | ( | double | inner,
|  | double | label
| ) |  |

Parameters

  inner | inner product between sample and model \(\bw^\top \bx\).
  label | sample label \(y\).

Returns
  value of the loss.

The interface is the same for a loss function, its derivative, or the conjugate loss.

See also
  SVM fundamentals

## ◆ vl_svm_hinge_loss_derivative()

double vl_svm_hinge_loss_derivative | ( | double | inner,
|  | double | label
| ) |  |

Parameters

  inner | inner product between sample and model \(\bw^\top \bx\).
  label | sample label \(y\).

Returns
  value of the loss.

The interface is the same for a loss function, its derivative, or the conjugate loss.

See also
  SVM fundamentals

## ◆ vl_svm_l1_conjugate_loss()

double vl_svm_l1_conjugate_loss | ( | double | u,
|  | double | label
| ) |  |

Parameters

  u | dual variable.
  label | label value.

Returns
  conjugate loss.

## ◆ vl_svm_l1_dca_update()

double vl_svm_l1_dca_update | ( | double | alpha,
|  | double | inner,
|  | double | norm2,
|  | double | label
| ) |  |

Parameters

  alpha | current value of the dual variable.
  inner | inner product \(\bw^\top \bx\) of the sample with the SVM model.
  norm2 | normalization factor \(\|\bx\|^2/\lambda n\).
  label | label \(y\) of the sample.

Returns
  incremental update \(\Delta\alpha\) of the dual variable.

See also
  Stochastic Dual Coordinate Ascent

## ◆ vl_svm_l1_loss()

double vl_svm_l1_loss | ( | double | inner,
|  | double | label
| ) |  |

Parameters

  inner | inner product between sample and model \(\bw^\top \bx\).
  label | sample label \(y\).

Returns
  value of the loss.

The interface is the same for a loss function, its derivative, or the conjugate loss.

See also
  SVM fundamentals

## ◆ vl_svm_l1_loss_derivative()

double vl_svm_l1_loss_derivative | ( | double | inner,
|  | double | label
| ) |  |

Parameters

  inner | inner product between sample and model \(\bw^\top \bx\).
  label | sample label \(y\).

Returns
  value of the loss.

The interface is the same for a loss function, its derivative, or the conjugate loss.

See also
  SVM fundamentals

## ◆ vl_svm_l2_conjugate_loss()

double vl_svm_l2_conjugate_loss | ( | double | u,
|  | double | label
| ) |  |

Parameters

  u | dual variable.
  label | label value.

Returns
  conjugate loss.

## ◆ vl_svm_l2_dca_update()

double vl_svm_l2_dca_update | ( | double | alpha,
|  | double | inner,
|  | double | norm2,
|  | double | label
| ) |  |

Parameters

  alpha | current value of the dual variable.
  inner | inner product \(\bw^\top \bx\) of the sample with the SVM model.
  norm2 | normalization factor \(\|\bx\|^2/\lambda n\).
  label | label \(y\) of the sample.

Returns
  incremental update \(\Delta\alpha\) of the dual variable.

See also
  Stochastic Dual Coordinate Ascent

## ◆ vl_svm_l2_loss()

double vl_svm_l2_loss | ( | double | inner,
|  | double | label
| ) |  |

Parameters

  inner | inner product between sample and model \(\bw^\top \bx\).
  label | sample label \(y\).

Returns
  value of the loss.

The interface is the same for a loss function, its derivative, or the conjugate loss.

See also
  SVM fundamentals

## ◆ vl_svm_l2_loss_derivative()

double vl_svm_l2_loss_derivative | ( | double | inner,
|  | double | label
| ) |  |

Parameters

  inner | inner product between sample and model \(\bw^\top \bx\).
  label | sample label \(y\).

Returns
  value of the loss.

The interface is the same for a loss function, its derivative, or the conjugate loss.

See also
  SVM fundamentals

## ◆ vl_svm_logistic_conjugate_loss()

double vl_svm_logistic_conjugate_loss | ( | double | u,
|  | double | label
| ) |  |

Parameters

  u | dual variable.
  label | label value.

Returns
  conjugate loss.

## ◆ vl_svm_logistic_dca_update()

double vl_svm_logistic_dca_update | ( | double | alpha,
|  | double | inner,
|  | double | norm2,
|  | double | label
| ) |  |

Parameters

  alpha | current value of the dual variable.
  inner | inner product \(\bw^\top \bx\) of the sample with the SVM model.
  norm2 | normalization factor \(\|\bx\|^2/\lambda n\).
  label | label \(y\) of the sample.

Returns
  incremental update \(\Delta\alpha\) of the dual variable.

See also
  Stochastic Dual Coordinate Ascent

## ◆ vl_svm_logistic_loss()

double vl_svm_logistic_loss | ( | double | inner,
|  | double | label
| ) |  |

Parameters

  inner | inner product between sample and model \(\bw^\top \bx\).
  label | sample label \(y\).

Returns
  value of the loss.

The interface is the same for a loss function, its derivative, or the conjugate loss.

See also
  SVM fundamentals

## ◆ vl_svm_logistic_loss_derivative()

double vl_svm_logistic_loss_derivative | ( | double | inner,
|  | double | label
| ) |  |

Parameters

  inner | inner product between sample and model \(\bw^\top \bx\).
  label | sample label \(y\).

Returns
  value of the loss.

The interface is the same for a loss function, its derivative, or the conjugate loss.

See also
  SVM fundamentals

## ◆ vl_svm_new()

VlSvm* vl_svm_new | ( | VlSvmSolverType | type,
|  | double const * | data,
|  | vl_size | dimension,
|  | vl_size | numData,
|  | double const * | labels,
|  | double | lambda
| ) |  |

Parameters

  type | type of SMV solver.
  data | a pointer to a matrix of data.
  dimension | dimension of the SVM model.
  numData | number of training samples.
  labels | training labels.
  lambda | regularizer parameter.

Returns
  the new object.

data has one column per sample, in `double` format. More advanced inputs can be used with vl_svm_new_with_dataset and vl_svm_new_with_abstract_data.

See also
  vl_svm_delete

## ◆ vl_svm_new_with_abstract_data()

VlSvm* vl_svm_new_with_abstract_data | ( | VlSvmSolverType | solver,
|  | void * | data,
|  | vl_size | dimension,
|  | vl_size | numData,
|  | double const * | labels,
|  | double | lambda
| ) |  |

Parameters

  solver | type of SMV solver.
  data | pointer to the data.
  dimension | dimension of the SVM model.
  numData | num training samples.
  labels | training samples labels.
  lambda | regularizer parameter.

Returns
  the new object.

After calling this function, vl_svm_set_data_functions must be used to setup suitable callbacks for the inner product and accumulation operations (

See also
  svm-data-abstraction).
  vl_svm_delete

## ◆ vl_svm_new_with_dataset()

VlSvm* vl_svm_new_with_dataset | ( | VlSvmSolverType | solver,
|  | VlSvmDataset * | dataset,
|  | double const * | labels,
|  | double | lambda
| ) |  |

Parameters

  solver | type of SMV solver.
  dataset | SVM dataset object
  labels | training samples labels.
  lambda | regularizer parameter.

Returns
  the new object.

See also
  vl_svm_delete

## ◆ vl_svm_set_bias()

void vl_svm_set_bias | ( | VlSvm * | self,
|  | double | b
| ) |  |

Parameters

  self | object.
  b | bias.

The function set the internal representation of the SVM bias to be equal to b (the bias multiplier is applied). The same remark that applies to vl_svm_set_model applies here too.

## ◆ vl_svm_set_bias_learning_rate()

void vl_svm_set_bias_learning_rate | ( | VlSvm * | self,
|  | double | rate
| ) |  |

Parameters

  self | object
  rate | bias learning rate (positive).

This parameter applies only to the SGD solver.

## ◆ vl_svm_set_bias_multiplier()

void vl_svm_set_bias_multiplier | ( | VlSvm * | self,
|  | double | b
| ) |  |

Parameters

  self | object
  b | bias multiplier.

The bias multiplier is the value of the constant feature appended to the data vectors to implement the bias (Adding a bias).

## ◆ vl_svm_set_conjugate_loss_function()

void vl_svm_set_conjugate_loss_function | ( | VlSvm * | self,
|  | VlSvmLossFunction | f
| ) |  |

## ◆ vl_svm_set_data_functions()

void vl_svm_set_data_functions | ( | VlSvm * | self,
|  | VlSvmInnerProductFunction | inner,
|  | VlSvmAccumulateFunction | acc
| ) |  |

Parameters

  self | object.
  inner | inner product function.
  acc | accumulate function.

See Data abstraction: working with compressed data.

## ◆ vl_svm_set_dca_update_function()

void vl_svm_set_dca_update_function | ( | VlSvm * | self,
|  | VlSvmDcaUpdateFunction | f
| ) |  |

## ◆ vl_svm_set_diagnostic_frequency()

void vl_svm_set_diagnostic_frequency | ( | VlSvm * | self,
|  | vl_size | f
| ) |  |

Parameters

  self | object.
  f | diagnostic frequency (`>=` 1).

A diagnostic round (to test for convergence and to printout information) is performed every f iterations.

## ◆ vl_svm_set_diagnostic_function()

void vl_svm_set_diagnostic_function | ( | VlSvm * | self,
|  | VlSvmDiagnosticFunction | f,
|  | void * | data
| ) |  |

Parameters

  self | object.
  f | diagnostic function pointer.
  data | pointer to data used by the diagnostic function.

## ◆ vl_svm_set_epsilon()

void vl_svm_set_epsilon | ( | VlSvm * | self,
|  | double | epsilon
| ) |  |

Parameters

  self | object
  epsilon | threshold (non-negative).

## ◆ vl_svm_set_iteration_number()

void vl_svm_set_iteration_number | ( | VlSvm * | self,
|  | vl_uindex | n
| ) |  |

Parameters

  self | object.
  n | iteration number.

If called before training, this can be used with SGD for a warm start, as the net effect is to slow down the learning rate.

## ◆ vl_svm_set_lambda()

void vl_svm_set_lambda | ( | VlSvm * | self,
|  | double | lambda
| ) |  |

Parameters

  self | object.
  lambda | regularizer parameter.

Note that lambda is usually set when calling a constructor for VlSvm as certain parameters, such as the maximum number of iterations, are tuned accordingly. This tuning is not performed when lambda is changed using this function.

## ◆ vl_svm_set_loss()

void vl_svm_set_loss | ( | VlSvm * | self,
|  | VlSvmLossType | loss
| ) |  |

Parameters

  self | object.
  loss | type of loss function.

See also
  Loss functions.

## ◆ vl_svm_set_loss_derivative_function()

void vl_svm_set_loss_derivative_function | ( | VlSvm * | self,
|  | VlSvmLossFunction | f
| ) |  |

## ◆ vl_svm_set_loss_function()

void vl_svm_set_loss_function | ( | VlSvm * | self,
|  | VlSvmLossFunction | f
| ) |  |

Parameters

  self | object.
  f | loss function callback.

Note that setting up a loss requires specifying more than just one callback. See Loss functions for details.

## ◆ vl_svm_set_max_num_iterations()

void vl_svm_set_max_num_iterations | ( | VlSvm * | self,
|  | vl_size | n
| ) |  |

Parameters

  self | object.
  n | maximum number of iterations.

## ◆ vl_svm_set_model()

void vl_svm_set_model | ( | VlSvm * | self,
|  | double const * | model
| ) |  |

Parameters

  self | object.
  model | model.

The function copies the content of the vector model to the internal model buffer. This operation can be used for warm start with the SGD algorithm, but has undefined effect with the SDCA algorithm.

## ◆ vl_svm_set_weights()

void vl_svm_set_weights | ( | VlSvm * | self,
|  | double const * | weights
| ) |  |

Parameters

  self | object.
  weights | data weights.

weights must be an array of non-negative weights. The loss of each data point is multiplied by the corresponding weight.

Set weights to `NULL` to weight the data uniformly by 1 (default).

Note that the weights array is not copied and must be valid througout the object lifetime (unless it is replaced).

## ◆ vl_svm_train()

void vl_svm_train | ( | VlSvm * | self | ) |

Parameters

  self | object.

The data on which the SVM operates is passed upon the cration of the VlSvm object. This function runs a solver to learn a corresponding model. See Getting started.