Documentation>C API
Fisher Vector encoding (FV)

Author
  David Novotny
  Andrea Vedaldi

fisher.h implements the Fisher Vectors (FV) image representation [22] [23] . A FV is a statistics capturing the distribution of a set of vectors, usually a set of local image descriptors.

Getting started demonstrates how to use the C API to compute the FV representation of an image. For further details refer to:

- Fisher vector fundamentals - Fisher Vector definition.
- Fisher vector derivation - Deriving the Fisher Vectors as a Fisher Kernel.
- Fisher kernel - The Fisher Kernel in general.

# Getting started

The Fisher Vector encoding of a set of features is obtained by using the function vl_fisher_encode. Note that the function requires a Gaussian Mixture Model (GMM) of the encoded feature distribution. In the following code, the result of the coding process is stored in the `enc` array and the improved fisher vector normalization is used.

float * means ;
float * covariances ;
float * priors ;
float * posteriors ;
float * enc;
// create a GMM object and cluster input data to get means, covariances
// and priors of the estimated mixture
gmm = vl_gmm_new (VL_TYPE_FLOAT) ;
vl_gmm_cluster (gmm, data, dimension, numData, numClusters);
// allocate space for the encoding
enc = vl_malloc(sizeof(float) * 2 * dimension * numClusters);
// run fisher encoding
vl_fisher_encode
(enc, VL_F_TYPE,
vl_gmm_get_means(gmm), dimension, numClusters,
vl_gmm_get_covariances(gmm),
vl_gmm_get_priors(gmm),
dataToEncode, numDataToEncode,
VL_FISHER_FLAG_IMPROVED
) ;
vl_fisher_encode
vl_size vl_fisher_encode(void *enc, vl_type dataType, void const *means, vl_size dimension, vl_size numClusters, void const *covariances, void const *priors, void const *data, vl_size numData, int flags)
Fisher vector encoding of a set of vectors.
Definition: fisher.c:550
VL_FISHER_FLAG_IMPROVED
#define VL_FISHER_FLAG_IMPROVED
Improved Fisher vector. This is the same as VL_FISHER_FLAG_SQUARE_ROOT|VL_FISHER_FLAG_NORMALIZED.
vl_malloc
void * vl_malloc(size_t n)
Allocate a memory block.
Definition: generic.c:1312
VL_TYPE_FLOAT
#define VL_TYPE_FLOAT
Definition: generic.h:35
vl_gmm_get_covariances
void const * vl_gmm_get_covariances(VlGMM const *self)
Get covariances.
Definition: gmm.c:513
vl_gmm_get_priors
void const * vl_gmm_get_priors(VlGMM const *self)
Get priors.
Definition: gmm.c:524
vl_gmm_cluster
double vl_gmm_cluster(VlGMM *self, void const *data, vl_size numData)
Run GMM clustering - includes initialization and EM.
Definition: gmm.c:1557
vl_gmm_get_means
void const * vl_gmm_get_means(VlGMM const *self)
Get means.
Definition: gmm.c:502
vl_gmm_new
VlGMM * vl_gmm_new(vl_type dataType, vl_size dimension, vl_size numComponents)
Create a new GMM object.
Definition: gmm.c:354

The performance of the standard Fisher Vector can be significantly improved [23] by using appropriate Normalization and improved Fisher vectors normalizations. These are controlled by the `flag` parameter of vl_fisher_encode.