SupervisedDescentFitter

class menpofit.sdm.SupervisedDescentFitter(images, group=None, bounding_box_group_glob=None, sd_algorithm_cls=None, reference_shape=None, diagonal=None, holistic_features=<function no_op>, patch_features=<function no_op>, patch_shape=(17, 17), scales=(0.5, 1.0), n_iterations=3, n_perturbations=30, perturb_from_gt_bounding_box=<function noisy_shape_from_bounding_box>, batch_size=None, verbose=False)

Bases: MultiScaleNonParametricFitter

Class for training a multi-scale Supervised Descent model.

Parameters

• images (list of menpo.image.Image) – The list of training images.

• group (str or None, optional) – The landmark group that corresponds to the ground truth shape of each image. If None and the images only have a single landmark group, then that is the one that will be used. Note that all the training images need to have the specified landmark group.

• bounding_box_group_glob (glob or None, optional) – Glob that defines the bounding boxes to be used for training. If None, then the bounding boxes of the ground truth shapes are used.

• sd_algorithm_cls (class, optional) –

  The Supervised Descent algorithm to be used. The possible algorithms are are separated in the following four categories:

  Non-parametric:

  Class	Regression
  NonParametricNewton	IRLRegression
  NonParametricGaussNewton	IIRLRegression
  NonParametricPCRRegression	PCRRegression
  NonParametricOptimalRegression	OptimalLinearRegression
  NonParametricOPPRegression	OPPRegression

  Parametric shape:

  Class	Regression
  ParametricShapeNewton	IRLRegression
  ParametricShapeGaussNewton	IIRLRegression
  ParametricShapePCRRegression	PCRRegression
  ParametricShapeOptimalRegression	OptimalLinearRegression
  ParametricShapeOPPRegression	ParametricShapeOPPRegression

  Parametric appearance:

  Class	Regression
  ParametricAppearanceProjectOutNewton	IRLRegression
  ParametricAppearanceProjectOutGuassNewton	IIRLRegression
  ParametricAppearanceMeanTemplateNewton	IRLRegression
  ParametricAppearanceMeanTemplateGuassNewton	IIRLRegression
  ParametricAppearanceWeightsNewton	IRLRegression
  ParametricAppearanceWeightsGuassNewton	IIRLRegression

  Parametric shape and appearance:

  Class	Regression
  FullyParametricProjectOutNewton	IRLRegression
  FullyParametricProjectOutGaussNewton	IIRLRegression
  FullyParametricMeanTemplateNewton	IRLRegression
  FullyParametricWeightsNewton	IRLRegression
  FullyParametricProjectOutOPP	OPPRegression

• reference_shape (menpo.shape.PointCloud or None, optional) – The reference shape that will be used for normalising the size of the training images. The normalization is performed by rescaling all the training images so that the scale of their ground truth shapes matches the scale of the reference shape. Note that the reference shape is rescaled with respect to the diagonal before performing the normalisation. If None, then the mean shape will be used.

• diagonal (int or None, optional) – This parameter is used to rescale the reference shape so that the diagonal of its bounding box matches the provided value. In other words, this parameter controls the size of the model at the highest scale. If None, then the reference shape does not get rescaled.

• holistic_features (closure or list of closure, optional) – The features that will be extracted from the training images. Note that the features are extracted before warping the images to the reference shape. If list, then it must define a feature function per scale. Please refer to menpo.feature for a list of potential features.

• patch_features (closure or list of closure, optional) – The features that will be extracted from the patches of the training images. Note that, as opposed to holistic_features, these features are extracted after extracting the patches. If list, then it must define a feature function per scale. Please refer to menpo.feature and menpofit.feature for a list of potential features.

• patch_shape ((int, int) or list of (int, int), optional) – The shape of the patches to be extracted. If a list is provided, then it defines a patch shape per scale.

• scales (float or tuple of float, optional) – The scale value of each scale. They must provided in ascending order, i.e. from lowest to highest scale. If float, then a single scale is assumed.

• n_iterations (int or list of int, optional) – The number of iterations (cascades) of each level. If list, it must specify a value per scale. If int, then it defines the total number of iterations (cascades) over all scales.

• n_perturbations (int, optional) – The number of perturbations to be generated from each of the bounding boxes using perturb_from_gt_bounding_box.

• perturb_from_gt_bounding_box (callable, optional) – The function that will be used to generate the perturbations from each of the bounding boxes.

• batch_size (int or None, optional) – If an int is provided, then the training is performed in an incremental fashion on image batches of size equal to the provided value. If None, then the training is performed directly on the all the images.

• verbose (bool, optional) – If True, then the progress of the training will be printed.

References

1

X. Xiong, and F. De la Torre. “Supervised Descent Method and its applications to face alignment”, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2013.

2

P. N. Belhumeur, D. W. Jacobs, D. J. Kriegman, and N. Kumar. “Localizing parts of faces using a consensus of exemplars”, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2011.

fit_from_bb(image, bounding_box, max_iters=20, gt_shape=None, return_costs=False, **kwargs)

Fits the multi-scale fitter to an image given an initial bounding box.

Parameters

• image (menpo.image.Image or subclass) – The image to be fitted.

• bounding_box (menpo.shape.PointDirectedGraph) – The initial bounding box from which the fitting procedure will start. Note that the bounding box is used in order to align the model’s reference shape.

• max_iters (int or list of int, optional) – The maximum number of iterations. If int, then it specifies the maximum number of iterations over all scales. If list of int, then specifies the maximum number of iterations per scale.

• gt_shape (menpo.shape.PointCloud, optional) – The ground truth shape associated to the image.

• return_costs (bool, optional) – If True, then the cost function values will be computed during the fitting procedure. Then these cost values will be assigned to the returned fitting_result. Note that the costs computation increases the computational cost of the fitting. The additional computation cost depends on the fitting method. Only use this option for research purposes.

• kwargs (dict, optional) – Additional keyword arguments that can be passed to specific implementations.

Returns

fitting_result (MultiScaleNonParametricIterativeResult or subclass) – The multi-scale fitting result containing the result of the fitting procedure.

fit_from_shape(image, initial_shape, max_iters=20, gt_shape=None, return_costs=False, **kwargs)

Fits the multi-scale fitter to an image given an initial shape.

Parameters

• image (menpo.image.Image or subclass) – The image to be fitted.

• initial_shape (menpo.shape.PointCloud) – The initial shape estimate from which the fitting procedure will start.

• max_iters (int or list of int, optional) – The maximum number of iterations. If int, then it specifies the maximum number of iterations over all scales. If list of int, then specifies the maximum number of iterations per scale.

• gt_shape (menpo.shape.PointCloud, optional) – The ground truth shape associated to the image.

• return_costs (bool, optional) – If True, then the cost function values will be computed during the fitting procedure. Then these cost values will be assigned to the returned fitting_result. Note that the costs computation increases the computational cost of the fitting. The additional computation cost depends on the fitting method. Only use this option for research purposes.

• kwargs (dict, optional) – Additional keyword arguments that can be passed to specific implementations.

Returns

fitting_result (MultiScaleNonParametricIterativeResult or subclass) – The multi-scale fitting result containing the result of the fitting procedure.

increment(images, group=None, bounding_box_group_glob=None, verbose=False, batch_size=None)

Method to increment the trained SDM with a new set of training images.

Parameters

• images (list of menpo.image.Image) – The list of training images.

• group (str or None, optional) – The landmark group that corresponds to the ground truth shape of each image. If None and the images only have a single landmark group, then that is the one that will be used. Note that all the training images need to have the specified landmark group.

• bounding_box_group_glob (glob or None, optional) – Glob that defines the bounding boxes to be used for training. If None, then the bounding boxes of the ground truth shapes are used.

• verbose (bool, optional) – If True, then the progress of training will be printed.

• batch_size (int or None, optional) – If an int is provided, then the training is performed in an incremental fashion on image batches of size equal to the provided value. If None, then the training is performed directly on the all the images.

property holistic_features

The features that are extracted from the input image at each scale in ascending order, i.e. from lowest to highest scale.

Type

list of closure

property n_scales

Returns the number of scales.

Type

int

property reference_shape

The reference shape that is used to normalise the size of an input image so that the scale of its initial fitting shape matches the scale of this reference shape.

Type

menpo.shape.PointCloud

property scales

The scale value of each scale in ascending order, i.e. from lowest to highest scale.

Type

list of int or float