DlibERT

class menpofit.dlib.DlibERT(images, group=None, bounding_box_group_glob=None, reference_shape=None, diagonal=None, scales=(0.5, 1.0), n_perturbations=30, n_dlib_perturbations=1, perturb_from_gt_bounding_box=<function noisy_shape_from_bounding_box>, n_iterations=10, feature_padding=0, n_pixel_pairs=400, distance_prior_weighting=0.1, regularisation_weight=0.1, n_split_tests=20, n_trees=500, n_tree_levels=5, verbose=False)[source]

Bases: MultiScaleNonParametricFitter

Class for training a multi-scale Ensemble of Regression Trees model. This class uses the implementation provided by the official DLib package (http://dlib.net/) and makes it multi-scale.

Parameters

  • images (list of menpo.image.Image) – The list of training images.

  • group (str or None, optional) – The landmark group that corresponds to the ground truth shape of each image. If None and the images only have a single landmark group, then that is the one that will be used. Note that all the training images need to have the specified landmark group.

  • bounding_box_group_glob (glob or None, optional) – Glob that defines the bounding boxes to be used for training. If None, then the bounding boxes of the ground truth shapes are used.

  • reference_shape (menpo.shape.PointCloud or None, optional) – The reference shape that will be used for normalising the size of the training images. The normalization is performed by rescaling all the training images so that the scale of their ground truth shapes matches the scale of the reference shape. Note that the reference shape is rescaled with respect to the diagonal before performing the normalisation. If None, then the mean shape will be used.

  • diagonal (int or None, optional) – This parameter is used to rescale the reference shape so that the diagonal of its bounding box matches the provided value. In other words, this parameter controls the size of the model at the highest scale. If None, then the reference shape does not get rescaled.

  • scales (float or tuple of float, optional) – The scale value of each scale. They must provided in ascending order, i.e. from lowest to highest scale. If float, then a single scale is assumed.

  • n_perturbations (int or None, optional) – The number of perturbations to be generated from each of the bounding boxes using perturb_from_gt_bounding_box. Note that the total number of perturbations is n_perturbations * n_dlib_perturbations.

  • perturb_from_gt_bounding_box (function, optional) – The function that will be used to generate the perturbations.

  • n_dlib_perturbations (int or None or list of those, optional) – The number of perturbations to be generated from the part of DLib. DLib calls this “oversampling amount”. If list, it must specify a value per scale. Note that the total number of perturbations is n_perturbations * n_dlib_perturbations.

  • n_iterations (int or list of int, optional) – The number of iterations (cascades) of each level. If list, it must specify a value per scale. If int, then it defines the total number of iterations (cascades) over all scales.

  • feature_padding (float or list of float, optional) – When we randomly sample the pixels for the feature pool we do so in a box fit around the provided training landmarks. By default, this box is the tightest box that contains the landmarks. However, you can expand or shrink the size of the pixel sampling region by setting a different value of padding. To explain this precisely, for a padding of 0 we say that the pixels are sampled from a box of size 1x1. The padding value is added to each side of the box. So a padding of 0.5 would cause the algorithm to sample pixels from a box that was 2x2, effectively multiplying the area pixels are sampled from by 4. Similarly, setting the padding to -0.2 would cause it to sample from a box 0.6x0.6 in size. If list, it must specify a value per scale.

  • n_pixel_pairs (int or list of int, optional) – P parameter from [1]. At each level of the cascade we randomly sample pixels from the image. These pixels are used to generate features for the random trees. So in general larger settings of this parameter give better accuracy but make the algorithm run slower. If list, it must specify a value per scale.

  • distance_prior_weighting (float or list of float, optional) – To decide how to split nodes in the regression trees the algorithm looks at pairs of pixels in the image. These pixel pairs are sampled randomly but with a preference for selecting pixels that are near each other. This parameter controls this “nearness” preference. In particular, smaller values will make the algorithm prefer to select pixels close together and larger values will make it care less about picking nearby pixel pairs. Note that this is the inverse of how it is defined in [1]. For this object, you should think of distance_prior_weighting as “the fraction of the bounding box will we traverse to find a neighboring pixel”. Nominally, this is normalized between 0 and 1. So reasonable settings are values in the range (0, 1). If list, it must specify a value per scale.

  • regularisation_weight (float or list of float, optional) – Boosting regularization parameter - nu from [1]. Larger values may cause overfitting but improve performance on training data. If list, it must specify a value per scale.

  • n_split_tests (int or list of int, optional) – When generating the random trees we randomly sample n_split_tests possible split features at each node and pick the one that gives the best split. Larger values of this parameter will usually give more accurate outputs but take longer to train. It is equivalent of S from [1]. If list, it must specify a value per scale.

  • n_trees (int or list of int, optional) – Number of trees created for each cascade. The total number of trees in the learned model is equal n_trees * n_tree_levels. Equivalent to K from [1]. If list, it must specify a value per scale.

  • n_tree_levels (int or list of int, optional) – The number of levels in the tree (depth of tree). In particular, there are pow(2, n_tree_levels) leaves in each tree. Equivalent to F from [1]. If list, it must specify a value per scale.

  • verbose (bool, optional) – If True, then the progress of building ERT will be printed.

References

1

V. Kazemi, and J. Sullivan. “One millisecond face alignment with an ensemble of regression trees.” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2014.

fit_from_bb(image, bounding_box, gt_shape=None)[source]

Fits the model to an image given an initial bounding box.

Parameters

  • image (menpo.image.Image or subclass) – The image to be fitted.

  • bounding_box (menpo.shape.PointDirectedGraph) – The initial bounding box from which the fitting procedure will start.

  • gt_shape (menpo.shape.PointCloud, optional) – The ground truth shape associated to the image.

Returns

fitting_result (MultiScaleNonParametricIterativeResult) – The result of the fitting procedure.

fit_from_shape(image, initial_shape, gt_shape=None)[source]

Fits the model to an image. Note that it is not possible to initialise the fitting process from a shape. Thus, this method raises a warning and calls fit_from_bb with the bounding box of the provided initial_shape.

Parameters

  • image (menpo.image.Image or subclass) – The image to be fitted.

  • initial_shape (menpo.shape.PointCloud) – The initial shape estimate from which the fitting procedure will start. Note that the shape won’t actually be used, only its bounding box.

  • gt_shape (menpo.shape.PointCloud, optional) – The ground truth shape associated to the image.

Returns

fitting_result (MultiScaleNonParametricIterativeResult) – The result of the fitting procedure.

property holistic_features

The features that are extracted from the input image at each scale in ascending order, i.e. from lowest to highest scale.

Type

list of closure

property n_scales

Returns the number of scales.

Type

int

property reference_shape

The reference shape that is used to normalise the size of an input image so that the scale of its initial fitting shape matches the scale of this reference shape.

Type

menpo.shape.PointCloud

property scales

The scale value of each scale in ascending order, i.e. from lowest to highest scale.

Type

list of int or float