LucasKanadeFitter¶
-
class
menpofit.lk.
LucasKanadeFitter
(template, group=None, holistic_features=<function no_op>, diagonal=None, transform=<class 'menpofit.transform.homogeneous.DifferentiableAlignmentAffine'>, scales=(0.5, 1.0), algorithm_cls=<class 'menpofit.lk.algorithm.InverseCompositional'>, residual_cls=<class 'menpofit.lk.residual.SSD'>)[source]¶ Bases:
MultiScaleNonParametricFitter
Class for defining a multi-scale Lucas-Kanade fitter that performs alignment with respect to a homogeneous transform. Please see the references for a basic list of relevant papers.
- Parameters
template (menpo.image.Image) – The template image.
group (str or
None
, optional) – The landmark group of the template that will be used as reference shape. IfNone
and the template only has a single landmark group, then that is the one that will be used.holistic_features (closure or list of closure, optional) – The features that will be extracted from the training images. Note that the features are extracted before warping the images to the reference shape. If list, then it must define a feature function per scale. Please refer to menpo.feature for a list of potential features.
diagonal (int or
None
, optional) – This parameter is used to rescale the reference shape (specified by group) so that the diagonal of its bounding box matches the provided value. In other words, this parameter controls the size of the model at the highest scale. IfNone
, then the reference shape does not get rescaled.scales (tuple of float, optional) – The scale value of each scale. They must provided in ascending order, i.e. from lowest to highest scale.
transform (subclass of
DP
andDX
, optional) – A differential homogeneous transform object, e.g.DifferentiableAlignmentAffine
.algorithm_cls (class, optional) –
The Lukas-Kanade optimisation algorithm that will get applied. The possible algorithms in menpofit.lk.algorithm are:
Class
Warp Direction
Warp Update
ForwardAdditive
Forward
Additive
ForwardCompositional
Forward
Compositional
InverseCompositional
Inverse
residual_cls (class subclass, optional) –
The residual that will get applied. All possible residuals are:
Class
Description
Sum of Squared Differences
Sum of Squared Differences on Fourier domain
Enhanced Correlation Coefficient
Image Gradient
Gradient Correlation
References
- 1
B.D. Lucas, and T. Kanade, “An iterative image registration technique with an application to stereo vision”, International Joint Conference on Artificial Intelligence, pp. 674-679, 1981.
- 2
G.D. Evangelidis, and E.Z. Psarakis. “Parametric Image Alignment Using Enhanced Correlation Coefficient Maximization”, IEEE Transactions on Pattern Analysis and Machine Intelligence, 30(10): 1858-1865, 2008.
- 3
A.B. Ashraf, S. Lucey, and T. Chen. “Fast Image Alignment in the Fourier Domain”, IEEE Proceedings of International Conference on Computer Vision and Pattern Recognition, pp. 2480-2487, 2010.
- 4
G. Tzimiropoulos, S. Zafeiriou, and M. Pantic. “Robust and Efficient Parametric Face Alignment”, IEEE Proceedings of International Conference on Computer Vision (ICCV), pp. 1847-1854, November 2011.
-
fit_from_bb
(image, bounding_box, max_iters=20, gt_shape=None, return_costs=False, **kwargs)¶ Fits the multi-scale fitter to an image given an initial bounding box.
- Parameters
image (menpo.image.Image or subclass) – The image to be fitted.
bounding_box (menpo.shape.PointDirectedGraph) – The initial bounding box from which the fitting procedure will start. Note that the bounding box is used in order to align the model’s reference shape.
max_iters (int or list of int, optional) – The maximum number of iterations. If int, then it specifies the maximum number of iterations over all scales. If list of int, then specifies the maximum number of iterations per scale.
gt_shape (menpo.shape.PointCloud, optional) – The ground truth shape associated to the image.
return_costs (bool, optional) – If
True
, then the cost function values will be computed during the fitting procedure. Then these cost values will be assigned to the returned fitting_result. Note that the costs computation increases the computational cost of the fitting. The additional computation cost depends on the fitting method. Only use this option for research purposes.kwargs (dict, optional) – Additional keyword arguments that can be passed to specific implementations.
- Returns
fitting_result (
MultiScaleNonParametricIterativeResult
or subclass) – The multi-scale fitting result containing the result of the fitting procedure.
-
fit_from_shape
(image, initial_shape, max_iters=20, gt_shape=None, return_costs=False, **kwargs)¶ Fits the multi-scale fitter to an image given an initial shape.
- Parameters
image (menpo.image.Image or subclass) – The image to be fitted.
initial_shape (menpo.shape.PointCloud) – The initial shape estimate from which the fitting procedure will start.
max_iters (int or list of int, optional) – The maximum number of iterations. If int, then it specifies the maximum number of iterations over all scales. If list of int, then specifies the maximum number of iterations per scale.
gt_shape (menpo.shape.PointCloud, optional) – The ground truth shape associated to the image.
return_costs (bool, optional) – If
True
, then the cost function values will be computed during the fitting procedure. Then these cost values will be assigned to the returned fitting_result. Note that the costs computation increases the computational cost of the fitting. The additional computation cost depends on the fitting method. Only use this option for research purposes.kwargs (dict, optional) – Additional keyword arguments that can be passed to specific implementations.
- Returns
fitting_result (
MultiScaleNonParametricIterativeResult
or subclass) – The multi-scale fitting result containing the result of the fitting procedure.
-
warped_images
(image, shapes)[source]¶ Given an input test image and a list of shapes, it warps the image into the shapes. This is useful for generating the warped images of a fitting procedure stored within a
LucasKanadeResult
.- Parameters
image (menpo.image.Image or subclass) – The input image to be warped.
shapes (list of menpo.shape.PointCloud) – The list of shapes in which the image will be warped. The shapes are obtained during the iterations of a fitting procedure.
- Returns
warped_images (list of menpo.image.MaskedImage or ndarray) – The warped images.
-
property
holistic_features
¶ The features that are extracted from the input image at each scale in ascending order, i.e. from lowest to highest scale.
- Type
list of closure
-
property
n_scales
¶ Returns the number of scales.
- Type
int
-
property
reference_shape
¶ The reference shape that is used to normalise the size of an input image so that the scale of its initial fitting shape matches the scale of this reference shape.
- Type
menpo.shape.PointCloud
-
property
scales
¶ The scale value of each scale in ascending order, i.e. from lowest to highest scale.
- Type
list of int or float