GenerativeAPS

class menpofit.aps.GenerativeAPS(images, group=None, appearance_graph=None, shape_graph=None, deformation_graph=None, holistic_features=<function no_op>, reference_shape=None, diagonal=None, scales=(0.5, 1.0), patch_shape=(17, 17), patch_normalisation=<function no_op>, use_procrustes=True, precision_dtype=<class 'numpy.float32'>, max_shape_components=None, n_appearance_components=None, can_be_incremented=False, verbose=False, batch_size=None)[source]

Bases: object

Class for training a multi-scale Generative Active Pictorial Structures model. Please see the references for a basic list of relevant papers.

Parameters
  • images (list of menpo.image.Image) – The list of training images.

  • group (str or None, optional) – The landmark group that will be used to train the AAM. If None and the images only have a single landmark group, then that is the one that will be used. Note that all the training images need to have the specified landmark group.

  • appearance_graph (list of graphs or a single graph or None, optional) – The graph to be used for the appearance menpo.model.GMRFModel training. It must be a menpo.shape.UndirectedGraph. If None, then a menpo.model.PCAModel is used instead.

  • shape_graph (list of graphs or a single graph or None, optional) – The graph to be used for the shape menpo.model.GMRFModel training. It must be a menpo.shape.UndirectedGraph. If None, then the shape model is built using menpo.model.PCAModel.

  • deformation_graph (list of graphs or a single graph or None, optional) – The graph to be used for the deformation menpo.model.GMRFModel training. It must be either a menpo.shape.DirectedGraph or a menpo.shape.Tree. If None, then the minimum spanning tree of the data is computed.

  • holistic_features (closure or list of closure, optional) – The features that will be extracted from the training images. Note that the features are extracted before warping the images to the reference shape. If list, then it must define a feature function per scale. Please refer to menpo.feature for a list of potential features.

  • reference_shape (menpo.shape.PointCloud or None, optional) – The reference shape that will be used for building the APS. The purpose of the reference shape is to normalise the size of the training images. The normalization is performed by rescaling all the training images so that the scale of their ground truth shapes matches the scale of the reference shape. Note that the reference shape is rescaled with respect to the diagonal before performing the normalisation. If None, then the mean shape will be used.

  • diagonal (int or None, optional) – This parameter is used to rescale the reference shape so that the diagonal of its bounding box matches the provided value. In other words, this parameter controls the size of the model at the highest scale. If None, then the reference shape does not get rescaled.

  • scales (float or tuple of float, optional) – The scale value of each scale. They must provided in ascending order, i.e. from lowest to highest scale. If float, then a single scale is assumed.

  • patch_shape ((int, int) or list of (int, int), optional) – The shape of the patches to be extracted. If a list is provided, then it defines a patch shape per scale.

  • patch_normalisation (list of callable or a single callable, optional) – The normalisation function to be applied on the extracted patches. If list, then it must have length equal to the number of scales. If a single patch normalization callable, then this is the one applied to all scales.

  • use_procrustes (bool, optional) – If True, then Generalized Procrustes Alignment is applied before building the deformation model.

  • precision_dtype (numpy.dtype, optional) – The data type of the appearance GMRF’s precision matrix. For example, it can be set to numpy.float32 for single precision or to numpy.float64 for double precision. Even though the precision matrix is stored as a scipy.sparse matrix, this parameter has a big impact on the amount of memory required by the model.

  • max_shape_components (int, float, list of those or None, optional) – The number of shape components to keep. If int, then it sets the exact number of components. If float, then it defines the variance percentage that will be kept. If list, then it should define a value per scale. If a single number, then this will be applied to all scales. If None, then all the components are kept. Note that the unused components will be permanently trimmed.

  • n_appearance_components (list of int or int or None, optional) – The number of appearance components used for building the appearance menpo.shape.GMRFModel. If list, then it must have length equal to the number of scales. If a single int, then this is the one applied to all scales. If None, the covariance matrix of each edge is inverted using np.linalg.inv. If int, it is inverted using truncated SVD using the specified number of components.

  • can_be_incremented (bool, optional) – In case you intend to incrementally update the model in the future, then this flag must be set to True from the first place. Note that if True, the appearance and deformation menpo.shape.GMRFModel models will occupy double memory.

  • verbose (bool, optional) – If True, then the progress of building the APS will be printed.

  • batch_size (int or None, optional) – If an int is provided, then the training is performed in an incremental fashion on image batches of size equal to the provided value. If None, then the training is performed directly on the all the images.

References

1

E. Antonakos, J. Alabort-i-Medina, and S. Zafeiriou, “Active Pictorial Structures”, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA, pp. 1872-1882, June 2015.

increment(images, group=None, batch_size=None, verbose=False)[source]

Method that incrementally updates the APS model with a new batch of training images.

Parameters
  • images (list of menpo.image.Image) – The list of training images.

  • group (str or None, optional) – The landmark group that will be used to train the APS. If None and the images only have a single landmark group, then that is the one that will be used. Note that all the training images need to have the specified landmark group.

  • batch_size (int or None, optional) – If an int is provided, then the training is performed in an incremental fashion on image batches of size equal to the provided value. If None, then the training is performed directly on the all the images.

  • verbose (bool, optional) – If True, then the progress of building the APS will be printed.

instance(shape_weights=None, scale_index=- 1, as_graph=False)[source]

Generates an instance of the shape model.

Parameters
  • shape_weights ((n_weights,) ndarray or list or None, optional) – The weights of the shape model that will be used to create a novel shape instance. If None, the weights are assumed to be zero, thus the mean shape is used.

  • scale_index (int, optional) – The scale to be used.

  • as_graph (bool, optional) – If True, then the instance will be returned as a menpo.shape.PointTree or a menpo.shape.PointDirectedGraph, depending on the type of the deformation graph.

random_instance(scale_index=- 1, as_graph=False)[source]

Generates a random instance of the APS.

Parameters
  • scale_index (int, optional) – The scale to be used.

  • as_graph (bool, optional) – If True, then the instance will be returned as a menpo.shape.PointTree or a menpo.shape.PointDirectedGraph, depending on the type of the deformation graph.

view_deformation_model(scale_index=- 1, n_std=2, render_colour_bar=False, colour_map='jet', image_view=True, figure_id=None, new_figure=False, render_graph_lines=True, graph_line_colour='b', graph_line_style='-', graph_line_width=1.0, ellipse_line_colour='r', ellipse_line_style='-', ellipse_line_width=1.0, render_markers=True, marker_style='o', marker_size=5, marker_face_colour='k', marker_edge_colour='k', marker_edge_width=1.0, render_axes=False, axes_font_name='sans-serif', axes_font_size=10, axes_font_style='normal', axes_font_weight='normal', crop_proportion=0.1, figure_size=(7, 7))[source]

Visualize the deformation model by plotting a Gaussian ellipsis per graph edge.

Parameters
  • scale_index (int, optional) – The scale to be used.

  • n_std (float, optional) – This defines the size of the ellipses in terms of number of standard deviations.

  • render_colour_bar (bool, optional) – If True, then the ellipses will be coloured based on their normalized standard deviations and a colour bar will also appear on the side. If False, then all the ellipses will have the same colour.

  • colour_map (str, optional) – A valid Matplotlib colour map. For more info, please refer to matplotlib.cm.

  • image_view (bool, optional) – If True the ellipses will be rendered in the image coordinates system.

  • figure_id (object, optional) – The id of the figure to be used.

  • new_figure (bool, optional) – If True, a new figure is created.

  • render_graph_lines (bool, optional) – Defines whether to plot the graph’s edges.

  • graph_line_colour (See Below, optional) –

    The colour of the lines of the graph’s edges. Example options:

    {r, g, b, c, m, k, w}
    or
    (3, ) ndarray
    

  • graph_line_style ({-, --, -., :}, optional) – The style of the lines of the graph’s edges.

  • graph_line_width (float, optional) – The width of the lines of the graph’s edges.

  • ellipse_line_colour (See Below, optional) –

    The colour of the lines of the ellipses. Example options:

    {r, g, b, c, m, k, w}
    or
    (3, ) ndarray
    

  • ellipse_line_style ({-, --, -., :}, optional) – The style of the lines of the ellipses.

  • ellipse_line_width (float, optional) – The width of the lines of the ellipses.

  • render_markers (bool, optional) – If True, the centers of the ellipses will be rendered.

  • marker_style (See Below, optional) –

    The style of the centers of the ellipses. Example options

    {., ,, o, v, ^, <, >, +, x, D, d, s, p, *, h, H, 1, 2, 3, 4, 8}
    

  • marker_size (int, optional) – The size of the centers of the ellipses in points.

  • marker_face_colour (See Below, optional) –

    The face (filling) colour of the centers of the ellipses. Example options

    {r, g, b, c, m, k, w}
    or
    (3, ) ndarray
    

  • marker_edge_colour (See Below, optional) –

    The edge colour of the centers of the ellipses. Example options

    {r, g, b, c, m, k, w}
    or
    (3, ) ndarray
    

  • marker_edge_width (float, optional) – The edge width of the centers of the ellipses.

  • render_axes (bool, optional) – If True, the axes will be rendered.

  • axes_font_name (See Below, optional) –

    The font of the axes. Example options

    {serif, sans-serif, cursive, fantasy, monospace}
    

  • axes_font_size (int, optional) – The font size of the axes.

  • axes_font_style ({normal, italic, oblique}, optional) – The font style of the axes.

  • axes_font_weight (See Below, optional) –

    The font weight of the axes. Example options

    {ultralight, light, normal, regular, book, medium, roman,
     semibold,demibold, demi, bold, heavy, extra bold, black}
    

  • crop_proportion (float, optional) – The proportion to be left around the centers’ pointcloud.

  • figure_size ((float, float) tuple or None optional) – The size of the figure in inches.

property n_scales

Returns the number of scales.

Type

int