Generative Topographic Mapping
From Biocrawler, the free encyclopedia.
In Statistics, GTM is basically a Bayesianised version of the Kohonnen network.
It assumes a latent (hidden) space which contains mixture components, arranged in space – like Kohonnen network nodes.
The Latent space is assumed to be non-linearly projected into data space, so that the nodes in this projected data space are near the data points. This is similar in spirit to the movement of Kohonnen nodes towards centers of data clusters. However GTM is properly probabilistic, so we treat the nodes as Gaussian mixture sources.
Additionally, we can now also put a prior on the possible deformations. Kohonnen nodes can wander around at will; GTM nodes are constrained by the allowable transformations and the probabilities on those transformations. If the deformations are well-behaved the the topology of the latent space is preserved.
In theory, an arbitrary nonlinear parametric deformation could be used. The optimal parameters could be found by gradient descent etc.
A computationally efficient alternative is (like with support vector machines) to create a large number of extra dimensions in the latent space, which are fixed arbitrary nonlinear transforms of the original latent dimensions. So the points in latent space acquire additional coordinates in these extra dimensions, given by the nonlinear transforms.
The deformation can then be taken as a linear transform of the enlarged latent space. This has the advantage that it can be optimised analytically, and integrated over to give the model posterior if required. By contrast, a finding nonlinear transforms generally requires heuristic computational optimization methods.
In data analysis, GTMs are like a nonlinear version of PCA, which allow high dimensional data to be modelled as resulting from Gaussian noise added to sources in lower-dimensional latent space. For example, to locate stocks in plottable 2D space based on their hi-D time-series shapes. Other applications may want to have fewer sources than data points, for example mixture models.
In generative deformational modelling, the latent and data spaces have the same dimensions, for example, 2D images or 1 audio sound waves. However we can still use GTM to model the deformation process. Extra 'empty' dimensions can be added to the source (known as the 'template' in this form of modelling), for example locating the 1D sound wave in 2D space. Further nonlinear dimensions are then added, got by combining the original dimensions. The enlarged latent space is then projected back into the 1D data space. The probability of a given projection is, as before, given by the product of the likelihood of the data under the Gaussian noise model with the prior on the deformation parameter. Note that unlike conventional spring-based deformation modelling, this has the advantage of being analytically optimizable. However it has the disadvantage of being a 'data-mining' approach, ie. the shape of the deformation prior is unlikely to be meaningful as an explanation of the possible deformations, as it is based on a very high, artificial- and arbitrarily constructed nonlinear latent space. For this reason the prior will have to be learned from data rather than created by a human expert, as is possible for spring-based models.
Note that the Kohonnen model was created as a biological model of neurons. GTMs have nothing to do with neuroscience or cognition, they are purely a statistical tool.
