Generative Models of Nonlinear Component Superpositions for Unsupervised Learning from Visual and Auditory Data

Abstract:
In the nervous system of humans and animals, sensory data are represented as combinations of elementary data components. While for data such as sound waveforms the elementary components combine linearly, other data can better be explained by non-linear forms of component super-positions.
Using examples of visual and auditory spectrogram data, I will motivate and define probabilistic generative models of superposition non-linearities. Learning in these models is based on novel approaches to Maximum Likelihood parameter optimization related to variational Expectation Maximization (EM). I show example applications of the derived learning algorithms and quantitative comparisons with other approaches including recent versions of principle component analysis (PCA), independent component analysis (ICA), and non-negative matrix factorization (NMF). Finally, I will discuss the relation of the novel algorithms to learning in neural network models.

back