Bag of Pursuits and Neural Gas for Improved Sparse Coding


Sparse coding employs low-dimensional subspaces in order to encode high-dimensional signals. Finding the optimal subspaces is a difficult optimization task. We show that stochastic gradient descent is superior in finding the optimal subspaces compared to MOD and K-SVD, which are both state-of-the art methods. The improvement is most significant in the difficult setting of highly overlapping subspaces. We introduce the so-called ``Bag of Pursuits'' that is derived from Orthogonal Matching Pursuit. It provides an improved approximation of the optimal sparse coefficients, which, in turn, significantly improves the performance of the gradient descent approach as well as MOD and K-SVD. In addition, the ``Bag of Pursuits'' allows to employ a generalized version of the Neural Gas algorithm for sparse coding, which finally leads to an even more powerful method.
Original languageEnglish
Title of host publicationProceedings of COMPSTAT'2010
EditorsYves Lechevallier, Gilbert Saporta
Number of pages10
Place of PublicationHeidelberg
PublisherPhysica-Verlag HD
Publication date01.2010
ISBN (Print)978-3-7908-2603-6
ISBN (Electronic)978-3-7908-2604-3
Publication statusPublished - 01.2010
Event19th International Conference on Computational Statistics - Paris, France
Duration: 22.08.201027.08.2010


Dive into the research topics of 'Bag of Pursuits and Neural Gas for Improved Sparse Coding'. Together they form a unique fingerprint.

Cite this