Recursive Autoconvolution for Unsupervised Learning of Convolutional Neural Networks

Boris Knyazev; Erhardt Barth; Thomas Martinetz

Recursive Autoconvolution for Unsupervised Learning of Convolutional Neural Networks

Boris Knyazev, Erhardt Barth, Thomas Martinetz

Institute of Neuro- and Bioinformatics

Abstract

In visual recognition tasks, such as image classification, unsupervised learning exploits cheap unlabeled data and can help to solve these tasks more efficiently. We show that the recursive autoconvolution operator, adopted from physics, boosts existing unsupervised methods by learning more discriminative filters. We take well established convolutional neural networks and train their filters layer-wise. In addition, based on previous works we design a network which extracts more than 600k features per sample, but with the total number of trainable parameters greatly reduced by introducing shared filters in higher layers. We evaluate our networks on the MNIST, CIFAR-10, CIFAR-100 and STL-10 image classification benchmarks and report several state of the art results among other unsupervised methods.

Original language	English
Publication status	Published - 02.06.2016
Event	International Joint Conference on Neural Networks (IJCNN 2017) - William A. Egan Civic and Convention Center , Anchorage, Alaska, United States Duration: 14.05.2017 → 19.05.2017 http://www.ijcnn.org/

Conference

Conference	International Joint Conference on Neural Networks (IJCNN 2017)
Country/Territory	United States
City	Anchorage, Alaska
Period	14.05.17 → 19.05.17
Internet address	http://www.ijcnn.org/

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Cite this

@conference{fa5dba9670ab4607a223580ff8eff313,

title = "Recursive Autoconvolution for Unsupervised Learning of Convolutional Neural Networks",

abstract = "In visual recognition tasks, such as image classification, unsupervised learning exploits cheap unlabeled data and can help to solve these tasks more efficiently. We show that the recursive autoconvolution operator, adopted from physics, boosts existing unsupervised methods by learning more discriminative filters. We take well established convolutional neural networks and train their filters layer-wise. In addition, based on previous works we design a network which extracts more than 600k features per sample, but with the total number of trainable parameters greatly reduced by introducing shared filters in higher layers. We evaluate our networks on the MNIST, CIFAR-10, CIFAR-100 and STL-10 image classification benchmarks and report several state of the art results among other unsupervised methods.",

author = "Boris Knyazev and Erhardt Barth and Thomas Martinetz",

note = "8 pages, accepted to International Joint Conference on Neural Networks (IJCNN 2017); International Joint Conference on Neural Networks (IJCNN 2017) ; Conference date: 14-05-2017 Through 19-05-2017",

year = "2016",

month = jun,

day = "2",

language = "English",

url = "http://www.ijcnn.org/",

}

TY - CONF

T1 - Recursive Autoconvolution for Unsupervised Learning of Convolutional Neural Networks

AU - Knyazev, Boris

AU - Barth, Erhardt

AU - Martinetz, Thomas

N1 - 8 pages, accepted to International Joint Conference on Neural Networks (IJCNN 2017)

PY - 2016/6/2

Y1 - 2016/6/2

N2 - In visual recognition tasks, such as image classification, unsupervised learning exploits cheap unlabeled data and can help to solve these tasks more efficiently. We show that the recursive autoconvolution operator, adopted from physics, boosts existing unsupervised methods by learning more discriminative filters. We take well established convolutional neural networks and train their filters layer-wise. In addition, based on previous works we design a network which extracts more than 600k features per sample, but with the total number of trainable parameters greatly reduced by introducing shared filters in higher layers. We evaluate our networks on the MNIST, CIFAR-10, CIFAR-100 and STL-10 image classification benchmarks and report several state of the art results among other unsupervised methods.

AB - In visual recognition tasks, such as image classification, unsupervised learning exploits cheap unlabeled data and can help to solve these tasks more efficiently. We show that the recursive autoconvolution operator, adopted from physics, boosts existing unsupervised methods by learning more discriminative filters. We take well established convolutional neural networks and train their filters layer-wise. In addition, based on previous works we design a network which extracts more than 600k features per sample, but with the total number of trainable parameters greatly reduced by introducing shared filters in higher layers. We evaluate our networks on the MNIST, CIFAR-10, CIFAR-100 and STL-10 image classification benchmarks and report several state of the art results among other unsupervised methods.

M3 - Conference Papers

T2 - International Joint Conference on Neural Networks (IJCNN 2017)

Y2 - 14 May 2017 through 19 May 2017

ER -

Recursive Autoconvolution for Unsupervised Learning of Convolutional Neural Networks

Abstract

Conference

UN SDGs

Fingerprint

Cite this