On the Problem of Finding the Least Number of Features by L1-norm Minimisation

Sascha Klement; Thomas Martinetz

doi:10.1007/978-3-642-21735-7_39

On the Problem of Finding the Least Number of Features by L1-norm Minimisation

Institute of Neuro- and Bioinformatics

Abstract

Recently, the so-called Support Feature Machine (SFM) was proposed as a novel approach to feature selection for classification. It relies on approximating the zero-norm minimising weight vector of a separating hyperplane by optimising for its one-norm. In contrast to the L1-SVM it uses an additional constraint based on the average of data points. In experiments on artificial datasets we observe that the SFM is highly superior in returning a lower number of features and a larger percentage of truly relevant features. Here, we derive a necessary condition that the zero-norm and 1-norm solution coincide. Based on this condition the superiority can be made plausible.

Original language	English
Title of host publication	Artificial Neural Networks and Machine Learning – ICANN 2011
Editors	Timo Honkela, Włodzisław Duch, Mark Girolami, Samuel Kaski
Number of pages	8
Volume	6791
Publisher	Springer Berlin Heidelberg
Publication date	2011
Pages	315-322
ISBN (Print)	978-3-642-21734-0
ISBN (Electronic)	978-3-642-21735-7
DOIs	https://doi.org/10.1007/978-3-642-21735-7_39
Publication status	Published - 2011
Event	21st International Conference on Artificial Neural Networks - Espoo, Finland Duration: 14.06.2011 → 17.06.2011

Access to Document

10.1007/978-3-642-21735-7_39

KlMa11Final published version, 177 KB

https://link.springer.com/chapter/10.1007/978-3-642-21735-7_39

Cite this

Klement, S., & Martinetz, T. (2011). On the Problem of Finding the Least Number of Features by L1-norm Minimisation. In T. Honkela, W. Duch, M. Girolami, & S. Kaski (Eds.), Artificial Neural Networks and Machine Learning – ICANN 2011 (Vol. 6791, pp. 315-322). (Lecture Notes in Computer Science; Vol. 6791). Springer Berlin Heidelberg. https://doi.org/10.1007/978-3-642-21735-7_39

@inproceedings{abd8171b5f3f4b1c8e983d0370c325eb,

title = "On the Problem of Finding the Least Number of Features by L1-norm Minimisation",

abstract = "Recently, the so-called Support Feature Machine (SFM) was proposed as a novel approach to feature selection for classification. It relies on approximating the zero-norm minimising weight vector of a separating hyperplane by optimising for its one-norm. In contrast to the L1-SVM it uses an additional constraint based on the average of data points. In experiments on artificial datasets we observe that the SFM is highly superior in returning a lower number of features and a larger percentage of truly relevant features. Here, we derive a necessary condition that the zero-norm and 1-norm solution coincide. Based on this condition the superiority can be made plausible.",

author = "Sascha Klement and Thomas Martinetz",

year = "2011",

doi = "10.1007/978-3-642-21735-7_39",

language = "English",

isbn = "978-3-642-21734-0",

volume = "6791",

series = "Lecture Notes in Computer Science",

publisher = "Springer Berlin Heidelberg",

pages = "315--322",

editor = "Honkela, {Timo } and Duch, {W{\l}odzis{\l}aw } and Girolami, {Mark } and Kaski, {Samuel }",

booktitle = "Artificial Neural Networks and Machine Learning – ICANN 2011",

note = "21st International Conference on Artificial Neural Networks, ICANN 2011 ; Conference date: 14-06-2011 Through 17-06-2011",

}

Klement, S & Martinetz, T 2011, On the Problem of Finding the Least Number of Features by L1-norm Minimisation. in T Honkela, W Duch, M Girolami & S Kaski (eds), Artificial Neural Networks and Machine Learning – ICANN 2011. vol. 6791, Lecture Notes in Computer Science, vol. 6791, Springer Berlin Heidelberg, pp. 315-322, 21st International Conference on Artificial Neural Networks, Espoo, Finland, 14.06.11. https://doi.org/10.1007/978-3-642-21735-7_39

On the Problem of Finding the Least Number of Features by L1-norm Minimisation. / Klement, Sascha; Martinetz, Thomas.
Artificial Neural Networks and Machine Learning – ICANN 2011. ed. / Timo Honkela; Włodzisław Duch; Mark Girolami; Samuel Kaski. Vol. 6791 Springer Berlin Heidelberg, 2011. p. 315-322 (Lecture Notes in Computer Science; Vol. 6791).

Research output: Chapters in Books/Reports/Conference Proceedings › Conference contribution › peer-review

TY - GEN

T1 - On the Problem of Finding the Least Number of Features by L1-norm Minimisation

AU - Klement, Sascha

AU - Martinetz, Thomas

PY - 2011

Y1 - 2011

N2 - Recently, the so-called Support Feature Machine (SFM) was proposed as a novel approach to feature selection for classification. It relies on approximating the zero-norm minimising weight vector of a separating hyperplane by optimising for its one-norm. In contrast to the L1-SVM it uses an additional constraint based on the average of data points. In experiments on artificial datasets we observe that the SFM is highly superior in returning a lower number of features and a larger percentage of truly relevant features. Here, we derive a necessary condition that the zero-norm and 1-norm solution coincide. Based on this condition the superiority can be made plausible.

AB - Recently, the so-called Support Feature Machine (SFM) was proposed as a novel approach to feature selection for classification. It relies on approximating the zero-norm minimising weight vector of a separating hyperplane by optimising for its one-norm. In contrast to the L1-SVM it uses an additional constraint based on the average of data points. In experiments on artificial datasets we observe that the SFM is highly superior in returning a lower number of features and a larger percentage of truly relevant features. Here, we derive a necessary condition that the zero-norm and 1-norm solution coincide. Based on this condition the superiority can be made plausible.

U2 - 10.1007/978-3-642-21735-7_39

DO - 10.1007/978-3-642-21735-7_39

M3 - Conference contribution

SN - 978-3-642-21734-0

VL - 6791

T3 - Lecture Notes in Computer Science

SP - 315

EP - 322

BT - Artificial Neural Networks and Machine Learning – ICANN 2011

A2 - Honkela, Timo

A2 - Duch, Włodzisław

A2 - Girolami, Mark

A2 - Kaski, Samuel

PB - Springer Berlin Heidelberg

T2 - 21st International Conference on Artificial Neural Networks

Y2 - 14 June 2011 through 17 June 2011

ER -

On the Problem of Finding the Least Number of Features by L1-norm Minimisation

Abstract

Access to Document

Fingerprint

Cite this