TY - JOUR

T1 - Multiple test procedures using an upper bound of the number of true hypotheses and their use for evaluating high-dimensional EEG data

AU - Hemmelmann, Claudia

AU - Ziegler, Andreas

AU - Guiard, Volker

AU - Weiss, Sabine

AU - Walther, Mario

AU - Vollandt, Rüdiger

N1 - Funding Information:
We are grateful to Manfred Horn for helpful discussions and for many valuable suggestions that greatly improved the clarity of the paper. We would also like to thank the referees for their helpful comments. This work was supported by the Deutsche Forschungsgemeinschaft (VO 683/2-1).

PY - 2008/5/15

Y1 - 2008/5/15

N2 - Frequency analyses of EEG data yield large data sets, which are high-dimensional and have to be evaluated statistically without a large number of false positive statements. There exist several methods to deal with this problem in multiple comparisons. Knowing the number of true hypotheses increases the power of some multiple test procedures, however the number of true hypotheses is unknown, in general, and must be estimated. In this paper, we derive two new multiple test procedures by using an upper bound for the number of true hypotheses. Our first procedure controls the generalized family-wise error rate, and thus is an improvement of the step-down procedure of Hommel and Hoffmann [Hommel G., Hoffmann T. Controlled uncertainty. In: Bauer P. Hommel G. Sonnemann E., editors. Multiple Hypotheses Testing, Heidelberg: Springer 1987;ISBN 3540505598:p. 154-61]. The second new procedure controls the false discovery proportion and improves upon the approach of Lehmann and Romano [Lehmann E.L., Romano J.P. Generalizations of the familywise error rate. Ann. Stat. 2005;33:1138-54]. By Monte-Carlo simulations, we show how the gain in power depends upon the accuracy of the estimate of the number of true hypotheses. The gain in power of our procedures is demonstrated in an example using EEG data on the processing of memorized lexical items.

AB - Frequency analyses of EEG data yield large data sets, which are high-dimensional and have to be evaluated statistically without a large number of false positive statements. There exist several methods to deal with this problem in multiple comparisons. Knowing the number of true hypotheses increases the power of some multiple test procedures, however the number of true hypotheses is unknown, in general, and must be estimated. In this paper, we derive two new multiple test procedures by using an upper bound for the number of true hypotheses. Our first procedure controls the generalized family-wise error rate, and thus is an improvement of the step-down procedure of Hommel and Hoffmann [Hommel G., Hoffmann T. Controlled uncertainty. In: Bauer P. Hommel G. Sonnemann E., editors. Multiple Hypotheses Testing, Heidelberg: Springer 1987;ISBN 3540505598:p. 154-61]. The second new procedure controls the false discovery proportion and improves upon the approach of Lehmann and Romano [Lehmann E.L., Romano J.P. Generalizations of the familywise error rate. Ann. Stat. 2005;33:1138-54]. By Monte-Carlo simulations, we show how the gain in power depends upon the accuracy of the estimate of the number of true hypotheses. The gain in power of our procedures is demonstrated in an example using EEG data on the processing of memorized lexical items.

UR - http://www.scopus.com/inward/record.url?scp=41049096921&partnerID=8YFLogxK

U2 - 10.1016/j.jneumeth.2007.12.013

DO - 10.1016/j.jneumeth.2007.12.013

M3 - Journal articles

C2 - 18279970

AN - SCOPUS:41049096921

VL - 170

SP - 158

EP - 164

JO - Journal of Neuroscience Methods

JF - Journal of Neuroscience Methods

SN - 0165-0270

IS - 1

ER -