Abstract
EMPIRE10 (Evaluation of Methods for Pulmonary Image REgistration 2010) is a public platform for fair and meaningful comparison of registration algorithms which are applied to a database of intrapatient thoracic CT image pairs. Evaluation of nonrigid registration techniques is a nontrivial task. This is compounded by the fact that researchers typically test only on their own data, which varies widely. For this reason, reliable assessment and comparison of different registration algorithms has been virtually impossible in the past. In this work we present the results of the launch phase of EMPIRE10, which comprised the comprehensive evaluation and comparison of 20 individual algorithms from leading academic and industrial research groups. All algorithms are applied to the same set of 30 thoracic CT pairs. Algorithm settings and parameters are chosen by researchers expert in the configuration of their own method and the evaluation is independent, using the same criteria for all participants. All results are published on the EMPIRE10 website (http://empire10.isi.uu.nl). The challenge remains ongoing and open to new participants. Full results from 24 algorithms have been published at the time of writing. This paper details the organization of the challenge, the data and evaluation methods and the outcome of the initial launch with 20 algorithms. The gain in knowledge and future work are discussed.
| Original language | English |
|---|---|
| Article number | 5782992 |
| Journal | IEEE Transactions on Medical Imaging |
| Volume | 30 |
| Issue number | 11 |
| Pages (from-to) | 1901-1920 |
| Number of pages | 20 |
| ISSN | 0278-0062 |
| DOIs | |
| Publication status | Published - 01.11.2011 |
Funding
Manuscript received March 04, 2011; revised May 10, 2011; accepted May 13, 2011. Date of publication May 31, 2011; date of current version November 02, 2011. The work of M. Peroni was supported by the Roberto Rocca Foundation. Asterisk indicates corresponding author. Due to space constraints, author affiliations are located at the end of the paper. Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/TMI.2011.2158349 ACKNOWLEDGMENT The authors would like to thank the following people for their assistance in various aspects of organizing the EMPIRE10 challenge: M. Staring, Leiden University Medical Center, The Netherlands, T. Kockelkorn and G. van Hoorn, Image Sciences Institute, University Medical Center Utrecht, The Netherlands. Dataset 17 was provided courtesy of the Léon Bérard Cancer Center and CREATIS lab, Lyon, France. Datasets 13, 16, and 23 are courtesy of the Department of Radiation Oncology, Stanford University School of Medicine and Philips Research.
