Analysis of Variability in Manual Liver Tumor Delineation in CT Scans

Jan Hendrik Moltz; Stefan Braunewell; Jan Rühaak; Frank Heckel; Sebastiano Barbieri; Lennart Tautz; Horst Karl Hahn; Heinz-Otto Peitgen

doi:10.1109/ISBI.2011.5872797

Analysis of Variability in Manual Liver Tumor Delineation in CT Scans

Jan Hendrik Moltz, Stefan Braunewell, Jan Rühaak, Frank Heckel, Sebastiano Barbieri, Lennart Tautz, Horst Karl Hahn, Heinz-Otto Peitgen

Institut für Mathematische Methoden der Bildverarbeitung

Abstract

Manual delineations by experts are often used as reference standards for validating segmentation algorithms, although it is well known that they always show some degree of variability. Our goal is to estimate the effects of using a limited number of expert segmentations. Given ten manual delineations of 13 liver tumors, we analyzed the volume error made by randomly selecting subsets of the ten segmentations compared to the complete set. We found that when using just one segmentation the expected error was 17% with a maximum of 35%. This means that it is questionable whether a comparison with a single reference allows a reliable validation. When three segmentations are chosen, the error is halved, so this might be a reasonable compromise between accuracy and viability of evaluation studies.

Originalsprache	Englisch
Titel	2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro
Seitenumfang	4
Erscheinungsort	Chicago, Illinois, USA
Herausgeber (Verlag)	IEEE
Erscheinungsdatum	01.03.2011
Seiten	1974-1977
ISBN (Print)	978-1-4244-4127-3
ISBN (elektronisch)	978-1-4244-4128-0
DOIs	https://doi.org/10.1109/ISBI.2011.5872797
Publikationsstatus	Veröffentlicht - 01.03.2011
Veranstaltung	2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro - Chicago, USA / Vereinigte Staaten Dauer: 30.03.2011 → 02.04.2011

UN SDGs

Dieser Output leistet einen Beitrag zu folgendem(n) Ziel(en) für nachhaltige Entwicklung

Zugriff auf Dokument

10.1109/ISBI.2011.5872797

https://ieeexplore.ieee.org/document/5872797/

Zitieren

Moltz, J. H., Braunewell, S., Rühaak, J., Heckel, F., Barbieri, S., Tautz, L., Hahn, H. K., & Peitgen, H.-O. (2011). Analysis of Variability in Manual Liver Tumor Delineation in CT Scans. in 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro (S. 1974-1977). (International Symposium on Biomedical Imaging: From Nano to Macro). IEEE. https://doi.org/10.1109/ISBI.2011.5872797

@inproceedings{55f132321e324b73a872e50caaa7d155,

title = "Analysis of Variability in Manual Liver Tumor Delineation in CT Scans",

abstract = "Manual delineations by experts are often used as reference standards for validating segmentation algorithms, although it is well known that they always show some degree of variability. Our goal is to estimate the effects of using a limited number of expert segmentations. Given ten manual delineations of 13 liver tumors, we analyzed the volume error made by randomly selecting subsets of the ten segmentations compared to the complete set. We found that when using just one segmentation the expected error was 17% with a maximum of 35%. This means that it is questionable whether a comparison with a single reference allows a reliable validation. When three segmentations are chosen, the error is halved, so this might be a reasonable compromise between accuracy and viability of evaluation studies.",

author = "Moltz, {Jan Hendrik} and Stefan Braunewell and Jan R{\"u}haak and Frank Heckel and Sebastiano Barbieri and Lennart Tautz and Hahn, {Horst Karl} and Heinz-Otto Peitgen",

year = "2011",

month = mar,

day = "1",

doi = "10.1109/ISBI.2011.5872797",

language = "English",

isbn = "978-1-4244-4127-3",

series = "International Symposium on Biomedical Imaging: From Nano to Macro",

publisher = "IEEE",

pages = "1974--1977",

booktitle = "2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro",

note = "2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro ; Conference date: 30-03-2011 Through 02-04-2011",

}

Moltz, JH, Braunewell, S, Rühaak, J, Heckel, F, Barbieri, S, Tautz, L, Hahn, HK & Peitgen, H-O 2011, Analysis of Variability in Manual Liver Tumor Delineation in CT Scans. in 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro. International Symposium on Biomedical Imaging: From Nano to Macro, IEEE, Chicago, Illinois, USA, S. 1974-1977, 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro, Chicago, Illinois, USA / Vereinigte Staaten, 30.03.11. https://doi.org/10.1109/ISBI.2011.5872797

Analysis of Variability in Manual Liver Tumor Delineation in CT Scans. / Moltz, Jan Hendrik; Braunewell, Stefan; Rühaak, Jan et al.
2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro. Chicago, Illinois, USA: IEEE, 2011. S. 1974-1977 (International Symposium on Biomedical Imaging: From Nano to Macro).

Publikation: Kapitel in Büchern/Berichten/Konferenzbänden › Konferenzbeitrag › Begutachtung

TY - GEN

T1 - Analysis of Variability in Manual Liver Tumor Delineation in CT Scans

AU - Moltz, Jan Hendrik

AU - Braunewell, Stefan

AU - Rühaak, Jan

AU - Heckel, Frank

AU - Barbieri, Sebastiano

AU - Tautz, Lennart

AU - Hahn, Horst Karl

AU - Peitgen, Heinz-Otto

PY - 2011/3/1

Y1 - 2011/3/1

N2 - Manual delineations by experts are often used as reference standards for validating segmentation algorithms, although it is well known that they always show some degree of variability. Our goal is to estimate the effects of using a limited number of expert segmentations. Given ten manual delineations of 13 liver tumors, we analyzed the volume error made by randomly selecting subsets of the ten segmentations compared to the complete set. We found that when using just one segmentation the expected error was 17% with a maximum of 35%. This means that it is questionable whether a comparison with a single reference allows a reliable validation. When three segmentations are chosen, the error is halved, so this might be a reasonable compromise between accuracy and viability of evaluation studies.

AB - Manual delineations by experts are often used as reference standards for validating segmentation algorithms, although it is well known that they always show some degree of variability. Our goal is to estimate the effects of using a limited number of expert segmentations. Given ten manual delineations of 13 liver tumors, we analyzed the volume error made by randomly selecting subsets of the ten segmentations compared to the complete set. We found that when using just one segmentation the expected error was 17% with a maximum of 35%. This means that it is questionable whether a comparison with a single reference allows a reliable validation. When three segmentations are chosen, the error is halved, so this might be a reasonable compromise between accuracy and viability of evaluation studies.

U2 - 10.1109/ISBI.2011.5872797

DO - 10.1109/ISBI.2011.5872797

M3 - Conference contribution

SN - 978-1-4244-4127-3

T3 - International Symposium on Biomedical Imaging: From Nano to Macro

SP - 1974

EP - 1977

BT - 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro

PB - IEEE

CY - Chicago, Illinois, USA

T2 - 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro

Y2 - 30 March 2011 through 2 April 2011

ER -

Moltz JH, Braunewell S, Rühaak J, Heckel F, Barbieri S, Tautz L et al. Analysis of Variability in Manual Liver Tumor Delineation in CT Scans. in 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro. Chicago, Illinois, USA: IEEE. 2011. S. 1974-1977. (International Symposium on Biomedical Imaging: From Nano to Macro). doi: 10.1109/ISBI.2011.5872797

Analysis of Variability in Manual Liver Tumor Delineation in CT Scans

Abstract

UN SDGs

Zugriff auf Dokument

Fingerprint

Zitieren