Multi-scale GANs for Memory-efficient Generation of High Resolution Medical Images

Hristina Uzunova; Jan Ehrhardt; Fabian Jacob; Alex Frydrychowicz; Heinz Handels

doi:10.1007/978-3-030-32226-7_13

Multi-scale GANs for Memory-efficient Generation of High Resolution Medical Images

Hristina Uzunova^*, Jan Ehrhardt, Fabian Jacob, Alex Frydrychowicz, Heinz Handels

^*Corresponding author for this work

Institute of Medical Informatics

Abstract

Currently generative adversarial networks (GANs) are rarely applied to medical images of large sizes, especially 3D volumes, due to their large computational demand. We propose a novel multi-scale patch-based GAN approach to generate large high resolution 2D and 3D images. Our key idea is to first learn a low-resolution version of the image and then generate patches of successively growing resolutions conditioned on previous scales. In a domain translation use-case scenario, 3D thorax CTs of size 512 ³ and thorax X-rays of size 2048 ² are generated and we show that, due to the constant GPU memory demand of our method, arbitrarily large images of high resolution can be generated. Moreover, compared to common patch-based approaches, our multi-resolution scheme enables better image quality and prevents patch artifacts.

Original language	English
Title of host publication	MICCAI 2019: Medical Image Computing and Computer Assisted Intervention – MICCAI 2019
Editors	Dinggang Shen, Tianming Liu, Terry M. Peters, Lawrence H. Staib, Caroline Essert, Sean Zhou, Pew-Thian Yap, Ali Khan
Number of pages	9
Volume	11769
Place of Publication	Cham
Publisher	Springer Verlag
Publication date	10.10.2019
Pages	112-120
ISBN (Print)	978-3-030-32225-0
ISBN (Electronic)	978-3-030-32226-7
DOIs	https://doi.org/10.1007/978-3-030-32226-7_13
Publication status	Published - 10.10.2019
Event	22nd International Conference on Medical Image Computing and Computer-Assisted Intervention - Shenzhen, China Duration: 13.10.2019 → 17.10.2019 Conference number: 232939

Research Areas and Centers

Academic Focus: Biomedical Engineering

Access to Document

10.1007/978-3-030-32226-7_13

Cite this

Uzunova, H., Ehrhardt, J., Jacob, F., Frydrychowicz, A., & Handels, H. (2019). Multi-scale GANs for Memory-efficient Generation of High Resolution Medical Images. In D. Shen, T. Liu, T. M. Peters, L. H. Staib, C. Essert, S. Zhou, P-T. Yap, & A. Khan (Eds.), MICCAI 2019: Medical Image Computing and Computer Assisted Intervention – MICCAI 2019 (Vol. 11769, pp. 112-120). (Lecture Notes in Computer Science ; Vol. 11769 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-030-32226-7_13

Uzunova, Hristina ; Ehrhardt, Jan ; Jacob, Fabian et al. / Multi-scale GANs for Memory-efficient Generation of High Resolution Medical Images. MICCAI 2019: Medical Image Computing and Computer Assisted Intervention – MICCAI 2019. editor / Dinggang Shen ; Tianming Liu ; Terry M. Peters ; Lawrence H. Staib ; Caroline Essert ; Sean Zhou ; Pew-Thian Yap ; Ali Khan. Vol. 11769 Cham : Springer Verlag, 2019. pp. 112-120 (Lecture Notes in Computer Science ).

@inproceedings{90b64a24b63e4bd9b522f6fdfcb50b2d,

title = "Multi-scale GANs for Memory-efficient Generation of High Resolution Medical Images",

abstract = "Currently generative adversarial networks (GANs) are rarely applied to medical images of large sizes, especially 3D volumes, due to their large computational demand. We propose a novel multi-scale patch-based GAN approach to generate large high resolution 2D and 3D images. Our key idea is to first learn a low-resolution version of the image and then generate patches of successively growing resolutions conditioned on previous scales. In a domain translation use-case scenario, 3D thorax CTs of size 512 3 and thorax X-rays of size 2048 2 are generated and we show that, due to the constant GPU memory demand of our method, arbitrarily large images of high resolution can be generated. Moreover, compared to common patch-based approaches, our multi-resolution scheme enables better image quality and prevents patch artifacts.",

author = "Hristina Uzunova and Jan Ehrhardt and Fabian Jacob and Alex Frydrychowicz and Heinz Handels",

note = "Publisher Copyright: {\textcopyright} 2019, Springer Nature Switzerland AG. Copyright: Copyright 2019 Elsevier B.V., All rights reserved.; 22nd International Conference on Medical Image Computing and Computer-Assisted Intervention , MICCAI 2019 ; Conference date: 13-10-2019 Through 17-10-2019",

year = "2019",

month = oct,

day = "10",

doi = "10.1007/978-3-030-32226-7_13",

language = "English",

isbn = "978-3-030-32225-0",

volume = "11769",

series = "Lecture Notes in Computer Science ",

publisher = "Springer Verlag",

pages = "112--120",

editor = "Shen, {Dinggang } and Liu, {Tianming } and {M. Peters}, {Terry } and Staib, {Lawrence H. } and Essert, {Caroline } and Zhou, {Sean } and Yap, {Pew-Thian } and Khan, {Ali }",

booktitle = "MICCAI 2019: Medical Image Computing and Computer Assisted Intervention – MICCAI 2019",

}

Uzunova, H , Ehrhardt, J , Jacob, F , Frydrychowicz, A & Handels, H 2019, Multi-scale GANs for Memory-efficient Generation of High Resolution Medical Images. in D Shen, T Liu, T M. Peters, LH Staib, C Essert, S Zhou, P-T Yap & A Khan (eds), MICCAI 2019: Medical Image Computing and Computer Assisted Intervention – MICCAI 2019. vol. 11769, Lecture Notes in Computer Science , vol. 11769 LNCS, Springer Verlag, Cham, pp. 112-120, 22nd International Conference on Medical Image Computing and Computer-Assisted Intervention , Shenzhen, China, 13.10.19. https://doi.org/10.1007/978-3-030-32226-7_13

Multi-scale GANs for Memory-efficient Generation of High Resolution Medical Images. / Uzunova, Hristina ; Ehrhardt, Jan ; Jacob, Fabian et al.
MICCAI 2019: Medical Image Computing and Computer Assisted Intervention – MICCAI 2019. ed. / Dinggang Shen; Tianming Liu; Terry M. Peters; Lawrence H. Staib; Caroline Essert; Sean Zhou; Pew-Thian Yap; Ali Khan. Vol. 11769 Cham: Springer Verlag, 2019. p. 112-120 (Lecture Notes in Computer Science ; Vol. 11769 LNCS).

Research output: Chapters in Books/Reports/Conference Proceedings › Conference contribution › peer-review

TY - GEN

T1 - Multi-scale GANs for Memory-efficient Generation of High Resolution Medical Images

AU - Uzunova, Hristina

AU - Ehrhardt, Jan

AU - Jacob, Fabian

AU - Frydrychowicz, Alex

AU - Handels, Heinz

N1 - Conference code: 232939

PY - 2019/10/10

Y1 - 2019/10/10

N2 - Currently generative adversarial networks (GANs) are rarely applied to medical images of large sizes, especially 3D volumes, due to their large computational demand. We propose a novel multi-scale patch-based GAN approach to generate large high resolution 2D and 3D images. Our key idea is to first learn a low-resolution version of the image and then generate patches of successively growing resolutions conditioned on previous scales. In a domain translation use-case scenario, 3D thorax CTs of size 512 3 and thorax X-rays of size 2048 2 are generated and we show that, due to the constant GPU memory demand of our method, arbitrarily large images of high resolution can be generated. Moreover, compared to common patch-based approaches, our multi-resolution scheme enables better image quality and prevents patch artifacts.

AB - Currently generative adversarial networks (GANs) are rarely applied to medical images of large sizes, especially 3D volumes, due to their large computational demand. We propose a novel multi-scale patch-based GAN approach to generate large high resolution 2D and 3D images. Our key idea is to first learn a low-resolution version of the image and then generate patches of successively growing resolutions conditioned on previous scales. In a domain translation use-case scenario, 3D thorax CTs of size 512 3 and thorax X-rays of size 2048 2 are generated and we show that, due to the constant GPU memory demand of our method, arbitrarily large images of high resolution can be generated. Moreover, compared to common patch-based approaches, our multi-resolution scheme enables better image quality and prevents patch artifacts.

UR - https://www.scopus.com/record/display.uri?eid=2-s2.0-85075832533&origin=inward&txGid=d85476197ed5d79e8876c3ef963898ee

U2 - 10.1007/978-3-030-32226-7_13

DO - 10.1007/978-3-030-32226-7_13

M3 - Conference contribution

SN - 978-3-030-32225-0

VL - 11769

T3 - Lecture Notes in Computer Science

SP - 112

EP - 120

BT - MICCAI 2019: Medical Image Computing and Computer Assisted Intervention – MICCAI 2019

A2 - Shen, Dinggang

A2 - Liu, Tianming

A2 - M. Peters, Terry

A2 - Staib, Lawrence H.

A2 - Essert, Caroline

A2 - Zhou, Sean

A2 - Yap, Pew-Thian

A2 - Khan, Ali

PB - Springer Verlag

CY - Cham

T2 - 22nd International Conference on Medical Image Computing and Computer-Assisted Intervention

Y2 - 13 October 2019 through 17 October 2019

ER -

Uzunova H , Ehrhardt J , Jacob F , Frydrychowicz A , Handels H. Multi-scale GANs for Memory-efficient Generation of High Resolution Medical Images. In Shen D, Liu T, M. Peters T, Staib LH, Essert C, Zhou S, Yap P-T, Khan A, editors, MICCAI 2019: Medical Image Computing and Computer Assisted Intervention – MICCAI 2019. Vol. 11769. Cham: Springer Verlag. 2019. p. 112-120. (Lecture Notes in Computer Science ). doi: 10.1007/978-3-030-32226-7_13

Multi-scale GANs for Memory-efficient Generation of High Resolution Medical Images

Abstract

Research Areas and Centers

Access to Document

Other files and links

Fingerprint

Cite this