Enhancing brain tumor detection using optical coherence tomography and variational autoencoders

Neurosurgical intervention is critical in brain tumor treatment, with long-term survival closely linked to the extent of tumor resection. The goal is to completely remove tumor tissue while preserving healthy tissue, a challenging task due to the diffuse nature of some brain tumors, such as glioblastoma, which infiltrate healthy tissue in ways that are difficult to distinguish histologically. Current intraoperative imaging techniques, including MRI and fluorescence microscopy, are limited in reliably identifying tumor tissue. Optical coherence tomography (OCT) offers a promising alternative, providing non-invasive, high-resolution cross-sectional images. This study investigates the use of a variational autoencoder (VAE) in combination with an evidential learning framework to enhance the classification of brain tissues in OCT images. The classification approach, applied to ex vivo OCT images captured at a wavelength of 1300 nm, achieved an average precision of 0.87 and a recall of 0.88 for the discrimination of healthy and tumorous brain tissue with consideration of prediction uncertainties. This method demonstrated improved discrimination between healthy white matter and tumor-infiltrated white matter compared to previous studies.