Skip to main navigation Skip to search Skip to main content

CoM3eT: A foundation model for medical image analysis through federated, multidimensional context integration

  • J Raphael Schäfer
  • , Kai Geissler
  • , Till Nicke
  • , Chiara Tappermann
  • , Karoline Heber
  • , Eike Petersen
  • , Habib Mergan
  • , Lars Ole Schwen
  • , Nick Weiss
  • , Annika Gerken
  • Fraunhofer MEVIS

Abstract

Medical foundation models improve generalization when training AI models with limited labeled data, but remain confined to a single specialty, such as pathology or radiology, and to either sparse or dense outputs, such as classification or segmentation. Here, we present CoM3eT (Co-representation Multidimensional Multitask Medical Transformer), a medical vision foundation model that unifies pathology and radiology, sparse and dense predictions, and two- and higher-dimensional inputs by modeling multidimensional context with attention. CoM3eT outperformed other medical foundation models in an open competition spanning five tomographic, four whole-specimen, and three two-dimensional datasets, covering sparse and dense prediction tasks as well as report generation. When adapted across diverse clinical applications, training fewer than 2.5% of parameters achieved performance comparable to full fine-tuning, enabling research without access to high-performance GPU clusters. Applied to federated learning across hospitals, this approach achieved performance comparable to pooled-data training over internet connections and with consumer-grade hardware.
Original languageUndefined/Unknown
JournalarXiv preprint arXiv:2608.16268
Publication statusPublished - 2026

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

  1. SDG 9 - Industry, Innovation, and Infrastructure
    SDG 9 Industry, Innovation, and Infrastructure

Cite this