Spatiotemporal Attention for Realtime Segmentation of Corrupted Sequential Ultrasound Data

Image-guided diagnostics with AI assistance, e.g. compression-ultrasound for detecting deep vein thrombosis, requires stable, robust and real-time capable analysis algorithms that best support the user. When using anatomical segmentations for user guidance the spatiotemporal consistency is of great importance, but point-of-care modalities deliver signal which in many frames is hard to interpret. Since 2D+t models with 3D CNNs are not applicable for many mobile end devices,we propose a newspatiotemporal attention approach that re-uses deep backbone features from previous frames to learn and optimally fuse all available image information. Proof-of-concept experiments demonstrate an improvement of over 8% for the segmentation compared to simpler 2D+t models (using several frames as multi-channel input).