Abstract:Surgical phase recognition is critical for assisting surgeons in understanding surgical videos. Existing studies focused more on online surgical phase recognition, by leveraging preceding frames to predict the current frame. Despite great progress, they formulated the task as a series of frame-wise classification, which resulted in a lack of global context of the entire procedure and incoherent predictions. Moreover, besides online analysis, accurate offline surgical phase recognition is also in significant clinical need for retrospective analysis, and existing online algorithms do not fully analyze the entire video, thereby limiting accuracy in offline analysis. To overcome these challenges and enhance both online and offline inference capabilities, we propose a universal Surgical Phase Localization Network, named SurgPLAN++, with the principle of temporal detection. To ensure a global understanding of the surgical procedure, we devise a phase localization strategy for SurgPLAN++ to predict phase segments across the entire video through phase proposals. For online analysis, to generate high-quality phase proposals, SurgPLAN++ incorporates a data augmentation strategy to extend the streaming video into a pseudo-complete video through mirroring, center-duplication, and down-sampling. For offline analysis, SurgPLAN++ capitalizes on its global phase prediction framework to continuously refine preceding predictions during each online inference step, thereby significantly improving the accuracy of phase recognition. We perform extensive experiments to validate the effectiveness, and our SurgPLAN++ achieves remarkable performance in both online and offline modes, which outperforms state-of-the-art methods. The source code is available at <a class="link-external link-https" href="https://github.com/lxj22/SurgPLAN-Plus" rel="external noopener nofollow">this https URL</a>.

Frequency-Based Temporal Analysis Network for Accurate Phase Recognition from Surgical Videos

SPHASE: Multi-Modal and Multi-Branch Surgical Phase Segmentation Framework based on Temporal Convolutional Network.

SurgPLAN: Surgical Phase Localization Network for Phase Recognition

Attention-based Spatial-Temporal Neural Network for Accurate Phase Recognition in Minimally Invasive Surgery: Feasibility and Efficiency Verification

SF-TMN: SlowFast temporal modeling network for surgical phase recognition

OSFENet: Object Spatiotemporal Feature Enhanced Network for Surgical Phase Recognition

Surgical Temporal Action-aware Network with Sequence Regularization for Phase Recognition

TeCNO: Surgical Phase Recognition with Multi-Stage Temporal Convolutional Networks

Surgical Phase Recognition of Short Video Shots Based on Temporal Modeling of Deep Features

Laparoscopic Video Analysis Using Temporal, Attention, and Multi-Feature Fusion Based-Approaches

Not End-to-End: Explore Multi-Stage Architecture for Online Surgical Phase Recognition

Friends Across Time: Multi-Scale Action Segmentation Transformer for Surgical Phase Recognition

SurgPLAN++: Universal Surgical Phase Localization Network for Online and Offline Inference

Neural Finite-State Machines for Surgical Phase Recognition

Surgformer: Surgical Transformer with Hierarchical Temporal Attention for Surgical Phase Recognition

DeepPhase: Surgical Phase Recognition in CATARACTS Videos

Surgical Workflow Recognition Using Two-Stream Mixed Convolution Network

PoCaPNet: A Novel Approach for Surgical Phase Recognition Using Speech and X-Ray Images

Thoracic Surgery Video Analysis for Surgical Phase Recognition

DACAT: Dual-stream Adaptive Clip-aware Time Modeling for Robust Online Surgical Phase Recognition

Cascade Multi-Level Transformer Network for Surgical Workflow Analysis