Abstract:\textit{Purpose} Estimating the interaction forces of instruments and tissue is of interest, particularly to provide haptic feedback during robot assisted minimally invasive interventions. Different approaches based on external and integrated force sensors have been proposed. These are hampered by friction, sensor size, and sterilizability. We investigate a novel approach to estimate the force vector directly from optical coherence tomography image volumes. \textit{Methods} We introduce a novel Siamese 3D CNN architecture. The network takes an undeformed reference volume and a deformed sample volume as an input and outputs the three components of the force vector. We employ a deep residual architecture with bottlenecks for increased efficiency. We compare the Siamese approach to methods using difference volumes and two-dimensional projections. Data was generated using a robotic setup to obtain ground truth force vectors for silicon tissue phantoms as well as porcine tissue. \textit{Results} Our method achieves a mean average error of 7.7 +- 4.3 mN when estimating the force vector. Our novel Siamese 3D CNN architecture outperforms single-path methods that achieve a mean average error of 11.59 +- 6.7 mN. Moreover, the use of volume data leads to significantly higher performance compared to processing only surface information which achieves a mean average error of 24.38 +- 22.0 mN. Based on the tissue dataset, our methods shows good generalization in between different subjects. \textit{Conclusions} We propose a novel image-based force estimation method using optical coherence tomography. We illustrate that capturing the deformation of subsurface structures substantially improves force estimation. Our approach can provide accurate force estimates in surgical setups when using intraoperative optical coherence tomography.

Deep learning-based 3D cellular force reconstruction directly from volumetric images

Imaging and Reconstruction of Cell Cortex Structures Near the Cell Surface

A Calibration-Informed Deep Learning Model for Three-Dimensional Particle Reconstruction of Volumetric Particle Image Velocimetry

Automated Quantification of Three-Dimensional Organization of Fiber-Like Structures in Biological Tissues.

Deep Learning Enabled Time-Lapse 3D Cell Analysis

Theoretical Analysis of Novel Quasi-3D Microscopy of Cell Deformation

Mechanical Behavior Study of Single Cell Contraction by Digital Image Correlation Technique

Deep Learning for Complex Displacement Field Measurement

High-Speed Automated Reconstruction of Drosophila Larval Brain from Volumetric EM Data

DeepLFM: Deep Learning-based 3D Reconstruction for Light Field Microscopy

Spatiotemporally Super-Resolved Volumetric Traction Force Microscopy

Force Estimation from OCT Volumes using 3D CNNs

Video-rate 3D imaging of living cells using Fourier view-channel-depth light field microscopy

Quantifying Cell Traction Forces at the Individual Fiber Scale in 3D: A Novel Approach Based on Deformable Photopolymerized Fiber Arrays

DMCVR: Morphology-Guided Diffusion Model for 3D Cardiac Volume Reconstruction

Visualizing and quantifying dynamic cellular forces with photonic crystal hydrogels

Cellular Traction Force Reconstruction Based on a Self-adaptive Filtering Scheme

Vision-Based Cellular Force Measurement Using An Elastic Microfabricated Device

Reconstructing 3D Biomedical Architectural Order at Multiple Spatial Scales with Multimodal Stack Input

Computational Reconstruction of Cell and Tissue Surfaces for Modeling and Data Analysis

Feature Extraction of 3T3 Fibroblast Microtubule Based on Discrete Wavelet Transform and Lucy–Richardson Deconvolution Methods